We have seen an increasing interest to get Flume natively running on

Windows

Operating Systems. Various IIS driven companies has developed some .NET stuff to send logs from a IIS to Flume agents, running on the box as the IIS runs, as example. 

Note that Flume 1.x is not officially supported on Windows and the following instructions are only meant to help interested parties - it does not mean that Flume is supported by the community on Windows. Please note that YMMV.

Prerequisites

Build system
maven 3x, git, jdk1.6.x, WinRAR (or similar program)

Apache Flume agent node
jdk1.6.x, WinRAR (or similar program), Ultraedit++ or similar texteditor

Prepare the Windows Build Box

  1. Download and install JDK 1.6x
  2. Set the environment variables
    1. => Start - type "env" into the search box, select "Edit system environment variables", click Environment Variables, Select "New" from the "Systems variables" box, type "JAVA_HOME" into "Variable name" and the path to your JDK installation into "Variable value" (Example: C:\Program Files (x86)\Java\jdk1.6.0_33)
  3. Download and install maven 3
  4. Set the environment variables
    1. => from the field "System variables" select New, Variable name "M2_HOME", value Path to your maven installation (Example: D:\Maven\apache-maven-3.0.4)
    2. => from the field "User variables for (your username)" select New, Variable name "M2", value "%M2_HOME%\bin"
    3. => from the field "User variables for (your username)" select New, Variable name "MAVEN_OPTS", value "-XX:MaxPermSize=1024M"
  5. Close the CMD if you have running one
  6. Download and install msysgit or use a similar program to checkout a git repo
  7. Change into the previously outchecked directory
  8. Build with "mvn clean" and "mvn package -DskipTests"

Start a flume agent

  1. Edit the log4j.properties file too, simple change the logger facility from
    flume.root.logger=INFO,LOGFILE
    into
    flume.root.logger=DEBUG,CONSOLE
  2. Start Flume:
    flume-1.3.0-SNAPSHOT>"c:\Program Files (x86)\Java\jdk1.6.0_33\bin\java.exe" -Xmx20m -Dlog4j.configuration=file:///%CD%\conf\log4j.properties -cp "lib*" org.apache.flume.node.Application -f conf\test1.conf -n syslog-agent
    Explanation:
    Path to java.exe + Java related options + log4j config in Windows Uri style + Flume libs + application + config

Available sinks

This section will be extend from time to time. These sinks where reported in from users as working:

  1. Syslog TCP
  2. Syslog UDP
  3. Exec
  4. Avro
  5. HDFS
    1. When a cygwin installed HDFS node is available on the same host as the agent runs