Flume Recipes

This page contains recipes that describe how to setup Flume for different kinds of input and output sources.

Using Flume with EC2 / S3

Here are some blogs on getting flume to write to S3.

• SimpleReach's instructions
• ShareThrough's instructions

Deployment and Configuration

• Justin Workman has several blog posts some of address problems folks have encountered.
  • instructions on scripting configurations
  • dealing with 0-length files
  • sending a file directly via command line"

• Chetan Sarva's repo with flume node puppet deployment recipes
  • https://github.com/chetan/puppet-flume_node

Using the data afterwards.

• Using Apache Pig to post process Flume data
• Using Apache Hive to post process Flume data