This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • Performance Measurements - round 2

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.


Measurements were taken to get an idea around the configuration that yields best performance. So took measurements only for all data points in the grid that made sense. For example it was not necessary to take measurements for multiple data dirs dataDirs at single sink, as it was evident more sinks is bettermultiple HDFS sink would better than single sink config.

2.     HDFS Sink:

Flume version: 1.4


Event Size



BatchSz: 1


BatchSz: 100

BatchSz: 1000



0.4 MB/s

0.5 MB/s


0.8 MB/s

0.8 MB/s

0.9 MB/s




6.     Kafka Source:

Flume version: 1.6

Channel: Memory