This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • Performance Measurements - round 2

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Measurements were taken to get an idea around the configuration that yields best performance. So took measurements only for all data points in the grid that made sense. For example it was not necessary to take measurements for multiple data dirs dataDirs at single sink, as it was evident more sinks is bettermultiple HDFS sink would better than single sink config.

2.     HDFS Sink:

Flume version: 1.4

...

Event Size

(bytes)

 

BatchSz: 1

 

BatchSz: 100


BatchSz: 1000

500

 

0.4 MB/s

0.5 MB/s

1000

0.8 MB/s

0.8 MB/s

0.9 MB/s

 

 

...

6.     Kafka Source:

Flume version: 1.6

Channel: Memory

...