This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • Disk space utilization guidance
Skip to end of metadata
Go to start of metadata
Num of NodesMETRIC_RECORD (MB)

METRIC_RECORD

_MINUTE (MB)

METRIC_RECORD

_HOURLY (MB)

METRIC_RECORD

_DAILY (MB)

METRIC_AGGREGATE (MB)

METRIC_AGGREGATE

_MINUTE (MB)

METRIC_AGGREGATE

_HOURLY (MB)

METRIC_AGGREGATE

_DAILY (MB)

TOTAL (GB)
505120270024510150030528110
10010240540049020150030528118
3003072016200147060150030528149
50051200270002450100150030528181
800819204320039201601500305281128

 

NOTE

  • The above guidance has been derived from looking at AMS disk utilization in actual clusters.
  • The ACTUAL numbers have been obtained by observing an actual cluster with the basic services (HDFS, YARN, HBase) installed along with Storm, Kafka and Flume.
  • Kafka and Flume generate metrics only while a job is running. If those services are being used heavily, additional disk space is recommended. We ran sample jobs with STORM and KAFKA while deriving these numbers to make sure there is some contribution.

 

Actual disk utilization data

Num of NodesMETRIC_RECORD (MB)

METRIC_RECORD

_MINUTE (MB)

METRIC_RECORD

_HOURLY (MB)

METRIC_RECORD

_DAILY (MB)

METRIC_AGGREGATE (MB)

METRIC_AGGREGATE

_MINUTE (MB)

METRIC_AGGREGATE

_HOURLY (MB)

METRIC_AGGREGATE

_DAILY (MB)

TOTAL (GB)
21201751715451361611
3294513.41104261.810.5
1010245404921433.63052813.3


  • No labels