...
- App Blacklisting - Blacklist metrics from one or more services. Other service metrics will be entirely allowed or controlled through a whitelist file.
ams-site : timeline.metrics.apps.blacklist = hbase,namenode
- App Whitelisting - Whitelist metrics from one or more services.
ams-site:timeline.metrics.apps.whitelist = nimbus,datanode
...
- Metric Whitelisting - Same as the whitelisting method in Ambari 2.4.3 (through a whitelist file).
In addition to supplying metric names in the whitelist file, patterns can also be supplied using the ._p_ perfix. For example, a pattern can be specified as follows
._p_dfs.FSNamesystem.*
._p_jvm.JvmMetrics*
An example of a metric whitelisting file that has both metrics and patterns - https://github.com/apache/ambari/blob/trunk/ambari-metrics/ambari-metrics-timelineservice/src/test/resources/test_data/metric_whitelist.dat.
These whitelisting/blacklisting techniques can be used together. For example,
- If you just have timeline.metrics.whitelist.file = <some_file>, only metrics in that file will be allowed (irrespective of whatever apps might be sending them).
- If you just have timeline.metrics.apps.blacklist = datanode, all datanode metrics will be disallowed. Metrics from all other services will be allowed.
- If you just have timeline.metrics.apps.whitelist = namenode, it is not useful since there is no blacklisting at all.
- If you have metric whitelisting enabled (through a file), and have timeline.metrics.apps.blacklist = datanode, all datanode metrics will be disallowed. The whitelisted metrics from other services will be allowed.
- If you have timeline.metrics.apps.blacklist = datanode, timeline.metrics.apps.whitelist = namenode and metric whitelisting enabled (through a file), datanode metrics will be blacklisted, all namenode metrics will be allowed, and whitelisted metrics from other services will be allowed.
Known Issues
- The Kafka Topics Grafana dashboard is unable to discover the Kafka Topics by default if whitelisting is enabled. The reason is that 'kafka.log.Log.*' metrics are used to filter the available Kafka Topics but those metrics are not enabled by default in the whitelist file. The issue can be remediated by adding '._p_kafka.log.Log.*' to the whitelist file and restarting the Metrics Collector.
...
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key AMBARI-25383 - The following metrics are erroneously filtered by the whitelisting despite they are defined whitelist file. Due to this issue the Kafka Topics Grafana dashboard is not showing data.
kafka.server.BrokerTopicMetrics.BytesInPerSec.topic.*.count
The issue can be worked around by adding the '._p_' prefix to the corresponding metrics in the whitelist file, e.g.
kafka.server.BrokerTopicMetrics.BytesOutPerSec.topic.*.count
kafka.server.BrokerTopicMetrics.MessagesInPerSec.topic.*.count
kafka.server.BrokerTopicMetrics.TotalProduceRequestsPerSec.topic.*.count
._p_kafka.server.BrokerTopicMetrics.BytesInPerSec.topic.*.count .Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key AMBARI-25383
...