Page History

...

1.     Create a data-structure on the Job-/TaskManager containing a metrics snapshot
2.     Transfer this snapshot to the WebInterface backendback-end
3.     Store the snapshot in the WebRuntimeMonitor in an easily accessible way
4.     Expose the stored metrics to the WebInterface via REST API

...

The MetricRegistry will contain a MetricDumper, which act similarly as acts like an unscheduled reporter.
The Dumper creates and returns can be used by the Job-/TaskManager to create a Key-Value representation of the entire metric space when queried by the Manager.

The keys represent the name of the metric; formatted according to the following scope format strings:

The initial number serves as a category for the WebInterface, and allows for faster handling as we don't have to parse the entire string before deciding what category it belongs to.
  0 = JobManager
  1 = TaskManager
  2 = Job
  3 = Task
  4 = Operator

For this to work we need to be able to use a different format than the one configured in the configuration, and also cache the resulting strings.
For now we can hard-code a separate scopeString field in the AbstractMetricGroup; a more general solution would be to allow separate ScopeFormat configurations for each reporter, which is a natural follow-up to

Transfer to the WebInterface

...

This will be done in a separate Thread inside the WebRuntimeMonitor, which also has the responsiblity responsibility to merge the returned dumps.

...

Storage in the WebRuntimeMonitor

My (rough) proposal for a datastructure data-structure is the following:

MetricStore {
	void addMetric(String name, Object value);

	JobManager jobManager;

	class JobManager {
		Map<String, Object> metrics;
	}

	Map<String, TaskManager> taskmanagers;

	class TaskManager {
		Map<String, Object> metrics;
	}

	Map<String, Job> jobs;

	class Job {
		Map<String, Object> metrics;
		Map<String, Task> tasks;
	}

	class Task {
		Map<String, Object> metrics;
		Map<String, Subtask>;
	}

	class Subtask {
		Map<String, Object> metrics;
	}
}

...