...
| Page properties | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Document the state by adding a label to the FLIP page with one of "discussion", "accepted", "released", "rejected".
|
Motivation
Currently users have to click on every operator and check how much data each sub-task is processing to see if there is data skew. This is particularly cumbersome and error-prone for jobs with big job graphs. Data skew is an important metric that should be more visible.
...
- dataSkewPercentage: This will be used to show an overall or historical data skew score under the proposed Data Skew tab (see the UI Changes section).
- The existing numRecordsIn metric can be used to build this new metric
- The existing numRecordsIn metric can be used to build this new metric
- dataSkewPercentagePerSecond: This will be used to show a "live" score on the Job Graph (see the UI Changes section).
- The existing numRecordsInPerSecond metric can be used to build this new metric
See the "rejected alternatives" section for other metrics that were considered.
...