Note |
---|
This is work in progress |
Problem Statement
The pinot schema timespec looks like this:
...
PRIMARY
: The primary date time column. This will be the date time column which keeps the milliseconds value. This will be used as the default time column, in references by pinot code (e.g. retention manager)SECONDARY
: The date time columns which are not the primary columns with milliseconds value. These can be date time columns in other granularity, put in by applications for their specific use cases.DERIVED
: The date time columns which are derived, say using other columns, generated via rollups, etc.
Examples
Code Block | ||
---|---|---|
| ||
“dateTimeFieldSpec”:
{
“name” : “Date”,
“dataType” : “LONG”,
“format” : “1:HOURS:EPOCH”,
“granularity” : “1:HOURS”,
“dateTimeType” : "PRIMARY"
}
“dateTimeFieldSpec”:
{
“name” : “Date”,
“dataType” : “LONG”,
“format” : “1:MILLISECONDS:EPOCH”,
“granularity” : “5:MINUTES”,
“dateTimeType” : "PRIMARY"
}
“dateTimeFieldSpec”:
{
“name” : “Date”,
“dataType” : “LONG”,
“format” : “1:DAYS:SIMPLE_DATE_FORMAT:yyyyMMdd”,
“granularity” : “1:DAYS”,
“dateTimeType” : "SECONDARY"
} |
Migration
Once this change is pushed in, we will migrate all our clients to start populating the new DateTimeFieldSpec, along with the TimeSpec.
...