DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
Below we roughly outline the steps in the phases, and discuss some considerations and challenges and the phase we will address them in.
...
Phase 1 Goal
Phase 1: - PR https://github.com/apache/incubator-pinot/pull/4527
i. Basic functional Order By support
...
ii. Basic trimming, equivalent to current code's trimming behavior (As explained in the trimming sections below)
iii. Work mostly on starting CombineGroupByOrderByOperator layer and above
Phase 2:
Advanced trimming strategies - on the fly ordering and trimming
Work on AggregationGroupByOperator and downwards.
Phase 3:
TODO
i) Expression evaluation in order by
ii) Smarter trimming:
- Inner segment - Use Map + PQ for on the fly trimming. Can OrderByTrimmingService be used? This would mean AggregationGroupByOperator gets refactored to understand GroupByRecord. Is this feasible?
- Inter segment - Basic trimming and ordering exists. Use Map + PQ for on the fly operation. Apply smarter limits.
- Broker level - Basic collect all + sort approach implemented currently. Move to PQ based approach.
iii) A single ResultSet object to host all results + schema. Possibly use this in all queries.
iv) Remove duplicate CombineGroupByOperator and CombineGroupByOrderByOperator. Keep just 1 path of execution.Expression evaluation in order by
Considerations and Challenges
...