Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Below we roughly outline the steps in the phases, and discuss some considerations and challenges and the phase we will address them in.

...


Phase 1 Goal

Phase 1: - PR https://github.com/apache/incubator-pinot/pull/4527

i. Basic functional Order By support

...

ii. Basic trimming, equivalent to current code's trimming behavior (As explained in the trimming sections below)

iii. Work mostly on starting CombineGroupByOrderByOperator layer and above

Phase 2:

Advanced trimming strategies - on the fly ordering and trimming 

Work on AggregationGroupByOperator and downwards.

Phase 3:


TODO

i) Expression evaluation in order by

ii) Smarter trimming:

  • Inner segment - Use Map + PQ for on the fly trimming. Can OrderByTrimmingService be used? This would mean AggregationGroupByOperator gets refactored to understand GroupByRecord. Is this feasible?
  • Inter segment - Basic trimming and ordering exists. Use Map + PQ for on the fly operation. Apply smarter limits.
  • Broker level - Basic collect all + sort approach implemented currently. Move to PQ based approach.

iii) A single ResultSet object to host all results + schema. Possibly use this in all queries.

iv) Remove duplicate CombineGroupByOperator and CombineGroupByOrderByOperator. Keep just 1 path of execution.Expression evaluation in order by

Considerations and Challenges

...