Query Processing Phases

• Query Parsing: Pinot supports a slightly modified version of SQL which we refer to as PQL.  We use Antlr to parse the query into a parse tree. In this phase, all syntax validations are performed and default values are set for missing elements.
• Logical Plan Phase: This phase takes in query parse tree and outputs a Logical Plan Tree. This phase is single threaded and is simple and constructs the appropriate logical plan operator tree based on the query type (selection, aggregation, group by etc) and metadata provided by the data source.
• Physical Plan Phase: This phase further optimizes the plan based on individual segment. The optimization applied in this phase can be different across various segments.
• Executor Service: Once we have per segment physical operator tree, executor service takes up the responsibility of scheduling the query processing tasks on each and every segment.

Query Plan

Based on the query type, we create three main query plans for aggregation query, aggregation group by query and selection query.

Aggregation Query

Aggregation Group By Query

Selection Query

Details

InstanceRequest contains a list of segments(from routing table in broker) to query. Query executor gets these segments from InstanceDataManager

Segment Pruners are applied based on segment metadata. TimeRangeSegmentPruner prunes segment based on time range specified in segment metadata level.

Inter segment query plan is used to represent how to process query crossing multiple segments. Inner-segment query plan is for how to query inside one segment.