...
During segment completion phase,
- SplitSegmentCommitter will check this value. If it exists, the segment committer will be able to finish segment commit successfully even if the upload to the segment store fails. The committer will report to the controller that the segment is available in peer servers.
- When Pinot servers in LLRealtimeSegmentDataManager fail to download segments from the segment store during goOnlineFromConsuming() transition, they also check this field's value. If it exists, it can
- First discover the segment location server URI u.
- Construct the complete uri using the configured scheme (http or https) and use the appropriate segment fetcher to download it.
Note this is a table level config. We will test the new download behavior in realtime tables in incremental fashion. Once fully proven, this config can be upgraded to server level config.
...
During segment completion phase,
- SplitSegmentCommitter will check this value and if true, behaves exactly like Option 1.
- When Pinot servers in LLRealtimeSegmentDataManager fails to download segments from the segment store during goOnlineFromConsuming() transition, it can use the PeerServerSegmentFetcher (if SERVER scheme is configured) to discover
...
- and download the segment from peers.
Failure cases and handling
...