Versions Compared

Old Version 38

changes.mady.by.user Ting Chen

Saved on

compared with

New Version 39

changes.mady.by.user Ting Chen

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Server no longer upload segment data. Instead during the commit step, only segment metadata is upload to the controller (to be written into zk).
  2. Server can submit an asynchronous task to upload the segment build to a configured deep storage.
  3. When a slow server is asked to download a segment, it will go through the segmentLocation list for the segment. It will first try to download from the deep storage. If not available, it will then download from the servers. 
  4. To facilitate download from servers, Pinot servers will have a new REST api for segment download. 

Image Added                         

    Image Removed


Pros:

  • Simple to implement
  • Minimal changes to the current controller/server FSMs used in the LLC segment protocols: some noticeable changes would be in the controller FSM – the Committer Uploading state now becomes obsolete for obvious reason and should be generalized to a state which reflects the segment is being persisted.  

...