Page History

...

Memory components

• A memory component start starts as an empty component with its state being either READABLE_WRITABLE (first component) or INACTIVE (other components).
• A READABLE_WRITABLE component becomes READABLE_UNWRITABLE prior to flushing it to allow for current writers inside the components to exit before flushing.
  • This operation is done through the primary index operation tracker and it affects all indexes/partitions of the dataset.
  • This setting of the components state happens from outside the components due to two factos:
    • There is a single primary opTracker that is used for synchronization on multiple partitions. 
    • Some operations don't affect all indexes of a dataset in a partition.
  • Hence, if a partition/index had no operationHence, , we still want to make it READABLE_UNWRITABLE to aviod going beyond the memory budget
    and an empty component can become READABLE_UNWRITABLE and schedule flush will get called.
• If schedule flush is called on an empty component, its state is switched back from READABLE_UNWRITABLE to READABLE_WRITABLE.
• On scheduling a flush, the memory component state is transitioned to READABLE_UNWRITABLE_FLUSHING. The previous state can either be
  READABLE_UNWRITABLE or (READABLE_WRITABLE: not clear when).
• When a flush is scheduled, the activation flag is set for the next component. If there is only a single component, then the flag is set for the current component.
• When any operation tries to enter a component that is INACTIVE and has the activation flag set, it will switch it to READABLE_WRITABLE.
• When an operation exits the memory component:
  • If it is a modification operation that succeeded, the component is the current mutable memory component and is currently READABLE_WRITABLE ,
    and the memory budget has been exceeded, the component state is set to READABLE_UNWRITABLE.
  • If it is a modification operation and the component is not the current mutable component or if the operation is a search or replicate operation and there are
    no more readers in the component, and it is currently UNREADABLE_UNWRITABLE, its state is set to INACTIVE.
  • If it is a failed flush operation, the state is reverted from READABLE_UNWRITABLE_FLUSHING to READABLE_UNWRITABLE. It will stay in this state until
    system shutdown.
  • If it is a successful flush operation and there are no more readers, the state becomes INACTIVE.
  • If it is a successful flush operation and there are readers inside the component, the state becomes UNREADABLE_UNWRITABLE.

Disk components

• A disk component starts with its state being READABLE_UNWRITABLE.
• When a merge is scheduled, the component's state is set to READABLE_MERGING.
• If a merge operation fails, the component state is reverted back to READABLE_UNWRITABLE.
• If an operation completes and there are no more readers inside the component and its state is currently READABLE_MERGING, then it becomes INACTIVE.