...
2. protocal design
2.1. layout
Update: reuse Reuse the current index layout and just treat deletemap as a new index file type
...
Extend the IndexMaintainer interface, and add create DeleteMapIndexMaintainer implements IndexMaintainer<KeyValue, DeleteIndex>
...
Considerations for implementation:
Currently, when set'changelog-producer' = 'lookup', the data write behavior is not atomic but divided into two steps: first, data is written to create snapshot1, then lookup compaction generates snapshot2. We need to consider the atomicity of this.- In most cases, the data will be transferred to level-0 first, and then rewritten. The writing overhead is a bit high, and perhaps some optimization can be done in this regard.
- If change log needs to be generated, in theory, change log and delete file can be produced simultaneously (without reading twice).
- The merge engine is still available.
...
- For each read task, load the corresponding deleteFile.
- Construct a the
map<fileName, bitmap>from deleteFile. - Get the bitmap based on the filename, then pass it to the reader.
...