...
Normal mode: The data is visible when the writing is successful.
Batch mode: The data is not visible until server commits.
Related Research
Detailed Design
Principles:
- Guarantee once data is accepted, it is eventually imported successfully.
...
- What scenario to use it?
Currently, we use it in log writing scene
- Why not use "begin... insert... commit"
From the user's view, the "begin... insert... commit" syntax ensures the atomicity of multi rows. Many use cases do not require such capabilities, and can use insert directly.
"begin... insert... commit" requires the user to control the commits, and also associates a transaction and RowSet, commits can not too frequently；auto_batch_load can commit data from multi clients.
Data may be lost if users don't commit when "begin... insert... commit"; auto_batch_load pre-write WAL to ensure that the data can be eventually loaded.
- The performance impact of too much rpc
Many KV systems are used in this way, and rpc has little effect;
In our tests, Fe can parse and redirect 20,000 + SQL per second . At the same time, Fe is scalable.
Detailed Design
...
.
Scheduling
specific implementation steps and approximate scheduling.