This page is meant as a template for writing a DSIP.
Status
Current state: [One of "Under Discussion", "Accepted", "Rejected"]
Discussion thread:
JIRA or Github Issue:
Released: <Doris Version>
Google Doc: <If the design in question is unclear or needs to be discussed and reviewed, a Google Doc can be used first to facilitate comments from others.>
Motivation
Cloud object storage is cheaper than multi replication local storage, thus we can put cold data to s3 to store much more data at lower price. To be more general, doris should not lose any feature due to putting cold data to s3.
Related Research
There is an implementation migrating data to s3, https://github.com/apache/incubator-doris/pull/9197. The implementation migrates whole data of tablet to s3, and once a tablet is migrated to s3, the tablet is not allowed to be written.
Detailed Design
The proposal aims to store cold rowsets in s3 without losing any feature, like updating and schema changing. The whole work can be divided into four parts.
Policy
Decision
Action
Result
Scheduling
specific implementation steps and approximate scheduling.