Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This page is meant as a template for writing a DSIP.

Status

Current state[One of "Under Discussion", "Accepted", "Rejected"]

...

Google Doc: <If the design in question is unclear or needs to be discussed and reviewed, a Google Doc can be used first to facilitate comments from others.>

Motivation

Cloud object storage is cheaper than multi replication local storage, thus we can put cold data to s3 to store much more data at lower price.  To be more general, doris should not lose any feature due to putting cold data to s3.

Related Research

There is an implementation migrating data to s3, https://github.com/apache/incubator-doris/pull/9197. The implementation migrates whole data of tablet to s3, and once a tablet is migrated to s3, the tablet is not allowed to be written.

Detailed Design

The proposal aims to store cold rowsets in s3 without losing any feature, like updating and schema changing.  The whole work can be divided into four parts.

Policy

Now there are cooldown and remote_cooldown, it is too 

Decision

Action


Result

Scheduling

specific implementation steps and approximate scheduling.