This page is meant as a template for writing a DSIP.

Status

Current state: Accepted Released

Discussion thread: 

JIRA or Github Issue: 

Released: <Doris Version>1.2.0

Google Doc: <If the design in question is unclear or needs to be discussed and reviewed, a Google Doc can be used first to facilitate comments from others.>

...

To avoid frequent changes of std::shared_ptr use count when switching TLS MemTrackers frequently, during an attach query, TLS caches all switched MemTrackers and uncommitted memory consumption. In the future, the mem tracker in TLS should be changed to a raw pointer to fundamentally solve this problem.

As of now, the new memory statistics framework will bring about a 1%-2% performance penalty.

• opening the Hook TCMalloc new/delete loses about 1%;
• Turning on verbose memory tracking loses about 1%;

Scheduling

step1: Refactor impl of MemTracker, and related use. (https://github.com/apache/incubator-doris/pull/8322)
step2: Hook TCMalloc new/delete automatically counts to MemTracker. (https://github.com/apache/incubator-doris/pull/8476)
step3: Switch TLS mem tracker to separate more detailed memory usage, part1. (https://github.com/apache/incubator-doris/pull/8605)
step4: Switch TLS mem tracker to separate more detailed memory usage, part2. (https://github.com/apache/incubator-doris/pull/8669)
step5: Fix accuracy of memory tracker in vectorization.

...