Page History
...
Ignite with enabled persistence uses following folder stucture
| 2.3+ | Older versions (2.1 & 2.2) | ||
|---|---|---|---|
Pst-subfolder name is same for all storage folders. Name is selected on start, may be based on node consistentId |
|
Subfolders generation
Subfolder name is generated on start.
By default new stile naming is used, for example node00-e819f611-3fb9-4dbe-a3aa-1f6de4af5d02, where 'node' is constant prefix, '00-node index' is incrementing counter of local nodes under same PST root folder, UUID is consistent ID.
For running ignite for clear persistence storage root folder, this naming is used (option 1).
Option 2 is used in case there is existing pst-subfolder with exact same name with compatible consistent ID (loca host IPs and ports list). If there is such folder, Ignite is started using this one, consistent ID is not changed.
Option 3 is applied in case there is preconfigured value from IgniteConfiguration
In case there is old style folder, but its name doesn't match with compatible consistent ID, following warning is generated.
| No Format |
|---|
There is other non-empty storage folder under storage base directory [work\db\127_0_0_1_49999, 299718 bytes, modified 10/04/2017 04:33 PM ] |
There are two file locks used in folders selection.
First one is used to check if there is no up and running node which is using same directory.
This lock is placed in work/db/{pst-subfolder}/lock (work/db may be still customized by storage folder property)
Second lock is placed in storage root folder: work/db/lock. This lock is held for short time when new pst-subfolder is being created. This protects from concurrent folder intialisation by nodes which are starting simultaneouly.
Page store
Ignite Durable Memory is basis for all data structures. There is no cache state saved on heap now.
To save cache state to disk we can dump all its pages to disk. First prototypes used this simple approach: stop all updates and save all pages.
Page store is the storage fo all pages related to particual cache and its partition.
Partitions of each cache have corresponding file in page store directory (particular node may own not all partitions).
...
- part-1.bin, part-2.bin (shown as P1,P2,PN at picture) - Cache partition pages.
- index.bin - index partition data, special partition with number 65535 is used for SQL indexes and saved to index.bin
- cache_data.dat - special file with stored cache data (configuration) . StoredCacheData includes more data than CacheConfiguration, e.g. Query entities
Persistence and Crash Recovery
Crash Recovery can be
...
and
...
Crash Recovery
Ignite Durable Memory is basis for all data structures. There is no cache state saved on heap now.
To save cache state to disk we can dump all its pages to disk. First prototypes used this simple approach: stop all updates and save all pages.
Checkpointing
Can be of two types
- Sharp Checkpointing - if checkpoint is completed all data structures on disk are consistent, data is consistent in terms of references and transactions.
- Fuzzy Checkpointing - means state on disk may require recovery itself
...
Operation is acknowleged after operation was logged, and page(s) update was logged. Checkpoint will be started later by its triggers.
Local Crash Recovery
Crash Recovery can be
- Local (most DB are able to do this)
- and distributed (whole cluster state is restored).
WAL records for recovery
Crash recovery involves following records writtent in WAL, it may be of 2 main types
...
Overview
Content Tools
Apps