View Source

This document describes some internal details of TDE.Phase 1 implementation [1].

I suggest that reader familiar with the general design described in IEP-18 [2].

Cache group key management and node join enhancements:

1. All group encryption keys are stored in MetaStore.
2. Joining node sends to cluster:
  1. Master key digest.
  2. All group keys saved locally (Map<Integer, byte[]>). Keys send over a network in encrypted form.
3. Coordinator on new node join:
  1. Compares new node master key digest with the local master key digest.
    If differs then new node join is rejected.
  2. Compares local and received group keys.
    If some key differs then new node join is rejected.
4. All server nodes:
  1. If some of received keys are new then they save locally.
Dynamic cache creation:
1. On server node - Encryption key is generated and added to DynamicCacheChangeRequest.
2. On client node:
  1. Prior to creation of DynamicCacheChangeRequest we have to get new encryption key from some server node.
  2. New request added to solve this - GenerateEncryptionKeyRequest.

WAL Record encryption:

New type of WAL record created – EncryptedRecord.
EncryptedRecord is a container that stores some WalRecordCacheGroupAware in encrypted form inside.
Write:
Each record for an encrypted group that implements WalRecordCacheGroupAware written to WAL in encrypted form.
Instead of that record we write EncryptedRecord with plain record inside(PageSnapshot, PageDeltaRecord, etc).
Read: There are 3 different cases on EncryptedRecord read:
1. WAL restore – we read EncryptedRecord, decrypt internal record and return it.
2. Offline WAL iteration(StandaloneWalRecordsIterator) - EncryptionSpi is null so we can’t decrypt EncryptedRecord and just return it from an iterator.
3. Meta storage restore process – On node start we restore MetaStore.
  When we do it no encryption keys are available, because they are stored in MetaStore.
  So we can’t decrypt EncryptedRecord and just return it from an iterator.
  We don't need decrypted record on this step to operate properly.

We have to write on disk pages aligned on 2^n (2kb, 4kb, etc) for gain maximum perfrormance.
There is a 16 byte overhead for and AES CBC mode.
So we have to preserve 16 bytes in page in memory to get encrypted page size equal to 2^n when written it to disk.
PageIO has many methods with pageSize parameter.
So for encrypted cache groups page size is calculated as cfg.getPageSize() - 16 byte.