Transparent Data Encryption

To meet the need for user data security, SynxDB Elastic supports Transparent Data Encryption (TDE).

Transparent Data Encryption (TDE) is a technology for encrypting database data files:

Data: Refers to the data within the database.
Encryption at rest: Data files are stored in an encrypted format on disk and are decrypted when read into memory. TDE protects data at rest.
Transparency: The encryption and decryption processes are managed automatically by TDE, so users and applications can use it without changing their operational habits or code.

Introduction to encryption algorithms

Basic concepts

DEK (Data Encryption Key): Used to directly encrypt data. It is generated by the database and stored in memory.
DEK plaintext: Synonymous with DEK, it can only be stored in memory.
Master Key: The primary key used to encrypt the DEK, ensuring its security.
DEK ciphertext: The DEK encrypted by the Master Key, which is persisted as ciphertext.

Key management module

The key management module is the core component of TDE. It uses a two-tier key structure with a Master Key and a DEK. In the cloud-native architecture of SynxDB Elastic, the Master Key is managed by an independent and secure key-service component deployed in a Kubernetes cluster. The DEK is used to encrypt database data, and its ciphertext is stored within the database.

Algorithm categories

Encryption algorithms are divided into symmetric and asymmetric encryption. Due to performance advantages, TDE primarily uses block cipher algorithms from symmetric encryption. SynxDB Elastic supports two industry-standard block cipher algorithms: AES and SM4.

AES encryption algorithm

AES is an international standard block cipher algorithm that supports 128, 192, and 256-bit keys. Common encryption modes include:

ECB: Electronic Codebook mode
CBC: Cipher Block Chaining mode
CFB: Cipher Feedback mode
OFB: Output Feedback mode
CTR: Counter mode

More ISO/IEC encryption algorithms

Other ISO/IEC algorithms include:

ISO/IEC 14888-3/AMD1 (that is, SM2): An asymmetric encryption algorithm based on ECC, with better performance than RSA.
ISO/IEC 10118-3:2018 (that is, SM3): A message digest algorithm similar to MD5, with a 256-bit output.
ISO/IEC 18033-3:2010/AMD1:2021 (that is, SM4): A symmetric encryption algorithm for wireless LAN standards, supporting 128-bit keys and block lengths.

How to use TDE

In SynxDB Elastic, TDE is specified when creating a database, enabling database-level encryption.

Before using TDE, you need to:

Have permission to execute CREATE DATABASE.
Ensure your version of SynxDB Elastic supports TDE.
Have access to the kubectl command-line tool to manage the key-service during verification.

When creating a database, use the WITH ENCRYPTION_ENABLE clause to enable TDE and specify the tablespace and encryption algorithm (for example, aes or sm4).

Create an encrypted database using the AES algorithm:

CREATE DATABASE encryptdb_aes WITH ENCRYPTION_ENABLE 'aes' tablespace default_global_synxdb_tablespace;

Create an encrypted database using the SM4 algorithm:

CREATE DATABASE encryptdb_sm4 WITH ENCRYPTION_ENABLE 'sm4' tablespace default_global_synxdb_tablespace;

Once created, all data stored in this database (including future tables and data) will be automatically encrypted.

How to verify

TDE is transparent to upper-layer applications. To verify that TDE can protect data security if the master key is lost, you can simulate a scenario where the key-service is unavailable.

Create an encrypted database and write data to the database.

First, create a database with AES encryption and switch to the database.

-- Create an encrypted database named encryptdb1
CREATE DATABASE encryptdb1 WITH ENCRYPTION_ENABLE 'aes' tablespace default_global_synxdb_tablespace;

-- Switch to the newly created database
\c encryptdb1

To see detailed encryption and decryption logs in the following steps, you can enable the debug_tde_print_encrypt_data debug parameter. This is an optional step, mainly for demonstration and debugging purposes.

-- Sets the warehouse and enable the debug parameter.
SET warehouse TO wl;
SET debug_tde_print_encrypt_data = true;

Next, create a table and insert data. At this point, you should see a notification in the terminal indicating the encryption operation.

-- Creates a test table.
encryptdb1=# create table t1(id int);
CREATE TABLE

-- Inserts data to trigger encryption.
encryptdb1=# INSERT INTO t1 SELECT generate_series(1,5);
NOTICE:  Encrypt data BLock, use encrypt algorithm: AES_256 (seg0 127.0.0.1:5433 pid=23938)
INSERT 0 5

Simulate the loss of the Master Key service.

The Master Key is managed by the key-service component. You can use the kubectl command to scale the number of replicas for this service down to 0, simulating service unavailability.
```
# Scales the number of replicas for the key-service deployment to 0.
kubectl scale --replicas=0 deployment/key-service -n <your-namespace>
```
Replace <your-namespace> with the namespace where your SynxDB Elastic system is running, for example, synxdb-system-4x.
Verify that data is inaccessible when the key is lost.

Because key information is cached in the database, you need to restart the database for the key-service shutdown to take effect. After the database restarts, try to query the data in the encrypted table.
```
encryptdb1=# select * from t1;
ERROR:  get master key fail, dboid: 16397, key_path:/16397/1  (hd_keys_manager.c:259)
```
The query will fail and return a “get master key fail” error. This clearly indicates that the database cannot decrypt the data without access to the master key from the key-service, thus verifying the security of TDE.

Restore the Master Key service and verify the data.

# Restores the number of replicas for the key-service deployment to 1.
kubectl scale --replicas=1 deployment/key-service -n <your-namespace>

Restart the database again to clear the error state and reload the key. After the database restarts, data queries should return to normal.

-- Successfully query data, triggering decryption.
encryptdb1=# SELECT * FROM t1; [cite: 70]
NOTICE:  Decrypt data BLock, use encrypt algorithm: AES_256 (seg0 slice1 127.0.0.1:5433 pid=23938)
id
----
(5 rows)

By following these steps, you can fully verify the TDE feature in SynxDB Elastic: it is transparent to users under normal conditions and effectively prevents data access when the key service is abnormal, ensuring the security of data at rest.