Transparent tablespace and log encryption on MariaDB 10.1 using Amazon Key Management Service

Jan Lindström, Principal Engineer, MariaDB Corporation

Amsterdam, Netherlands | October 5, 2016

2

Agenda

1. Introduction

2. Concepts

3. InnoDB/XtraDB

4. Encryption Plugins

5. Amazon AWS KMS

6. Configuration

Introduction

4

What is transparent encryption?

• Transparent to application

• Application does’t know anything about keys, algorithm, etc

• Anyone that can connect to MariaDB can dump data

• Not data-in-transit encryption (SSL/TLS)

• Not per-column encryption

• Not application-side encryption

• No encryption functions needed (AES_ENCRYPT())

5

All data written to disk should beencrypted

• InnoDB tablespaces (per-file and system)

• InnoDB log files

• Aria tables

• Temporary files

• Temporary tables

• Binary log• No mysqlbinlog, though!

6

What’s missing?

• Aria logs

• Audit log

• Error log

• Slow query log

• General query log

• MyISAM tables

• CONNECT, CSV, et. Al.

• Galera gcache

7

Implementation

• MariaDB has a new interface for encryption plugins• Key management

• Encryption/decryption

• Implemented co-operation together with Google and Eperi

• https://mariadb.com/kb/en/mariadb/encryption-plugins/

Concepts

9

Concepts

• Key ID• ID 1 for system data, like InnoDB redo logs, binary logs, etc

• ID 2 (if available) for temporary data, like temporary files and temporary tables

• Other Ids as configured when creating tables, etc.

• Key Version (for rotation)

• Encryption algorithm• Default AES_CBC

• Support for these items may vary across plugins!

InnoDB/XtraDB

11

InnoDB/XtraDB

• ON/OFF/FORCE• innodb-encrypt-tables = [ON | OFF | FORCE];

• Encrypt log• innodb-encrypt-log = [ON | OFF];

• Monitoring (IS)• innodb-tablespaces-scrubbing

• innodb-tablespaces-encryption

12

InnoDB/XtraDB

• Optional background rotation• innodb-encryption-threads = n;• innodb-encryption-rotate-key-age= n;

- ”Age” in key versions

• innodb-encryption-rotation-iops = n;

• Optional data scrubbing• innodb-background-scrub-data-compressed = [ON | OFF];• innodb-background-scrub-data-uncompressed = [ON | OFF];• innodb-immediate-scrub-data-uncompressed = [ON | OFF];• innodb-scrub-log = [ON|OFF];• innodb-scrub-log-speed=n;• innodb-background-scrub-data-check-interval=n;

https://mariadb.com/kb/en/mariadb/xtradb-innodb-data-scrubbing/

Encryption plugins

14

Encryption plugins

• File key management• https://mariadb.com/kb/en/mariadb/data-at-rest-encryption/#file_key_management-

plugin

• AWS KMS plugin• https://mariadb.com/kb/en/mariadb/aws-key-management-encryption-plugin/

• Eperi plugin• http://eperi.de/en/products/database-encryption/

• Custom plugins to meet customer needs?

15

File_key_management

• Keys stored in a local file (note that this file could be on USB stick)

• No support for key rotation/version

• Key file itself can be encrypted (but used key in my.cnf)

• Do you feel good having your encryption keys sitting next to your data ?

16

Eperi plugin

• Separate Eperi gateway software• Licenses and downloads from Eperi’s web portal

• KMS• Plugin opens listener that the KMS connects to in order to authenticate the

connecting MariaDB instance

• Page encryption server• InnoDB actually sends pages to the Eperi gateway node to be encrypted!

Amazon KMS Encryption Plugin

18

AWS KMS Encryption Plugin

• Amazon Web Services Key Management Service

• CloadTrail & CloudWatch• Logging

• Auditing

• Notifications

• Identity and Access Management (IAM)

• Interesting possibilities• MFA for MariaDB startup

• IAM roles to read keys

• AWS logging & alerts

19

Requirements

• You need to sign up for Amazon Web Services

• You need to create IAM user• MariaDB server will use these credentials to authenticate AWS server

• You need to create a master encryption key• Used to encrypt the actual encryption keys that will be used by MariaDB

• You will need to configure AWS credentials

• You will need to configure MariaDB (naturally)

20

AWS KMS Plugin

• Writes enrypted keys to local disk• MariaDB must connect to KMS to decrypt keys

- MariaDB startup

- Creating a table that uses a new key

• Supports key rotation

• Limited platform support due to C++11 requirement of AWS SDK• Requires C++11 compiler: gcc4.7+, clang 3.3+ or VS2013+

• RHEL

• CentOS 7

• ~600 lines• Great reference for people who want to write their own plugins

21

Credentials Management

• Identify and Access Management (IAM) policy for keys• Authorized source addresses

• IAM users w/ restricted privileges

• Multi-Factor Authentication (2FA/MFA)

• AWS SDK• Config file, environment variables, etc.

• Flexible wrapper program

• EC2 (Elastic Compute Cloud) instance IAM role

Configuration

23

Install, enable, and configure

$ cat /etc/my.cnf.d/aws_key_management.cnf

[mariadb]

plugin-load-add=aws_key_management.so

aws-key-management

aws-key-management-master-key-id = alieas/mariadb2

# aws_key_management_log_level = Trace

ignore-db-dirs=.pki

!include /etc/my.cnf.d/enable_encryption.preset

24

Turn on encryption settings

$ cat /etc/my.cnf.d/enable_encryption.preset

[mariadb]

aria-encrypt-tables

encrypt-binlog

encrypt-tmp-disk-tables

encrypt-tmp-files

loose-innodb-encrypt-log

loose-innodb-encrypt-tables

25

Encrypted system tablespace

$ sudo –u mysql mysql_install_db

…

2016-09-29 11:40:00 [Note] AWK KMS plugin: generated encrypted datakey for keyid=1, version=1

2016-09-29 11:40:00 [Note] AWK KMS plugin: loaded key 1, version 1, key length128 bit

…

2016-09-29 11:40:01 [Note] AWK KMS plugin: generated encrypted datakey for keyid=2, version=1

2016-09-29 11:40:01 [Note] AWK KMS plugin: loaded key 2, version 1, key length128 bit

2016-09-29 11:40:01 [Note] Using encryption key id 2 for temporary files

…

26

Why encrypt data ?

MariaDB [db]> create table client_credit_card(id int not null primary key, credit_card varchar(20)) engine=innodb encrypted=no;

MariaDB [db]> insert into client_credit_cards values(20071992, ’5275-0000-0000-0000’):

…

$ sudo strings /var/lib/mysql/db/client_credit_cards.ibd

infimum

supremum

5275-0000-0000-0000

27

Automatic key generation

MariaDB [db]> create table client_credit_card(id int not null primary key, credit_cardvarchar(20)) engine=innodb encrypted=yes encryption_key_id=3;

MariaDB [db]> insert into client_credit_card values(20071992, ’5275-0000-0000-0000’):

…

$ sudo strings /var/lib/mysql/db/client_credit_card.ibd

{7fgh

k6klj

B_0=

…

28

I_S table for encryption info

MariaDB [(none)]> select * from information_schema.innodb_tablespaces_encryption where name='db/client_credit_card'\G*************************** 1. row ***************************

SPACE: 6NAME: db/client_credit_card

ENCRYPTION_SCHEME: 1KEYSERVER_REQUESTS: 1

MIN_KEY_VERSION: 1CURRENT_KEY_VERSION: 1

KEY_ROTATION_PAGE_NUMBER: NULLKEY_ROTATION_MAX_PAGE_NUMBER: NULL

CURRENT_KEY_ID: 31 row in set (0.00 sec)

29

Key rotation

MariaDB [(none)]> show variables like 'aws%';+----------------------------------+----------------+| Variable_name | Value |+----------------------------------+----------------+| aws_key_management_key_spec | AES_128 || aws_key_management_log_level | Off || aws_key_management_master_key_id | alias/mariadb2 || aws_key_management_rotate_key | 0 |+----------------------------------+----------------+4 rows in set (0.00 sec)

MariaDB [(none)]> set global aws_key_management_rotate_key=3;Query OK, 0 rows affected (0.27 sec)

30

Key rotation

MariaDB [db]> set global innodb_encryption_threads=4;Query OK, 0 rows affected (0.00 sec)

MariaDB [db]> set global innodb_encryption_rotate_key_age=0;Query OK, 0 rows affected (0.00 sec)

MariaDB [db]> select * from information_schema.innodb_tablespaces_encryptionwhere name like 'db/c%'\G*************************** 1. row ***************************

SPACE: 6NAME: db/client_credit_card

ENCRYPTION_SCHEME: 1KEYSERVER_REQUESTS: 2

MIN_KEY_VERSION: 2CURRENT_KEY_VERSION: 2

KEY_ROTATION_PAGE_NUMBER: NULLKEY_ROTATION_MAX_PAGE_NUMBER: NULL

CURRENT_KEY_ID: 31 row in set (0.00 sec)

31

Documentation

•https://mariadb.com/kb/en/mariadb-enterprise/mariadb-enterprise-aws-kms-encryption-plugin-setup-guide/•https://mariadb.com/kb/en/mariadb-enterprise/mariadb-enterprise-aws-kms-encryption-plugin-advanced-usage/•https://mariadb.com/kb/en/mariadb/data-at-rest-encryption/•https://mariadb.com/kb/en/mariadb/xtradb-innodb-data-scrubbing/

Q/A