Linux on system z cryptographic support - IBMpublic.dhe.ibm.com/software/dw/linux390/perf/crypto.pdf · 2010-10-11 · Linux on system z cryptographic support. ... (CEX2A) for Secure

Martin Kammerer [email protected] 7/20/10

Linux on system z cryptographic support

visit us at http://www.ibm.com/developerworks/linux/linux390/perf/index.html

© 2010 IBM Corporation

mailto:[email protected]

Linux on System z Performance Evaluation

System z hardware

For the use of cryptographic services, IBM System z9 supports the cryptographic hardware functions Central Processor Assist for Cryptographic Function (CPACF) and Crypto Express2 (CEX2) Feature.

CPACF– CPACF provides support for symmetric ciphers and secure hash algorithms (SHA) on every

central processor. Hence the potential encryption/decryption throughput scales with the number of central processors in the system.

CEX2– The two PCI-X adapters on a CEX2 feature can be configured in two ways: Either as

cryptographic Coprocessor (CEX2C) for secure key encrypted transactions, or as cryptographic Accelerator (CEX2A) for Secure Sockets Layer (SSL) acceleration. A CEX2A works only in clear key mode.

Both adapters can be of the same type, or you can configure one adapter as CEX2A and the other as CEX2C.

2 July 20, 2010 Linux on system z cryptographic support © 2010 IBM Corporation

© 2010 IBM Corporation3 Linux on system z cryptographic supportJuly 20, 2010


SSL acceleration

The SSL communication protocol was designed to provide a secure communication over an open insecure network like the Internet. OpenSSL is the Open Source implementation of the SSL protocol. OpenSSL may use the hardware accelerated cryptographic services to speed up an SSL communication. Other applications or protocols may use these cryptographic services using different APIs (e.g. PKCS#11).

OpenSSL needs the engine ibmca to communicate with the interface library (libICA). The libICA library then communicates with CPACF or via the Linux generic device driver zcrypt with a CEX2 (if available). Hence the device driver zcrypt must be loaded in order to use CEX2 features.



Cryptographic support – Linux SSL stack flow

CPACF

Central Processor

CEX2A

zcryptdevice driver

User Space

SharedSystemLibraries

KernelSpace

CryptographicHardware

Application , e.g. Apache w/ mod_ssl

LibICA

engine ibmcaOpenSSL

softw

are

im

ple

me

nta

tio

n

sym

me

tric

cip

he

rs



Performance tests

The Linux SSL performance tests show the benefit of a Crypto Express2 Feature (CEX2) configured for SSL acceleration (CEX2A) in combination with the Central Processor Assist for Cryptographic Function (CPACF).

Our tests compare the performance with and without System z hardware support. The throughput numbers for a CEX2 configured as a cryptographic coprocessor (CEX2C) are also available for reference. However a CEX2C is designed and optimized for secure encrypted transactions.

The SSL protocol for secure network connections establishes a connection between a client and a server node with an SSL handshake.

For data encryption, SSL uses a symmetric cipher for performance reasons. For the SSL handshake process, an asymmetric cipher (RSA) is used which is very cost intensive. The SSL handshake exchanges the keys for the symmetric cipher and chooses the symmetric cipher for the data encryption. The symmetric ciphers DES, TDES, and AES are supported through the System z CPACF function.

A CEX2 adapter can be configured as an SSL Accelerator (CEX2A), which gives best performance in supporting the SSL handshake (RSA acceleration). An adapter configured as Coprocessor (CEX2C) speeds up SSL handshakes as well, but is optimized to support secure key encrypted transactions.



Cryptographic support – Exemplary Workload

workload emulates a secured webserver the connection between client and server is SSL secured scaling over number of parallel connections HTML files of different sizes are exchanged

– 40 Bytes (SSL handshake)– 20 KB (small data portion)– 250 KB and 500KB (medium data portions)– 1 MB (big data portion)

System p client System z webserver

OpenSSL s_time program

OpenSSLs_server program

OSA Express

SSL secured connection



1 2 4 8 16 320

500100015002000250030003500 4 CPUs - 40B data

No HW1 CEX2A

number of parallel connections

con

nec

tio

ns/

s

no HW CEX2A0%

20%

40%

60%

80%

100%32 parallel connections

idlesystemuserC

PU

lo

ad

Crypto Express2 Accelerator (CEX2A) - SSL handshakes

CEX2A accelerates SSL handshake process (asymmetric cipher RSA) the number of SSL handshakes is up to 4x higher with CEX2A support in the 32 connections case we save about 50% of the CPU resources



Generic cryptographic device driver polling thread (1)

zcrypt device driver has a configurable polling thread – introduced with driver version 2.1.0 (SLES10 SP1 and RHEL 5.1); default was enabled– since SLES10 SP2 and RHEL 5.2 disabled per default

– check state: cat /sys/bus/ap/poll_thread 1==enabled; 0==disabled

enabled– polls cryptographic adapter for finished cryptographic requests– best utilization of CEX2 cryptographic adapter– uses one CPU for the thread when polling– only active during outstanding adapter requests

– enable: echo 1 > /sys/bus/ap/poll_thread

disabled– finished requests are fetched with Linux timer interrupt– poor performance when cryptographic adapter is not fully utilized– no further CPU costs for polling

– disable: echo 0 > /sys/bus/ap/poll_thread



Generic cryptographic device driver polling thread (2)

performance degradations until adapter limit is reached for poll=no

1 2 4 8 160

500

1000

1500

2000

2500

3000

35001 CEX2A / 40b data

poll nopoll yes


con

nec

tio

ns/

s ec



CP Assist for Cryptographic Function (CPACF) (1)

z10 newly supports AES-192, AES-256 use your cryptographic hardware! – software configuration issue

TDES-SHA

DES-SHA

AES128-SHA

AES256-SHA

RC4-SHA *

0

1

2

3

4

5

64 CPUs - 1MB data - OpenSSL

* only SHA portion is calculated by CPACF

no CPACF1 CPACF

no

rma

lize

d t

hro

ug

hp

ut




reduced CPU costs for fully supported block ciphers TDES most expensive cipher when calculated in software

DES-SHA

TDES-SHA

AES128-SHA

AES256-SHA

RC4-SHA *

0

2

4

6

8

10

12

144 CPUs - 1MB data - OpenSSL

no CPACF1 CPACF

mo

rma

l ize

d C

PU

co

st s

* only SHA portion is calculated by CPACF




supported ciphers and secure hash functions per System z machine

System z machine supportszSeries z890, z990 DES, TDES, SHA-1

z9DES, TDES, AES-128, SHA-1, SHA-256 DES, TDES, AES-128, AES-192, AES-256, SHA-1, SHA-224,

z10SHA-256, SHA-384, SHA-512



SSL traffic with CEX2A and CPACF

CEX2A accelerates SSL handshakes CPACF accelerates data encryption (symmetric ciphers) use of both hardware features can double the throughput using pure software encryption costs up to 6x more CPU

No HW CPACF CPACF+CEX2A

01234567

no

rmal

ized

CP

U c

ost

s

No HW CPACF CPACF+CEX2A

0

0.5

1

1.5

2

2.520K 1MB

no

rmal

ized

th

rou

gh

pu

t

AES-128



IPsec

IPsec is an extension of the IP protocol providing security to IP and upper-layer protocols. IPsec uses in-kernel cryptographic algorithms.The IPsec protocols use secure hash

algorithms and symmetric ciphers. The following charts show results for three different network workload types with a secure IP

connection between the client and server. The workload types are:

– rr200x1k emulates an online transaction– crr64x8k emulates a website request– rr200x32k emulates a database query

For each workload type three sequences were measured:– no IPsec IPsec is not used– IPsec - no CPACF HW support for in-kernel cryptography is disabled in the kernel– IPsec - CPACF HW support for in-kernel cryptography is enabled in the kernel

The chart shows normalized transaction rates. When doing IPsec, there is an additional overhead to secure the network connection, which decreases the number of achievable transactions. Enabling the HW support for in-kernel crypto dramatically absorbs this decrease.



Linux in-kernel crypto (1)

example: IP security (IPsec) is done in the Linux kernel Linux kernel itself is capable of exploiting CPACF carefully choose a CPACF supported cipher and hash function strong performance impacts with pure software in-kernel crypto

rr200x1k transaction

crr64x8k webserver

rr200x32k DB_query

00.20.40.60.8

11.2

no IPsecIPsec - no CPACFIPsec - CPACF

workload type

no

rma

lize

d t

o n

o I

Pse

c

IPsec config:HMAC-SHA256AES-128




IPsec overhead is significant CPU costs can be 16x higher with software in-kernel crypto by using CPACF the CPU costs can be reduced up to 4x IPsec config:

HMAC-SHA256AES-128

rr200x1k transaction

crr64x8k webserver

rr200x32k DB_query

02468

1012141618


workload type

no

rma

li ze

d C

PU

co

st s




response times <= 5 ms for emulated DB request with IPsec/CPACF

IPsec config:HMAC-SHA256AES-128

rr200x32k0

5

10

15

20

25


workload type

res

po

ns

e t

ime

in

ms



Generic cryptographic device driver AP interrupts (1)

zcrypt device driver uses AP interrupts instead of polling mechanism– since System z10 EC GA2, z10 BC– only in LPAR, z/VM delivers no AP interrupts – supported with driver version 2.1.x (SLES10 SP3, SLES11 SP1 and RHEL 5.4, RHEL6)– if available, zcrypt polling thread cannot be activated

reduces the CPU costs for CEX2 crypto operations– no additional CPU costs for the polling thread

same throughput rates compared to an enabled polling thread new read-only attribute shows status

– cat /sys/bus/ap/devices/cardxy/subsystem/ap_interrupts

– 1 == interrupts used– 0 == no interrupts used



Generic cryptographic device driver AP interrupts (2)

SSL handshake rates similar to polling thread reduced CPU costs

1 2 4 8 1602468

101214

no

rmal

ized

CP

U c

ost

s


1 2 4 8 160

500100015002000250030003500 1 CEX2A / 40b data

poll threadAP interruptsno HW


co

nn

ec

tio

ns

/se

c



Useful crypto display commands (1)

icainfo – CPACF supported operations by libICA on this system – supported since libica library version 1.3.8

Sample command output:

System z9:

# icainfoThe following CPACF operations are supported by libica on this system:SHA-1: yesSHA-256: yesSHA-512: noDES: yesTDES-128: yesTDES-192: yesAES-128: yesAES-192: noAES-256: noPRNG: yes

System z10:

# icainfoThe following CPACF operations are supported by libica on this system:SHA-1: yesSHA-256: yesSHA-512: yesDES: yesTDES-128: yesTDES-192: yesAES-128: yesAES-192: yesAES-256: yesPRNG: yes



Useful crypto display commands (2)

icastats – libICA request statistics – supported since libica library version 2.0.1

Sample command outputs for a OpenSSL workload:

-

no engine ibmca used

# icastats function | # hardware | # software----------+------------+----------- SHA1 | 0 | 0 SHA224 | 0 | 0 SHA256 | 0 | 0 SHA384 | 0 | 0 SHA512 | 0 | 0 RANDOM | 1 | 0 MOD EXPO | 0 | 0 RSA CRT | 0 | 0 DES ENC | 0 | 0 DES DEC | 0 | 0 3DES ENC | 0 | 0 3DES DEC | 0 | 0 AES ENC | 0 | 0 AES DEC | 0 | 0

# -

engine ibmca + CPACF

# icastats function | # hardware | # software ----------+------------+------------ SHA1 | 549736 | 0 SHA224 | 0 | 0 SHA256 | 0 | 0 SHA384 | 0 | 0 SHA512 | 0 | 0 RANDOM | 1 | 0 MOD EXPO | 0 | 0 RSA CRT | 0 | 6247 DES ENC | 0 | 0 DES DEC | 0 | 0 3DES ENC | 0 | 0 3DES DEC | 0 | 0 AES ENC | 31235 | 0 AES DEC | 18741 | 0

engine ibmca + CPACF + CEX2

icastatsfunction | # hardware | # software ---------+------------+------------ SHA1 | 1082664 | 0 SHA224 | 0 | 0 SHA256 | 0 | 0 SHA384 | 0 | 0 SHA512 | 0 | 0 RANDOM | 1 | 0 MOD EXPO | 0 | 0 RSA CRT | 12303 | 0 DES ENC | 0 | 0 DES DEC | 0 | 0 3DES ENC | 0 | 0 3DES DEC | 0 | 0 AES ENC | 61515 | 0 AES DEC | 36909 | 0



Linux cryptographic support - Summary

Crypto Express2 Accelerator (CEX2A) – optional feature for System z machines – executes cryptographic requests asynchronously to the Central Processor (CP) – accelerates public key operations used for the SSL protocol (SSL handshake)– requires generic zcrypt device driver– zcrypt device driver with enabled polling thread utilizes adapter best

CP Assist for Cryptographic Function (CPACF) – supports several block ciphers and secure hash functions– executes cryptographic requests synchronously to the Central Processor (CP)– Linux kernel can use CPACF as well (in-kernel crypto)– software must be configured appropriately to exploit the hardware

CEX2A and CPACF can be combined – for example: SSL uses symmetric and asymmetric ciphers– best throughput results with both cryptographic features together



Trademarks

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Other product and service names might be trademarks of IBM or other companies.

Source: If applicable, describe source origin

http://www.ibm.com/legal/copytrade.shtml

Documents

Linux on system z cryptographic support - IBMpublic.dhe.ibm.com/software/dw/linux390/perf/crypto.pdf · 2010-10-11 · Linux on system z cryptographic support. ... (CEX2A) for Secure