SafeBricks: Shielding Network Functions in the Cloud · NF TLS IPSec TLS SafeBricks also supports “direct” delivery of traffic IPSec. Outsourcing NFs using hardware enclaves 39

SafeBricks: Shielding Network Functions in the Cloud

Rishabh Poddar, Chang Lan, Raluca Ada Popa, Sylvia Ratnasamy

UC Berkeley

Network Functions (NFs) in the cloud

�2

DestinationEnterprise

Clients


�3

DestinationEnterprise

Clients


�4

NF providersNF providersNF providersNF providers

Enterprise

Clients

Destination

Problem: Security

�5


Enterprise

Clients

Destination

Email

�6


Enterprise

Clients

Destination

Email

Need to protect traffic from the cloud provider1

Hackers /curious employees

Problem: Security

Problem: Security

�7


Enterprise

Clients

Destination

Email

Need to protect traffic from the NF providers

Exfiltration

2

Problem: Security

�8


Enterprise

Clients

Destination

Email

Need to protect NF code and rulesets from client enterprise and cloud3

Cryptographic solutions do not suffice

�9


Enterprise

Clients

Destination

Email


�10


Enterprise

Clients

Destination

Email

Standard encryption: e.g. end-to-end TLS1


�11


Enterprise

Clients

Destination

Email

Standard encryption: e.g. end-to-end TLS • Functionality: Doesn’t allow any computation on encrypted payload

?

1


�12


Enterprise

Clients

Destination

Email

Standard encryption: e.g. end-to-end TLS • Functionality: Doesn’t allow any computation on encrypted payload • Security: Unencrypted fields (e.g. IP headers) still leak information

?

1


�13

Specialized encryption: e.g. BlindBox, Embark [Sherry et al. (SIGCOMM’15)]

[Lan et al. (NSDI’16)]

2


�14

Specialized encryption: e.g. BlindBox, Embark • Too limited in functionality!

2

• Header-based comparisons

• Keyword matching

• Regular expressions

• Cross-flow analysis

• Statistical computations

How to achieve full functionality and

our security goals simultaneously?

�15

SafeBricks

�16

2

3

Protects traffic from the NF providers

Protects NF source code and rulesets from client enterprise and cloud

Protects traffic from the cloud provider1

SafeBricks

�17

2

3



Hardware enclaves + language-based isolation


• Secure region of memory (enclaves) protected by hardware

Background: Hardware enclaves (e.g. Intel SGX)

�18

Operating System (untrusted)

Application (untrusted)

Enclave (trusted)


�19



Enclave (trusted)

Secretdata

Trustedcode



�20



Enclave (trusted)

Secretdata

Trustedcode

Client

• Remote attestation by clients



�21



Enclave (trusted)

Secretdata

Trustedcode

Client

• Remote attestation by clients • Remotely verify enclave contents



�22



Enclave (trusted)

Secretdata

Trustedcode

Client

• Remote attestation by clients • Remotely verify enclave contents • Establish a secure channel with enclave


Background: NetBricks

�23

[Panda et al. (OSDI’16)]

I/O interface

Poll for I/O

Programming abstractions

State abstractions

Scheduler

DPDK

NICs

NetBricks• Framework for developing arbitrary NFs

�24


• Framework for developing arbitrary NFs • MapReduce like programming

abstractions (operators) for packet processing

I/O interface

Poll for I/O


State abstractions

Scheduler

DPDK

NICs

NetBricks


�25


• Framework for developing arbitrary NFs • MapReduce like programming

abstractions (operators) for packet processing

• NFs represented as a directed graph with operators as nodes

I/O interface

Poll for I/O


State abstractions

Scheduler

DPDK

NICs

NetBricks


�26

• Written in Rust

• Fast, safe, zero-copy semantics

• Isolates NFs deployed in a chain while running them in the same address space

NICs

NetBricks

NF 1 NF 1


SafeBricks

�27


Protects traffic from the cloud provider

2

3

1


SafeBricks

�28



2

3

1


Outsourcing NFs using hardware enclaves

�29 Enterprise

Clients

Gateway

Enclave

OS (untrusted)

Cloud (untrusted)

Destination


�30 Enterprise

Clients

Gateway

Enclave

OS (untrusted)

Cloud (untrusted)

Destination

NF


�31 Enterprise

Clients

Gateway

Enclave

OS (untrusted)

Cloud (untrusted)

Destination

NF


�32 Enterprise

Clients

Gateway

Enclave

OS (untrusted)

Cloud (untrusted)

Destination

NF

IPSec

IPSec


�33 Enterprise

Clients

Gateway

Enclave

OS (untrusted)

Cloud (untrusted)

Destination

NF

IPSec

IPSec

TLS

Interception proxy


�34 Enterprise

Clients

Gateway

Enclave

OS (untrusted)

Cloud (untrusted)

Destination

NF

TLS

IPSec

TLS

IPSec


�35 Enterprise

Clients

Gateway

Enclave

OS (untrusted)

Cloud (untrusted)

Destination

NF

TLSTLSTLSTLS

IPSecIPSec

TLS

IPSec


�36 Enterprise

Clients

Gateway

Enclave

OS (untrusted)

Cloud (untrusted)

Destination

NF

TLSTLSTLSTLS

IPSecIPSec

TLS

IPSec

Packet headers also encrypted


�37 Enterprise

Clients

Gateway

Enclave

OS (untrusted)

Cloud (untrusted)

Destination

NF

TLSTLSTLSTLS

IPSec

TLS

IPSec

TLS

IPSec


�38 Enterprise

Clients

Gateway

Enclave

OS (untrusted)

Cloud (untrusted)

Destination

NF

TLSTLSTLSTLS

IPSec

TLS

SafeBricks also supports “direct” delivery of traffic

IPSec


�39 Enterprise

Clients

Gateway

Enclave

OS (untrusted)

Cloud (untrusted)

Destination

NF

TLSTLSTLSTLS

IPSec

TLS

Can use general purpose frameworks,

e.g. Haven, Scone

IPSec

TLS

IPSec

Challenges

�40

Small trusted computing base (TCB) — enclave should contain minimal amount of code

1

Challenges

�41

High performance — Transitioning into / out of enclaves is expensive!


2

1

Challenges

�42


Illegal enclave instructions — SGX does not support system calls or instructions that may lead to a VMEXIT


2

3

1

Challenges

�43

2

3


Illegal enclave instructions — SGX does not support system calls or instructions that lead to a VMEXIT


1

�44

I/O interface

Poll for I/O


State abstractions

Scheduler

DPDK

NICs

NetBricks1

�45

I/O interface

Poll for I/O


State abstractions

Scheduler

DPDK

NICs

Enclave1

• Maximal TCB: NetBricks stack entirely within enclave

�46

I/O interface

Poll for I/O


State abstractions

Scheduler

DPDK

NICs

Enclave1

• Minimal TCB: Only security-critical components within enclave

• One enclave transition per node per packet batch

• Intermediate TCB • One enclave transition per

packet batch

�47

I/O interface

Poll for I/O


State abstractions

Scheduler

DPDK

NICs

Enclave1

�48

I/O interface

Poll for I/O


State abstractions

Scheduler

DPDK

NICs

Glue code (trusted)

Glue code (untrusted)

SafeBricksenclave(trusted)

SafeBrickshost

(untrusted)

1

• Partitioned NetBricks framework; glue code connects trusted and untrusted code

�49

I/O interface

Poll for I/O


State abstractions

Scheduler

DPDK

NICs

Glue code (trusted)

Glue code (untrusted)

SafeBricksenclave(trusted)

SafeBrickshost

(untrusted)

1

• Partitioned NetBricks framework; glue code connects trusted and untrusted code

Two new operators for packet transfer to/from enclave:

and toHosttoEnclave

Challenges

�50

2

3



1


SafeBricks host

SafeBricks enclave

�51

NICs

2

toHosttoEnclave

NF

• One enclave transition per packet batch

SafeBricks host

SafeBricks enclave

�52 NICs

2

toHosttoEnclave

NF

• Shared queues in non-enclave heap

• Separate enclave and host threads

• Access queues without exiting enclave — zero enclave transitions

send recv

Enclave I/O

Host I/O

Challenges

�53

2

3



1


�54

3

Observation: NFs in general do not require support for system calls / instructions that lead to VMEXITs

�55

3

Observation: NFs in general do not require support for system calls / instructions that lead to VMEXITs, except:

• Logging

• Timestamps (using rdtsc)

�56

Observation: NFs in general do not require support for system calls / instructions that lead to VMEXITs, except:

• Logging

• Timestamps (using rdtsc)

3

SafeBricks designs custom solutions for these operations without enclave transitions

�57

2

3

1

SafeBricks




Problem: Malicious NFs within enclaves

�58

Malicious NFs inside the enclave can exfiltrate or tamper with packets!!


�59

Observation: NFs typically need access only to specific packet fields

• E.g. Firewall needs read-only access to TCP/IP headers

• E.g. NAT needs both read-write access to headers but not to packet payload



�60





IP addresses; TCP ports; HTTP

payload


�61





SafeBricks enforces least privilege across NFs within the enclave

toEnclave toHost

Host

�62

Run NFs within the same enclave

Least privilege enforcement

Firewall NAT

�63

Run NFs within the same enclaveFirewall

toEnclave

NAT

toHost

Host


�64

Run NFs within the same enclave

• Stitch NFs together interspersed with an operator ( wList ) that embeds a vector of permissions in packets — two bits per packet field

Firewall

toEnclave

NAT

toHost

Host

wList wList

wList


�65

Enforce permissions by mediating access to

packets using Rust’s ownership model

SafeBricks Controller


�66



• Controller module holds ownership of packet buffers


Packet buffer


�67




• NFs borrow references to packet fields from the Controller, which checks permissions vector in packet


NAT Firewall

Packet buffer


�68




• NFs borrow references to packet fields from the Controller, which checks permissions vector in packet


Packet buffer

NAT Firewall

Returns an immutable reference for read-only access, and a mutable reference for

write access


Assumption: Trusted compilation of NFs

�69

Least privilege guarantees only hold if NFs are built using a compiler that prohibits unsafe operations!


�70


E.g. Check array bounds, no pointer arithmetic, no

unsafe type casts


�71


• Possible solution: Client obtains NF source codes from providers and assembles them locally


Client Enterprise

NF code + rulesets


�72


• Possible solution: Client obtains NF source codes from providers and assembles them locally

• Problem: This violates the confidentiality of NF source code!


Enterprise

NF code + rulesets

�73

2

3

1

SafeBricks




Assembling NFs

�74

• Key idea: Build NFs within a special “meta”-enclave in the cloud using an agreed upon compiler

Assembling NFs

�75

• Key idea: Build NFs within a special “meta”-enclave in the cloud using an agreed upon compiler

• Both client and NF providers can verify the agreed upon compiler using remote attestation

Assembling NFs

�76

Enterprise


Loader Compiler

Assembly enclave

Assembling NFs

�77

Enterprise


Loader Compiler

Assembly enclave

Remote attestation

Assembling NFs

�78

Enterprise


Loader Compiler

Assembly enclave

Remote attestation

Assembling NFs

�79

Enterprise


Loader Compiler

Assembly enclave

Remote attestation

Assembling NFs

�80

Enterprise


Loader Compiler

Assembly enclave

NF code + rulesets

Assembling NFs

�81

Enterprise


Loader Compiler

Assembly enclave

NF code + rulesets

Config

Assembling NFs

�82

Enterprise


Loader Compiler

Assembly enclave

NF code + rulesets

Config

Placement of NFs, least privilege policies per NF

Assembling NFs

�83

Enterprise


Loader Compiler

Assembly enclave

NF code + rulesets

Config

Assembling NFs

�84

Enterprise


Loader Compiler

Assembly enclave

Deployment enclave

SafeBricks

�85

2

3




Performance

�86

Throughput decline across NFs

�87~0–15% overhead across applications for different packet sizes

Thro

ughp

ut d

eclin

e

0%

10%

20%

30%

40%

Packet size (bytes)64 256 512 1024

Firewall NATLoad Balancer DPI

DPI performance with increasing no. of rules

�88 Overhead spikes when NF working set exceeds enclave memory

Thro

ughp

ut d

eclin

e

0%

25%

50%

75%

100%

Number of rules

0 500010000

1500020000

25000

94MB64MB8MB

DPI performance with increasing no. of rules

�89 Overhead spikes when NF working set exceeds enclave memory

Thro

ughp

ut d

eclin

e

0%

25%

50%

75%

100%

Number of rules

0 500010000

1500020000

25000

94MB64MB8MB

Not a fundamental limitation

SafeBricks uses a combination of hardware enclaves

and language-based isolation to:

• Protect client traffic from the cloud provider

• Enforce least privilege across NFs

• Protect the confidentiality of NF code and rulesets

�90

Summary

Modest overhead across a range of applications

[email protected]

mailto:[email protected]

Documents

SafeBricks: Shielding Network Functions in the Cloud · NF TLS IPSec TLS SafeBricks also supports “direct” delivery of traffic IPSec. Outsourcing NFs using hardware enclaves 39