Panoply:Low-TCBLinuxApplica5ons

WithSGXEnclavesShwetaShindeDatLeTien

Shru5ToplePrateekSaxena

Na#onalUniversityofSingapore

TCB:Hos5ngaWebServer

Hypervisor

Opera5ngSystem

Webserver10MLOC

150KLOC

183KLOC

CurrentsystemshavealargeTCB

2

SGX:Hardware-rootofTrust

3

Opera5ngSystem/VMM

RAM

WebServerEnclave

Ring0-2

Ring3

TrustedUntrusted

EPC

OtherApplica5ons

SGX:Hardware-rootofTrust

Hardware

Hypervisor

Opera5ngSystem

Webserver10MLOC

150KLOC

183KLOC

Confiden5alityandIntegrity

Enclave

4

SGXeliminatesallthenon-applica5onsoUwarefromTCB....butlimitstheexpressivenessofthe

applica5ons(e.g.,nosyscalls)

5

TCB&ExpressivenessTrade-off

Expressiveness

TCB

~KLOC

~100KLOC

~1MLOC

ContainersScone[OSDI’16]

Ryoan[OSDI’16]

LibraryOSHaven[OSDI’14] Graphene-SGX[EuroSys’14]

?

6

Contribu5ons

•  Panoply– Expressiveness:AllstandardPOSIXAPIs– LowTCB:2ordersofmagnitudesmallerthanLibraryOS– Library-enclavesforfine-grainedTCB

•  EvaluaCon– Absolute24%and5-10%comparedtoLibraryOS

7

Problem

8

ChallengeI:ExpressivenessvsTCB

Webserver

Syscalls

Threading

EventHandling

Forking

TCB

SyscallEmula5on

ThreadHandling

EventManagement

ForkEmula5on

WebServer

EnclaveLegacyApplica5onDesign

9

ChallengeI:Expressivenessvs.TCB

Expressiveness TCB

10

ChallengeII:Mul5-EnclaveApplica5ons

WebServer libssl

libssl

libcrypto

libevent

zlib

WebServer

SingleEnclaveApplica5on

WebServer

libssl

libcrypto

libevent

zlib

Mul5-EnclaveApplica5on

Opera5ngSystem

11

AjacksonMul5-EnclaveApplica5ons

SSLLibraryEnclave

WebserverEnclave

session_tsession;certificate_credentials_txcred;/*Specifycallbackfunction*//*InitializeTLSsession*/init(&session,TLS_CLIENT);

certificate_set_verify_function(...);

SetSSLCallback OS

[SSLLib]

12

AjacksonMul5-EnclaveApplica5ons

OS

SSLLibraryEnclave

WebserverEnclave

Drop

Spoof

Replay13

OurSolu5on:Panoply

14

PanoplyRun5me

Micronskeeplibcoutsidetheenclave

15

Enclave-boundLogicPanoplyShimLib TrustedSGXLib

libc.so Non-enclaveLogic

UntrustedSGXLib

LinuxUser-levelProcess

Enclave

Non-Enclave

Micron

Overview

Panoply

CompilerInstrumentation

CreatingEnclaves

E1

E2

1 2

AddcallstoPanoplyAPI

AddFlowChecks

Source Code

Programmer Annotations

IntelSGXSDK

PanoplyShim

Enclave-boundCode

16

PanoplyApplication

Ap

E1

E3E2

ChallengeI:Expressiveness

17

Enclave

Syscalls

On-demandthreading

Mul5-processing

EventHandling

Delegateratherthanemulate

Expressiveness:PanoplyAPIsCoreServices

ProcessCrea5onandControl 5

Signals 6

Timers 5

FileandDirectoryOpera5ons 37

Pipes 4

CLibrary(StandardC) 66I/OPortInterfaceandControl 40

Real-CmeExtensionsReal-TimeSignals 4

ClocksandTimers 1

Semaphores 2

MessagePassing 7

SharedMemory 6AsynchronousandSynchronousI/O

29

MemoryLockingInterface 6ThreadExtensions

ThreadCrea5on,Control,andCleanup

17

ThreadScheduling 4

ThreadSynchroniza5on 10

SignalDelivery 2

SignalHandling 3

POSIXAPIsSupportedfor

CommodityLinuxApps18

ExpressivenessExample:Fork

ParentProcess

ChildProcess

OSPageCopy

ParentEnclave

ChildEnclave

PageCopyPIDMgmt

OS

PageCopyPIDMgmt

ForkSeman5cs LibraryOSForkImplementa5on

PIDMgmt

19

LibraryOSesemulateforkseman5cs

ExpressivenessExample:Delega5ngFork

PanoplyFork OS PIDMgmt

ParentEnclave ChildEnclave

ParentProcess ChildProcess

20

•  Crea5ngchildprocessandchildenclave

•  Childenclavehasacleanmemorystate

OS

ChildEnclave

ExpressivenessExample:AchievingForkSeman5cs

•  Mirroringparent’smemoryinchildenclave– AUertheforkcall,beforeresumingexecu5on

21

ParentEnclave

ChildProcess

Stack

ParentProcess

SealedData

Heap Data Stack Heap Data

ExpressivenessExample:AchievingForkSeman5cs

•  Mirroringparent’smemoryinchildenclave– Fullreplica:defaultmodeinPanoply

•  Alterna5vestrategiestofullreplica– Copyondemand:Requirespage-faultsupportfromSGXv2

– Copyonneed:Replicateselectedaddresseswhicharepre-determinedbysta5canalysis

22

ExpressivenessExample:Mul5-Threading

PanoplyShim

TCS1

TCS2

TCS3

Thread Control

Manager Micron

SharedVariables

VirtualThreads

Micron A

23

PanoplyShim

TCS1

TCS2

TCS3

Micron A’

ChallengeII:Mul5-enclaveApplica5ons

24

SecuringMul5-EnclaveApps

Enclave2Enclave1

OS

EnclaveIden5ty

CallAck

Pair-wiseNonce

25

ALack SecurityPropertySpoofing Sender/ReceiverAuthen5ca5onReplay MessageFreshness

SilentDrops ReliableDelivery

Evalua5on

26

Benchmarks•  Real-worlduse-casesforSGX–  4apps:Tor,H2Owebserver,FreeTDS,OpenSSL

•  Opera5ngsystemstresstes5ng–  26LMBenchbenchmarkstests–  17metricsformemory,network,signal,syscallAPIs

27

TCBEvalua5onGraphene-SGXPanoply

Component LOCPanoplyLibrary 10425APIWrappers 9788Total 20213

Component LOC

Glibc 1156740libPal-LinuxSGX 16901libPal-enclave 33103Total 1206744

28

PanoplyreducesTCBby2ordersofmagnitude

PerformanceEvalua5on

•  Createdeletetakeslargefrac5onofthe5me•  OverheadincreaseswithnumberofOut-Calls

29

App Panoply EmptyEnclave

Overhead(%increase)

OpenSSL 3.16 2.79 13H2O 8.79 6.56 34FreeTDS 8.74 8.60 1Tor 6.72 4.54 48

Average 24

Panoplyincurs24%overhead

ThroughputEvalua5on

30

ThroughputEvalua5on

31

OverheadforSGX-appsispropor5onaltothesizeofrequests

ComparisonwithGraphene-SGX

32

ComparisonwithGraphene-SGX

Panoplyperformancevariesby5-10%ascomparedtoGraphene-SGX

33

Conclusion

Expressiveness

TCB

~KLOC

~100KLOC

~1MLOC

Panoply

34

-254APIs-20KLOC-24%Overhead

Contact•  ShwetaShindeshweta24@comp.nus.edu.sg

•  PanoplyBenchmarks&Case-studies:hjp://shwetasshinde24.github.io/Panoply/

ThankYou!35

References•  [OSDI’14]A.Baumann,M.Peinado,andG.Hunt,ShieldingApplica5onsfromanUntrustedCloudwithHaven

•  [OSDI’16]S.Arnautov,B.Trach,F.Gregor,T.Knauth,A.Mar5n,C.Priebe,J.Lind,D.Muthukumaran,D.O’Keeffe,M.L.S5llwell,D.Goltzsche,D.Eyers,R.Kapitza,P.Pietzuch,andC.Fetzer,SCONE:SecureLinuxContainerswithIntelSGX

•  [OSDI’16]T.Hunt,Z.Zhu,Y.Xu,S.Peter,andE.Witchel,Ryoan:ADistributedSandboxforUntrustedComputa5ononSecretData

•  [EuroSys’14]Graphene-SGXLibraryOS-alibraryOSforLinuxmul5-processapplica5ons,withIntelSGXsupport

36