Student: Cuong PhamPIs: Prof. Zbigniew Kalbarczyk, Prof. Ravi K. Iyer

ACC Meeting, Oct 2015

Monitoring Hypervisor Integrity at Runtime

Hypervisor

OS

App

OS

App

OS

App

VirtualMachine

VirtualMachine

VirtualMachine

x86 Architecture Server

Motivation - Server Virtualization Trend

2

x86 servers werevirtualized in 2012Source: 451 Research's TheInfoPro service reports

Source: Derivative analysis based on Worldwide Virtual Machine 2013–2017 Forecast: VirtualizationBuildout Continues Strong IDC #242762 / Aug 2013

Building Secure & Reliable VMs

3

Chain of trust must be built from bottom up … and continuously through time

Firmware/Bios

OS/Hypervisor

VM

Hardware

VM VM…

Layer

HyperTap (DSN ‘14)

Hprobes (EDCC ‘15)

Attack Surface

TimeLoad-time Execution

Building Secure & Reliable VMs

4

Chain of trust must be built from bottom up … and continuously through time

Firmware/Bios

OS/Hypervisor

VM

Hardware

VM VM…

Layer

TPM

Intel TXT

HyperTap (DSN ‘14)

HProbes (EDCC ‘15)

Attack Surface

TimeLoad-time Execution

Physical security

Protect Hypervisors: Existing approaches

5

Did I say “continuously through time”?

VM VM VM…

LayerAttack Surface

HyperSentry/SICE …

Periodicallymeasure

hypervisor integrity

Firmware/Bios

OS/Hypervisor

HardwareTPM

Intel TXT

TimeLoad-time Execution

Physical security

Vulnerable to transient attacks

Introducing hShield

6

Assumption: Hardware is trusted TPM, Intel TXT are enabled Physical security

VM VM VM…

Layer

HyperTap

HProbes

Attack Surface

hShield

Continuouslymeasure

hypervisor integrity

Firmware/Bios

OS/Hypervisor

HardwareTPM

Intel TXT

Physical security

TimeLoad-time Execution

Threat Model: VM Escape Attacks

7

Hardware Assisted Virtualization

Attackers have full control of guest OS

Violate hypervisor Control Flow Integrity (CFI)

VM-exit

VM-entry

PrivilegedOperation

Virtual Machine Hypervisor

Hypervisor execution(handle VM-exit)

Attack entry point

Attack code

Example: Venom VM Attack (CVE-2015-3456)

8

VM Exit

VM Entryioctl()io(port 0x35,

command 0x8e)

fdctrl->fifo[fdctrl->data_pos++] = value;

QEMUBHFunc *bh->cb

Attack codecallq bh->cb

overwrite

point to

Virtual Machine KVM/XEN QEMU (version containsVenom vulnerability)

DescriptionThough the VENOM vulnerability is also agnostic of the guest operating system, an attacker (or an attacker’s malware) would need to have administrative or root privileges in the guest operating system in order to exploit VENOMImpact Type: Allows unauthorized disclosure of information; Allows unauthorized modification; Allows disruption of service

Source: https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-3456

Threat Model: VM Escape Attacks

9

Attack code DoS host Access other co-located VMs (e.g., sniffing network traffic, stealing images) Install backdoors, access secrets in host…

VM Exit

VM Entry

PrivilegedOperation

Virtual Machine Hypervisor

Hypervisor execution(handle VM Exit)

Attack entry point

Attack code

VM Escape-enabling CVEs…

10

CVE-2007-1744 – Directory traversal vulnerability in shared folders feature CVE-2008-0923 – Path traversal vulnerability in VMware’s shared folders

implementation CVE-2009-1244 – Cloudburst (VMware virtual video adapter vulnerability) CVE-2012-0217 – 64-bit PV guest privilege escalation vulnerability CVE-2014-0983 – Oracle VirtualBox 3D acceleration multiple memory

corruption vulnerabilities CVE-2015-(2336-2340) – Escaping VMware Workstation through COM1 (5

CVE!!!) CVE-2015-3456 – QEMU heap overflow flaw in floppy disk driver CVE-2015-5154 – QEMU heap overflow flaw while processing certain ATAPI

commands.

hShield Design Goals

11

1. Resistance to zero-day VM escape attacks2. Detect both transient and persistent attacks3. Small performance overhead in attack-free executions4. Support target randomization

hShield Approach

12

Continuous monitoring Detect both persistent and transient attacks

White-list monitoring Detect unknown attacks

Hardware extension Hardware isolation and performance

…

OS/Hypervisor

HardwarehShield

TPM

hShield Overview

13

Detect a VM-escape attack right at the end of the exploited VM-exit: defeat transient attacks.

Virtual Machine Hypervisor

Start measurementsession

Stop measurementsession

Hypervisor execution

(handle VM-Exit)

VM-exit

VM-entry

Execution tracei1i2i2…im

Hi

Whitelist

Check

PrivilegedOperation

hShield Counter

hShield Auditor

Match

No match

White-list vs. Black-list

14

White-list No one can access except the white-listed Prevents unknown attacks E.g., Control Flow Integrity (CFI) techniques

Black-list Everyone can access except the black-listed Prevents known (black-listed) attacks E.g., Signature-based malware detection

2

3

6

5

4

1

X

ControlFlowGraph(CFG)

Signature based detection

Current CFI Techniques

15

Heavily relies on static analysis to construct CFG No interactions with dynamic libraries, OS Heuristic (e.g., pointer analysis is imperfect) Scalability issues (e.g., large binaries)

High runtime overhead Check at every branch

Compatibility issues against Address Space Layout Randomization (ASLR) Programs relying on dynamic binary re-writing (e.g., Linux kernels)

2

3

6

5

4

1

X

ControlFlowGraph(CFG)

hShield Approach

16

Dynamic analysis to construct CFG Full system coverage Load-time binary rewriting compatible

Check at the sink of executions Reduce runtime overhead

Use hashes of basic blocks instead of addresses ASLR compatible

2

3

6

5

4

1

2

3

6

5

1

2

6

5

4

1

=+

Check

Source(VM-exit)

Sink(VM-entry)

hShield Approach

17

2

3

6

5

4

1

2

3

6

5

1Hash HiWorkloads

VM

CFGConstruction

Hypervisor

HotTableH1H2…Ht

Most popularpaths

Profiling Runtime checking

GoodBad

Yes

Yes

No

No

RecomputedHi in CFG

Find Hi inHotTable

VM-exit

VM-entry

Profiling Result: Path Popularity

18

Setup: Qemu (HW) – Qemu(Host) – Linux VM (Guest)Workloads: Boot Linux kernel + UnixBench

High hit rate of HotTable• 1% paths – 97% of exits• 0.1% paths – 95% of exits

Increase HotTable Hit Rate

19

Execution Pattern Inference: Hash = a pattern of similar executions Noise reduction (e.g., exclude interrupt handlers) Loop rerolling

Execution tracei1i2i2…im

Pattern of executionsbb1bb2…bbn

Hi

Noise reductionLoop rerolling

Hash

Loop Rerolling Example

20

a = 0;for (i = 1..n)

a = a + i;return a;

a = 0i = 1

a = a + ii = i + 1

i <= n

return a

Y

N

Basic block 1 (BB1)

BB2

BB3

BB4

BB1 BB2 BB4BB1 BB2 BB3 BB4BB1 BB2 BB3 BB2 BB3 BB4…|Paths| = 1 + |Range(n)|

a = 0i = 1

a = a + ii = i + 1

i <= n

return a

Y

N

Loop Rerolling Example

21

a = 0;for (i = 0..n)

a = a + i;return a;

Basic block 1 (BB1)

BB2

BB3

BB4

Solution: Loop rerolling

BB1 BB2 BB4BB1 BB2 BB3 BB4BB1 BB2 BB3 BB2 BB3 BB4…|Paths| = 2

hShield Approach

22

2

3

6

5

4

1

2

3

6

5

1Hash HiWorkloads

VM

CFGConstruction

Hypervisor

HotTableH1H2…Ht

Most popularpaths

Profiling Runtime checking

GoodBad

Yes

Yes

No

No

RecomputedHi in CFG

Find Hi inHotTable

VM-exit

VM-entry Small size Fast Lookup High hit rate (>95%)

hShield Approach

23

2

3

6

5

4

1

2

3

6

5

1Hash HiWorkloads

VM

CFGConstruction

Hypervisor

HotTableH1H2…Ht

Most popularpaths

Profiling Runtime checking

GoodBad

Yes

Yes

No

No

RecomputedHi in CFG

Find Hi inHotTable

VM-exit

VM-entry

How to efficiently verify Hi is recomputable in the CFG?

Execution Path Reconstruction

24

Input Hash Hi and CFG G=<V, E>

Question: Exist a path P ∈ G: Hash(P) = Hi

Naïve solution: Traverse G until P is found - impractical

hShield Solution:Using incremental hashing to efficiently reconstruct P from

Hi and G

Execution Path Reconstruction

25

2

3

6

5

4

1

Source

Sink

Hash Hi

2

4

6

5

1

Is first basic block?Yes, update( , Hi) == Hi

1

1

Is first basic block?Yes, update( , Hi) == Hi

2

2

Is first basic block?No, update( , Hi) != Hi

3

3

Is first basic block?Yes, update( , Hi) == Hi

4

4

…. end at 6

Execution Representation

26

E ∈ Exe: E = I1I2..In Ii: Instruction byte code (e.g.,

x86)

Range: Fixed length output

f: Exe Range

• f requirements• Collision resistant• Interactive – online construction• Incremental – online update• Facilitate loop rerolling

implementation

Incremental Collision-free Hashing [1]

27

Randomization – Derived from standard cryptographic functions (e.g., SHA, MD5)

Combination – Algebraic operation Incrementality: Allow update

results when a portion of input changed without re-computing from scratch

Collision-free [1]

[1] M. Bellare and D. Micciancio, “A new paradigm for collision-free hashing: Incrementality at reduced cost,” in Advances in Cryptology EU- ROCRYPT97. Springer, 1997, pp. 163–192.

h(<1>B1) h(<n>Bn)h(<2>B2) …

⊗

Randomization

Combination

Message E

f(E)

hShield Counter Hash Function

B’1=<1>.s.B1 B’n=<n>.s.BnB’2=<2>.s.B2 …

Execution E

B2

[address: instruction]

y2 = sha1(salt.<2>.b2)

MuHASH(y1, y2) = y1⊗ y2

hShield Counter Hash Function

29

B = Basic block – facilitate loop rerolling

s: salt – individualize target

⊗: modular multiplication (MuHash)

B’1=<1>.s.B1 B’n=<n>.s.BnB’2=<2>.s.B2 …

⊗

Execution E

f(E)

sha1(B’1) sha1(B’n)sha1(B’2) …

F(E) = ⊗i=1..n(sha1(<i>.s.Bi))

Security Evaluation

30

Attack Model Change execution E —> E’: f(E’) ∈ F(E)Solution Find E’ complexity = Discrete Log problem (assuming h is ideal) Must harder to find an E’ which is valid x86 code

E h ⨷ salt F(E)

Know ✔ ✔ ✔ ✔ ✔Change ✔ ✕ ✕ ✕ ✕

Scopes

31

Focus on design of the monitoring framework What/Where/How to monitor (Answers: VM Exit/Hardware/Whitelist)

Design for flexible future hardware implementation Make best effort to conduct measurement on actual hardware

E.g., obtained supporting data on physical systems.

Make best effort to anticipate problems in actual hardware implementation. E.g., issues with speculative execution, memory size constraints.

Assume that we can place hooks in some basic signals in the hardware. E.g., intercept all interrupts and exceptions.

Prototype the proposal in QEMU Software emulation of x86 processors and many external devices.

More to come…

32

Archirectural Design Evaluation with real attacks Performance evaluation