Securing your cloud with Xen's advanced security features

Intro Network path Bootloader Device model Xen Conclusion

Securing your cloud with Xen’s advanced securityfeatures

George Dunlap

Brussels – February 2-3, 2013


Intro

I Xen is an enterprise-grade Type I hypervisor

I Built for the Cloud before it was called the CloudI A number of advanced security features

I Driver domains, stub domains, FLASK, &c

I Most of them aren’t or can’t be turned on by default

I Although simple to use, sometimes seem complicated

Brussels – February 2-3, 2013 Securing your cloud with Xen’s advanced security features 2 / 28


Goal

I Tools to think about security in Xen

I Know some key security features of Xen

I Equipped with the knowledge to get them working



Outline

I Overview of the Xen architecture

I Brief introduction to principles of security analysisI Consider some attack surfaces and Xen features we can use to

mitigate themI Driver domainsI pvgrubI stub domainsI PV vs HVMI FLASK example policy



Xen Architecture

Xen Hypervisor

Hardware

device model(qemu)

toolstack

dom 0

HardwareDrivers

I/O Devices CPU Memory

Paravirtualized(PV)

Domain

Fully Virtualized

(HVM)Domainnetback

blkbacknetfrontblkfront



Security Overview

I Threat ModelI Attacker can access networkI Attacker controls one guest OS

I Security considerationsI How much code is accessible?I What is the interface like? (e.g., pointers vs scalars)I Defense-in-depth



Example System

I Hardware setupI Two networks: control network, guest networkI IOMMU with interrupt remapping (AMD or Intel VT-d v2)

I Default configurationI Network drivers in dom0I PV guests with pygrubI HVM guests with qemu running in domain 0



Attack surface: Network path

Xen Hypervisor

Hardware

toolstackdom 0

NICDriver

Control NIC

RogueDomain

netback netfront

Guest NIC

bridgeiptables

Domain

netfront

I How to break in?I Bugs in hardware driverI Bugs in bridging / filteringI Bugs in netback via the ring protocol



Attack surface: Network path

Xen Hypervisor

Hardware

toolstackdom 0

NICDriver

Control NIC

RogueDomain

netback netfront

Guest NIC

bridgeiptables

Domain

netfront

I What does it buy you?I Control of domain 0 kernelI Pretty much control of the whole system



Security feature: Driver Domains

Xen Hypervisor

Hardware

toolstack

dom 0

NICDriver

Control NIC

RogueDomain

netback netfront

Guest NIC

bridgeiptables

Domain

netfront

NICDriver

Driver Domain

I What is it?I Unprivileged VM which drives hardware, provides access to

guests



Security feature: Driver Domains

Xen Hypervisor

Hardware

toolstack

dom 0

NICDriver

Control NIC

RogueDomain

netback netfront

Guest NIC

bridgeiptables

Domain

netfront

NICDriver

Driver Domain

I Now an exploit buys you:I Control of a PV VM (PV hypercall interface)I Guest network trafficI Control of NICI Opportunity to attack netfront of other guests



HowTo: Driver Domains

I Create a VM with appropriate driversI Any distro supporting dom0 should do

I Install the xen-related hotplug scriptsI Just installing the xen tools in the VM is usually good enough

I Give the VM access to the physical NIC with PCI pass-throughI Configure the network topology in the driver domain

I Just like you would for dom0

I Configure the guest vif to use the new domain IDI Add backend=domnet to vif declaration

vif = [ ’type=pv, bridge=xenbr0, backend=domnet’ ]

I http://wiki.xen.org/wiki/Driver Domain



Attack surface: Pygrub

Xen Hypervisor

toolstackdom 0

Paravirtualized(PV)

Domain

domainbuilder

pygrub

guestdisk

I What is it?I grub implementation for PV guestsI Python program running in domain 0I Reads guest FS, parses grub.conf, presents menuI Passes resulting kernel image to domain builder




Xen Hypervisor

toolstackdom 0

Paravirtualized(PV)

Domain

domainbuilder

pygrub

guestdisk

I How to break in?I Bugs in file system parserI Bugs in menu parserI Bugs in domain builder




Xen Hypervisor

toolstackdom 0

Paravirtualized(PV)

Domain

domainbuilder

pygrub

guestdisk

kernel

I What does it buy you?I Control of domain 0 user spaceI Pretty much control of the whole system



Security practice: Fixed kernels

Xen Hypervisor

toolstackdom 0

Paravirtualized(PV)

Domain

domainbuilder

kernelimage

guestdisk

I What is it?I Passing a known-good kernel from domain 0

I Removes attacker avenue to domain builder



Security practice: Fixed kernels

Xen Hypervisor

toolstackdom 0

Paravirtualized(PV)

Domain

domainbuilder

kernelimage

guestdisk

I DisadvantagesI Host admin must keep up with kernel updatesI Guest admin can’t pass kernel parameters, custom kernels,



Security feature: pvgrub

Xen Hypervisor

toolstackdom 0

domainbuilder

guestdisk

MiniOS

pvgrub

I What is it?I MiniOS + pv port of grub running in a guest contextI PV equivalent of HVM “BIOS + grub”

I Now an exploit buys you:I Control of your own VM



HowTo: pvgrub

I Make sure that you have the pvgrub imageI pvgrub-$ARCH.gzI Normally lives in /usr/lib/xen/bootI Included in Fedora Xen packagesI Debian-based: need to build yourself

I Use appropriate pvgrub as bootloader in guest config

kernel="/usr/lib/xen/boot/pvgrub-x86_32.gz"

I http://wiki.xen.org/wiki/Pvgrub



Attack surface: Device model (qemu)

I How to break in?I Bugs in NIC emulator parsing packetsI Bugs in emulation of virtual devices

I What does it buy you?I Domain 0 privileged userspaceI Pretty much control of the whole system



Security feature: qemu stub domains

I What is it?I Stub domain: a small “service” domain running just one

applicationI qemu stub domain: run each qemu in its own domain

I Now an exploit buys you:I Control of the stubom VMI ...which we assume you control



HowTo: qemu stub domains

I Make sure that you have the pvgrub image:I ioemu-$ARCH.gzI Normally lives in /usr/lib/xen/bootI Included in Fedora Xen packagesI Debian-based: need to build yourself

I Specify stub domains in your guest config

device_model_stubdomain_override = 1

I http://wiki.xen.org/wiki/Device Model Stub Domains



Attack Surface: Xen

I PV guestsI PV Hypercalls

I HVM guestsI HVM hypercalls (Subset of PV hypercalls)I Instruction emulation (MMIO, shadow pagetables)I Emulated platform devices: APIC, HPET, PITI Nested virtualization

I Security practice: Use PV VMs



Security feature: FLASK example policy

I What is FLASK?I Xen Security Module (XSM): Xen equivalent of LSMI FLASK: Framework for XSM developed by NSAI Xen Equivalent of SELinuxI Uses same concepts, tools as SELinuxI Allows a policy to restrict hypercalls

I What can FLASK do?I Basic: Restricts hypercalls to those needed by a particular

guestI Advanced: Allows more fine-grained granting of privileges

I FLASK example policyI This contains example roles for dom0, domU, stub domains,

driver domains, &c



HowTo: Use the example FLASK policy

I Build Xen with XSM enabled

I Build the example policyI Add the appropriate label to guest config files

I seclabel=[foo]I stubdom label=[foo]

I WARNING: In 4.2, the example policy not extensively tested.Use with care!

I http://wiki.xen.org/wiki/Xen Security Modules : XSM-FLASK



Outline

I Overview of the Xen architecture

I Brief introduction to principles of security analysisI Consider some attack surfaces and Xen features we can use to

mitigate themI Driver domainsI pvgrubI stub domainsI PV vs HVM

I Experimental: Example policy for FLASK



Goal

I Tools to think about security in Xen

I Know some key security features of Xen

I Equipped with the knowledge to get them working



Questions

Questions?

More info at http://wiki.xen.org/wiki/Securing XenCheck out our blog: http://blog.xen.org/


Technology

Securing your cloud with Xen's advanced security features