Module: Cloud Computing Security

Systems and Internet Infrastructure Security Laboratory (SIIS) Page

Module: Cloud Computing Security

Professor Trent JaegerPenn State University

1

1

Systems and Internet Infrastructure Security (SIIS) Laboratory Page

Cloud Computing Is Here

2

2-1



2

2-2



2

2-3



2

2-4



2

2-5



2

2-6



2

2-7



2

2-8



2

2-9



2

2-10



2

Why not use it?

2-11


What’s Happening in There?

3

3


From Data Center to Cloud

4

4-1



4

4-2



4

4-3


Reasons to Doubt

• History has shown they are vulnerable to attack

‣ SLAs, audits, and armed guards offer few guarantees

‣ Insiders can subvert even hardened systems

5

Data Loss Incidents

‘06 ‘07 ‘08 ‘09 ‘10 ‘11

903

678695

986

770641

Incident Attack Vector

External54%

Unknown7%

Insider16%

Accidental23%

Credit: The Open Security Foundation datalossdb.org

5


Cloudy Future

• New problem or new solution?

‣ New challenges brought on by the cloud (plus old ones)

‣ Utility could provide a foundation for solving such challenges

6

6


What is Cloud Computing?• Cloud vendor provides managed computing

resources for rent by customers

• What do you want to rent?

‣ (Virtualized) Hosts (Infrastructure as a Service)

• Rent cycles: Amazon EC2, Rackspace Cloud Servers, OpenStack

‣ Environment (Platform as a Service)

• Rent instances: Microsoft Azure, Google App Engine

‣ Programs (Software as a Service)

• Rent services: Salesforce, Google Docs

• Other variations can be rented7

7


What is Cloud Computing?

8

8


IaaS Platform: OpenStack

9

Client

SchedulerNetworkController

CloudDatabase

Message Queue

VolumeStore

ImageStore

Cloud API

CloudCustomer

CloudNode

Instances

CloudVendor

9


How to Build an IaaS Cloud?• Vendors obtain hardware resources for

‣ Various cloud services: API, Messages, Storage, Network, ...

‣ Compute nodes for running customer workloads

• Install your hardware

‣ Need to choose software configurations specific for services and compute nodes

• Start your hosts

‣ Join the cloud - services and available compute nodes

• Now your cloud is running

‣ Have fun! Customers are ready to use your services and nodes

10

10


How to Use an IaaS Cloud?• Customers choose an OS distribution

‣ These are published by the cloud vendor and others

‣ Obtain cloud storage necessary to store these and your data

• Configure your instance (VM)

‣ Prior to starting - enable you to login and others to access the instance’s services

• Start your instance

‣ Boots the chosen OS distribution with the configurations

• Now your instance is running

‣ Have fun! Login via SSH or ready for your clients

11

11


Cloud Complexity• Cloud environment challenges

‣ Opaque, Complex, Dynamic

‣ Insiders, Instances, Co-hosting

12

Client Service

12-1





12

Client

CloudNode

CloudNode

CloudNode

CloudNodeCloud

Platform

12-2





12

Client

CloudNode

CloudNode

CloudNode

CloudNode

12-3





12

Client

CloudNode

CloudNode

CloudNode

CloudNode

VM

12-4





12

Client

CloudNode

CloudNode

CloudNode

CloudNode

VM

12-5


VM




12

Client

CloudNode

CloudNode

CloudNode

CloudNode

VM

12-6


VM




12

Client

CloudNode

CloudNode

CloudNode

CloudNode

VM

12-7


VM




12

Client

CloudNode

CloudNode

CloudNode

CloudNode

VM

VMVM

12-8


VM




12

Client

CloudNode

CloudNode

CloudNode

CloudNode

VM

VM

VM

12-9


What Could Go Wrong?• What do customers depend on from the cloud?

‣ Trust Model

‣ Are those parties worthy of our trust?

• Who are potential adversaries in the cloud?

‣ Threat Model

‣ Are customers protected from their threats?

• What would be ideal from a security standpoint?

‣ Ideal Security Model

‣ How many trusted parties and how many threats?

13

13


Published Instances

14

our case, operates the IaaS cloud infrastructure, authenti-cates users and bills them for the resources they consumed.

The Publisher creates and publicly o↵ers cloud apps, calledAmazon Machine Images (AMIs). For this, he selects an ex-isting AMI (AMI-1 in Fig. 1), instantiates it (Instance-1AMI-1),logs into the running instance to configure it, and finallypublishes a snapshot as a new AMI (AMI-2).

The Consumer selects this AMI from a list of availableAMIs, instantiates it (Instance-2AMI-2), and uses it for herpurposes. Optionally, a Publisher can declare an AMI aspaid AMI to earn money from Consumers invoking it.

!"#$%&'((&)*#+,&

-.&/#012$+,& 3.&405*6076*,&

8.&$5,&

9.&($:"45;&

<.&405*6076*,&

!"#$%&'()*

+,-&".()*

!),/%0()*

=05*60/,>3'?=>3&

=05*60/,>-'?=>-&

'?=>3&

'?=>-&

Figure 1: Basic System Model of Cloud App Store

The Cloud App Store poses security challenges for both,Consumers and Publishers (see also [48, 17]).

Security of Consumer. The Consumer must trust thePublisher not to include any malware into the AMI. Sucha malicious AMI could contain a Trojan horse that spieson or modifies the Consumer’s data, or a backdoor for mali-cious remote login. Even though full protection against suchmalicious AMIs is almost impossible, filters, virus scanners,and rootkit detectors could provide at least some level ofprotection [48].1

Security of Publisher. The Publisher on the other handmight accidentally publish AMIs that contain highly sensi-tive information. Examples include keys, credentials, pass-words, command history/log files, or source code.

Although Amazon’s user guide recommends to ensure thatall confidential information is removed before publishing anAMI [12, Sharing AMIs Safely], many users seem to be un-aware of the crucial consequences of ignoring these recom-mendations, do not have the appropriate tools at hand, orsimply forgot private data in their AMIs.

The Gap between Theory and Practice. The Pro-vider could filter AMIs for Trojans, backdoors, or confiden-tial information to reduce the chance of malicious or sen-sitive data within AMIs. This was proposed in [48], butalthough the automated filtering system presented in thatpaper seems to be used already within the IBM Smart-Cloud [32], the explicit filtering rules are not available tothe public.

In contrast, Amazon currently does not provide automatedscanning of public AMIs as they are not responsible/liablefor what users do with their own data. Though Amazonquickly reacts on incidents reported to their security hotline1In principle, this is similar to mobile app stores wheredownloaded apps must be trusted as well. Recently,Google’s mobile app store withdrew 25 Android apps thatwere infected with malware [13]. As such attacks alsoharm the reputation of the mobile app store provider, someproviders already review new apps submitted to the store toensure that they perform as expected [9, 31].

and informs a↵ected customers, e.g., those running an AMIin which a backdoor was found [15].2

In this paper we show that these previously reported inci-dents are only the tip of the iceberg and many of the publiclyavailable AMIs have severe security vulnerabilities leakinghighly sensitive data.

Our Contribution and Outline.After summarizing related work in §2 and giving back-

ground information on the Amazon Web Services (AWS)in §3 we present the following contributions.Extraction of Sensitive Information from Public

AMIs (cf. §4). Through an extensive analysis we were ableto extract highly sensitive information from several publiclyavailable EC2 AMIs. To make the analysis cost and timee↵ective we developed an automated tool that uses di↵erentsearch strategies and exploits technology specific aspects ofthe Amazon cloud. The costs for running our attack wereless than $20 while the information we extracted from theAMIs would allow an attacker to cause financial damage ofseveral $10, 000 per day and could severely harm the reputa-tion of several companies that operate services in the cloud.After testing overall 1225 AMIs we got hold of the sourcecode repositories, administrator passwords and other typesof credentials of various web service providers.SSH Vulnerabilities in AMIs (cf. §5). We discovered

several vulnerabilities in AMIs that are introduced by incor-rect usage and configuration of SSH. About one third of thetested 1100 public AMIs in Europe and the US-East regioncontain an SSH backdoor, i.e., a (forgotten) public key thatallows remote login for the Publisher. We identified multi-ple instances that use the same SSH host key which allowsan external attacker to correlate these instances running thesame or a similar AMI, identify candidates for correspondingpublic AMIs, and mount several attacks, e.g., host imper-sonation.Countermeasures (cf. §6). We provide several mech-

anisms to protect against our attacks on public AMIs. Be-sides organizational measures we propose to use our tools toenhance the security of the interfaces for publishing AMIsand also extensions to the interface of the Cloud App Store.

2. RELATED WORKIn this section we briefly revisit previous work on the se-

curity challenges of publicly sharing Virtual Machine (VM)images (AMIs in our terminology) on which we build ourpractical attacks. Afterwards we review the main relatedwork on general cloud security, security aspects specific tothe Amazon cloud, and methods for searching private data.

VM Image Analysis.As summarized in §1, security and privacy risks for the

Consumer and Publisher when sharing VM images havebeen identified in [48]. Shared VM images may contain ei-ther malware that was intentionally or unintentionally in-cluded by the Publisher. To protect against these threats,the authors propose filtering of VM images by the Providerwhich has been implemented in the Mirage image manage-

2“For security reasons, we (Amazon) recommend that anyinstance based on a publicly available AMI that is dis-tributed with an included SSH public key should be con-sidered compromised and immediately terminated.” [14]

390

Consumers use published instances

Who do you trust? What are threats?

14


SSH Study [AmazonIA]

15

• Publisher left an SSH user authentication key in their AMI

• Fortunately, Amazon agreed that this is a violation

‣ Unfortunately, it was not an isolated problem

• 30% of 1100 AMIs checked contained such a key

‣ Also, pre-configured AMIs had SSH host keys

• Thus, all instances use the same host key pair

• Implications?

15


Security Configuration ‣ Zillions of security-relevant configurations for instances

• Do you have the right code and data installed?

• Are you running the expected code?

• Discretionary access control

• Firewalls

• Mandatory access control

‣ SELinux, AppArmor, TrustedBSD, Trusted Solaris, MIC

• Application policies (e.g., Database, Apache)

• Pluggable Authentication Modules (PAM)

• Application configuration files

‣ Plus new configuration tasks for the cloud - e.g., storage 16

16


Cloud Service Vulnerabilities• Vulnerabilities have been found in cloud services

‣ E.g., OpenStack identity service, web interface, and API service

• Adversaries who compromise such services may launch a variety of attacks

‣ E.g., Key Injection Attack

17

Response Modification. Compromised cloud servicescan tamper with responses returned by compute the ser-vice, causing the client to see attacker crafted responsesinstead of the legitimate compute service reponse (i.e.Console Output Attack).

In next few sections, we exemplify and analyze eachcategory.

A. Snapshot Attack

In this attack, attacker can obtain sensitive datasaved on instance’s filesystem through taking a snapshotof it.

Background. The cloud supports a quick and easy wayto create new instance image by taking a snapshot of arunning instance. The new instance image will containall files on the running instances filesystem at the timeof snapshot creation. This is useful as for backup andimage building purposes. The detailed procedure to takea snapshot is shown as follows: Client makes a requestto cloud API service to take a snapshot. Cloud APIservice invokes the snapshot_instance method ofthe compute service that hosts the instance. Computeservice will interface with local virtualization driver(e.g. Libvirt Driver) to take a snapshot of the instance.

Attack. A malicious cloud API service under controlof attacker can obtain sensitive data from a client’sinstance by directly making snapshot request to computeservice. This request is not on behalf of the client.Nevertheless, compute service will still serve it as itperforms no authentication and authorization (the cloudidentity service only authenticates and authorizes theclient’s initial request)

Launching this attack requires the attacker to invokethe The difficulty for the attacker to launch Snap-shot Attack relies on attackers ability to invoke thesnapshot_instance method of the compute ser-vice. Unfortunately, this is fairly straightforward. SinceOpenStack cloud services are interconnected throughthe message queue, any service can publish messages onthe queue to the compute service. The compute serviceneither verifies nor authorizes the message as it believesthe request is already authenticated and authorized.Therefore, attacker only needs to compromise one singlecloud service that is connected to the message queue.

There are many similar security sensitive meth-ods supported by compute service. For example, theset_admin_password method of compute serviceallows resetting the admin password of an instanceand inject_file allows injecting arbitrary files intoa running instance. These security sensitive methodsare open to attackers once they compromise any cloudservice connected to the message queue.

Attack Analysis. The root cause of the Snapshot Attackis the lack of authorization of incoming requests to

APIService

ComputeService

Database

APIService

nova keypair-add

mykey

nova boot --key-name

mykey

mykey : ssh-rsa ABC

mykey : ssh-rsa ABC

ssh-rsa ABCssh-rsa DEF

Step 1

Step 2

Fig. 3: Key Injection Attack

the compute service. The compute service serves anyrequest from any service without verifying that therequest is on behalf of the client.

B. Key Injection Attack

Even if the request is verified to be on behalf ofclients, services can still modify the request in maliciousways. An example is the key injection attack. In thisattack, attackers can gain privileged access to client’sinstance by modifying the SSH connection key injectedinto instance.

Background. When client starts up an instance, it isoften based on a clean image retrieved from the cloudimage service that contains no user-specific credentials.To enable client login, the cloud provides a mechanismto dynamically inject SSH connection key (a publickey belonging to client) into the instance. The clientcan later can use his private key to authenticate andlog in to the instance. The detailed procedure is shownin Figure 3. First, the client makes a request to cloudAPI service to create a key pair. The keypair is named“mykey” in this case. While client keeps the private key,the public key will be uploaded to cloud and saved incloud database. In the second step, client starts up anew instance with “mykey”. Cloud API services querythe database to retrieve the actual public key that mapsto “mykey” and sends it to the compute service. Thecompute service injects the public key into the instanceimage and boots up the instance. The client can thenlogin to her instance by establishing an SSH connectionusing her private key.

Attack. We now consider a malicious cloud API servicecontrolled by an attacker. A malicious API service canmodify the public key during either step 1) saving it todatabase or step 2) retrieving it from database. With amodified public key (a public key belongs to attacker)injected into client’s instance, attacker can thus log in toclients instance as privileged user. Attacker can increasethe subtlety of the attack through appending a publickey instead of modifying it. Therefore both client and

17


Insiders‣ Although the vendor may have a good reputation, not every

employee may

18

Embracing the cloud Trust me with your

code & data

Cloud Provider Client

You have to trust us as well

Cloud operators

Problem #1 Client code & data secrecy and integrity vulnerable to attack

11"

18


Insider Threats• May trust the cloud vendor company

‣ But, do you trust all its employees?

• Insiders can control platform

‣ Determine what software runs consumers’ code

• Insiders can monitor execution

‣ Log instance operation from remote

• Insiders may have physical access

‣ Can monitor hardware, access physical memory, and tamper secure co-processors

19

19


Co-Hosting Threats• An instance co-hosted on the same physical

platform could launch attacks against your instance

• Co-hosted instances share resources

‣ Computer

• CPU, Cache, Memory, Network, etc.

• Shared resources may be used as side channels to learn information about resource or impact its behavior

20

20


Resource Freeing Attacks• Setup

• Victims

‣ One or more VMs with public interface

• Beneficiary

‣ VM whose performance we want to improve (contend over target resource)

• Helper

‣ Mounts attack using public interface

21

Helper&

VM#

VM#

Vic&m#

Beneficiary#

21


Resource Freeing Attacks• Resource contention over the CPU

‣ Schedule beneficiary more frequently

• Attack: shift resource usage via public interface

‣ Helper can choose requests to send to victim

‣ Approach lower scheduling priority

• Make victim appear CPU-bound

22

RFA$intensi*es$–$*me$in$ms$per&second&

196%$slowdown$

86%$slowdown$

60%$Performance$Improvement$

22


Preventing Vulnerabilities• How would you prevent these threats?

‣ Misconfigured instances

‣ Compromised cloud services

‣ Insiders

‣ Side channels

23

23


Verifiable Computation

24

ServiceClient

24-1



• Your services are black boxes - to the cloud!

24

ServiceClient

24-2




‣ Send a program and encrypted data

24

ServiceClient

24-3





‣ Program computes over encrypted data

24

ServiceClient

24-4






‣ Scheme: KeyGen (for Program), Compute (Program), Verify

24

ServiceClient

24-5







24

ServiceDataClient

24-6







24

ServiceDataClient

24-7







24

ServiceClient

24-8







24

ServiceClient

Depends on heavy crypto - homomorphic encryption

24-9


Pinocchio [Oakland 2013]

25

25-1



• New cryptographic protocol for general-purpose public verifiable computation with support for zero-knowledge arguments

25

25-2




• Big advance: Performance

25

25-3





‣ History: PCP (2007) = 72 trillion years, GGP (2010) = 37 centuries, Pepper/Ginger (2012) = 6 oom improvement, Pinocchio = 7 oom improvement (often ~10ms for verification)

25

25-4






• Encoding in “quadratic programs”; signature depends only on security constant

25

25-5







‣ Idea behind quadratic arithmetic programs: each multiplication gate is a “small expression”. Construct polynomials that encode the equations, such that if the evaluation is correct, then D(z) / P(z). Then the protocol just checks divisibility randomly

25

25-6







‣ Idea behind quadratic arithmetic programs: each multiplication gate is a “small expression”. Construct polynomials that encode the equations, such that if the evaluation is correct, then D(z) / P(z). Then the protocol just checks divisibility randomly

• Beats local C execution (for verification)25

25-7


Excalibur• Policy-sealed data [USENIX Sec 2012b]

‣ Do not release my data to the cloud until that cloud satisfies my requirements

‣ Customer-chosen policy

• How to ensure that only nodes that satisfy customer-chosen policy get data?

‣ Attribute-based encryption

‣ Encrypt data using ABE description of load-time configuration

‣ A verifiable monitor is trusted to delegate correct credentials to nodes (using hardware-based attestations - e.g., via TPM)

26

26


Excalibur Approach

27

4/19/13 Nuno Santos

Excalibur Architecture!

13

!  Check node configurations !  Monitor attests

nodes in background

!  Scalable policy enforcement !  CP-ABE

operations at client-side lib

Monitor

Customer Policy-Sealed Data

+

seal

unseal attest & send

credential

Datacenter

From Nuno Santos’ slides

27


Runtime Monitoring• Excalibur does not address runtime issues with instance

‣ Customers may want to ensure that clients of their services only receive communications from satisfactory instances

‣ Customer may want to take remediative actions

28

28


Self-Service Clouds• Customizable cloud platform stack [CCS 2012]

29

Why do these problems arise?

Hardware

Hypervisor

Management$VM$(dom0)$

Work"VM"

Work"VM"

Work"VM"

14"

Slides courtesy of Vinod Ganapathy

29



30

Our solution

Hardware

Hypervisor

Management$VM$ Client’s$VMs$

19"

SSC: Self-service cloud computing

30



‣ UDom0 boots customer-defined Service VMs

31

An SSC platform

Hardware

SSC Hypervisor

25"

SDom0$

Work$VM$

Work$VM$UDom0$

Client’s$metaBdomain$

Service$VM$

Equipped$with$a$Trusted$Plaiorm$Module$(TPM)$chip$

31


Take Away• Cloud computing is here to stay

‣ In some form

• May be a solution or a problem or both

‣ Introduces new types of vulnerabilities into systems we ran on data centers - which had vulnerabilities to begin with

• Ultimately, have to improve service providers’ jobs

‣ Make it easy to ensure that systems perform as expected

• Two possible methods

‣ Verifiable computation and instance monitoring

32

32

Documents

Module: Cloud Computing Security