Team 6: (DDoS) The Amazon Cloud Attack

Kevin Coleman, Jeffrey Starker, Karthik Rangarajan, Paul Beresuita, Arunabh Verma

and Amay Singhal

What Happened?

Bitbucket was down for over 19 hours

DDoS took down the connection between Bitbucket and The Amazon Elastic Computing Cloud (EC2)

UDP packets and TCP SYN connection packets

The Attack

What was the impact?

Because of this attack, Bitbucket received over 19 hours of downtime

Their customers could not access any of their source code hosted by Bitbucket

This attack showed that cloud computing is not as safe as most people think. Although, this is one of the first times the attack has happened and it only affected Bitbucket.

Why did the attack succeed?

Initial complaint from Bitbucket dismissed as temporary

Tech support at Amazon denied anything was wrong with their system, asking Bitbucket to look at their own

8 hours after the problem was reported, Amazon accepted that the problem was on their system

Because of this initial dismissal, it took Amazon some time to figure out the attack pattern

There are now confirmed reports that the EC2 service was exposed to external Internet traffic

Why did the attack succeed?

Jesper Nøhr, owner of Bitbucket, says Amazon’s OoS system failed when the cloud came under attack

Amazon also did not have measures to detect a large number of UDP packets targeted to the same IP address

Having this measure could have easily prevented this attack from happening

While it is largely clear how the attack succeeded, it is still not clear how the internal EC2 and EBS were exposed to external internet traffic

EC2 and EBS were considered secure from such attacks since they are on the internal network between Amazon and its customers

Faint rumors still do rounds that it might have been one of Amazon’s customers that launched this attack, but this possibility is unlikely

What happened in the aftermath?

Bitbucket, at one point, was considering switching service and received offers from various providers

Nohr (creator Bitbucket) speculates the fact that their storage share common network interface with the one that connects the site with the outside world

Amazon issued a statement for the incidence

Discussions followed which raised some concerns about the service

Amazon’s statement

Amazon issued the following statement:

• " .....one of our customers reported a problem with their Amazon Elastic Block Store (EBS). This issue was limited to this customer's single Amazon EBS volume ....….  While the customer perceived this issue to be slowness of their EBS volume………. but rather that the customer's Amazon EC2 instance was receiving a very large amount of network traffic…….... we worked with the customer ….. to help mitigate the unwanted traffic they were receiving…. apply network filtering techniques which have kept their site functioning properly….…. continue to improve the speed with which we diagnose issues like this… use features like Elastic Load Balancing and Auto-Scaling to architect their services to better handle this sort of issue…."

What was done to make system less

vulnerable?

Transparency - Network Traffic information

Improved Customer Support

Better data filters and detection systems

Threat Prevention: Transparency

Providing to the customer:

Network traffic information

Technical support for attack detection

Elastic Load Balance

Auto-Scaling

Distribute instances in multiple availability zones and regions.

Threat Prevention: Improved Customer

Support

Amazon’s technical support failed to properly diagnose the issue quickly

Amazon didn’t trust Bitbucket’s information, which in-correct time wasting diagnoses

11 hours were lost due to poor diagnoses

Threat Prevention: Diversify Server Farms

Relying on specific cloud provider is dangerous

Having a second provider accelerates website recovery time after a DDoS attack

Spreading resources between providers prevents a complete system failure.

Threat Prevention: Improve Data Filters

Detecting harmful packets must be improved

Stopping harmful packets from reaching sensitive equipment reduces system vulnerability

What chapter in the book will be helpful?

Chapter 7, specially 7.2

Sources

http://www.theregister.co.uk/2009/10/05/amazon_bitbucket_outage/

http://www.thewhir.com/web-hosting-news/100609_Outage_Hits_Amazon_Cloud_Customer_Hard

http://www.theregister.co.uk/2009/10/09/amazon_cloud_bitbucket_ddos_aftermath/

http://www.networkworld.com/community/node/45891

http://blog.bitbucket.org/2009/10/04/on-our-extended-downtime-amazon-and-whats-coming/