AWS Performance Tuning

Markus Albe @PerconaFernando Ipar @Percona

Ryan Lowe @Square

PLNY 2012

Tuesday, October 2, 12

Amazon Web Services

• Cloud Formation

• CloudFront

• CloudSearch

• CloudWatch

• DirectConnect

• DynamoDB

• ec2

• ElastiCache

• Elastic Beanstalk

• Elastic MapReduce

• Glacier

• IAM

• RDS

• Route 53

• S3

• SES

• SNS

• SQS

• Storage Gateway

• SWF

• VPC

Tuesday, October 2, 12

Amazon Web Services

• Cloud Formation

• CloudFront

• CloudSearch

• CloudWatch

• DirectConnect

• DynamoDB

• ec2

• ElastiCache

• Elastic Beanstalk

• Elastic MapReduce

• Glacier

• IAM

• RDS

• Route 53

• S3

• SES

• SNS

• SQS

• Storage Gateway

• SWF

• VPC

Tuesday, October 2, 12

Amazon Web Services

• Cloud Formation

• CloudFront

• CloudSearch

• CloudWatch

• DirectConnect

• DynamoDB

• ec2

• ElastiCache

• Elastic Beanstalk

• Elastic MapReduce

• Glacier

• IAM

• RDS

• Route 53

• S3

• SES

• SNS

• SQS

• Storage Gateway

• SWF

• VPC

Tuesday, October 2, 12

Amazon Web Services

• Cloud Formation

• CloudFront

• CloudSearch

• CloudWatch

• DirectConnect

• DynamoDB

• ec2

• ElastiCache

• Elastic Beanstalk

• Elastic MapReduce

• Glacier

• IAM

• RDS

• Route 53

• S3

• SES

• SNS

• SQS

• Storage Gateway

• SWF

• VPC

Tuesday, October 2, 12

Amazon Web Services

• Cloud Formation

• CloudFront

• CloudSearch

• CloudWatch

• DirectConnect

• DynamoDB

• ec2

• ElastiCache

• Elastic Beanstalk

• Elastic MapReduce

• Glacier

• IAM

• RDS

• Route 53

• S3

• SES

• SNS

• SQS

• Storage Gateway

• SWF

• VPC

Tuesday, October 2, 12

Amazon Web Services

• Cloud Formation

• CloudFront

• CloudSearch

• CloudWatch

• DirectConnect

• DynamoDB

• ec2

• ElastiCache

• Elastic Beanstalk

• Elastic MapReduce

• Glacier

• IAM

• RDS

• Route 53

• S3

• SES

• SNS

• SQS

• Storage Gateway

• SWF

• VPC

Tuesday, October 2, 12

Amazon Web Services

• Cloud Formation

• CloudFront

• CloudSearch

• CloudWatch

• DirectConnect

• DynamoDB

• ec2

• ElastiCache

• Elastic Beanstalk

• Elastic MapReduce

• Glacier

• IAM

• RDS

• Route 53

• S3

• SES

• SNS

• SQS

• Storage Gateway

• SWF

• VPC

Tuesday, October 2, 12

Amazon Web Services

• Cloud Formation

• CloudFront

• CloudSearch

• CloudWatch

• DirectConnect

• DynamoDB

• ec2

• ElastiCache

• Elastic Beanstalk

• Elastic MapReduce

• Glacier

• IAM

• RDS

• Route 53

• S3

• SES

• SNS

• SQS

• Storage Gateway

• SWF

• VPC

Tuesday, October 2, 12

Amazon Web Services

• Cloud Formation

• CloudFront

• CloudSearch

• CloudWatch

• DirectConnect

• DynamoDB

• ec2

• ElastiCache

• Elastic Beanstalk

• Elastic MapReduce

• Glacier

• IAM

• RDS

• Route 53

• S3

• SES

• SNS

• SQS

• Storage Gateway

• SWF

• VPC

Tuesday, October 2, 12

Amazon Web Services

• Cloud Formation

• CloudFront

• CloudSearch

• CloudWatch

• DirectConnect

• DynamoDB

• ec2

• ElastiCache

• Elastic Beanstalk

• Elastic MapReduce

• Glacier

• IAM

• RDS

• Route 53

• S3

• SES

• SNS

• SQS

• Storage Gateway

• SWF

• VPC

Tuesday, October 2, 12

Regions & Availability Zones

Tuesday, October 2, 12

Regions

• US (x3)

• EU (Ireland)

• APAC (x2)

• South America (Sao Paulo)

• GovCloud (US Gov’t & Contractors)

Tuesday, October 2, 12

AZ Differences

• Performance Characteristics

• Hardware

• Pricing

• Availability

Tuesday, October 2, 12

Tuesday, October 2, 12

AZ Strategy

• Multi-AZ Deployment

• Assume Entire AZs fail

• Smart Software

• Cost Optimization

Tuesday, October 2, 12

ec2

Tuesday, October 2, 12

ec2(Elastic Compute Cloud)

• The Cloud

• On-Demand computers with minimal administration

• Best for organizations with light uptime requirements, highly variable workloads, horizontally distributable data and/or little time/skill to manage hosts

Tuesday, October 2, 12

ec2 Instance Types

• Micro

• Small

• Medium

• Large

• X-XXXXL

• High-Memory

• Provisioned IOPS

• EBS

• SSD

Tuesday, October 2, 12

ec2 Instance Pricing

• Micro $0.025

• Small $0.105

• Large $0.415

• XL $0.830

• XXL $1.170

• XXXXL $2.340 ($1600/month)

Tuesday, October 2, 12

EBS(Elastic Block Store)

• Persistent Data Storage

• Up to 1TB*

• Limited to single availability zone

• Automatic “Replication”

• Multiple EBS per ec2 Instance

Tuesday, October 2, 12

EBS Pricing

• Standard EBS

• $0.10 per GB provisioned

• $0.10 per million IO

• Provisioned IOPs

• $0.125 per GB provisioned

• $0.10 per provisioned IOPS-month

• EBS Snapshots to S3

• $0.125 per GB-month of data stored

Tuesday, October 2, 12

ec2 First Write Penalty

Tuesday, October 2, 12

ec2 Hardware

• Measured in Cores with Compute Units

• “One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor”

• Performance Highly Variable. Empirical evidence suggests > 1% unusable!

Tuesday, October 2, 12

ec2 Gotchas

• Many organizations don’t know what their monthly bill will be until it’s too late.

• Overprovisioning

• Underprovisioning

• Forgetting mandatory maintenance

• Never testing restores

Tuesday, October 2, 12

16G 16T 8F [rw]/sec(Large w/RAID10)

0

37500

75000

112500

150000

Random Reads Random R/W Random Writes Seq RSeq W

Tuesday, October 2, 12

16G 16T 8F [rw]/sec(Large w/EBS)

0

75000

150000

225000

300000

Random Reads Random R/W Random Writes Seq RSeq W

Tuesday, October 2, 12

EBS vs. Hardware

IOPS

0 375000 750000 1125000 1500000

EBS RAID10 SSD SAN

Tuesday, October 2, 12

Benchmark Scripts

• https://launchpad.net/~vadim-tk

Tuesday, October 2, 12

RDS

Tuesday, October 2, 12

RDS (Relational Database Service)

• SQL Server, MySQL, Oracle

• If you need “A relational database service with minimal administration”

• Best for organizations with light uptime requirements & little time/skill to manage databases

Tuesday, October 2, 12

RDS Caveats

• Mandatory 30-minute maintenance window PER WEEK

• Magic Backups

• Auto-Upgrade Minor Versions!

Tuesday, October 2, 12

RDS Performance

• Relatively Sane Defaults

• Somewhat limited access to MySQL configuration variables

• Very close to the corresponding ec2 size

Tuesday, October 2, 12

Monitoring

Tuesday, October 2, 12

Monitoring ec2

• http://status.aws.amazon.com

Tuesday, October 2, 12

Monitoring Instances

• Basic Monitoring: 5-minute granularity

• Detailed Monitoring: 1-minute granularity

• Custom Metrics & Alarms

• AutoScaling based on Metrics!!

Tuesday, October 2, 12

AutoScaling

• To autoscale or not to autoscale?

• Monitoring/Alerting on cost

• Detailed monitoring on some %

• Regular Reviews

Tuesday, October 2, 12

Architecture

Tuesday, October 2, 12

Partition by Function

• ElastiCache for sessions

• CloudSearch for ... wait for it ... Search

• Per-Service data stores

Tuesday, October 2, 12

Shard

• Active-Active AZs

• Shard by GeoIP?

Tuesday, October 2, 12

Chaos Monkey

• https://github.com/Netflix/SimianArmy

Tuesday, October 2, 12

Conclusions

• No magic scaling bullet

• Common sense (and $) will get you far

• Understand all AWS offerings

• Software Architecture & Design FTW

Tuesday, October 2, 12

Questions?

Tuesday, October 2, 12