Optimizing for Cost in the Cloud

Optimizing for Cost in the Cloud

Jinesh Varia@jinman

Technology Evangelist

Multiple dimensions of optimizations

CostPerformanceResponse timeTime to marketHigh-availabilityScalabilitySecurityManageability…….

Optimizing for Cost

When you turn off your cloud resources, you actually stop paying for them

Continuous optimization in your architecture results in recurring savings in your next month’s bill

Elasticity is one of the fundamental properties of the cloud that drives many of its

economic benefits

#1 Use only what you need (use Auto Scaling Service, modify–db)

Optimizing for Cost…

Turn off what you don’t need (automatically)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 240

2

4

6

8

10

12

14

Daily CPU Load

Hour

Lo

ad 25% Savings

Optimize by the time of day

Availability Zone #2


Auto Scaling group : App Tier

Auto Scaling group : Web Tier

Elastic Load Balancer

www.MyWebSite.com(dynamic data)

media.MyWebSite.com (static data)

Amazon Route 53(DNS)

Amazon EC2

Amazon RDS AmazonRDS

Amazon S3

AmazonCloudFront

1 5 9 13 17 21 25 29 33 37 41 45 49

Week

Web

Ser

vers

Optimize during a year

50% Savings

Auto scaling : Types of Scaling

Scaling by Schedule• Use Scheduled Actions in Auto Scaling Service

• Date• Time• Min and Max of Auto Scaling Group Size

• You can create up to 125 actions, scheduled up to 31 days into the future, for each of your auto scaling groups. This gives you the ability to scale up to four times a day for a month.

Scaling by Policy• Scaling up Policy - Double the group size • Scaling down Policy - Decrement by 1

Auto scaling Best Practices

Use Auto Scaling TagsUse Auto scaling Alarms and Email NotificationsScale up and down symmetricallyScale up quickly and scaling down slowlyAuto Scaling across Availability ZonesLeverage Suspend and Resume Processes

Scale up by 10%

if CPU utilization is greater than 60%

for 5 minutes,

Scale down by 10%

if CPU utilization is less than 30%

for 20 minutes.

Example:

Ag

g.

CP

UIn

stan

ces

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Days of the Month

RD

S D

B S

erve

rs75% Savings

Optimize during a month

End of the month processing

Expand the cluster at the end of the month• Expand/Shrink feature in Amazon Elastic MapReduce

Vertically Scale up at the end of the month• Modify-DB-Instance (in Amazon RDS) (or a New RDS DB Instance )• CloudFormation Script (in Amazon EC2)

Tip: Use “Reminder scripts”

Disassociate your unused EIPs Delete unassociated EBS volumes Delete older EBS snapshots Leverage S3 Object Expiration

Pick the Right Instance Type

Basic recommendations on Instance Type

Choose the EC2 instance type that best matches the resources required by the application• Start with memory requirements and architecture type (32bit or 64-

bit)• Then choose the closest number of virtual cores required

Scaling across AZs• Smaller sizes give more granularity for deploying to multiple AZs

AWS Support – Trusted Advisor – Your personal cloud assistant

Tip – Instance Optimizer

Instance

Amazon CloudWatchAlarm

Free MemoryFree CPUFree HDDAt 1-min intervals

Custom Metrics

PUT 2 weeks

“You could save a bunch of money by switching to a small instance, Click on CloudFormation Script to Save”


#2 Invest time in Reserved Pricing analysis (EC2, RDS)


Your Best Option: Reserved + On-Demand

Save more when you reserve

On-demandInstances

• Pay as you go

• Starts from $0.02/Hour

ReservedInstances

• One time low upfront fee + Pay as you go

• $23 for 1 year term and $0.01/Hour

1-year and 3-year terms

Heavy Utilization RI

Medium Utilization RI

Light Utilization RI

Utilization Sweet Spot Feature Savings over On-Demand

<10% On-Demand No Upfront Commitment

10% - 40% Light Utilization RI Ideal for Disaster Recovery Up to 56% (3-Year)

40% - 75% Medium Utilization RI Standard Reserved Capacity Up to 66% (3-Year)

>75% Heavy Utilization RI Lowest Total Cost Ideal for Baseline Servers

Up to 71% (3-Year)

0% 10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

$-

$2,000

$4,000

$6,000

$8,000

$10,000

$12,000

$14,000

Heavy UtilizationMedium UtilizationLight UtilizationOn-Demand

Utilization

Co

st

m2.xlarge running Linux in US-East Region over 3 Year period Break-even

point

Recommendations

Steady State Usage Pattern• For 100% utilization

• 3-Year Heavy RI (for maximum savings over on-demand)

Spiky Predictable Usage Pattern• Baseline

• 3-Year Heavy RI (for maximum savings over on-demand)• 1-Year Light RI (for lowest upfront commitment) + savings over on-demand

• Peak: On-Demand

Uncertain and unpredictable Usage Pattern• Start out small with On-Demand Instances (risk-free and commitment-free)• Switch to some combination of Reserved and On-Demand, if application is

successful • If not successful, you walk away having spent a fraction of what you would pay

to buy your own technology infrastructure

Example: Simple 3-Tier Web Application

Description Option 1 Option 2 Option 3 Option 4

2 Web servers 2 On-Demand 2 On-Demand 1 On-Demand and 1 Reserved Medium Utilization

1 On-Demand and 1 Reserved Light Utilization

2 App servers 2 On-Demand 2 On-Demand 1 On-Demand and 1 Reserved Medium Utilization

1 On-Demand and 1 Reserved Light Utilization

2 Database servers 2 On-Demand 2 Reserved Medium Utilization

2 Reserved Medium Utilization

2 Reserved Heavy Utilization

Savings Option 1 Option 2 Option 3 Option 4

Calculator Calculator Calculator Calculator

Monthly Cost $702.72 $374.78 $256.20 $238.63

One-Time Cost 1 Year Term - $1280.00 $1600.00 $1698.00

3 Year Term - $2000.00 $2500.00 $2612..60

Total Cost 1 Year Term (x12) $8432.64 $5777.36 $4674.40 $4561.56

3 Year Term (x36) $25297.92 $15492.08 $11723.20 $11203.28

Savings(Over Option 1)

1 Year Term n/a 32% 44% 45%

3 Year Term n/a 39% 54% 54%

Example: Simple 3-Tier Web Application

http://calculator.s3.amazonaws.com/calc5.html?key=calc-A358B9F9-9246-4B4B-B14D-3A873960A01D

http://calculator.s3.amazonaws.com/calc5.html?key=calc-1C64DBE0-0301-4698-AB4C-EBCEEDF53B86

http://calculator.s3.amazonaws.com/calc5.html?key=calc-B4B87AD7-48F4-4D58-B900-8CD3CF82C975

http://calculator.s3.amazonaws.com/calc5.html?key=calc-20399300-8604-4555-A7A1-84308E837EA8



#3 Architect for Spot Instances (bidding strategies)


Optimize by using Spot Instances

Heavy Utilization RI

Medium Utilization RI

Light Utilization RI

1-year and 3-year terms

On-demandInstances

• Pay as you go

• Starts from $0.02/Hour

ReservedInstances

• One time low upfront fee + Pay as you go

• $23 for 1 year term and $0.01/Hour

SpotInstances

• Requested Bid Price and Pay as you go

• $0.005/Hour as of today at 9 AM

What are Spot Instances?

Availability Zone

Region

Availability Zone

Unused

Unused

Unused

Unused

Unused

Unused

Sold at 50% Discount!






What is the tradeoff?

Availability Zone

Region

Availability Zone

Unused

Unused

Unused

Unused

Unused

Unused

Reclaimed

Reclaimed

Spot Use cases

Use Case Types of Applications

Batch Processing Generic background processing (scale out computing)

Hadoop Hadoop/MapReduce processing type jobs (e.g. Search, Big Data, etc.)

Scientific Computing Scientific trials/simulations/analysis in chemistry, physics, and biology

Video and Image Processing/Rendering

Transform videos into specific formats

Testing Provide testing of software, web sites, etc

Web/Data Crawling Analyzing data and processing itFinancial Hedgefund analytics, energy trading, etcHPC Utilize HPC servers to do embarrassingly parallel jobs

Cheap Compute Backend servers for Facebook games

Save more money by using Spot Instances

Reserved Hourly Price > Spot Price < On-Demand Price

Spot: Example Customers

63%

50%

57%

50%

50%

66%

56%

50%

Typical Spot Bidding Strategies

1. Bid near the Reserved Hourly Price

2. Bid above the Spot Price History

3. Bid near On-Demand Price

4. Bid above the On-Demand Price30

.0%

40.0

%50

.0%

60.0

%70

.0%

80.0

%90

.0%

100.

0%11

0.0%

120.

0%13

0.0%

140.

0%15

0.0%

160.

0%17

0.0%

180.

0%19

0.0%

200.

0%

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

20%

Bid Distribution (for last 3 months)

Bid Price as Percentage of the On-Demand Price

Pe

rce

nta

ge

of

the

Dis

trib

uti

on

$ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $$ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $$ $ $ $ $ $

1. Bid Near the Reserved Hourly Price

66% Savings over On-Demand

2. Bid above the Spot Price History


3. Bid near the On-Demand Price


4. Bid above the On-Demand Price


Managing Interruption

Amazon Elastic MapReduceHadoop Cluster

HDFS

TaskNode

TaskNode

Core Node

CoreNode

Input Data

OutputData

Amazon S3

Metadata

Amazon SimpleDB

BI Apps

Upload large datasets or log files directlyData

Source

Code/Scripts

Amazon S3

Service

Amazon ElasticMapReduce

HiveQLPig LatinCascading

Mapper Reducer

Runs multiple JobFlow Steps

NameNode

JDBC/ODBC

HiveQLPig Latin

Query

Amazon EMR (Hadoop): Run Task Nodes on Spot

#1: Cost without Spot4 instances *14 hrs * $0.45 = $25.20

Allocate 4 instances

Job Flow

14 HoursDuration:

Scenario #1

Add 5 Spot Instances

Duration:

Job Flow

7 Hours

Scenario #2

#2: Cost with Spot4 instances *7 hrs * $0.45 = $12.60 +5 instances * 7 hrs * $0.225 = $7.875Total = $20.475

Time Savings: 50% Cost Savings: ~19%

Amazon EMR: Reducing Cost with Spot

Use Case: Web crawling/Search using Hadoop type clusters. Use Reserved Instances for their DB workloads and Spot instances for their indexing clusters. Launch 100’s of instances.Bidding Strategy: Bid a little above the On-Demand price to prevent interruption. Interruption Strategy: Restart the cluster if interrupted

Made for each other: MapReduce + Spot


On-demand + Spot

Amazon S3

Amazon SQS

Amazon SimpleDB

Job

Metadata

Status

Raw D

ata

Amazon S3

Amazon SQS

Amazon SimpleDB

CompletedJob

LogsExceptions

Proce

ssed

Data

ReportsWebsite

Amazon CloudWatch

Amazon Elastic Compute Cloud

Amazon EC2

Amazon EC2

Amazon EC2

Input Queue

Output Queue

InputBucket

OutputBucket

Website(Job

Manager)

Intranet

Video Transcoding Application Example

Use of Amazon SQS in Spot Architectures

VisibilityTimeOut Amazon EC2 Spot Instance

Optimizing Video Transcoding Workloads

Free Offering• Optimize for reducing cost• Acceptable Delay Limits

Implementation• Set Persistent Requests• Use on-demand Instances, if

delay

Maximum Bid Price < On-demand RateGet your set reduced price for your workload

Premium Offering Optimized for Faster response times No Delays

Implementation Invest in RIs Use on-demand for Elasticity

Maximum Bid Price >= On-demand RateGet Instant Capacity for higher price

Persistent Requests

Architecting for Spot Instances : Best Practices

Manage interruption• Split up your work into small increments• Checkpointing: Save your work frequently and periodically

Test Your ApplicationTrack when Spot Instances Start and StopSpot Requests• Use Persistent Requests for continuous tasks• Choose maximum price for your requests




#4 Leverage Application Services (ELB, SNS, SQS, SWF, SES)


Optimize by converting ancillary instances into services

Monitoring: CloudWatchNotifications: SNSQueuing: SQSSendMail: SESLoad Balancing: ELBWorkflow: SWFSearch: CloudSearch

Elastic Load Balancing

Elastic Load BalancingPros

Elastic and Fault-tolerantAuto scalingMonitoring included

ConsFor Internet-facing traffic only

Software LB on EC2Pros

Application-tier load balancer

ConsSPOFElasticity has to be implemented manuallyNot as cost-effective

Web Servers

vs.

$0.08per hour

(small instance)

Availability Zone

$0.025per hour

Web Servers

Availability Zone

EC2 instance+ software LB


DNS

DNS

Application Services

SNS, SQS, SES, SWFPros

Pay as you goScalabilityAvailabilityHigh performance

Software on EC2Pros

Custom features

ConsRequires an instanceSPOFLimited to one AZDIY administration

vs.

Producer

SQS queue

Consumers

Consumers

Producer

EC2 instance+ software queue

$0.01 per

10,000 Requests ($0.000001 per Request)

$0.08per hour

(small instance)




#4 Leverage Application Services (ELB, SNS, SQS, SWF, SES)

#5 Implement Caching (ElastiCache, CloudFront)


Optimize for performance and cost by page caching and edge-caching static content

caching

When am I charged?Paris

Singapore

London

Amazon Simple Storage Service

(S3)

Edge Location

Edge Location

Edge Location

Client

Client

Client


(EC2)

When content is popular…Paris

Singapore

London

Amazon Simple Storage Service

(S3)

Edge Location

Edge Location

Edge Location

Client

Client

Client


(EC2)

Architectural Recommendations

Use Amazon S3 + CloudFront as it will reduce the cost as well as reduce latency for static data• Depends on cache-hit ratio

For Video Streaming, use CloudFront as there is no need of a separate streaming server running Adobe FMSUse managed caching service (Amazon ElastiCache)




#4 Leverage Application Services (ELB SNS, SQS, SWF, SES)

#5 Implement Caching (ElastiCache, CloudFront)

Number of ways to further save with AWS…

[email protected]: @jinman

Thank you!

http://aws.amazon.com

Steady State Usage Pattern

(Example: Corporate Website)

Web Application Usage Patterns

Spiky Predictable Usage Pattern

(Example: Marketing Promotions Website)

Uncertain unpredictable Usage Pattern

(Example: Social game or Mobile Website)



Auto Scaling group : App Tier

Auto Scaling group : Web Tier


www.MyWebSite.com(dynamic data)

media.MyWebSite.com (static data)

Amazon Route 53(DNS)

Amazon EC2

Amazon RDS AmazonRDS

Amazon S3

AmazonCloudFront

Example: TCO of a 3-tier Web Application

Documents

Optimizing for Cost in the Cloud