This One Weird API Request Will Save You Thousands

This One Weird API Request Will

Save You Thousands

Joshua Burgin, General Manager EC2 Spot

Zeev Stolin, DevOps, Gett

On-Demand

Pay for compute

capacity by the hour

with no long-term

commitments

For spiky workloads,

or to define needs

AWS EC2 Consumption Models

Reserved

Make a low, one-time

payment and receive

a significant discount

on the hourly charge

For committed

utilization

Spot

Bid for unused capacity,

charged at a Spot Price

which fluctuates based

on supply and demand

For time-insensitive,

transient and cost

sensitive workloads

Spare capacity at scale

AWS has more than a

million active customers

in 190 countries.

On Average, every

week, AWS customers

are using more compute

capacity on EC2 Spot

Instances than

customers in 2012 were

running across all of

EC2.

With Spot the rules are simple

Markets where the price of compute changes based on

supply and demand

You’ll never pay more than your bid. When the market exceeds your bid you get 2 minutes to

wrap up your work

$0.27 $0.29$0.50

1b 1c1a

8XL

$0.30 $0.16$0.214XL

$0.07 $0.08$0.082XL

$0.05 $0.04$0.04XL

$0.01 $0.04$0.01L

C3

$1.76

On

Demand

$0.88

$0.44

$.22

$0.11

Show me the markets!

Each instance family

Each instance size

Each Availability Zone

In every region

Is a separate Spot Market

50% Bid

75% Bid

You pay the

market

price

Bid Price Vs Market Price

25% Bid

JUST BID ON-DEMAND!

Example customer cases

Why use Spot – customer examples

“The company has saved tens of thousands of dollars. That’s

between 20 and 30 percent of our total monthly AWS bill.”

Gal Aviv Research & Development Group Manager

Why use Spot – customer examples

The raw data from the CMS experiment in the Large Hadron

Collider (LHC) is recorded every 25 nanoseconds at a rate of

approximately 1 petabyte per second.

- Spot Bid Advisor

- Spot fleet

- Spot blocks

- New! Spot console

Recent Innovations

Spot Bid Advisor

1) We make this easy using the

Spot bid advisor

2) With deliberate pool

selection and bidding, you

will keep your Spot instance

as long as you need to.

3) And with new features like

Spot fleet diversified we do

the heavy lifting for you...

Spot Bid Advisor – aws-spot-labs

Spot fleet – fly like a pro

Launch Thousands of Spot Instanceswith one RequestSpotFleet call.

Get Best PriceFind the lowest priced horsepower that works for you.

or

Get Diversified ResourcesDiversify your fleet. Grow your availability.

And

Apply Custom WeightingCreate your own capacity unit based on your application

needs

Spot fleet – continued innovation

One-Time Fleets [May 2016]

CloudWatch Metrics for Spot Fleets [Mar 2016]

Modify Your Fleet [Oct 2015]

Distribute Your Fleet Across Multiple Capacity Pools [Sep 2015]

Weighted Bidding for EC2 Spot Instances [Aug 2015]

Spot instances in the lowest priced Availability Zone [Jul 2015]

Spot fleet – super easyaws ec2 request-spot-fleet --spot-fleet-request-config file://config.json {

"IamFleetRole": "arn:aws:iam::781603563322:role/fleet-role", "TargetCapacity":

"100", "SpotPrice": "0.03", "ValidFrom": "2015-09-15T00:56:19Z", "ValidUntil":

"2016-09-14T07:00:00Z", "TerminateInstancesWithExpiration": true,

"LaunchSpecifications": [ { "ImageId": "ami-0d4cfd66", "InstanceType":

"c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-d0dc51fb" }, { "ImageId":

"ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId":

"subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large",

"WeightedCapacity": 2, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-

0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId":

"subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge",

"WeightedCapacity": 4, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-

0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId":

"subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge",

"WeightedCapacity": 16, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-

0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId":

"subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge",

"WeightedCapacity": 16, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-


"subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge",

"WeightedCapacity": 32, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-


"subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge",

"WeightedCapacity": 8, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-


"subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge",

"WeightedCapacity": 8, "SubnetId": "subnet-0b1b8052" } ] }

An easy to use interface that

lets you launch Spot instances,

fleets & Spot blocks in seconds

Helps you select and bid on the

EC2 instances that meet your

applications requirements

Simple to use dashboard lets

you modify and manage your

application’s compute capacity

Spot Console – New [June 2016]

Spot Launch Wizard – with fleet and blocks

Spot Console – Target Capacity & Automated Bidding

Spot Console – Dashboard

Using a single

additional Parameter

Run continuously

for up to 6 hours

Save up to 50% off

On-Demand pricing

Spot blocks

$1

Spot in action...

Best Practices

Example Workloads

• Hadoop

• Stateless Applications (e.g. web tiers)

EC2 Best practices

Fault tolerance

for Spot

Stateless Multi-AZ Loosely coupledInstance Flexibility

EC2 Spot – Hadoop

Core nodes

Master

Node

Master instance group

Hadoop cluster

Core instance group

HDFS HDFS

Core nodes run

TaskTracker and

Datanode (HDFS)

Process Data with

mappers and

reducers, store data

with HDFS or

DataNode

Core nodes

Master

Node

Master instance group

Hadoop cluster

Core instance group

HDFS HDFS

Can Add Core

Nodes:

More CPU

More Memory

More HDFS Space

HDFS

Task Nodes – Multiple Instance Types

Master

Node

Hadoop cluster

HDFS HDFS

YARN -

heterogeneous

instances

Can add and

remove task nodes

c3.8xl, r3.8xl, r3.4xl, etc

Spot the

opportunity

Core instance group

Multiple capacity pools

How flexible are you?

2. Across Families

3. Across Zones

1. Within family

Review and launch

{

"AllocationStrategy": "diversified",

"TargetCapacity": 1000,

"SpotPrice": "0.005",

"TerminateInstancesWithExpiration": true,

"LaunchSpecifications": [

{

"ImageId": "ami-0d4cfd66",

"InstanceType": "c3.xlarge",

"WeightedCapacity": 4,


"SubnetId": "subnet-d0dc51fb"

},

{


"InstanceType": "c3.2xlarge",

"WeightedCapacity": 8,


"SubnetId": "subnet-d0dc51fb"

},

{

Results - Hadoop

Requested 1000

vCores over 30 days

Minimum 848 vCores

Mode 1008 vCores

Average 1005 vCores

Average Price of

$0.0118 per vCore

Savings of over 81%

Capitalizing on two minute warning

When the Spot price exceeds

your bid price, the instance will

receive a two-minute warning

Check for the 2 minute spot

instance termination

notification every 5 seconds

leveraging a script invoked at

instance launch

Sample script – two minutes left!

1) Check for 2 minute warning

2) If YES, run shutdown scripts

3) OTHERWISE, do nothing

4) Then sleep for 5 seconds

#!/bin/bash

while true

do

if curl -s http://169.254.169.254/latest/meta-

data/spot/termination-time | grep -q .*T.*Z; then

/env/bin/runterminationscripts.sh;

else

# Spot instance not yet marked for termination.

sleep 5

fi

done

• No need to scale HDFS – Capacity

– Replication for durability

• Amazon S3 scales with your data– Both in IOPs and data storage

– Massively parallel

EMRFS - Amazon

S3 as HDFS Spot block for HDFS

• For core nodes if HDFS

cluster lives for less than

6 hours

Hadoop on EC2 Spot – takeaways

Your Work

Run task nodes separately with EC2 Spot fleet

Spot blocks for core/HDFS clusters that live less than 6 hours

What EC2 Spot fleet does for you

Saves you money

Heterogeneous instance management

Scale on the unit that matters to you

Accelerate results (time is money)

Web Applications with Spot

Stateless Web Application

Elastic Load

Balancing

Stateless

Web Servers

(Spot)

Stateless

Web Servers

(Spot)

Session

State Data

Spot fleet

Availability Zone A

Availability Zone B

Stateless

Web Servers

(Spot)

Stateless

Web Servers

(Spot)

Diversification with EC2 Spot fleet

Multiple EC2 Spot instances

selected

Multiple Availability Zones

selected

Pick the instances with similar

performance characteristics e.g.

c3.large, m3.large, m4.large,

r3.large, c4.large.

Multiple capacity pools

How flexible are you?

2. Across Families

3. Across Zones

1. Within family

Review and Launch{

"AllocationStrategy": "diversified",

"TargetCapacity": 50,


"LaunchSpecifications": [

{


"InstanceType": "c3.large",

"SpotPrice": "0.105"

},

{


"InstanceType": "c4.large",

"SpotPrice": "0.11"

},

{


"InstanceType": "m3.large",

"SpotPrice": "0.133"

},

{

Results - Web Application

50 instances requested, over 30 days.

- Never dropped below 45 instances

- 85% discount if you wanted 50 and could withstand dropping to 45

0

0.02

0.04

0.06

0.08

0.1

0.12

30

35

40

45

50

55

Instances Average Price Per Instance

- If you only wanted 45 the discount is still 83%

Some additional considerations

Elastic Load Balancing

Two minute warning

Since Spot fleet is configured to span across multiple Availability Zones, we highly recommend enabling cross-zone load balancing for the load balancer.

To allow in-flight requests to complete when de-registering Spot instances that are about to be terminated, connection drainingcan be enabled on the load balancer with a timeout of 90 seconds.

Elastic Load Balancing

Capitalizing on two minute warning

When the Spot price exceeds

your bid price, the instance will

receive a two-minute warning

Check for the 2 minute spot

instance termination

notification every 5 seconds

leveraging a script invoked at

instance launch

Sample script – two minutes left!

1) Check for 2 minute

warning

2) If YES, detach instance

from ELB

3) OTHERWISE, do nothing

4) Sleep for 5 seconds

$ if curl -s http://169.254.169.254/latest/meta-

data/spot/termination-time | \

grep -q .*T.*Z; then instance_id=$(curl -s

http://169.254.169.254/latest/meta-data/instance-id); \

aws elb deregister-instances-from-load-balancer \

--load-balancer-name my-load-balancer \

--instances $instance_id;

/env/bin/flushsessiontoDBonterminationscript.sh; fi

For those of you - Using Auto Scaling

Two Auto Scaling groups

• On-demand + Reserved for base use

• Add an additional Auto Scaling group with Spot

Both Auto Scaling groups behind the same Elastic Load Balancer.

Use the bid advisor to select the

right instance time for your

application.

Web Application Architecture with Spot

Elastic Load

Balancing

Stateless

Web Servers

Stateless

Web Servers

On Demand Auto

Scaling group

Session

State Data

Stateless Web

Servers (Spot)

Stateless Web

Servers (Spot)

Spot Auto

Scaling group

Availability Zone A

Availability Zone B

On-Demand

ASG

Spot ASG

Gett's AWS Spot Instances

Usage

2016

Gett is the largest and fastest

growing

on-demand mobility company in

EMEA

• 5,000+ corporate accounts

• 300% annual growth since

inception

• $500+ million in funding

• $500 million annual

revenue

• 50,000+ cabs globally

• 60 cities worldwide

• 30M+ passengers

Gett is the leader in On-

demand

mobility in Israel

• 7,500 vehicles

• National coverage

• 50,000+ rides daily

• 1.5 million users

• 1,700+ corporate

accounts

• 80% brand recognition

Why use spots?

Gett has a small Mobile App

That Generates a Lot of traffic

Why use spots?

… a traffic that requires much CPU Power and Memory to process:

In Production:

~300 EC2 Instances

For BI + Staging + Development:

~350-400 Additional EC2 Instances

In Production

We replaced ~70% of the On-Demand Instances with a Spot

Instances. We left only 3 On-Demand Instances per service and the

rest are Spots

Result:

65% Cost Saving for production

Availability is improved (due to the additional HW redundancy)

Latency is improved (due to the additional HW resources)

In Production (numbers)

~300 servers for ~30 services

3 on demand instances per service ~ 90 for high availability

200 spots of m3.large, m3.xlarge, c3.2xlarge ~ 400k $ saved annually

We replaced almost all of the On-Demand Instances with Spot

Instances.

Result:

85% Cost Saving for these environments.

Spots allows us to run as much staging environments as we need.

The cost saving - is tremendous!

In BI, Staging, and Development

In BI, Staging, and Development (numbers)

In order to use agile methodology we need a lot of staging environments

On demand : 15 environments 20 m3.large servers each. ($0.146 per Hour X

20 X 15 X 24 X 365 ~ 400k $ annually)

Spot: 15 environments 20 m3.large servers each. ($0.0211 per Hour X 20 X

15 X 24 X 365 ~ 50k $ annually)

~ 350k $ saved annually

Other Notes

Using spot-request resources for each service (for persistence).

Smart Bid price mechanism based on Spot bid advisor.

→ Bid price is based on instance type.

We use fulfillment option of spot-requests for persistency.

We use Terraform (from HashiCorp) and Green/Blue deployment.

Wrapping it all up

Using Spot, OD and RI Together

Data Science

New app development Test and Development

Internal IT

- CloudTrail events

- Dedicated Instance on Spot for HIPAA workloads

- ECS to automatically scale Spot fleet using

CloudWatch Metrics

Coming soon!

Getting started

Try the Bid Advisor Fire up the Spot Console Block some time

[email protected]

Technology

This One Weird API Request Will Save You Thousands