Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) | AWS re:Invent 2013

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

Dynamic Content Acceleration: Lightning-Fast Web Apps with Amazon CloudFront and Amazon Route 53 Kalyanaraman Prasad Parviz Deyhim November 13, 2013

Fundamental Facts

Any web application must have… • Tight Security • High Availability • High Performance

Why Does Availability Matter? • If your application is not available, your revenue loss is

100%.

• Impact to customer loyalty and your brand image.

How AWS Helps? Use Amazon Route 53 to health-check your origin

webservers, with automatic failover.

Use Amazon CloudFront to front your origins to reduce load on your origins.

Use Amazon CloudFront to customize your error pages.

Amazon CloudFront will automatically serve stale content when origin is unavailable.

Why Performance Matters? • Performance translates to …

• Higher Page Views • Better Customer Experience • Higher Conversion Rates

Why Performance Matters?


Great amount of time and money spent improving back-end infrastructure performance


80% of user’s perceived latency comes from front-end

How do we Improve Performance ? A Typical Web Application Has … • Static or Re-Usable Content

• High TTLs • Low TTLs (Customized Content)

• Dynamic or Unique Content • Zero TTL

Static or Re-Usable Content

Can be cached (High TTLs or Low TTLs)

Typical Architecture

Dynamic or Unique Content

Cannot be cached - BUT affects 100% of your viewers!

Why Not…?

Why Don’t Customers Use a CDN for Dynamic Content?

I don’t see the value - each request is unique and must go back to the origin web server.

AND/OR

I see the value, but my current CDN charges premium rates for dynamic content acceleration, with many additional fees.

AND/OR

Configuring a CDN for dynamic content acceleration requires expensive professional services and is not self-service.

How Can Amazon CloudFront Help? TCP/IP optimizations for the network path

Keep-Alive Connections to reduce RTT

SSL Termination close to viewers

POST/PUT upload optimizations

Latency Based Routing

Low prices, same as static content delivery!

Unique or Reusable Contents?

STATIC or REUSABLE A given content where the state of the content

does NOT change for a given period of time

t0 t1

DYNAMIC OR UNIQUE A given content where the state of the content

changes as soon as it gets created

t0 t1

Example

Example Index.jsp (dynamic)

Images (static)

. sec

Page Load Time?

. Sec

Improving Web Application Performance

Accelerating static contents Accelerating dynamic contents

Brief Introduction to Waterfall Graphs

Waterfall Graphs

• Most important tool for web-performance measurement • Most browsers provide waterfall graph plug-ins

What Happens?

Typing the Address Browser renders

Understanding Waterfall Graphs

DNS Lookup

TCP Connection

Time to First Byte

Content Download

Understanding Waterfall Graphs

Index.jsp

Optimizing Static Content


Index.jsp

Images, JS and CSS


Index.jsp

Optimize By Caching With Amazon CloudFront

Caching

Origin

Edge Location

User Request A

Caching

Origin

Edge Location

Get Image

User Request A

Caching

Origin

Edge Location

Get Image

Get Image

User Request A

Caching

Origin

Edge Location

Get Image

Get Image

Image

User Request A

Caching

Origin

Edge Location

Get Image

Get Image

Image

Image

User Request A

Caching

Origin

Edge Location

User Request B

Get Image

Caching

Origin

Edge Location

Get Image Image

User Request B

Optimizing Static Content with Caching

Brings content closer to the users


Improves the experience and performance


Offloads your infrastructure

Before Caching = 1.46sec

After Caching = 770ms

Are We Done?

NOT SO FAST . sec GOAL:

Cache As Much As You Can

How? I’m Caching All My Images, CSS

and JS

Steps to Find Cachable Content

1. Collect web (W3C) logs from your web tier

2. Run a report on your logs (Amazon EMR, Amazon RDS, or Amazon Redshift)

3. Identify top N URLs

Steps to Find Cachable Content

Example of query:

Select count(url) count, url from logs_table Group by url sort by count;

Site Content

STATIC or REUSABLE A given content where the state of the content

does NOT change for a given period of time

t0 t1

Caching for Smaller Time Units • Goal: Find contents that can be cached for any

given period of time

• Hours • Minutes • Seconds

• CloudFront can cache content for any period of

time

Content with Query Strings

Reusable?

110 /factor/create_image?name=book1&size=10x10

Content with Query Strings

• CloudFront can cache content with query strings • Every unique query string combination is a new

object in CloudFront’s cache

API Calls

Reusable?

100 /api/GetBooks?category=math

API Calls

Reusable?

80 /api/GetBooks?top=10

Caching for Smaller Time Units

• Imagine your have a read heavy API GETS Hit 100 or 1000 RPS

• Offload your web-tier from handling 1000 RPS

• Offload your load balancer: Elastic Load Balancing or any

other LB

• Provision less capacity and reduce cost

1000 /api/GetBooks?top=10

Base Page (First HTML page)

Reusable?

220 /index.jsp

Optimizing Dynamic Content

Dynamic Content

Index.jsp

Optimizing Dynamic Content

Index.jsp

Can Dynamic Content Be Optimized?

Dynamic content is not cachable Content proxied by CDN to the origin and back


User Request

Origin

Edge Location

Poke

Poke

Ok

Ok

Poke Poke

User Request


That adds latency? How to optimize dynamic content?

How to Optimize Dynamic Content?

DNS Lookup

TCP Connection

Time to First Byte

Content Download

How to Optimize Dynamic Delivery?

Route 53

Keep-Alive Connections & SSL Termination

Faster Response Time = Reduced DNS Time + Reduced Connection Time + Reduced First Byte Time + Reduced Content Download Time

Keep-Alive Connections TCP/IP Optimization

Rute53

Route 53

After Caching/Before CloudFront Dynamic Acceleration = 770ms

How to Improve DNS Time?

DNS Lookup

Index.jsp

With Amazon Route 53

Route 53

Optimizing DNS Response Time • Amazon Route 53 managed DNS offering

• Designed to be fast

• Low latency DNS resolution

• Global network of DNS servers

• Queries routed to the nearest DNS server

Without Amazon Route 53

With Amazon Route 53

How to Improve TCP Connection and First Byte Time?

TCP Connection

Index.jsp

With CloudFront’s Keep-Alive Connections

TCP/IP Hand Shake

• HTTP Runs on TCP/IP

• TCP has the concept of TCP handshake

• Every HTTP Connection has to complete TCP Handshake

• TCP/IP Hand Shake Penalizes Dynamic Contents

Two Users without CloudFront SYN

SYN-ACK

ACK

GET /index.jsp

ACK

SYN-ACK

GET /index.jsp

2nd User SYN

90ms

360ms

360ms

Without CloudFront

• Every user is new connection

• More users = more TCP connections

Without Keep-Alive Connections

• Puts load on your web servers: Memory/CPU

Without Keep-Alive Connections

• Load on your web server increases the time to first byte

Time to First Byte

SYN

SYN-ACK

ACK

GET /index.jsp GET /index.jsp

Keep Alive Connections

2nd Request

CloudFront Keep Alive SYN

SYN-ACK

ACK

GET /index.jsp

ACK

SYN-ACK

GET /index.jsp

SYN

30ms

SYN

SYN-ACK

ACK

GET /index.jsp

GET /index.jsp

60ms

2nd User

360ms

180ms

• More users More TCP connections

• Offloads your web tier’s CPU/memory • Improves response time

2 connections Without CloudFront 1 connection With CloudFront

720 ms Without CloudFront 540 ms With CloudFront

CloudFront Keep Alive

Test CPU Util. % Without CloudFront 20% With CloudFront 6%

How to Optimize My SSL Connections?

TCP Connection

Index.jsp

With CloudFront’s SSL Termination

SSL Optimization with CloudFront

• CloudFront has the ability to support SSL traffic

• Use CloudFront cert or bring your own • SSL traffic gets terminated at the closest

CloudFront location

CloudFront SSL Optimization Benefits

• Taking advantage of keep-alive connections

• SSL introduces additional TCP handshake packets

• Keep alive eliminates additional SSL TCP handshake packets

• Offloading your infrastructure from terminating 1000s of end-users SSL connections

SSL Optimization Patterns with CloudFront

Two optimization patterns:

1. Half bridge SSL termination

2. Full bridge SSL termination

Half Bridge SSL Termination

HTTP CloudFront

Half Bridge SSL Termination Benefits

• Better performance by leveraging HTTP connections to origin

HTTP CloudFront

Full Bridge SSL Termination

HTTPs

How to Improve Content Download Time?

Content Download

Index.jsp

With Amazon CloudFront Slow-Start Optimization

TCP Slow Start Packet1

Packet 1 ACK

Packet 2

Packet 3 ACK

Packet 3

Packet 4

Packet 5

Packet 6

Packet 7

• CloudFront can optimize slow start

• Slow start impacts new connections not the existing ones

• CloudFront uses existing connections so users can skip slow start

Slow-Start Optimization with CloudFront

Packet1

Packet 1 ACK

Packet 2

Packet 3 ACK

Packet 3

Packet 4

Packet 5

Packet 6

Packet 7

Packet1

Packet 2

Packet 4 ACK

Packet 3

Packet 4

Packet 5

Packet 6

Packet 7

Packet 8

Packet 9

Performance Test 1: Without CloudFront

Performance Test 2: With CloudFront

Performance Results

Test # Of Packets Response Time Per Request Response Time For 200 Requests

Without CloudFront 2605 170 ms 33.876 ms

With CloudFront 896 96 ms 19.24 ms

How to Optimize PUT/POST? TCP Connection

Index.jsp

With Amazon CloudFront PUT/POST Verb Optimization

PUT/POST Optimization with CloudFront

• CloudFront supports verbs: PUT, POST, DELETE, OPTIONS, and PATCH

• Data won’t get cached

• CloudFront proxies data to origin


• Dynamic content optimizations apply

– Keep-alive connections

– TCP slow-start optimization

– Close proximity connection termination


• Optimizing form POSTs

• Optimizing AJAX POST requests

• Optimizing content upload – Uploading to Amazon S3

PUT/POST Optimization Test

CloudFront


Uploading 10 MB data from an instance in US East region to US West region

Avg. result: 5sec


Uploading 10MB data from an instance in US East region to the closest CloudFront location

Avg. result: 3.5sec

CloudFront

How to Improve Content Download Time Even More?

Content Download

Index.jsp

With Amazon Route 53 Latency-based Routing

Route 53

Latency-based Routing (LBR) • Run multiple stacks of your application in different Amazon

EC2 regions around the world

• Create LBR records for each location and tag the location with geo information

• Route 53 will route end users to the endpoint that provides the lowest latency

112

LBR Benefits • Better performance than running in a single region

• Improved reliability relative to running in a single region

• Easier implementation than traditional DNS solutions

• Much lower prices than traditional DNS solutions

113

LBR For End-Users

114

LBR For End-Users

115

CloudFront and Route 53 • Use CloudFront for dynamic content optimization

• Host your origin at multiple AWS locations (or data

centers)

– US – Europe

CloudFront and Route 53 • Create Origin DNS records in Route 53 for each

location

• Route 53 measures the latency between CloudFront and all configured origins

• Route 53 resolves origin’s hostname to the closest location

• Reduce content download time

LBR For End-Users

118

LBR For End-Users

119

Lower Latency with CloudFront and Route 53







After CloudFront Dynamic Content Optimization = 555 ms

Example: somecompany.com

. sec

Design for Failure with CloudFront and Route 53

Route 53

Design for Failure Normal interaction: 1. Users connect to

CloudFront 1. CloudFront

connects to Origin

CloudFront

Design for Failure • What happens if the origin fails to respond back

to CloudFront?

CloudFront

Design for Failure • With Amazon Route 53, you can health-check

your origin

Design for Failure • Failures can be detected by Route 53 health

checks

CloudFront

Design for Failure • The traffic shifts to the healthy instances or load

balancers instead

CloudFront

Can mix health check and latency-based routing

Can apply the same logic to multiregion deployments

Users get connected to the closest region if both regions are healthy

Route 53 detects failures via health checks

Users get connected to the

healthy regions if the

closest region is not

passing the health check

Design for Failure: Caching • Caching improves performance

• Can also improve availability

• If your infrastructure is experiencing failure, CloudFront can serve cached content instead of 5xx,4xx and etc

148

Design for Failure: Caching • Going back to “cache as much as you can”

• More caching = better availability

149

Design for Failure: Serve Cached Content

Origin

Edge Location

User Request A


Origin

Edge Location

Get Image

User Request A


Origin

Edge Location

Get Image

Get Image

User Request A


Origin

Edge Location

Get Image

Get Image

Image

User Request A


Origin

Edge Location

Get Image

Get Image

Image

Image

User Request A


Origin

Edge Location

Image

User Request B


Origin

Edge Location

Get Image

Get Image

User Request B


Origin

Edge Location

Get Image

Get Image

User Request B


Origin

Edge Location

Get Image

Get Image Image

User Request B

Summary

• Accelerate all your content with CloudFront

• Use CloudFront with Route 53 latency-based routing to improve your performance

• Design for failure with CloudFront and Amazon Route 53

Customer Stories

Customer Story: Low TTLs

Customer Story: Query Strings

Customer Story: Cookies

Customer Story: POST/PUT “We are excited to use CloudFront's new POST, PUT, PATCH, and DELETE capabilities to accelerate our RESTful APIs on Amazon EC2. With these new HTTP methods we can now take advantage of CloudFront’s global footprint and optimized connections back to our origin servers in AWS. Routing our customers’ API requests via a CloudFront edge location near them will help improve their experience by minimizing packet loss and upload latency. This will help provide

Customer Story: Custom SSL

Customer Story: Health Checks & Failover

“Amazon Route 53’s DNS Failover feature provides high availability across our multiple AWS regions and gives us the ability to offload our origins.”

Please give us your feedback on this presentation

As a thank you, we will select prize winners daily for completed surveys!

ARC309

Technology

Dynamic Content Acceleration: Amazon CloudFront and Amazon Route 53 (ARC309) | AWS re:Invent 2013