CloudFront BEST PRACTICES & ANTI-PATTERNS

CloudFront BEST PRACTICES & ANTI-PATTERNS

Abhishek Tiwari@abhishektiwari

http://abhishek-tiwari.com

Wednesday, 7 August 13

https://twitter.com/abhishektiwari

https://twitter.com/abhishektiwari



About me


About me

Solutions Architect


About me

Solutions Architect Early AWS adopter (2007)


About me

Solutions Architect Early AWS adopter (2007)Built Cotton On Group’s AWS Infrastructure (2012)


Agenda

CloudFront in a nutshellCloudFront Best Practices/Anti-Patterns


CloudFront (CF)


CloudFront (CF)Content Delivery Network


CloudFront (CF)Content Delivery NetworkServes Static and Dynamic Content


CloudFront (CF)Content Delivery NetworkServes Static and Dynamic ContentGlobal, Redundant, Scalable


CloudFront (CF)Content Delivery NetworkServes Static and Dynamic ContentGlobal, Redundant, ScalableLow Latency, High Bandwidth


CloudFront (CF)Content Delivery NetworkServes Static and Dynamic ContentGlobal, Redundant, ScalableLow Latency, High BandwidthCost Effective: Transparent, PAYG, Price Classes


CloudFront (CF)Content Delivery NetworkServes Static and Dynamic ContentGlobal, Redundant, ScalableLow Latency, High BandwidthCost Effective: Transparent, PAYG, Price ClassesNO Edge Side Includes (ESI)


CF Concepts


CF ConceptsDistribution



Delivery Method (Download or Streaming)




CNAME Aliases (1-10)





Origin Server/s (M:M)






Domain Name, Origin ID, Origin Protocol







Object/s (Object Path/Cache Key)








Cache Behavior/s (1-10, 1:1, Ordered)









Path Patterns, Origin ID, Viewer Protocol, Minimum TTL, Cookies/Query String Forwarding










Edge Location (42+)










Edge Location (42+)

Viewer (GET, Range GET, HEAD Requests)Wednesday, 7 August 13

CF Concepts

http://mycontent.cloudfront.net/photos/profile.pngCache Key: photos/profile.png (Same for HTTP/HTTPS)

Cache Key: object path relative to origin + forwarded query string/cookies


http://mycontent.cloudfront.net/profile.png








CF Concepts

Dynamic Content












CF Concepts

Dynamic Content

Static Content












How it works1. Client request file

2. If file in cache CF return fileNearest Edge

Location

a. If file not in cache, CF request to origin

b. Origin returns file, status 200, cached in CF

1. Client request file

2. If file in cache CF return fileNearest Edge

Location

a. If file in cache but expired, CF request to origin

b. Origin returns 304, CF keeps file in cache

Routing based on low latency/price class

Routing based on low latency/price class

Condition GET


Distribution


DistributionDownload Distribution



Viewer Protocol HTTP/HTTPS



Viewer Protocol HTTP/HTTPSOrigin: S3 or Custom



Viewer Protocol HTTP/HTTPSOrigin: S3 or CustomCookies/Query String Forwarding




Streaming Distribution




Streaming DistributionViewer Protocol: Adobe RTMP




Streaming DistributionViewer Protocol: Adobe RTMPOrigin: S3 only (No Custom)




Streaming DistributionViewer Protocol: Adobe RTMPOrigin: S3 only (No Custom)Dynamic/Adoptive Streaming (Adjust bit rate during playback)


Query String Forwarding


Query String ForwardingForward query string to origin


Query String ForwardingForward query string to originMultiple parameters, order/case sensitive


Query String ForwardingForward query string to originMultiple parameters, order/case sensitiveOrder sensitive: Two objects in CF



http://d111111abcdef8.cloudfront.net/images/image.jpg?height=a&width=b


http://d111111abcdef8.cloudfront.net/images/image.jpg?parameter1=a&parameter2=b




http://d111111abcdef8.cloudfront.net/images/image.jpg?width=b&height=a




http://d111111abcdef8.cloudfront.net/images/image.jpg?parameter2=b&parameter1=a





Case sensitive: Two objects in CF









Case sensitive: Two objects in CFhttp://d111111abcdef8.cloudfront.net/images/image.jpg?type=png






http://d111111abcdef8.cloudfront.net/images/image.jpg?parameter1=a





Case sensitive: Two objects in CFhttp://d111111abcdef8.cloudfront.net/images/image.jpg?type=png

http://d111111abcdef8.cloudfront.net/images/image.jpg?type=PNG








http://d111111abcdef8.cloudfront.net/images/image.jpg?parameter1=A

http://d111111abcdef8.cloudfront.net/images/image.jpg?parameter1=A

Cookies Forwarding


Cookies ForwardingForwarded cookies (FC): NV pairs, not attributes


Cookies ForwardingForwarded cookies (FC): NV pairs, not attributesIf choose to forward cookies to origin,



Can forward all cookies, OR



Can forward all cookies, OROnly specified cookies




Object path+FC uniquely identify in CF




Object path+FC uniquely identify in CFNV pair/Case sensitive


Origin


OriginS3 Origin: Best for Static media content (Image, Video, Audio)


OriginS3 Origin: Best for Static media content (Image, Video, Audio)Custom Origin: Best for Static application content (JS, CSS, Fonts)


OriginS3 Origin: Best for Static media content (Image, Video, Audio)Custom Origin: Best for Static application content (JS, CSS, Fonts)Custom Origin: Best for Dynamic media content (HTML, Thumbnails)


Make a note


Make a note

Eventual Consistency


Make a note

Eventual Consistency10 Origin Servers per CF Distribution


Make a note

Eventual Consistency10 Origin Servers per CF Distribution10 Cache Behavior per CF Distribution


Make a note

Eventual Consistency10 Origin Servers per CF Distribution10 Cache Behavior per CF Distribution10 CNAME per CF Distribution


Make a note

Eventual Consistency10 Origin Servers per CF Distribution10 Cache Behavior per CF Distribution10 CNAME per CF Distribution10 Cookies forwarded per Cache Behavior


Best Practices


Best PracticesVersioning


Best PracticesVersioningCompression


Best PracticesVersioningCompressionDomain Sharding


Best PracticesVersioningCompressionDomain ShardingExpiration


Best PracticesVersioningCompressionDomain ShardingExpirationBucket Organization


Best PracticesVersioningCompressionDomain ShardingExpirationBucket OrganizationLogging


Best PracticesVersioningCompressionDomain ShardingExpirationBucket OrganizationLoggingPerformance Testing


Anti-Patterns

Cache Invalidation


Cache Invalidation


Cache Invalidation

Programmatic Cache Invalidation


Cache Invalidation

Programmatic Cache InvalidationCache Invalidation is expensive


Cache Invalidation

Programmatic Cache InvalidationCache Invalidation is expensive Unmanageable if you have object dependencies


Cache Invalidation

Programmatic Cache InvalidationCache Invalidation is expensive Unmanageable if you have object dependencies3 invalidation requests at any given time


Cache Invalidation

Programmatic Cache InvalidationCache Invalidation is expensive Unmanageable if you have object dependencies3 invalidation requests at any given timeEach invalidation request can included maximum of 1000 files


Cache Invalidation


Cache Invalidation

Invalidation takes time to propagate across all edge locations (5-15 minutes)


Cache Invalidation

Invalidation takes time to propagate across all edge locations (5-15 minutes)Deal breaker: browser cache


Cache Invalidation

Invalidation takes time to propagate across all edge locations (5-15 minutes)Deal breaker: browser cacheVersioning is best way to avoid the invalidation related issues


Versioning


Versioning

CF relies on Origin Servers for versioning


Versioning

CF relies on Origin Servers for versioningS3 supports versioning


Versioning

CF relies on Origin Servers for versioningS3 supports versioningVersion enabled Custom Origin


Versioning


VersioningVersioning using query strings



File name plus query string with version /static/profile.png?versionID=123




Versioning using content based hash key




Versioning using content based hash keyFile name as unique key based on file content /static/712vds57tr18929812312enb.png





Versioning using prefix/suffix





Versioning using prefix/suffixFile name with version prefix/suffix /static/profile_123.png






File name with version prefix/suffix /static/profile_ 712vds57tr18929812312enb.png






File name with version prefix/suffix /static/profile_ 712vds57tr18929812312enb.png

You can use a mixed versioning approachWednesday, 7 August 13

Versioning using query strings



CF supports versioning using query strings forwarding



CF supports versioning using query strings forwardingVersioning enabled Origin (Custom/S3)



CF supports versioning using query strings forwardingVersioning enabled Origin (Custom/S3)For S3 Content




On object upload version enabled S3 bucket will return x-amz-version-id





profile.png





profile.png Upload to S3





profile.png Upload to S3 static/images/profile.png





profile.png Upload to S3 static/images/profile.png Return x-amz-version-id





profile.png Upload to S3 static/images/profile.png Return x-amz-version-id static/images/profile.png?versionId=644c69e148





Versionify Application content



Versionify Application contentUse Build Number or Hash of GIT/HG Head




photo.css, app.css, profile.cssphoto.js, app.js, profile.js




photo.css, app.css, profile.cssphoto.js, app.js, profile.js Minify+Combine




photo.css, app.css, profile.cssphoto.js, app.js, profile.js Minify+Combine main.css

main.js





main.js Versionify





main.js Versionify main.css?v=468df6bmain.js?v=468df6b







File name is an unique hash key generated from file content (MD5)



File name is an unique hash key generated from file content (MD5)For S3 content




Before uploading to S3 generate MD5 hash for file




Before uploading to S3 generate MD5 hash for fileUse hash as file name to upload on S3




Before uploading to S3 generate MD5 hash for fileUse hash as file name to upload on S3Avoid content duplication by comparing MD5 hashes





profile.jpg





profile.jpg MD5 Hash File Name





profile.jpg MD5 Hash File Name b723eh0f0df.png





profile.jpg MD5 Hash File Name b723eh0f0df.png Upload to S3





profile.jpg MD5 Hash File Name b723eh0f0df.png Upload to S3 static/images/b723eh0f0df.png





For application content use a pipeline



For application content use a pipelinefile names generated using MD5 hash of file content




photo.css, app.css, profile.cssphoto.js, app.js, profile.js




photo.css, app.css, profile.cssphoto.js, app.js, profile.js Minify+Combine





main.js





main.js MD5 Hash File Name





main.js MD5 Hash File Name 8972bw7dyf2h.css67bfwu9hhuw.js




Compression


CompressionCompressed content is served faster and uses less bandwidth


CompressionCompressed content is served faster and uses less bandwidthCF can serve both compressed and uncompressed version of files


CompressionCompressed content is served faster and uses less bandwidthCF can serve both compressed and uncompressed version of filesDepends on viewer or browser request



To receive compressed content browser must include Accept-Encoding: gzip in the request header




For compression CF relies on the custom origin servers




For compression CF relies on the custom origin serversS3 does not perform the compression


Compression


CompressionS3 can store both gzip and non-gzip versions of the file in the same bucket


CompressionS3 can store both gzip and non-gzip versions of the file in the same bucketWhen uploading gzip file to Amazon S3 set Content-Encoding to gzip


CompressionS3 can store both gzip and non-gzip versions of the file in the same bucketWhen uploading gzip file to Amazon S3 set Content-Encoding to gzipOn-the-fly compression by custom origin server (Nginx, Apache)


CompressionS3 can store both gzip and non-gzip versions of the file in the same bucketWhen uploading gzip file to Amazon S3 set Content-Encoding to gzipOn-the-fly compression by custom origin server (Nginx, Apache)Compress: HTML, CSS, JS etc


CompressionS3 can store both gzip and non-gzip versions of the file in the same bucketWhen uploading gzip file to Amazon S3 set Content-Encoding to gzipOn-the-fly compression by custom origin server (Nginx, Apache)Compress: HTML, CSS, JS etcCompress Image: JPEG, WebP (Q= 0-100)


Expiration


ExpirationUse one/both of these



On Origin server set Cache-Control max-age header



On Origin server set Cache-Control max-age headerIn Cache Behavior set Minimum TTL




Date header/Age header (RFC 2616)




Date header/Age header (RFC 2616)Avoid Expire header


Expiration


ExpirationBrowser cachability issues (known bug)


https://forums.aws.amazon.com/thread.jspa?messageID=312798&%23312798


ExpirationBrowser cachability issues (known bug)Set Cache-Control max-age or Minimum TTL very far future




ExpirationBrowser cachability issues (known bug)Set Cache-Control max-age or Minimum TTL very far futureDepending on request frequency, CloudFront might remove the object before its expiration




ExpirationBrowser cachability issues (known bug)Set Cache-Control max-age or Minimum TTL very far futureDepending on request frequency, CloudFront might remove the object before its expirationDefault expiration 24hrs




ExpirationBrowser cachability issues (known bug)Set Cache-Control max-age or Minimum TTL very far futureDepending on request frequency, CloudFront might remove the object before its expirationDefault expiration 24hrsObjects are cached for max {Cache-Control max-age directive, Cache Behavior Minimum TTL}




Domain Sharding


Domain ShardingTo increase client side performance


Domain ShardingTo increase client side performanceOn average 100+ resources on an ecommerce page


Domain ShardingTo increase client side performanceOn average 100+ resources on an ecommerce pageBrowsers limit the number of simultaneous connections to one server


Domain ShardingTo increase client side performanceOn average 100+ resources on an ecommerce pageBrowsers limit the number of simultaneous connections to one serverHence limit the parallel download from one server


Domain Sharding


Domain ShardingUse two or more CNAME aliases to serve the content from a CF distribution


Domain ShardingUse two or more CNAME aliases to serve the content from a CF distributionBrowser treat them as different servers


Domain ShardingUse two or more CNAME aliases to serve the content from a CF distributionBrowser treat them as different serversSharding across 2 CNAME aliases will double the parallel download (Best 2-3)


Domain ShardingUse two or more CNAME aliases to serve the content from a CF distributionBrowser treat them as different serversSharding across 2 CNAME aliases will double the parallel download (Best 2-3)More than 4 CNAME will degrade browser performance (high CPU and memory uses)


Domain Sharding


Domain Sharding

Use CF CNAME alias feature


Domain Sharding

Use CF CNAME alias featureUse Route 53 for CNAME Aliasing


Domain Sharding

Use CF CNAME alias featureUse Route 53 for CNAME AliasingCustom Wild Card SSL Certificate


Domain Sharding

Use CF CNAME alias featureUse Route 53 for CNAME AliasingCustom Wild Card SSL CertificateOn application side: enable CF sharding logic (template compilation etc)


Domain Sharding

 <a href="http://mysite.com/about"><img src="http://cf.mycloudfront.net/about.jpg" alt=""/></a>

 <a href="http://mysite.com/blogs"><img src="http://cf.mycloudfront.net/blogs.jpg" alt=""/></a>

 <a href="http://mysite.com/contact"><img src="http://cf.mycloudfront.net/cont.jpg" alt=""/></a>

 <a href="http://mysite.com/about"><img src="http://cf1.mycloudfront.net/about.jpg" alt=""/></a>

 <a href="http://mysite.com/blogs"><img src="http://cf2.mycloudfront.net/blogs.jpg" alt=""/></a>

 <a href="http://mysite.com/contact"><img src="http://cf1.mycloudfront.net/cont.jpg" alt=""/></a>

 <a href="http://mysite.com/about"><img src="{{ CDN_DOMAINS }}/about.jpg" alt=""/></a>

 <a href="http://mysite.com/blogs"><img src="{{ CDN_DOMAINS }}/blogs.jpg" alt=""/></a>

 <a href="http://mysite.com/contact"><img src="{{ CDN_DOMAINS }}/cont.jpg" alt=""/></a>

 <a href="http://mysite.com/about"><img src="http://mysite.com/about.jpg" alt=""/></a>

 <a href="http://mysite.com/blogs"><img src="http://mysite.com/blogs.jpg" alt=""/></a>

 <a href="http://mysite.com/contact"><img src="http://mysite.com/cont.jpg" alt=""/></a>

CDN with domain shardingCDN without domain sharding

HTML Template

Without CDN

Compiled ORDOM Manipulation

Compiled ORDOM Manipulation


http://mysite.com/about


http://cf.mycloudfront.net/about.jpg

http://cf.mycloudfront.net/about.jpg

http://mysite.com/blogs


http://cf.mycloudfront.net/blogs.jpg

http://cf.mycloudfront.net/blogs.jpg

http://mysite.com/contact


http://cf.mycloudfront.net/cont.jpg

http://cf.mycloudfront.net/cont.jpg



http://cf1.mycloudfront.net/about.jpg

http://cf1.mycloudfront.net/about.jpg



http://cf2.mycloudfront.net/blogs.jpg

http://cf2.mycloudfront.net/blogs.jpg



http://cf1.mycloudfront.net/cont.jpg

http://cf1.mycloudfront.net/cont.jpg









http://mysite.com/about.jpg

http://mysite.com/about.jpg



http://mysite.com/blogs.jpg

http://mysite.com/blogs.jpg



http://mysite.com/cont.jpg

http://mysite.com/cont.jpg

Domain Sharding


Domain Sharding


Domain ShardingApply domain sharding only where it’s needed most


Domain ShardingApply domain sharding only where it’s needed mostEvenly distribute the resources amongst the shards


Domain ShardingApply domain sharding only where it’s needed mostEvenly distribute the resources amongst the shardsConsider performance overhead due to the DNS lookup and connection time


Domain ShardingApply domain sharding only where it’s needed mostEvenly distribute the resources amongst the shardsConsider performance overhead due to the DNS lookup and connection timeSharding may not be good for Mobile Browsers (CPU, Battery Life)


Bucket Organisation


Bucket Organisation

Limited Cache Behavior per CF Distribution


Bucket Organisation

Limited Cache Behavior per CF DistributionPlan your bucket organisation (object keys)


Bucket Organisation

Limited Cache Behavior per CF DistributionPlan your bucket organisation (object keys)Watch out for the upload paths


Bucket Organisation


/video/uploads/, /audio/uploads/, /images/uploads/ BAD


Bucket Organisation


/video/uploads/, /audio/uploads/, /images/uploads/ BAD/uploads/video/, /uploads/audio/, /uploads/image/ GOOD


Bucket Organisation


/video/uploads/, /audio/uploads/, /images/uploads/ BAD/uploads/video/, /uploads/audio/, /uploads/image/ GOOD

Use wild card patterns (/uploads/*)


Logging


LoggingEnable logging, store access log file to S3


LoggingEnable logging, store access log file to S3One log file per distribution


LoggingEnable logging, store access log file to S3One log file per distributionCombine log files and analyze logs (EMR)


LoggingEnable logging, store access log file to S3One log file per distributionCombine log files and analyze logs (EMR)W3C extended log file format


LoggingEnable logging, store access log file to S3One log file per distributionCombine log files and analyze logs (EMR)W3C extended log file format Streaming log vs Download log


LoggingEnable logging, store access log file to S3One log file per distributionCombine log files and analyze logs (EMR)W3C extended log file format Streaming log vs Download logContent Analytics: Popularity, Error etc


Performance Testing


Performance TestingTradition Load Testing not useful



Same DNS request, same set of IPs



Same DNS request, same set of IPsBackbone Testing (Ideal Word)




CDN Server → Backbone




CDN Server → BackboneLast Mile (Real World, User Experience)





CDN Server → Backbone → Device





CDN Server → Backbone → Device3rd party service (Gomez), application monitor






Load Testing (Edge Location, Elastic IP Aware)






Load Testing (Edge Location, Elastic IP Aware)Clients request from different geolocations






Load Testing (Edge Location, Elastic IP Aware)Clients request from different geolocationsIndependent DNS request, different set of IPs






Load Testing (Edge Location, Elastic IP Aware)Clients request from different geolocationsIndependent DNS request, different set of IPsDistribute load across set of IPs


Thank YouQ & A

Technology

CloudFront BEST PRACTICES & ANTI-PATTERNS