AWS re:Invent 2013 Recap

A harrowing tale of heroic geeks embarking on a never ending quest for knowledge and a steak dinner

Putting things in perspective

• Basics of application constraints• Address them with software architecture• THEN re:Invent– how these tools solve those problems

Server Constraints

• Performance– Processor (super fast)– RAM (super fast)– Disk I/O• Standard Hard Disk (super slow)• SSD (moderately slow, but no seek time)

– Bandwidth / Network (fast)• Disk Space • Disk Reliability

Better Living through Architecture• Efficient Code and Queries

– Lower Processor Usage– Reduce disk I/O– Minimize RAM footprint

• Caching– Reduce disk I/O– Avoid reprocessing the same code/query– Avoid calling and waiting for responses from the same external services– Increase RAM usage

• Background Processes and Queues– Reduce disk I/O– This is a low priority and can wait until other resources aren’t occupied– Predictable processing (no such thing as an overloaded queue)

• Throughput Optimization– Reduce disk I/O– Reduce bandwidth usage– Optimized images: use less disk space, bandwidth– Query only what you need: minimize bandwidth between database and application server

GOAL: Minimize Disk I/O

Response Time Limits• 0.1 second

– Limit for having the user feel that the system is reacting instantaneously– No special feedback is necessary except to display the result

• 1.0 second– About the limit for the user's flow of thought to stay uninterrupted– User will notice the delay– No special feedback is necessary during delays > 0.1s but < 1.0s– User does lose the feeling of operating directly on the data.

• 10 seconds– About the limit for keeping the user's attention focused on the dialogue– Users will want to perform other tasks while waiting– Should be given feedback indicating expected completion– Feedback during the delay is more important for variable response times

Anatomy of a Web Request

Better Living through “Cloud”

• Content Delivery Networks (CDN)• Geographic DNS / Routing• Dynamic Server Capacity• Dynamic Load Balancing• Unlimited Storage• Redundant Storage• Scalable Bandwidth Limitations• Scalable Disk I/O

Reduce Lookup TimeGeographic DNS

Remove Unnecessary RequestsCDN

• Combine & Minify JS and CSS files to limit requests• Use public CDNs for common libraries to leverage browser cache (ex - jQuery via Google)

• Removes library from your rolled JS which shrinks the download on redeploy• Use Image Sprites to minimize requests for multiple images

Remove Server StressDynamic Server CapacityDynamic Load BalancingUnlimited StorageRedundant StorageScalable Disk I/ODistributed Caching

The server request is the bottleneck preventing all other page loads from triggering

AWS Overview

What is AWS?with Mark B.

AWS Overview

AWS Overview

AWS Overview

AWS Overview

AWS Infrastructure

• S3/Glacier – Cloud File Storage• Cloudfront – CDN• Heroku – PaaS Built on top of AWS• RDS – Multi-AZ MySQL (plus read replica)• EC2 – Virtual Compute Resources• Route 53 / ELB – DNS and Load Balancing• Elasticache – Memcached – DB Caching• Cloud Search – People/Group Search• Cloud Watch – Monitoring/Metrics

NOW…TO AWS RE:INVENT 2013!Viva Las Vegas

So what exactly is re:Invent?

Major News from Amazon

Tons of Sessions

Hands on Labs

• Chances to experiment• Variable complexity• Lots to choose from

Vendors…so many vendors

• Demoing and answering questions about their products

• AWS Marketplace– License software on your

own instances

• Hosted services using AWS cloud– Use their APIs

• Services for managing AWS via AWS APIs

And SWAG…lots of

We get T-shirts They get your e-mail address

YES…YES WE DIDSo did we learn anything?

Quick Hits

• New at Amazon– RDS for PostgreSQL

*huge applause*

– DynamoDB• NoSQL as a service• Across availability zones

– Virtual Desktops– AppStream

• Stream graphic intensive apps to mobile devices

– Kinesis• Drink from the real time

data firehose

• Notables already there– Redshift

• Data warehouse• Interface mirrors Postgres

– Glacier• Dirt cheap long term

storage• In house appliance to

mimic/replace tape• Auto-backup from S3

– MySQL RDS adds streaming read replicas across availability zones

Scaling for the 1st 10 MillionNo Users – start small• What does your app do?• Do everything on one server

> 100 Users• Start infrastructure separation• Web and Database

> 1K Users• Start setting up redundancy• Multiple webs in different zones• Elastic load balancing across webs• Multi AZ database

> 10K Users• Use more redundancy• Lots of caching

– DB and session caching (Memcached/Elasticache)– Static assets to CDN– Use online storage for most things (S3)

> 500K Users• Service Oriented Architecture

– “Move services into their own tiers of modules. Treat each of these as 100% separate piece of infrastructure and scale independently.”

– Loose coupling, using messaging as a buffer (SQS)

> 5M Users• Typical to run into issues on DB write master• Consider sharding and/or other DB technologies

– PostgreSQL XC– MongoDB– DynamoDB

Musts at ALL levels• Good monitoring, metrics and logging tools

Things that are helpful on most levels• Auto scaling – increase/decrease your resources as

load requires

Dynamic CDN?

• Cloudfront CDN speeds up dynamic content– Geographic SSL endpoint– Use a single keep-alive– Hold fewer connections due to consistent user

bandwidth• Connections to your server are always Tier 1• Cloudfront handles the longer download times

SOA w/ SQS + SNS

• SOA = Service Oriented Architecture• SNS = Simple Notification Service• SQS = Simple Queue Service• Each service has a queue• Each service has a notification service• The queues subscribe to whatever notifications

that particular service cares about• A job processes the messages in the queue

Automating Media FlowsIngest – Fast file transfers• Companies:

• Aspera (used by Netflix)• Attunity Cloudbeam• Signiant

• Open Source: Tsunami UDP

Process• Elastic Transcoder• Reduced Redundancy Storage (S3)• Backup with Glacier• Select a Thumbnail

• Extract several samples• Ask Mechanical Turk to choose

• Turks are people too

Deliver• Save in Database• Add to CloudSearch index• Serve with Application• Streamed via Cloudfront CDN

Amazon Services used…• S3• Glacier• RDS/DynamoDB• Elastic Transcoder• Simple Workflow (SWF)• Mechanical Turk• EC2• Cloudfront• CloudSearch

Big Data Analytics• Amazon Redshift

– Analytic, Indexless Database– Huge Queries, Fast– Load data, process data, kill instance– Query interface mirrors PostgreSQL

• Amazon Elastic Map Reduce (EMR) / Hadoop– Mortar Framework, pig, and lipstick (for pig)

• Track timing on each piece of each job• Visually breakdown how the job is working• Identify time constraints / bottlenecks• Schedule cluster lifetime• Keeps historical operations data after cluster

destroyed• Store results in Dynamo/Mongo instead of S3

• JasperSoft– Open source Business Intelligence (BI)– Available in AWS Marketplace– Works with almost any backend

• Hadoop, EMR, Redshift, PostgreSQL

Amazon Kinesis

Overview• Streaming map/reduce• Routes incoming data by type to

ensure appropriate processing• Auto-provision instances to

handle streaming load• Allows failover for several

seconds for streaming data• Integrates with DynamoDB to

store incoming results• Uses Java to implement the

processing logic

Use Case: Twitter Firehose

…from the movie UHF…funny movie

And yes that guy is Kramer from Seinfeld

MONyog

• Monitors MySQL • Makes recommendations based on– Current configuration– Best practices

• Sort of like an auto-tune for MySQL

Hybrid Cloud

Eucalyptus• Open source Amazon

infrastructure• Develop and deploy against

Amazon APIs in your own datacenter

• Portable automation code

VPN• Setup Virtual Private Cloud

(VPC) within Amazon network

• Setup dedicated VPN connection between our datacenter and Amazon datacenters

Controlling the Flood

• DynamoDB– Fast write NoSQL database– Provisioned by preselecting throughput level

• Reads/writes per second

– When you go over…tough

• Simple Queue Service– Auto-scaling queuing system– Handles high load fast-writes– When writes fail due to throughput threshold, stick them in the

queue– Have a background worker keep trying to write to DynamoDB until

throughput is available

Key Lesson: Automate Everything

• 1 server…configure manually• 2 servers…configure each one…maybe• > 2 servers…automate or pray– Puppet: Cross platform provisioning automation• Yes, even Windows

– Vagrant: Excellent tool for using production virtual machines in development environments• Also great for experimenting with Puppet

Key Lesson: AWS does not mean best

• Amazon provides a lot of tools…not all of them are perfect• Amazon tools are usually better integrated• Elastic Beanstalk was heavily made fun of by PRESENTERS for its

weaknesses• Systems that have standard APIs are more portable…for example

– Elasticache is Memcached and Redis– Elastic Map Reduce is Hadoop– EC2 instances are just virtual servers running an OS

• Many infrastructure vendors within AWS datacenters• A lot of services are extremely easy to configure compared to

paying somebody else a markup to do it for you– Offload the complexity

Key Lesson: Load Test & Monitor

• You cannot know your bottlenecks until your application is under stress

• You cannot know your bottlenecks unless you are monitoring your infrastructure

Research it Yourself

Find MOST presentations

http://reinvent.awsevents.com/recap2013.html

Some are missing. There was an amazing session from Loggly that is not present.Update: Here it is… http://slidesha.re/1cTl8v7

Recommended Presentations• Coding Tips you should know before distributing your HTML5 Web App on Mobile

– http://bit.ly/1amFtID• Automated Media Workflows in the Cloud

– http://bit.ly/1iWQglf• Dynamic Content Acceleration using Cloudfront and Route53

– http://bit.ly/IEklGU• Controlling the Flood: Massive Message Processing with SQS and DynamoDB

– http://bit.ly/19mzn0o• Building Scalable Windows and .NET Apps

– http://bit.ly/1eitluH• 7 use cases in 7 minutes: The Power of Workflow Automation

– http://bit.ly/1bwvdwT• Professional Grade Cloud for your Hybrid IT needs

– http://bit.ly/1gKI2Jt• Scaling for your first 10 million users

– http://bit.ly/1dA4SiW• Drinking Our Own Champagne: How w00t, an Amazon subsidiary, uses AWS

– http://slidesha.re/1kstOSu• Building a Scalable Digital Asset Management Platform (PBS)

– http://bit.ly/19DiGbd• Instrumenting Your App Stack in a Dynamically Scaling Environment

– http://bit.ly/1gEchEv• Intrusion Detection in the Cloud

– http://bit.ly/18C6nf4

And about that steak dinner

• 7 hungry men wandered the entire Las Vegas strip for 3 hours (give or take 2 hours) trying to make a decision on where to eat a steak…

• The quest finally ended at…

Wolfgang Puck at the MGM Grand

Credits

• My incredible team at ACS Technologies, Inc.• Presenters at re:Invent 2013