Upload
wr0ngway
View
5.290
Download
3
Embed Size (px)
DESCRIPTION
A presentation for the AWS Startup Challenge tour on how we use AWS to deploy and operate snapmylife.com
Citation preview
Deploying and Scaling with AWS Matt Conway, VP Engineering
About Us
Company Profile • Founded 2007 • Tier 1 venture-backed • Experienced team of eleven in
Needham, MA • Developer of direct-to-consumer
mobile services
SnapMyLife Service • Mobile photo sharing community • Best mobile user experience • Premium demographic • Rapidly growing local & global
audience
2
Key Stats
• Launched April 2008
• Mobile website (no download required), iPhone web/app, Internet
• 1.5+ million unique visitors / month
• 15+ million monthly page views (Aug 2008)
• Over 90% traffic from mobile devices
• Visitors from every country in the world
• 400,000+ registered users
Screenshots
Mobile iPhone
Desktop
Overview
EC2 • Entire web serving stack • Daemons for background/batch processing • Internal infrastructure
S3 • Cheap CDN • Backups for EC2
Mechanical Turk • Content filtering • Content classification
4
EC2 Costs
Xen based cluster on leased hardware • Time sink • Not as flexible • Slow to provision (Days vs Minutes) • More expensive (~$1.8K/month)
EC2/S3 • We hardly ever think about it, just use it • Unlimited playground without fear of breakage • Clone your entire production stack in minutes • Works out cheaper (~$1.2K/month)
5
EC2/S3 Architecture
Fairly standard architecture DNS – Web – App – Memcached - Mysql Assume and plan for failure Use a role based architecture to easily scale
roles independently of each other (e.g. add an app server or db slave)
Roles scale in as well as out – incredibly useful to have production cloned to a single instance for testing new features.
6
EC2 Instance Allocation
Static Vs Dynamic instance allocation • We went static route • Our traffic graph pretty flat due to global nature • Easier, dynamic cost savings not worth the extra
complexity for us • We watch our load carefully, given a critical alert, we
can scale up in a matter of minutes by increasing the instance count for the impacted roles.
All instance configuration lives in our source tree. New instances to handle any combination of roles is only a couple of commands and a few minutes away
We open sourced our framework for doing so: The rubber project on github - http://github.com/wr0ngway/rubber/wikis
7
Mechanical Turk
All of our content gets classified through turk Some of our content gets tagged We interact with turk solely through
automation – large quantities preclude use of web interface
Results were better than anticipated Incredibly quick turnaround, 24x7 Very hard to match this performance given
the hiring constraints of a startup
8
Mturk costs
Depends a lot on how you do it Keep questions simple, pay less Multiple questions, better result Example of a months usage • ~64K picture moderations (x3) • ~28K pictures tagged • Total Cost ~$3.3K (AWS commission $1.1K)
9
Things we’d like to see
A reliable way to send mail from EC2 Faster S3 response - New AWS CDN
should address this Automatic billing for Mturk A better plan for large volume mturk usage –
commission is normally 10%, but the half cent minimum makes it 50% if all you do are 1c HITs
10