Embed Size (px)
DESCRIPTION
Scaling a Web Application is a very hard problem, especially for small project and teams who do not have sufficient manpower, money, and time to solve this problem. Luckily Amazon already had to solve this problem in their datacenters and offers their services to other developers. This talk will introduce the two most important Amazon Web Services, the Elastic Compute Cloud (EC2) and the Simple Storage Service (S3), and will present different ways to leverage them in your own web application. Ruby on Rails will be used for the examples but they will apply to any web framework. Several use-cases will be covered that show how S3 and EC2 can be used to move load from your servers to Amazon's or even to completely host your application at Amazon.
Citation preview
Use Somebody Else's Infrastructure:
Utilizing Amazon S3 and EC2
Jonathan Weiss
Peritor Wissensmanagement GmbH, www.peritor.comWeb 2.0 Expo Berlin, 2007
2
Peritor
• Consultancy from Berlin, Germany
• Specialized on Architecture, Scaling andRuby on Rails
• A lot of our clients use Amazon WebServices
3
www.peritormail.com
We run a Webmail portal on top of Amazon Web Services
4
Starting Point
One machine:
• Apache
• Ruby on Rails / PHP / Perl / Java / …
• MySQL
5
Worst Case Populariy
6
A Difficult Path
7
Problem: Backup
• High availability
• Redundancy
• Very big data sets
8
Problem: File System
• Important files have to be accessed by many servers
• NFS / Samba not practical
9
Problem: Messaging
• Many servers on different locations have to communicate
• Creating a reliable and scalable message queue is really hard
10
Problem: Spontaneous Traffic
11
Problem: Load Fluctuation
12
Don‘t reinvent the wheel!
13
Amazon Web Services
Simple Storage Service - S3
Redundant Data Store
$0,15 per GB data permonth
$0,10 - $0,20 per GBtraffic
Virtual server per hour
$0,10 per CPU hour
$0,10 - $0,20 per GBtraffic
Elastic Compute Cloud - EC2
…….
E-Commerce..
Reliable Message System
$0,10 per 1,000 messages
$0,10 - $0,18 per GBtraffic
Simple Queue Service - SQS
14
S3 - Simple Storage Service
• Redundant storage - as much as you like
• max. 5 GB per object
• Organized in „Buckets“
• Web Service API for uploads
• Downloads via
• Web Service
• HTTP / HTTPS
• BitTorrent
15
S3 - Buckets
• Unique over all S3
• Contains manykey-value-metadata tupel
• Cannot contain other buckets!
• Key can contain „/“
MyBucket_name
S3
foto_5.jpg
backups/januar.zip
fotos/2007/001.png
www.peritor.com
site/screen.css
site/script.js
FreeBSD-6.2.iso
16
S3 with AWS::S3Upload
Download
17
EC2 - Elastic Compute Cloud
• Based on XEN virtualization
• On demand virtual servers - controlled with Web Service API
• Use your favorite Linux distro (Linux 2.6.16),Amazon Machine Images (AMI) are stored on S3
• ACLs for hosts/ports access control
18
EC2 ToolsList availabe images
Start a new instance
Login with SSH
Shutdown instance
19
amazon-ec2 gemSetup
Usage
20
SQS - Simple Queue Service
• Distributed, highly scalable and reliable message system
• Unlimited number of queues and messages
• Locking / releasing built-in
• 256 KB message payload
• Basic permission model
21
SQS gemSetup
Usage
22
How does this solvemy problems?
23
S3 - Backup
• s3sync.rb
• Brackup
• Jungle Disk
• S3 FUSE
• s3DAV
• Duplicity
• S3Browser
• Firefox S3 Organizer extension
• …
24
s3sync.rb
Backup
Restore
25
S3 Asset Host
26
S3 Asset Host
Setup DNS
Rails configuration
27
S3 Asset Host
welcome.rhtml template
Output
28
S3 - Authenticated User Data
29
attachment_fu Rails plugin
Setup
30
attachment_fu Rails plugin
Upload
Download
31
On-Demand Computing with EC2
Time based, e.g. with cron
32
On-Demand Computing with EC2
Load based, e.g. with Monit
33
On-Demand Computing with EC2
34
EC2 for extra capacity
35
Peritor Wissensmanagement GmbH
Lenbachstrasse 212157 Berlin
Internet: www.peritor.comEmail: [email protected]
© Peritor Wissensmanagement GmbH - All Rights Reserved
Phone: +49 (0)30 69 40 11 94Fax: +49 (0)30 69 40 11 95
