Embed Size (px)
DESCRIPTION
IT systems provide tremendous value, and maintaining data integrity and guaranteeing business continuity is of utmost importance for any organization. However, in today's world, those systems have grown in complexity and cost, while the business demands IT agility and lower costs. In this talk, AWS technology evangelist, Simone Brunozzi, joined by AWS customers, will explore how organizations should approach backup and disaster recovery, and how these two aspects can be implemented in the cloud to improve efficiency and flexibility. The talk starts with general concepts, and then dives into technical details, culminating in real customer examples that showcase some tips and tricks and the benefits of a cloud-based approach.
Citation preview
2 2
"The mind is not a vessel to be filled, but a fire to be ignited." - Plutarch
Agenda
I. Prologue
II. Lessons
III. Customer Story
IV. Earthquake
V. Lessons
VI. Conclusions
The story of Monte Cassino
Backup
Shaw Media
What happened to my Parmigiano?
Disaster Recovery
... And a little surprise!
Prologue
Part I
5
6
Abbey of Monte
Cassino
7 7
Why is Monte Cassino important? ] [
8
The Treasure of Monte Cassino ] [ 8
9
The Treasure of Monte Cassino ] [
800 papal documents 20,500 volumes in the Old Library 60,000 in the New Library 200 manuscripts on parchment 100,000 prints and paintings (including 11 Titians) 500 incunabula
A book printed before 1501 C.E.
Gutenberg’s Bible was printed in 1455
C.E.
Titian, one of the most influential
painters ever
x
10
Business continuity continuum ] [
High availability
Backup storage
Disaster recovery
11
High Availability : Keeping services alive
Business continuity continuum ] [
12
High Availability : Keeping services alive Backing up : Process of copying and archiving of data so it may be used to restore the original after a data loss event
Business continuity continuum ] [
13
High Availability : Keeping services alive. Backing up : Process of copying and archiving of data so it may be used to restore the original after a data loss event. Disaster recovery : Recovery of technology infrastructure critical to an organization after a natural or human-induced disaster.
Business continuity continuum ] [
14
Origin of Backup ] [
Monastery : Brilliant, scalable, low-cost, highly durable backup system Origin of Universities (Charlemagne, 814 C.E.) The Empire
needs educated people
Let’s ask the Church!
Edict: Free education in
cathedrals and monasteries
Lots of books (and backups)
x
15
Origin ] [
Monastery : Brilliant, scalable, low-cost, highly durable backup system. Origin of Universities (Charlemagne, 814 a.C.) Indoctrination : One of the first critical function within an organization (Catholic Church) that needed continuation after any natural or human-induced disaster. It needed backup of books (Bibles, etc.) in order to function.
Barbarians, pestilences, fires, invasions, wars,
famines, revolts, etc.
16 16
Why is Monte Cassino important? ] [
17
World War II ] [
18 18
Dec 1942: Many “treasures” are transported from Rome and other places to Monte
Cassino, for safety
The Treasure of Monte Cassino ] [
19 19
Lost in translation ] [
It means “Military Division”
(abbreviated)
Intercepted German message: “Ist der Abt noch im Kloster?”
“Ja.”
It also means “Abbot”
(abbreviated)
20 20
Abbey of Monte Cassino ] [
21 21
The Treasure of Monte Cassino ] [
Feb 1944: Schlegel and Becker (Panzer-Division Hermann Göring) had the treasures transferred to the Vatican
x
22 22
Escape from Monte Cassino ] [
23 23
Escape from Monte Cassino ] [
Lt. Col. Julius Schlegel
(an Austrian Roman Catholic)
Capt. Maximilian Becker
(a Protestant surgeon)
“Biggest bombing against a single target of all time”
25 25
Monte Cassino after bombing (1944) ] [
26 26
Restoration in 1954 ] [
27 27
The Abbey of Monte Cassino today ] [
End of Prologue
Lessons from Monte Cassino
Part II
30 30
1. My backup should be accessible
a.k.a. the pain of physical data
transfer
AWS
1. My backup should be accessible
API AWS Direct Connect
AWS Storage Gateway
Customer owns the data
Redundancy
AWS Import/Export
AWS Storage Gateway ] [
GW-stored volumes
z
GW-Cached volumes
GW-stored volumes
“Cool” storage
“Cold”
w
VPN
Public / AWS Direct Connect
AWS Import/Export
z
36 36
2. My backup should be able to scale
37
Lessons from Monte Cassino ] [
2. My backup should be able to scale
• “Infinite” scale with Amazon S3 and Amazon Glacier • Scale to multiple regions • Seamless • No need to provision • Cost tiers (cheaper at scale)
38
Regions (8) GovCloud Regions (1)
(as of Nov 27th, 2012)
Global AWS Infrastructure ] [
39
Availability Zones (23)
Global AWS Infrastructure ] [ (as of Nov 27th, 2012)
40
Edge Locations (38)
Dallas (2)
St.Louis
Miami
Jacksonville Los Angeles (2)
Palo Alto
Seattle
Ashburn (2)
Newark New York (2)
Dublin
London Amsterdam (2) Stockholm
Frankfurt (2) Paris
Singapore (2)
Hong Kong
Tokyo
São Paulo
South Bend
San Jose
Osaka
Milan
Sydney
Madrid
Global AWS Infrastructure ] [ (as of Nov 27th, 2012)
41 41
3. My backup should be safe
42
Lessons from Montecassino ] [
3. My backup should be safe
• SSL Endpoints (Amazon S3 and Amazon Glacier) • Signed API calls • Store encrypted files • Server-side encryption • Durability: multiple copies across different data centers • Local/cloud with AWS Storage Gateway
43
3. My backup should be safe
44 44
4. My backup should work with a DR policy
45
Lessons from Monte Cassino ] [
4. My backup should work with a DR policy
• Easy to integrate within AWS or Hybrid • AWS Storage Gateway: Run services on Amazon EC2 (DR) • Clear costs • Reduced costs • I decide redundancy/availability in relation to costs
47
Lessons from Monte Cassino ] [
5. Someone should care about it
• Clear ownership • Permissions with IAM: Users, groups -> roles • Logs • AWS support
48
Lessons from Monte Cassino ] [
1. My backup should be accessible
2. My backup should be able to scale
3. My backup should be safe
4. My backup should work with a DR policy
5. Someone should care about it
A customer story
Part III
50 50
Augusto Rosa Manager, Server Operations - Shaw Media
augusto.rosa @ shawmedia.ca
51 51
Shaw Media ] [
52 52
Who we are ] [ • Shaw Media: Division of Shaw Communications Inc. • It reaches almost 100% of Canadians; 18 specialty channels • Global national newscast: 1+ million viewers every weekday • Access to full episodes: 20 websites, 4 video-on-demand • It engages with 25+ million Canadians per week
53 53
Before AWS ] [ • Data centers in Winnipeg and Toronto • Challenge to manage, frequent power outages, downtime • Expensive hosting fees inherited from parent company • Technology was old and in disarray (total revamp needed)
54 54
Mission Impossible? ] [
55 55
Mission ] [ • Implement a new CMS • Empower the editorial team • Business objectives •Time frame of 9 months • Be agile and cost effective
AWS
Amazon SQS Amazon SNS Amazon SES AWS Marketplace Amazon FPS Amazon DevPay Amazon Mechanical Turk Amazon Route 53 Amazon VPC Amazon Direct Connect Amazon S3 Amazon Glacier Amazon EBS AWS Import/Export AWS Storage Gateway AWS Support
Amazon EC2 Amazon EMR Auto Scaling Elastic Load Balancing Amazon CloudFront Amazon RDS Amazon DynamoDB Amazon SimpleDB Amazon ElastiCache Amazon IAM Amazon CloudWatch Amazon Elastic Beanstalk Amazon CloudFormation Amazon CloudSearch Amazon SWF Alexa WIS and Alexa Top Sites
Amazon SQS Amazon SNS Amazon SES AWS Marketplace Amazon FPS Amazon DevPay Amazon Mechanical Turk Amazon Route 53 Amazon VPC Amazon Direct Connect Amazon S3 Amazon Glacier Amazon EBS AWS Import/Export AWS Storage Gateway AWS Support
Amazon EC2 Amazon EMR Auto Scaling Elastic Load Balancing Amazon CloudFront Amazon RDS Amazon DynamoDB Amazon SimpleDB Amazon ElastiCache Amazon IAM Amazon CloudWatch Amazon Elastic Beanstalk Amazon CloudFormation Amazon CloudSearch Amazon SWF Alexa WIS and Alexa Top Sites
58 58
Phase One ] [ • Fast deployment of servers, network rules, load balancers • First site under new CMS: Live in 4 weeks from scratch • Full migration of 29 sites from a physical DC in 9 months
59 59
Phase Two ] [ • Full migration of 6 other websites and web services • From 2nd physical DC into AWS in 2 months • Migration: Windows ‘03/SQL ‘05 -> Windows ‘08/SQL ’08 • Creating new web farms takes 1 to 5 days (versus months) • Takes longer to procure licenses than the infrastructure • Ability to scale and automate
60 60
Benefits of Using AWS ] [ • Increased uptime from 98.8% to 99.99% • Scale to success, quicker response to business needs • 1+ M$ saved in capital and operational cost • No physical investment, smaller teams • Allowed using service management 3rd party companies • Easy backup on AWS -> 3 years retention (tax credits)
61 61
AWS Architecture ] [
62 62
Some Numbers ] [ • 50+ EC2 instances (various sizes) • 25+ TB traffic/month • 40M+ Route53 queries • 10+ TB backup on Amazon S3
... And growing!
63 63
Lessons Learned ] [ • Architecting for AWS in mind from start • Use all Availability Zones in area you choose to host; divide across all • Plan for failures: Be crazy about it (things fail) • Backup backup backup • Monthly AMI • Windows/SQL Server workarounds (failover cluster, AD, etc.) • Engage with AWS Solutions Architects early
64 64
Disaster Recovery ] [ • Learn from outages all the time • Implement changes to prevent failures at cloud level • Document how you recover from failures • Single component may fail; architecture shouldn’t
65 65
Backup ] [ • Daily snapshots of all volumes automatically • VIP volumes: snapshots every 4 hours • Keep the last 10 snapshots • Dell Replay: It backs up file system files every 1 hour • Volumes replicated to Amazon S3 (Oregon) every 2 hours • SQL Server backup every 30 minutes • SQL Server backup volumes moved to Amazon S3 every 2 hours
66 66
Future ] [ • Move from public cloud to VPC • Auto Scaling on Amazon EC2 • Amazon S3 as image repository for all sites • Second cloud vendor as DR (instead of in-house) • Amazon ElastiCache for central caching for ASP.net apps
67 67
Augusto Rosa Manager, Server Operations - Shaw Media
augusto.rosa @ shawmedia.ca
The 2012 Emilia Earthquake
Part IV
69 69
May 20th, 2012: Earthquake in Italy ] [
70
71 71
Parmigiano warehouse (0.5B € damage) ] [
72 72
“Let’s do something NOW” ] [
73 73
Buy 1 Kg of Parmigiano for 1 Euro ] [
74 74
Everybody helped ] [
Lessons from an Earthquake
Part V
76
1. You NEED a DR in place!
2. Testing your DR
3. Reducing costs
4. You can have different DR solutions
Lessons from an Earthquake ] [
77 77
1. You NEED a DR in place!
DR with High Availability
App DR with Standby
80 80
Business Impact Analysis (RTO, RPO)
81
Lessons from an Earthquake ] [
• RTO (Recovery Time Objective): 1) Time for trying to fix the problem 2) The recovery itself 3) Testing 4) Tell users • RPO (Recovery Point Objective): how much data I can lose
Business Impact Analysis (RTO, RPO)
82
Lessons from an Earthquake ] [
1) Backup and Restore 2) “Pilot light” for quick recovery into AWS (Cold standby) 3) Warm standby solution on AWS 4) Multi-site hybrid solution (AWS + on premises)
Different Types of DR Architecture
Cost ($/GB/month) Performance Durability
Amazon S3 0.125 *** ***** Amazon Glacier 0.01 * *****
AWS Storage Gateway
0.125 (+ 125/GW) **** ***
Amazon EBS 0.10 **** *** Amazon EBS
(PIOPS) 0.125 ***** ***
84 84
2. Testing your DR
85
Lessons from an Earthquake ] [
• Dev/test in the cloud is super easy • Spin up capacity only for the test • Regularly test your DR • Cost is minimal • What about data transfer speed?
2. Testing your DR
86
s3cmd ls --recursive
s3://datasets.elasticmapreduce/ngrams/b
ooks/ | awk '{print $4;
sub(/s3:\/\/datasets.elasticmapreduce/,
"/array", $4); print $4}' | parallel -
j0 -N2 --progress /usr/bin/s3cmd --no-
progress get {1} {2}
Special thanks to Craig Carl, AWS Solutions Architect
87
s3cmd ls --recursive
s3://datasets.elasticmapreduce/ngrams/b
ooks/ | awk '{print $4;
sub(/s3:\/\/datasets.elasticmapreduce/,
"/array", $4); print $4}' | parallel -
j0 -N2 --progress /usr/bin/s3cmd --no-
progress get {1} {2}
Lists every object in the bucket
88
s3cmd ls --recursive
s3://datasets.elasticmapreduce/ngrams/b
ooks/ | awk '{print $4;
sub(/s3:\/\/datasets.elasticmapreduce/,
"/array", $4); print $4}' | parallel -
j0 -N2 --progress /usr/bin/s3cmd --no-
progress get {1} {2}
Gets the path to the Amazon S3 object and the local
destination path
89
s3cmd ls --recursive
s3://datasets.elasticmapreduce/ngrams/b
ooks/ | awk '{print $4;
sub(/s3:\/\/datasets.elasticmapreduce/,
"/array", $4); print $4}' | parallel -
j0 -N2 --progress /usr/bin/s3cmd --no-
progress get {1} {2}
Runs parallel with as many threads as possible, '-N2' tells
parallel there were two arguments on stdin and
assigns them to {1} and {2}
90
s3cmd ls --recursive
s3://datasets.elasticmapreduce/ngrams/b
ooks/ | awk '{print $4;
sub(/s3:\/\/datasets.elasticmapreduce/,
"/array", $4); print $4}' | parallel -
j0 -N2 --progress /usr/bin/s3cmd --no-
progress get {1} {2}
It’s the command that GNU Parallel will run, '{1}' is
substituted with the Amazon S3 object path, '{2}' is
substituted with the local destination path
91
s3cmd ls --recursive
s3://datasets.elasticmapreduce/ngrams/b
ooks/ | awk '{print $4;
sub(/s3:\/\/datasets.elasticmapreduce/,
"/array", $4); print $4}' | parallel -
j0 -N2 --progress /usr/bin/s3cmd --no-
progress get {1} {2}
Copying 2.4 TB down from 48 hours to 9 hours (5x faster)
92 92
3. Reducing costs
93
Lessons from an Earthquake ] [
1) AWS cost reduction (e.g., S3 cost reduction on Nov 28th) 2) Reduced redundancy (Amazon S3) 3) Retention policy 4) Hot/warm/cool/cold backup 5) Reserved capacity/tiers
3. Reducing costs
0-1 TB 0.125 0.093
1-50 TB 0.110 0.083
50-500 TB 0.95 0.073
500-1,000 TB 0.90 0.063
1-5 PB 0.80 0.053
5+ PB 0.55 0.037
Amazon S3 Standard $/GB/Month
Reduced $/GB/Month
95 95
4. You can have different DR solutions
96
Lessons from an Earthquake ] [
• Easy to integrate existing vendors with DR on AWS • Approach: One vendor/hybrid/multiple vendors • One region/multi-regions (if you need geodiversity)
4. You can have different DR solutions
97
1. You NEED a DR in place!
2. Testing your DR
3. Reducing costs
4. You can have different DR solutions
Lessons from an Earthquake ] [
Conclusions
Part VI
Backups Disaster Recovery
Action items
Agility Cost savings Control
x
Parmigiano, a Monastery, Love and Faith
Simone Brunozzi Senior Technology Evangelist, Amazon Web Services
@simon
Technical lessons on how to do Backup and Disaster Recovery in the Cloud
We are sincerely eager to
hear your feedback on
this presentation and on
re:Invent.
Please fill out an
evaluation form when you
have a chance.
