Upload
amazon-web-services
View
1.274
Download
3
Tags:
Embed Size (px)
Citation preview
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
DevOps, Continuous Integration &
Deployment on AWS
Leo Zhadanovsky, Senior Solutions Architect, AWS@leozh
Dave Beck, Technical Director, National Novel Writing Month@DaveB_NaNoWriMo
What is DevOps?
• « DevOps is the practice of operations and
development engineers participating together in
the entire service lifecycle, from design through
the development process to production support
»
- theagileadmin.com
What is Continuous Integration?
• Changes to code automatically deployed to mainline branch– After passing unit and mock tests
• Makes changes to code, and deployments iterative, not monolithic
• Bugs are detected quickly
• Helps automate deployments
• Allows rapid development and deployment
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
DEVELOPER
PICK
TASKS
SUBMIT
CODE
SCHEDULE
BUILD
RECURRENT
BUILDS
CODE
FETCHCODE QUALITY
TESTS
TEST
RESULTS
BUILD OUTPUT
DOCS
BINARIES
& PACKAGES
DEV FACING
NOTIFICATIONS
CLOUDFORMATION
AMIS or CONTAINERS
AWS code services
AWS CodeCommit
Coming soon
AWS CodePipeline
Coming soon
AWS CodeDeploy
Launched Nov 2014
Cloud software development lifecycle
10/13/14 10
MonitorProvisionDeployTestBuildCode
AWS Elastic Beanstalk
AWS OpsWorks
Amazon
CloudWatch
AWS
CloudFormation
?
Source ControlPrivate Git repositories hosted on Amazon S3
• Full Git compatibility (use with existing tools)
• All the benefits of the cloud
(scalable, durable, reliable, low pay as you go
pricing)
• No size limits on repositories (store binary files)
• Online code tools with browse, edit, diff
10/13/14 11
Same Git experience
$ git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/aws-cli
Cloning into 'aws-cli'...
Receiving objects: 100% (16032/16032), 5.55 MiB | 1.25 MiB/s, done.
Resolving deltas: 100% (9900/9900), done.
Checking connectivity... done.
$ nano README.rst
$ git commit -am 'updated README'
[master 4fa0318] updated README
1 file changed, 1 insertion(+)
$ git push
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 297 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote:
To https://git-codecommit.us-east-1.amazonaws.com/v1/repos/aws-cli
4dacd6d..4fa0318 master -> master
Continuous DeliveryCustomizable release automation, with integrated build and test
• Model and visualize custom release workflow
(source build beta gamma prod)
• Automate builds, tests, and deployments
• Enforce custom rules, approvals, and gates
• Integrate with third-party and custom tools
10/13/14 13
DeploymentCoordinate software updates to fleets of EC2 instances
• Rolling updates for no downtime
• Deployment health checks and easy rollback
• Auto Scaling integration
• Works with any application
• Reuse existing setup tools
(Bash, Powershell, Chef, Puppet…)
10/13/14 15
Cloud software development lifecycle
10/13/14 17
MonitorProvisionDeployTestBuildCode
AWS Elastic Beanstalk
AWS OpsWorks
CloudWatchCloudFormationCodeDeploy
CodeCommit CodePipeline
PAIN POINTS• UNIT TESTS INCOMPLETE
• MOCK TESTS MAINTENANCE
• EXPENSIVE TEST ENVIRONMENT
• TEST ENVIRONMENT ≠
PRODUCTION
• DEPLOYMENT CYCLES
A lot of options…
• Configuration Management Systems– Puppet
– Chef
– Saltstack
• Deployment Frameworks– CodeDeploy
– AWS Elastic Beanstalk
– AWS OpsWorks
– Ansible
– Fabric
– Capistrano
• Infrastructure Management– CloudFormation
• Containers– Amazon EC2 Container Service
Bake an Image Configure dynamically
Time consuming configuration (startup time)
Static configurations (less change management)
Bootstrapping
Bake an Image Configure dynamically
Continuous deployment (latest code)
Environment specific (dev-test-prod)
Bootstrapping
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
AWS CLOUDFORMATIONSTACK-BASED DEPLOYMENT SERVICE
{"Description" : "Create RDS with username and password","Resources" : {
"MyDB" : {"Type" : "AWS::RDS::DBInstance","Properties" : {
"AllocatedStorage" : "500","DBInstanceClass" : "db.m1.small","Engine" : "MySQL","EngineVersion" : "5.5","MasterUsername" : "MyName","MasterUserPassword" : "MyPassword"
}}
}}
"AWS::CloudFormation::Init" : { "config" : {
"packages" : {"yum" : {
"mysql" : [],"mysql-server" : [],"httpd" : [],"php" : [],"php-mysql" : []
}},"sources" : {"/var/www/html" :
"https://s3.amazonaws.com/my-builds/build-v4.zip"}
}
{"Parameters" : {"KeyName" : {
"Description" : "Name of an existing EC2KeyPair to enable SSHaccess to the instance",
"Type" : "String"}
},}
CLOUDFORMATION
TEMPLATE
PROCEDURAL
DEFINITIONCreate it programmatically
KNOWN
CONFIGURATIONStore stack configuration in
source control
PARAMETER
DRIVENDynamic and user-driven
templates
COLLABORATIONShare templates with ease
as just files
11.6s
Mean time
between
deployments
(weekday)
1,079
Max number of
deployments in a
single hour
10,000
Mean number of
hosts
simultaneously
receiving a
deployment
30,000
Max number of
hosts
simultaneously
receiving a
deployment
DEPLOYMENTS AT
AMAZON.COM
(in 2011)
SPEED AND AGILITY
Experiment
Often
Fail quickly at
a low cost
More
Innovation
Experiment
Infrequently
Failure is
expensive
Less
Innovation
“ON-PREMISE”
At a glanceWhat is NaNoWriMo?
• National
• Novel
• Writing
• Month
We are a community of novelists.
Every November, hundreds of
thousands of writers gather virtually
and in-person to write 50,000 words
each. (The Great Gatsby, by
comparison, is 47,094 words.)
Last year, 180,000 started new
novels, and 40,000 finished.
It all started in 1999
Our founder, Chris Baty,
challenged 20 over-caffeinated
friends to write 50,000 words each
in a single month.
Six of them finished.
NaNoWriMo now
NaNoWriMo Main: Ruby 1.9.3, Rails 3.2.2, MySQL 5.6
Camp NaNoWriMo: Ruby 1.9.3, Rails 3.2.2, MySQL 5.6
The Young Writers Program: Drupal 5, MySQL 5.6
NaNoWriMo Store: Drupal 5, MySQL 5.6
November in detail
• Nov 1: ~500k sessions as people begin• All month: ~50k forum posts per day• Nov 30: Last chance to post word counts
Annual Site Crash
• Every year from 2003 to 2011 the site would crash on
November 1.
• In 2012 we migrated the sites to Amazon Web Services.
• The result: No more outages!
• Naturally, this is how our participants reacted…
Keeping NaNoWriMo Online
• Proxy servers (Varnish) on Debian 7.5 EC2 instances
• Ruby on Rails on EC2 (Nginx)
• Worker servers running Resque on EC2
• Support servers running Resque, Redis, Sphinx
• MySQL on RDS instances
• Domains and subdomains on Route 53
• Image assets on S3 instances via EBS
• Deployment via Capistrano, Puppet
Next Steps: Goals
To increase uptime for all services
To make server management less time-consuming
To reliably and quickly respond to user-driven events
To reduce EC2 costs through autoscaling
To create a faster and more dynamic email system
To improve site search
Next Steps: Goodbye Drupal
What: Porting Drupal sites to Ruby 2.2, Rails 4.2
Why: The old Drupal-based sites are slow and use more
than their share of server resources.
Next Steps: Emailing via Amazon SES
What: Migrating outgoing emails from PHPlist/Sendmail to
SES
Why: Event-driven emails on the sites are backgrounded to
support servers, which has proved slow and unreliable.
Mass emails rely on an old and very sluggish application
which has limited layout tools and metrics.
Next Steps: Load Balancing
What: Converting proxy servers to Elastic Load Balancers
Why: We will benefit from better AWS dashboard tools,
metrics, and notifications. It is part of our autoscaling
strategy.
Next Steps: CodeDeploy
What: Using CodeDeploy for deployment process
Why: Currently deployment relies on many custom shell
scripts and tends to be unreliable. We intend to move to
stateless instance spin-ups as part of our autoscaling
strategy.
Next Steps: Autoscaling
What: Implement an Auto Scaling process
Why: In the past we’ve had to predict our server needs
ahead of time. If we underestimated, the sites would
crash; if we overestimated, we weren’t using our small
budget effectively. And the planning process consumed
too much staff time.
Next Steps: Amazon SNS Notifications
What: Increased use of Amazon SNS service
Why: Last year SNS event messaging helped us respond
to emerging crises very quickly. With our expanded AWS
presence, we will link event notifications to all of the
services we use.
Next Steps: On the Horizon
What: Amazon CloudSearch; AWS Lambda
Why: We are interested in easy-to-manage alternatives to
Sphinx – CloudSearch is one option. Lambda looks
intriguing for a variety of possible applications.
Potential future architecture
Internet
Load Balancer
ELB EC2 instances
Code servers
EC2 instances
Databases
RDS DB instances
Search
CloudSearch instance
Outgoing email
Amazon SES service