Hosting Ruby Web Apps
Lessons learned from 8 years of
Overview
• System architecture
• Initial setup & deploy
• Keep it running and moving forward
Next Weekend! startupweekend.jp/swtokyo-personal-cloud/
Build a community through your events.
DB (psql)
memcache
nginxunicorn
unicornRails App (unicorn)
job workerMTA (postfix)
Contenders
Contenders
Contenders
Contenders
Contenders
Please Note
• Not an exhaustive list of all hosting providers(it’s not even everyone we’ve been using)
• Moving targets
Choosing Server Sizewww.flickr.com/photos/jonrb/7864016624
Choosing Server Size
• good CPU performance
• choose 2GB RAM or more
Choosing Server Size
• 1 ECU/core is a bit frustrating
• 2 ECU/core is OK
• >2 ECU/core is better
Choosing Server Size
• 3 different sizes
• choose standard size if you don’t have specific requirements
Hardware Failures
How can we make things robust?
www.flickr.com/photos/doegox/4551458930
Hardware Failures“We’re trying to prevent failures, please
make backups for worst case”
!
• provides load balancer
• fault tolerant setup example:2 web instances + 2 DB instances
• need to configure DB replication and failover yourself
Hardware Failures“Failures will happen, build your infrastructure
so they won’t impact you”
• provides load balancer (ELB)
• provides managed DB instances (RDS)
• replication support (not for psql yet)
• DB snapshots (can’t download though)
• fault tolerant setup example: 2 web instances + multi-AZ RDS
Hardware Failures“Failures will happen, let us help you build an
infrastructure so they won’t impact you”
• HA proxy on web instances
• one-click setup for DB (mysql/psql)
• replication support
• DB snapshots (downloadable)
• fault tolerant setup example:2 web instances + 2 DB instances
Hardware Failures“You don’t need to worry about failures”
!
!
• everything managed
• fault tolerant setup example: 2 dynos + premium DB (psql)
Initial Setup & Deploy
www.flickr.com/photos/thedailyenglishshow/6013713229
Initial Setup & Deploy
• manual setup feasible:
• install OS, ruby (rvm), libs, DB
• setup nginx
Initial Setup & Deploy
• let’s choose Capistrano for deploying
• Capistrano config goes into source repository
• Initial deploy:
cap deploy:setup cap deploy:cold
Initial Setup & Deploy
• let’s choose OpsWorks
• based on chef, provides predefined set of recipes
• more high level than Cloud Formation, more flexible than Elastic Beanstalk
Initial Setup & Deploy• initial deploy:
• create stack
• define layers
• create instances
• create app
• deploy
Initial Setup & Deploy
• define base layer
• assign it to all instances
• use it for any common recipes like creating swap, NewRelic, ...
Tip
Initial Setup & Deploy• you need to handle asset compilation
• use deploy hook:
Tip
# deploy/before_migrate.rb !rails_env = new_resource.environment["RAILS_ENV"] Chef::Log.info("Precompiling assets for RAILS_ENV=#{rails_env}...") !execute "rake assets:precompile" do cwd release_path command "bundle exec rake assets:precompile" environment "RAILS_ENV" => rails_env end !!
Initial Setup & Deploy• provides toolchain based on chef
• initial deploy:
• create an application
• select environment layout
• add plugins (NewRelic, ...)
• deploy
Initial Setup & Deploy
Heroku comes with its’ own toolchain:
heroku create my-awesome-app heroku addons:add … git remote add heroku … git push heroku master
What you’ll probably also need
• App configuration (for secrets and endpoints)
• Sending email
• Job queue
App Configuration
• no predefined way: DIY
• put settings.yml into shared config dir
• link it into app when deploying
App Configuration
• uses environment variables
heroku config:add MY_SECRET=topsecret heroku config
• in your code:
ENV[‘MY_SECRET’]
Sending Email
• use an email sending service (SendGrid, AWS SNS)
• EY and Heroku provide plugins for easy installation
Sending Email Yourself
• reverse IP lookup
• port 25 is restricted
• make sure your IP isn’t blacklisted
Job Queue
• DIY
• define custom layer in OpsWorks, need to get 3rd party recipe
Job Queue
• sample recipes are provided
• worker instances
Job Queue
• provides worker dynos
• requires setup in Procfile
Background mailer
• use database transaction to atomically create mailer job and related data
• rollback if something goes wrong
• reduces request response time
• job can retry sending email
Tip
Up and running :-)
Continuous Deployment
www.flickr.com/photos/layos/3743880081
Continuous Deployment
• keep deploys cheap
• automate deploy
• easy deployment trigger
• good test coverage - use CI
• use rolling / zero downtime deploy
Continuous Deployment
• deploy command:
git pull cap deploy
• unicorn for rolling deploy
Continuous Deployment
• unicorn is configured for rolling deploys
• deploy command:
aws --region=‘us-east-1’ opsworks create-deployment --stack-id=‘<STACK_UUID>’ --app-id=‘<APP_UUID>’ --instance-ids=‘[“<INSTANCE_UUID>”]’ --command='{"Name": "deploy"}
Continuous Deployment
gem 'aws-sdk' gem 'parseconfig' !PRODUCTION_APP_ID = "09afbde1-322b-4816-a1e9-xxxxxxxxxxxx" !config = ParseConfig.new(File.expand_path("~/.aws/config")) AWS.config( :access_key_id => config['default']['aws_access_key_id'], :secret_access_key => config['default']['aws_secret_access_key']) !@ops = AWS::OpsWorks.new.client @ops_app = @ops.describe_apps(app_ids: [PRODUCTION_APP_ID])[:apps].first @ops_stack = @ops.describe_stack_summary(stack_id: @ops_app[:stack_id])[:stack_summary] @ops_inst = @ops.describe_instances(stack_id: @ops_app[:stack_id])[:instances] @ops_inst_ids = @ops_instances.map do |instance| instance[:instance_id] if instance[:status] == 'online' end.compact !puts "Deploying #{@ops_app[:name]} to #{@ops_stack[:name]}" !deploy_options = { command: { name:' deploy' }, comment: "deploy from #{Socket.gethostname}", stack_id: @ops_app[:stack_id], app_id: @ops_app[:app_id], instance_ids: @ops_inst_ids } @ops.create_deployment deploy_options !!
or some simple ruby script:
Continuous Deployment
• deploy command:
ey deploy
• in config/ey.yml:
maintenance_on_migrate: false
Continuous Deployment
• deploy command:
git pull git push heroku master
• no-downtime deploys experimental:
heroku labs:enable -a myapp preboot
Continuous Deployment
• “fork” in code
• on/off switch for features
• slow rollout of new features
Tip
Rolling Deploys with!Database Migrations
www.flickr.com/photos/edwardshousemovers/6704586649
Rolling Deploys with Database Migrations
orchestrated deployment flow:
1. copy code, keep old version
2. run migrations
3. switch to new code
➜ one-step deployments
Rolling Deploys with Database Migrations
• asynchronous deployment flow
• two-step deployment
1. deploy old code + DB migrations(one instance only)
2. deploy new code(all instances)
Rolling Deploys with Database Migrations
• migrations are run via:
heroku run rake db:migrate
• two-step deployment
1. deploy old code + DB migrations
2. deploy new code
Down for!Maintenance
www.flickr.com/photos/metrolibraryarchive/4128611963
Down for Maintenance
• Maintenance page done right:
• include contact and progress info (use 3rd party service like Twitter)
• serve page with 503 error code
• use asset host or inline all assets
Tip
Down for Maintenance• add ‘capistrano-maintenance’ gem
• configure nginx
• then:
cap maintenance:enable
server { // nginx server config ! location @maintenance { rewrite ^(.*)$ /system/maintenance.html last; break; } if (-f $document_root/system/maintenance.html) { return 503; } ! error_page 503 @maintenance;
Down for Maintenance
• Need custom recipe / script
• Can use similar approach as with Capistrano
• 503 won’t pass ELB health check!
• Alternative: use Route53 to fail over to instance serving maintenance page
Down for Maintenance
• Part of the platform:
ey web disable
heroku maintenance:on
Getting Support
Getting Support
• provides ticket system for any platform related issues
• forum for anything else
Getting Support
• forums for everything
• access to tickets only available if certain health checks fail
Getting Support
• tickets for everything
• provides even hosting related help on application level
• “extension of your team”
Conclusion• there is no silver bullet
• a lot depends on your app
• be ready to get your hands dirty
• your production environment is your app
• balance ops / dev
Thank you!
Contact:
Michael [email protected] @mreinsch
Questions?More awesome events:
• ijaws.doorkeeper.jp
• Startup Weekend Tokyo Personal Cloud
Recommended
