Download pdf - Building Cloud Castles - LRUG

ben scofield / @bscofield / LRUG / 11 April 2011

BUILDINGCLOUD

CASTLES

http://www.flickr.com/photos/natlockwood/1006351276/

WORK IN PROGRESSthis presentation is a



not just

the cloud allowsLIMITED ACCESS

FINDING PROBLEMS

LOCAL$ ssh [email protected] production.server #1 SMP Sat Dec 5 16:04:55 UTC 2009 i686

To access official Ubuntu documentation, please visit:http://help.ubuntu.com/Last login: Fri Jan 28 16:33:49 2011 from local.hostdeploy@production:~$ cd /var/log/apache2deploy@production:/var/log/apache2$ tail error.log[Sun Jan 23 06:25:02 2011] [notice] Apache/2.2.12 (Ubuntu) Phusion_Passenger...[Tue Jan 25 15:21:42 2011] [error] [client 118.129.166.97] Invalid URI in ...[Fri Jan 28 12:01:50 2011] [error] [client 85.132.70.133] client sent HTTP/1...[Sun Jan 30 06:25:06 2011] [notice] SIGUSR1 received. Doing graceful restart

mailto:[email protected]


http://help.ubuntu.com


NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

CLOUD$ heroku logs --remote production2011-04-11T01:34:22-07:00 app[web.1]: Rendered text template within layout...2011-04-11T01:34:22-07:00 app[web.1]: Completed 200 OK in 54ms2011-04-11T01:34:30-07:00 app[web.3]: Rendered text template within layout...2011-04-11T01:34:30-07:00 app[web.3]: Completed 200 OK in 111ms (Views:57....2011-04-11T08:34:42+00:00 heroku[router]: GET devcenter.heroku.com/article...2011-04-11T01:34:42-07:00 heroku[nginx]: GET /articles/facebook HTTP/1.0 |...2011-04-11T01:34:42-07:00 app[worker.1]: [Worker(host:railgun64.53370 pid:...2011-04-11T01:34:42-07:00 app[worker.1]: [Worker(host:railgun64.53370 pid:...

http://web.1

http://web.1

http://web.1

http://web.1

http://web.3

http://web.3

http://web.3

http://web.3

http://hoptoadapp.com

http://newrelic.com

source

FIXING PROBLEMS

LOCAL$ ssh [email protected] production.server #1 SMP Sat Dec 5 16:04:55 UTC 2009 i686

To access official Ubuntu documentation, please visit:http://help.ubuntu.com/Last login: Fri Jan 28 16:33:49 2011 from local.hostdeploy@production:~$ cd /var/www/app/current/deploy@production:/var/www/app/current$ rails console productionLoading production environment (Rails 3.0.3)>> Article.count => 112>> Article.where(:problem => true).update_attributes(:problem => false)





CLOUDrequire 'test_helper'

class ArticleTest < ActiveSupport::TestCase context 'Broken articles' do setup do 5.times.do { Factory :broken_article } end

should 'be fixable' do assert_equal 5, Article.where(:problem => true).count Article.fix_problem_articles assert_equal 0, Article.where(:problem => true).count end endend

class Article def self.fix_problem_articles where(:problem => true).update_attributes(:problem => false) endend

the cloud isUNRELIABLE locally

LOCALclass Comic < ActiveRecord::Base has_attached_file :cover, :styles => { :thumb => "80x120>", :medium => "300x450>" }end

$ cd public/system$ ls /covers10/ 12/ 53/ 81/$ ls /covers/10/medium/ original/ thumb/$ ls /covers/10/mediumbatman-450.png

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

EVENTUAL CONSISTENCY

CLOUDclass Comic < ActiveRecord::Base has_attached_file :cover, :storage => s3, :s3_credentials => { :access_key_id => ENV['S3_KEY'], :secret_access_key => ENV['S3_SECRET'] }, :bucket => 'comicsapp', :url => ":s3_path_url", :s3_headers => { 'Expires' => 1.year.from_now.httpdate }, :styles => { :thumb => "80x120>", :medium => "300x450>" }end

SERVER SERVER

SERVER SERVER

STORAGE

the cloud can beEXPENSIVE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

CACHING

SERVER

SERVER

CACHING

the cloud prefersSMALL APPS

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

NODE

REST

NODE

NODE

NODE

NODE

NODE

NODEEXTERNALSERVICE

SERVER

SERVER SERVER

SERVICESERVER

the cloud isUNRELIABLEremotely

CLOUDclass Searcher cattr_accessor :index

def self.index @api ||= IndexTank::Client.new(ENV['INDEXTANK_API_URL']) @index = @api.indexes 'articles' end

def self.search(term) raw = self.index.search(term, :function => 1) results = raw['results'].to_a

article_ids = results.map {|result| result['docid'] }

unsorted = Article.published.where(:id => article_ids) results.map { |result| unsorted.find {|u| u.id.to_i == result['docid'].to_i} }.compact endend

EXPECT FAILURE

CLOUDclass Searcher # ...

def self.search(term) results = begin raw = self.index.search(term, :function => 1) raw['results'].to_a rescue URI::InvalidURIError # An IndexTank error occurred search_by_sql(term)['results'] end

# ... end

def self.search_by_sql(term) {'results' => Article.where(['content ILIKE ?', "%#{term}%"]). map {|a| {'docid' => a.id}}} endend

DEFENSIVE CODING

the cloud isALWAYS CURRENT

RAISE YOUR HAND if

INFRASTRUCTURE

PATTERNS and VIRTUES

HUMILITY

LAZINESS

PARANOIA

SOLIDsingle responsibility principleliskov substitution principleinterface segregation principle

CLOUD PRACTICESareBEST PRACTICES

THANK YOU!

Ben Scofield / @bscofieldhttp://benscofield.comhttp://speakerrate.com/t/7123