Upload
jonatas-paganini
View
434
Download
1
Embed Size (px)
Citation preview
ø downtimeJônatas Davi Paganini
jonatasjonatasdp
8 dev teams
10 deploys / day
RD Station
+2m mail / day
+700 external services
integrations
+N callbacks
Availability is
REQUIRED
99,9%
99,5%
99,0%
98,6%
98,0%
97,0%
99,99%
00:43:00
03:36:00
07:12:00
10:00:00
14:24:00
21:36:00
00:04:32
Availability
99.999???
availability =( total_time - timeout / total_time ) * 100
ø downtime
mindset
NO schedulemaintenance
avoid: all || nothing
build reversible things
build incremental migrations
RubyConf 2012
compatible versions
http://shipit.resultadosdigitais.com.br/blog/migrando-com-zero-downtime
talk is <cheap>!
show me the code!!!
Migration
add_column :people, :full_name, :string
Update data
Person.all.each do |person| person.full_name = "#{person.first_name} #{person.last_name}" person.saveend
350 million updates!!!
WTF time?!
Let’s improve it!
Person.all.each do |person| person.full_name = "#{person.first_name} #{person.last_name}" person.saveend
Select RIGHT attributes!
Update 0.1
Person.select("id,first_name,last_name").each do |person| person.full_name = "#{person.first_name} #{person.last_name}" person.saveend
update_attribute instead of save
Update 0.2
Person.select("id,first_name,last_name").each do |person| person.update_attribute "full_name", "#{person.first_name} #{person.last_name}"end
find in batches
avoid transaction
overhead
Update 0.3
Person.select("id,first_name,last_name").find_in_batches do |people| People.transaction do people.each do |person| person.update_attribute "full_name", "#{person.first_name} #{person.last_name}" end endend
FAILURE resilient
Update 0.4
Person.where(full_name: nil).select("id,first_name,last_name").find_in_batches do |people| People.transaction do people.each do |person| person.update_attribute "full_name", "#{person.first_name} #{person.last_name}" end endend
explore connection pool
Update 0.5
update_sql = "UPDATE people set full_name ="
Person.where(full_name: nil).select("id,first_name,last_name").find_in_batches do |people| People.transaction do people.each(ActiveRecord::Base.connection_config[:pool]) do |person| ActiveRecord::Base.connection_pool.with_connection do |conn| set_full_name = conn.quote("#{person.first_name} #{person.last_name}") conn.execute("#{update_sql} #{set_full_name} where id = #{person.id} ") end end endend
model hook
Model
class Person < ActiveRecord::Base before_save :update_full_name
def update_full_name self.full_name = "#{first_name} #{last_name}" endend
rollout control
Rollout
class Rollout def enabled?(feature, context) redis.sismember(feature, context) rescue Redis::BaseError false end
def enable(feature, context) redis.sadd(feature) end def disable(feature, context) redis.srem(feature) end
private def redis Redis::Namespace.new("rollout", redis: $redis) endend
rollout use
if:
before_save :update_full_name, if: -> { Rollout.enabled? "started_migration", "global" }
deploy steps
smoothlysteps
PR #1
add new column
PR #2
write old & new columns
PR #3
migrate data
PR #4
read from new column
PR #5
remove old column
PR #6
remove rollout code, cache
Conclusion
Embrace migrations ● smoothly● incremental● safe● resilient
Avoid migrations
● all || nothing● ! roll outable features
REFERENCES
blog.codeship.com/rails-migrations-zero-downtime/
shipit.resultadosdigitais.com.br/blog/migrando-com-zero-downtime/
confreaks.tv/videos/railsconf2012-zero-downtime-deploys-for-rails-apps
?
Thanks!jonatasjonatasdp
ideia.meshipit.resultadosdigitais.com.br