Upload
ilya-grigorik
View
21.121
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Multi-threaded servers compete for the global interpreter lock (GIL) and incur the cost of continuous context switching, potential deadlocks, or plain wasted cycles. Asynchronous servers, on the other hand, create a mess of callbacks and errbacks, complicating the code. But, what if, you could get all the benefits of asynchronous programming, while preserving the synchronous look and feel of the code – no threads, no callbacks?
Citation preview
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
No Callbacks, No Threads & Ruby 1.9
Ilya Grigorik@igrigorik
async & co-operative web-servers
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
The slides… Twitter My blog
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
The state of art is not good enough.(we’ve been stuck in the same local minima for several years)
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
require "active_record” ActiveRecord::Base.establish_connection( :adapter => "mysql", :username => "root", :database => "database", :pool => 5) threads = []10.times do |n| threads << Thread.new { ActiveRecord::Base.connection_pool.with_connection do |conn| res = conn.execute("select sleep(1)") end }end threads.each { |t| t.join }
The “Experiment”vanilla everything…
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
require "active_record” ActiveRecord::Base.establish_connection( :adapter => "mysql", :username => "root", :database => "database", :pool => 5) threads = []10.times do |n| threads << Thread.new { ActiveRecord::Base.connection_pool.with_connection do |conn| res = conn.execute("select sleep(1)") end }end threads.each { |t| t.join }
# time ruby activerecord-pool.rb## real 0m10.663s# user 0m0.405s# sys 0m0.201s
5 shared connections
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
BHP
WHP
% power loss
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
> 50% power loss!?
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Drivers
Ruby VM
Network
MySQLMongo
PSQL Couch…
MongrelPassenger
Unicorn
…
Threads
Fibers GIL …
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Drivers
Ruby VM
Network
MySQLMongo
PSQL Couch…
MongrelPassenger
Unicorn
…
Threads
Fibers GIL …
We’re as fast as the slowest component
1
2
3
4
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Concurrency is a myth in Ruby(with a few caveats, of course)
Global Interpreter Lock is a mutual exclusion lock held by a programming language interpreter thread to avoid sharing code that is not thread-safe with other threads.
There is always one GIL for one interpreter process.
http://bit.ly/ruby-gil
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Concurrency is a myth in Rubystill no concurrency in Ruby 1.9
N-M thread pool in Ruby 1.9…Better but still the same problem!
http://bit.ly/ruby-gil
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Concurrency is a myth in Rubystill no concurrency in Ruby 1.9
Nick – tomorrow @ 11:45am
http://bit.ly/ruby-gil
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Avoid locking interpreter threads at all costslet’s say you’re writing an extension…
Blocks entireRuby VM
Not as bad, butavoid it still..
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Will fetch_xyz() block the VM?when was the last time you asked yourself this question?
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
mysql.gem under the hood
require 'rubygems’require 'sequel'
DB = Sequel.connect('mysql://root@localhost/test')
while true DB['select sleep(1)'].select.firstend
22:10:00.218438 mysql_real_query(0x02740000, "select sleep(1)", 15) = 0 <1.001100>22:10:01.241679 mysql_real_query(0x02740000, "select sleep(1)", 15) = 0 <1.000812>
ltrace –ttTg -x mysql_real_query –p ./example.rb
Blocking 1s call!
http://bit.ly/c3Pt3f
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
gem install mysqlwhat you didn’t know…
1. Blocking calls to mysql_real_query2. mysql_real_query requires an OS thread3. Blocking on mysql_real_query blocks the Ruby VM4. Aka, “select sleep(1)” blocks the entire Ruby runtime for 1s
(ouch)
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
mysqlplus.gem under the hood
static VALUE async_query(int argc, VALUE* argv, VALUE obj) { ... send_query( obj, sql ); ... schedule_query( obj, timeout); ... return get_result(obj); }
static void schedule_query(VALUE obj, VALUE timeout) { ... struct timeval tv = { tv_sec: timeout, tv_usec: 0 };
for(;;){ FD_ZERO(&read); FD_SET(m->net.fd, &read);
ret = rb_thread_select(m->net.fd + 1, &read, NULL, NULL, &tv); ... if (m->status == MYSQL_STATUS_READY) break; }}
send query and block
Ruby: select() = C: rb_thread_select()
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
mysqlplus.gem + ruby select
spinning in select
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Step 1: Fix the driversMany of our DB drivers don’t respect the underlying Ruby VM. Don’t blame the VM.
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
WEBRick
Drivers
Ruby VM
Network
Mongrel made Rails viablestill powers a lot of Rails apps today
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Rails rediscovers Apachethe worker/forker model…
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
An exclusive Ruby VM for EACH requestam I the only one who thinks this is terrible?
…
*nix IPC is fast! Woo!
Full Ruby VM
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
An exclusive Ruby VM for EACH requestam I the only one who thinks this is terrible?
Robustness? That sounds like a bug.
“Does not care if your application is thread-safe or not, workers all run within their own isolated address space and only serve one client at a time for maximum robustness.”
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Step 2: consider entire stackThe driver, the web-server, and the networkmust all work together.
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
1. Node imposes the full-stack requirements2. Node imposes async drivers3. Node imposes async frameworks
Surprise: Node is “fast”
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
We can ignore the performanceissues at our own peril
or, we can just fix the problem
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
I’ll take Ruby over JSgem install eventmachine
>
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
EventMachine Reactorconcurrency without thread
EventMachine: The Speed DemonWednesday @ 11:45am – Aman Gupta
while true do timers
network_ioother_io
end
p "Starting"
EM.run do p "Running in EM reactor"end
p ”won’t get here"
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Non-blocking IO requires non-blocking drivers:
AMQP http://github.com/tmm1/amqpMySQLPlus http://github.com/igrigorik/em-mysqlplus Memcached http://github.com/astro/remcached DNS http://github.com/astro/em-dns Redis http://github.com/madsimian/em-redis MongoDB http://github.com/tmm1/rmongo HTTPRequest http://github.com/igrigorik/em-http-request WebSocket http://github.com/igrigorik/em-websocket Amazon S3 http://github.com/peritor/happening
And many others: http://wiki.github.com/eventmachine/eventmachine/protocol-implementations
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
em-mysqlplus: exampleasync MySQL driver
EventMachine.run do conn = EventMachine::MySQL.new(:host => 'localhost') query = conn.query("select sleep(1)") query.callback { |res| p res.all_hashes } query.errback { |res| p res.all_hashes }
puts ”executing…”end
# > ruby em-mysql-test.rb## executing…# [{"sleep(1)"=>"0"}]
gem install em-mysqlplus
callback fired 1s after “executing”
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
em-mysqlplus: under the hoodmysqlplus + reactor loop
non-blocking driverrequire 'mysqlplus'
def connect(opts) conn = connect_socket(opts) EM.watch(conn.socket, EventMachine::MySQLConnection, conn, opts, self)end
def connect_socket(opts) conn = Mysql.init
conn.real_connect(host, user, pass, db, port, socket, ...) conn.reconnect = false connend
EM.watch: reactor will poll & notify
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
em-mysqlplusmysqlplus + reactor loop
Features:
- Maintains C-based mysql gem API- Deferrables for every query with callback & errback- Connection query queue - pile 'em up! - Auto-reconnect on disconnects- Auto-retry on deadlocks
http://github.com/igrigorik/em-mysqlplus
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
and this callback goes to…
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
We can do better than node.jsall the benefits of evented code without the drawbacks
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Ruby 1.9 Fibersand cooperative scheduling
Ruby 1.9 Fibers are a means of creating code blocks which can be paused and resumed by our application (think lightweight threads, minus the thread scheduler and less overhead).
f = Fiber.new { while true do Fiber.yield "Hi” end}
p f.resume # => Hip f.resume # => Hip f.resume # => Hi
Manual / cooperative scheduling!
http://bit.ly/d2hYw0
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Ruby 1.9 Fibersand cooperative scheduling
http://bit.ly/aesXy5
Fibers vs Threads: creation time much lower
Fibers vs Threads: memory usage is much lower
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Untangling Evented Code with Fibershttp://bit.ly/d2hYw0
def query(sql) f = Fiber.current conn = EventMachine::MySQL.new(:host => 'localhost') q = conn.query(sql) c.callback { f.resume(conn) } c.errback { f.resume(conn) } return Fiber.yieldend EventMachine.run do Fiber.new { res = query('select sleep(1)') puts "Results: #{res.fetch_row.first}" }.resumeend
Exception, async!
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Untangling Evented Code with Fibershttp://bit.ly/d2hYw0
def query(sql) f = Fiber.current conn = EventMachine::MySQL.new(:host => 'localhost') q = conn.query(sql) c.callback { f.resume(conn) } c.errback { f.resume(conn) } return Fiber.yieldend EventMachine.run do Fiber.new { res = query('select sleep(1)') puts "Results: #{res.fetch_row.first}" }.resumeend
1. Wrap into a continuation
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Untangling Evented Code with Fibershttp://bit.ly/d2hYw0
def query(sql) f = Fiber.current conn = EventMachine::MySQL.new(:host => 'localhost') q = conn.query(sql) c.callback { f.resume(conn) } c.errback { f.resume(conn) } return Fiber.yieldend EventMachine.run do Fiber.new { res = query('select sleep(1)') puts "Results: #{res.fetch_row.first}" }.resumeend
2. Pause the continuation
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Untangling Evented Code with Fibershttp://bit.ly/d2hYw0
def query(sql) f = Fiber.current conn = EventMachine::MySQL.new(:host => 'localhost') q = conn.query(sql) c.callback { f.resume(conn) } c.errback { f.resume(conn) } return Fiber.yieldend EventMachine.run do Fiber.new { res = query('select sleep(1)') puts "Results: #{res.fetch_row.first}" }.resumeend
3. Resume the continuation
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
em-synchrony: simple evented programmingbest of both worlds…
Good news, you don’t even have to muck around with Fibers!
gem install em-synchrony
- Fiber aware connection pool with sync/async query support- Multi request interface which accepts any callback enabled client - Fibered iterator to allow concurrency control & mixing of sync / async- em-http-request: .get, etc are synchronous, while .aget, etc are async- em-mysqlplus: .query is synchronous, while .aquery is async- remcached: .get, etc, and .multi_* methods are synchronous
http://github.com/igrigorik/em-synchrony
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Untangling Evented Code with Fibershttp://bit.ly/d2hYw0
Async under the hood
require "em-synchrony/em-mysqlplus"
EventMachine.synchrony do db = EventMachine::MySQL.new(host: "localhost")
res = db.query("select sleep(1)") puts res
EventMachine.stopend
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Untangling Evented Code with Fibershttp://bit.ly/d2hYw0
require "em-synchrony/em-mysqlplus"
EventMachine.synchrony do db = EventMachine::MySQL.new(host: "localhost")
res = db.query("select sleep(1)") puts res
EventMachine.stopend
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Drivers
Ruby VM
Network
Fibers
EM-HTTP, EM-MySQL, EM-Jack, etc.
Async-rack
Thin
One VM, full concurrency, network-boundRuby 1.9, Fibers, Thin: in production!
Goliath
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Async Railswith EventMachine & MySQL
development: adapter: em_mysqlplus database: widgets pool: 5 timeout: 5000
git clone git://github.com/igrigorik/em-mysqlplus.gitgit checkout activerecordrake install
database.yml
require 'em-activerecord’require 'rack/fiber_pool'
# Run each request in a Fiberconfig.middleware.use Rack::FiberPoolconfig.threadsafe!
environment.rb
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Async Railswith EventMachine & MySQL
class WidgetsController < ApplicationController def index Widget.find_by_sql("select sleep(1)") render :text => "Oh hai” endend
ab –c 5 –n 10 http://127.0.0.1:3000/widgets
Server Software: thinServer Hostname: 127.0.0.1Server Port: 3000
Document Path: /widgets/Document Length: 6 bytes
Concurrency Level: 5Time taken for tests: 2.210 secondsComplete requests: 10Failed requests: 0Requests per second: 4.53 [#/sec] (mean)
woot! Fiber DB pool at work.
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Not only is it doable… it already works.
git clone git://…./igrigorik/mysqlplusgit checkout activerecordrake install
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
Ruby 1.9 + Rails 3 + new stack
=
Order of magnitude better performance
(aka, enough of a reason to actually switch)
No Callbacks, No Threads & Ruby 1.9 @igrigorikhttp://bit.ly/ruby-stack
What do you think?
The state of art is not good enough, in fact, it’s terrible!
Let’s fix it.
Untangling Evented Code with Ruby Fibers:http://www.igvita.com/2010/03/22/untangling-evented-code-with-ruby-fibers/
Fibers & Cooperative Scheduling in Ruby:http://www.igvita.com/2009/05/13/fibers-cooperative-scheduling-in-ruby/
EM-Synchrony:http://github.com/igrigorik/em-synchrony