0-60 with Goliath: Building High Performance Ruby Web-Services

0-60 with Goliath @igrigorik

0-60 with Goliath

Ilya Grigorik@igrigorik

building high-performance (Ruby) web services


HTTP Quizthis is not a trick question…

conn = EM::HttpRequest.new('http://gogaruco.com')

r1 = conn.get :path => "speakers.html", :keepalive => true # 250 msr2 = conn.get :path => "schedule.html" # 300 ms

# wait until done …

Total execution time is:

(a) 250 ms(b) 300 ms(c) 550 ms

~ 65% truthiness~ 25% truthiness *~ 10% truthiness **

Answer:

All of the above!


“SPDY? We can’t even get HTTP right…”the why, how, and how-to of Goliath


Client

HTTP 1.0RFC 1945 (1996)

Server

20 msTCP handshake

HTTP Request• headers, body

Multi-part body (*)

Terminate connection

40 msprocessing

+ 40ms TCP setup (network)+ 20ms request (network) + 40ms processing + 20ms response (network)

66% of time in network overhead


Benchmark client RTT, not just the server processing time

A public service announcement…


HTTP 1.1RFC 2616 (1999)

Keep-alive• Re-use open connection• No multiplexing, serial• Default to “on”

Pipelining• No multiplexing• Parallel requests


Keep-aliveRFC 2616 (1999)

+ 40ms TCP setup (network)

+ 20ms request (network) + 40ms processing + 20ms response (network)

x 40ms TCP setup (network)


200ms for two requests

Small win over HTTP 1.0

* One gotcha…


Keep-aliveRFC 2616 (1999)

Connection: close < ugh!

Net:HTTP


PipeliningRFC 2616 (1999)

+ 40ms TCP setup (network)


60% of time in network overhead

120ms for two requests – 50% improvement!


Pipelining Quiz RFC 2616 (1999)

Connection setup: 50ms

Request 1: 300msRequest 2: 250ms

Total time:

(a) ~250 ms(b) ~300 ms(c) ~350 ms(d) ~600 ms


Making HTTP Pipelining Usable on the Open Web

http://tools.ietf.org/html/draft-nottingham-http-pipeline-01

There is just one small gotcha…




HTTP in the wildit’s a sad state of affairs

conn = EM::HttpRequest.new('http://gogaruco.com')

r1 = conn.get :path => "speakers.html", :keepalive => true # 250 msr2 = conn.get :path => "schedule.html" # 300 ms

Total execution time is:

(a) 250 ms(b) 300 ms(c) 550 ms

~ 65% truthiness~ 25% truthiness *~ 10% truthiness **

Keep-alive what? HTTP 1.0!

Good: Keep-alive + PipeliningBad: Keep-alive + Garbage

“I’m confused”

Keep-alive: mostly works – yay!Pipelining: disabled (except in Opera)


HTTP can be a high-performance transportGoliath is our attempt to make it work


- “Social Analytics”- Rails frontend- Ruby backend- 95%+ of traffic via API’s

Brief HistoryGoliath @ PostRank

+ +

- Goliath == v3 API stack- Open source in 2011- Growing community


Rails

HTTP API

SQL

SQLSQLSQL

HTTP API

SQLSQLSolr

HTTP API

…

HTTP API …

• Separate logical & physical services• Easy to tune, easy to maintain, easy to “scale”• Stable code, fault-tolerance

• Higher upfront ops cost• Lots of cross-service communication

PRO

CON



• Single responsibility web-services

• Async HTTP response streaming + progressive notifications• Async HTTP request streaming + progressive notifications

• Multiple requests within the same VM

• Keep-alive support• Pipelining support

• Ruby API & “X-Ruby friendly”• Easy to maintain & test

… full HTTP 1.1 support

www.goliath.io

… lower ops costs

… Ruby polyglot!

http://www.goliath.io/


Network

EventMachine

(streaming) HTTP Parser

Middleware Routing

(optional) Fibers

Client API

Ruby, JRuby, Rubinius …

async-rack

“Sync API”

GoliathOptimize bottom up + minimal client API

optional async

HTTP 1.1 0.3 ms


Goliath: Hands on…


require 'goliath'

class Hello < Goliath::API

def response(env) [200, {}, "Hello World"] end

end

$> ruby hello.rb -sv –p 8000 –e production

Hello WorldSimple Goliath server


class Hello < Goliath::API

use Goliath::Rack::Params use Goliath::Rack::JSONP use Goliath::Rack::Validation::RequestMethod, %w(GET) use Goliath::Rack::Validation::RequiredParam, {:key => 'echo'} def response(env) [200, {}, {pong: params['echo’]}] end end

MiddlewareNo rackup file


class Bonjour < Goliath::API def response(env) [200, {}, "bonjour!"] endend

class RackRoutes < Goliath::API

map '/version' do run Proc.new { |env| [200, {}, ["Version 0.1"]] } end

get "/bonjour", Bonjour

not_found('/') do # run Proc. new { ... } end

end

Routingsimple, but powerful


class AsyncUpload < Goliath::API

def on_headers(env, headers) env.logger.info 'received headers: ' + headers end

def on_body(env, data) env.logger.info 'received data chunk: ' + data end

def on_close(env) env.logger.info 'closing connection' end

def response(env) # called when request processing is complete end end Async Request Processing

don’t need to wait for the full request…


class Stream < Goliath::API

def response(env) pt = EM.add_periodic_timer(1) { env.stream_send("hello") } EM.add_timer(10) do pt.cancel

env.stream_send("goodbye!") env.stream_close end

streaming_response 202, {'X-Stream' => 'Goliath’} end end

Async/Streaming Responsedon’t need to render full response…


class Websocket < Goliath::WebSocket def on_open(env) env.logger.info ”WebSocket opened” end

def on_message(env, msg) env.logger.info ”WebSocket message: #{msg}” end

def on_close(env) env.logger.info ”WebSocket closed” end

def on_error(env, error) env.logger.error error end end

Web-Socketssimple backend extension

* Goliath::SPDY


Ruby 1.9: Fibers


def response(env) conn = EM::HttpRequest.new(’http://google.com/') r1 = conn.get :query => {:q => 'gogaruco'}

r1.callback do r2 = conn.get :query => {:q => 'rubyconf'} r2.callback { ... } r2.errback { ... } end r2.errback { ... }

streaming_response 200, {} end Async fun

{ { {} } } …


def response(env) conn = EM::HttpRequest.new(’http://google.com/') r1 = conn.get :query => {:q => 'gogaruco'} if r1.error? r2 = conn.get :query => {:q => 'rubyconf'} else # ... end

[200, {}, r2.response]end

Async + FibersRuby gives us a choice


EM-Synchrony:

multi = EventMachine::Synchrony::Multi.new

multi.add :a, db.aquery("select sleep(1)")multi.add :b, db.aquery("select sleep(1)")

res = multi.perform

• em-http-request• em-memcached• em-mongo• em-jack• mysql2• redis• …

• ConnectionPool• MultiRequest• Iterators• Inline async• TCPSocket• …

http://github.com/igrigorik/em-synchrony




describe HttpLog do it 'forwards to our API server' do with_api(HttpLog, api_options) do |api|

get_request({}, err) do |c| c.response_header.status.should == 200 c.response_header[’X-Header'].should == 'Header' c.response.should == 'Hello from Responder' end end end end

Integration Testingsimple end-to-end testing


thoughts, questions?now or later: [email protected]

Goliath• http://github.com/postrank-labs/goliath • http://goliath.io/

• Google Group: http://groups.google.com/group/goliath-io • Examples: GIT repo / examples

mailto:[email protected]

http://github.com/postrank-labs/goliath

http://goliath.io/

http://groups.google.com/group/goliath-io

http://groups.google.com/group/goliath-io

Technology

0-60 with Goliath: Building High Performance Ruby Web-Services