Embed Size (px)
Citation preview
Using Coroutines to Create Efficient, High-Concurrency Web Applications
Matt Spitzmeebo, inc.
2
What’s a Web Application, Anyway?
Database
Application
Application
3
High-Concurrency Web Applications
Database
Application
Application
4
High-Concurrency Web Applications
• Many requests per second• Optimization opportunities– Hardware cost– Response time– Concurrency– Database impact
5
Meebo Bar
6
Meebo Bar
• 1000+ sites• Quantcast: 197 MM monthly uniques*• LOTS of pageviews• LOTS of ad requests
* http://bit.ly/xAPXx
7
Meebo’s Ad Server
• Given– User features– Available ads
• Objective– Maximize revenue
• P(click)• Price
– Satisfy advertisers• Respect targeting• Smooth campaign delivery
• Complex application• Lots of concurrent requests
8
Sample App: FortuneTeller
9
Sample App: FortuneTeller
• Given– Username– Available fortunes
• Objective– Select fortune for user• JaccardSimilarity(username, fortune)
• username=PyConIsForLovers– “Generosity and perfection are your everlasting
goals.”
10
Hosting FortuneTeller
• Apache CGI• Apache mod_wsgi• Twisted• gevent + gunicorn
11
Hosting FortuneTeller
• Evaluation metrics– Code complexity– Library support– Memory efficiency– Multi-core support
12
Take One: Apache CGI
• One process per request• O/S schedules CPU
13
Take One: Apache CGI
• Advantages– Straightforward, synchronous code– Isolated requests
• Disadvantages– Process overhead– Cold cache
14
EvaluationCode
ComplexityLibrarySupport
MemoryEfficiency
MulticoreSupport
Apache CGI Apache mod_wsgi
Twisted
gevent + gunicorn
15
Performance
• Environment– 4-core VM, 1 GB RAM– Ubuntu Server 10.10– MySQL on host machine– 25ms interface delay
• 1024 requests, X concurrent
16
Performance
1 2 4 8 16 32 64 128
apache_cgi
Concurrent Requests
Med
ian
Resp
onse
Tim
e
17
Take Two: Apache mod_wsgi
• Using mpm_prefork– Worker processes handle requests– One concurrent request per process– Memory cached between requests
• O/S schedules CPU
18
Take Two: Apache mod_wsgi
• Advantages– Straightforward, synchronous code– Cached memory
• Disadvantages– Resource inefficient
• Need working set in each process• Cold cache on restart
– Managing worker count• Too few: 502• Too many: OOM? Database DoS?
19
EvaluationCode
ComplexityLibrarySupport
MemoryEfficiency
MulticoreSupport
Apache CGI Apache mod_wsgi Twisted
gevent + gunicorn
20
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgi
Concurrent Requests
Med
ian
Resp
onse
Tim
e
21
Take Three: Twisted
• Asynchronous framework– Events and callbacks– Twisted orchestrates context switches
• Twisted server– Single event loop– Concurrent requests
22
Quick Break: Event Loopss = socket.socket(…)s.setblocking(ISBLOCKING)s.connect((HOST, PORT))greeting = s.recv(1024)s.close()
• Blocking– Wait for data
• Nonblocking– Initiate, return immediately
• Data (if available)• Exception: “I’m not done yet”
– Requires more plumbing
23
Process events that are ready (select/poll).
Quick Break: Event Loops
• Nonblocking sockets in an event loop
Events
f(x):s = NonBlockingSocket(…)greeting = s.recv(1024)print x, “|”, greeting
fd=18, fp=f, {x: 80}
Call recv().
1.
2.Create context, add to the event loop.3.
4.
Return to context when data is ready.5.
“80 | Hello from socket s!”6.
fd=5, fp=g, {s: ‘hi’, a: 5}
fd=2, fp=f, {x: 8080}
fd=3, fp=myfunc, {}
fd=18, fp=f, {x: 80}
fd=3, fp=myfunc, {}
fd=5, fp=g, {s: ‘hi’, a: 5}
fd=18, fp=f, {x: 80}
24
Take Three: Twisted
• Asynchronous framework– Events and callbacks– Twisted orchestrates context switches
• Twisted server– Single event loop– Concurrent requests
25
Take Three: Twisted
• Advantages– Shared memory– User space context switches
• Disadvantages– Develop asynchronously
• Stuck in the framework
– Asynchronous libraries• No I/O in C
– Unfair scheduling– Using multiple cores
26
EvaluationCode
ComplexityLibrarySupport
MemoryEfficiency
MulticoreSupport
Apache CGI Apache mod_wsgi Twisted gevent + gunicorn
27
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwisted
Concurrent Requests
Med
ian
Resp
onse
Tim
e
28
Take Four: gevent + gunicorn
• gevent– Networking library• Uses event loop• Synchronous API
– Synchronous code running asynchronously• Monkey patching
– Rewrites standard modules
• Coroutines for function context– Lightweight threads, no stack– greenlet implementation
29
Take Four: gevent + gunicorn
• gunicorn (“Green Unicorn”)– Lightweight WSGI server– Multiple worker processes• Share queued requests
– gevent support
30
Take Four: gevent + gunicorn
• Advantages– Best of both worlds!
• mod_wsgi– Straightforward, synchronous code– No framework, just python– Multicore support
• Twisted– Shared memory– User space context switches
• Disadvantages– Pure-python libraries– Unfair scheduling
31
EvaluationCode
ComplexityLibrarySupport
MemoryEfficiency
MulticoreSupport
Apache CGI Apache mod_wsgi Twisted gevent + gunicorn
32
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwistedgunicorn_1
Concurrent Requests
Med
ian
Resp
onse
Tim
e
33
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwistedgunicorn_1gunicorn_2
Concurrent Requests
Med
ian
Resp
onse
Tim
e
34
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwistedgunicorn_1gunicorn_2gunicorn_4
Concurrent Requests
Med
ian
Resp
onse
Tim
e
35
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwistedgunicorn_1gunicorn_2gunicorn_4gunicorn_6
Concurrent Requests
Med
ian
Resp
onse
Tim
e
36
Performance
1 2 4 8 16 32 64 128
apache_cgimod_wsgitwistedgunicorn_1gunicorn_2gunicorn_4gunicorn_6gunicorn_8
Concurrent Requests
Med
ian
Resp
onse
Tim
e
37
“Evented” Development
• Synchronous code still runs asynchronously– Requests aren’t independent
• Things to keep in mind– Duplicate work– Socket caching– CPU hogging
38
gunicorn + gevent in Production• Managing gunicorn
– greins• Randall Leeds (tilgovi): github/meebo/greins
– Multiple apps• URL routing
– Server hooks• Worker launch• Pre/post requests
– Daemon interface• Debugging gevent
– gevent-profiler• Shaun Lindsay (srlindsay): github/meebo/gevent-profiler
– Execution trace– Time spent
39
Load-tested, unicorn-approved!
• Blocking code is simple• Nonblocking code is efficient• gevent + gunicorn– Simple – Efficient– Reliable
40
Load-tested, unicorn-approved!
41
Thanks!
