Upload
walter-liu
View
101
Download
0
Tags:
Embed Size (px)
Citation preview
Super Fast GeventBy Walter Liu
Agenda• Simple Example• About Gevent• About libevent• Monkey patch of gevent• Other notes• About monkey patch
C10K problem and Gevent Performance
Simple Example
About GEvent• Fast event loop based on libevent or libevent2• Light weight execution unit based on greenlets• Use Monkey patch• See monkey patch to know what it works on.
• Simple API• Fast WSGI server
libevent• /dev/poll• kqueue(2)• event ports• POSIX select(2)• Windows select()• poll(2)• epoll(4)
• Diff in different platform
Gevent Monkey Patch• socket (dns=True)• ssl• os.fork()• time.sleep()• select• threading => become co-routine• _thread_local.local
• subprocess• sys• stdin, stdout, stderr
Other Notes• Monkey patch• It’s patching sys.modules, so it’s global patching.• Apply monkey patch before other import. (especially on thread)• Apply monkey patch after fork/multi-process.• Be noted about 3rd party library compatibility
• Be noted about blocking functions in your program.• Gevent with Python web framework• Played uWSGI with gevent. (very good performance)• Tried using it in Django to accelerate crawling.• How about gunicorn?
Monkey patch
Monkey patch example
Good for unit test
Tips• Life time of monkey patch is till the end of process.• You may chose not to monkey unpatch, but it may breaks the
whole program.• Basically, it’s a dirty workaround.• Thus, only use it on unit test or something like gevent.