Super Fast Gevent Introduction

Super Fast GeventBy Walter Liu

Agenda• Simple Example• About Gevent• About libevent• Monkey patch of gevent• Other notes• About monkey patch

C10K problem and Gevent Performance

Simple Example

About GEvent• Fast event loop based on libevent or libevent2• Light weight execution unit based on greenlets• Use Monkey patch• See monkey patch to know what it works on.

• Simple API• Fast WSGI server

libevent• /dev/poll• kqueue(2)• event ports• POSIX select(2)• Windows select()• poll(2)• epoll(4)

• Diff in different platform

Gevent Monkey Patch• socket (dns=True)• ssl• os.fork()• time.sleep()• select• threading => become co-routine• _thread_local.local

• subprocess• sys• stdin, stdout, stderr

Other Notes• Monkey patch• It’s patching sys.modules, so it’s global patching.• Apply monkey patch before other import. (especially on thread)• Apply monkey patch after fork/multi-process.• Be noted about 3rd party library compatibility

• Be noted about blocking functions in your program.• Gevent with Python web framework• Played uWSGI with gevent. (very good performance)• Tried using it in Django to accelerate crawling.• How about gunicorn?

Monkey patch

Monkey patch example

Good for unit test

Tips• Life time of monkey patch is till the end of process.• You may chose not to monkey unpatch, but it may breaks the

whole program.• Basically, it’s a dirty workaround.• Thus, only use it on unit test or something like gevent.