Upload
yc-ling
View
538
Download
3
Embed Size (px)
Citation preview
淺談Ruby Process Fork應用
大貓 (YC Ling, @miaout17)
Applica@on Concurrency
• Event-‐Driven I/O (Ex. Eventmachine, goliath) – Everything need to be non-‐blocking.
• Worker Threads (Ex. Rainbows with proper config) – Everything need to be thread-‐safe.
• Worker Processes (Ex. unicorn) – OS process is expensive – Reality: You are using a lot of 3rd party libraries, some of them are blocking and not thread-‐safe
– S@ll mainstream today – I don’t like it very much, but it just works
Worker Processes Memory Usage
Process 1
80MB
Worker Processes Memory Usage
Process 1
80MB
Process 2
80MB
Process 3
80MB
Process 4
80MB
Process 5
80MB
Process 6
80MB
Process 7
80MB
Process 8
80MB
Forked Worker Processes (Ex. Unicorn) Memory Usage: Copy on Write
Process 1
Process 2
Process 3
Process 4
Process 5
Process 6
Process 7
Process 8
Master Process
Normal Memory Page (CoW)Read-‐Only Page
God Monit
Worker Processes Startup Time
Process 1 Process 2 Process 3 Process 4 Process 5 Process 6 Process 7 Process 8 Time
• Assuming 2 cores • The 8 processes could be started simultaneously and the
startup will be execute concurrently with content switching.
Forked Worker Processes (Ex. Unicorn) Startup Time
Process 1 Process 2 Process 3 Process 4 Process 5 Process 6 Process 7 Process 8 Time
Master Process
• Load the applica@on ONCE • Fork is FAST
Loading environment
Forking
Resque
Master Process
Worker Process
• The master process loads the environment. • Whenever get a new job, fork a worker process. • Whenever the job is done, the worker process exits. • For isola@on: Resque assumes chaos
Got a job!
Fork Process
Job done Worker process terminates
50 resque workers?
• Don’t do this: – COUNT=50 QUEUE=* rake resque:workers
• 80MB*50 = 4GB
Master Process
80 MB
Master Process
80 MB
Master Process
80 MB
Master Process
80 MB
Master Process
80 MB
Master Process
80 MB
…
Worker Process
Worker Process
Worker Process
resque-‐pool
• hgps://github.com/nevans/resque-‐pool
Master Process
Master Process
Master Process
Master Process
Master Process
Master Process
Pool Manager
Fork
Worker Process
Worker Process
Worker Process
Fork
Summary
• Process forking in Ruby – For isola@on
• Prevent process status corrup@on • See also: spork-‐rails
– For concurrency • Faster startup • Smaller memory footprint