Upload
menno-van-der-sman
View
560
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Presentation given by Menno van der Sman from Wakoopa at Ams.rb
Citation preview
Lessons Learnedwhile building
How To Run Your Big App on OneAndAHalf Server
or...
Menno van der SmanDeveloper at Wakoopa
since May 2007
Wakoopa is about software
MacPC
Linux
200GB
Does it scale?
Scaling the front is easy
Scaling the back is harder
Limits spawn creativity
Serversat Railsmachine
16GB RAM8 Cores
5GB RAM4 Cores
dedicatedstaging
productiondatabase
sharedarchiving
Keeping Your Database Happy
or...
Queuing
DelayedJob for anything else
processing multiple items at a time
efficient cache update
Smart queries
INSERT INTO hourly_usage (...) VALUES (...) ON DUPLICATE KEY UPDATE active_seconds = active_seconds + ..., idle_seconds = idle_seconds + ...
UPDATE developers SET active_seconds = active_seconds + ...
Allows for concurrency
Aim for a Small Rowsize
choose a correct datatype
pick the right indexes
sometimes `id` shouldn’t be the PK
Allows efficient PK
Original implementation by Dr NicMaintained by Darrin Holst
http://github.com/drnic/composite_primary_keys
class HourlyUsage < ActiveRecord::Base set_primary_keys :user_id, :software_id, :used_at ...end
composite_primary_keys
SELECT ... FROM hourly_usage WHERE used_at < '...' INTO OUTFILE '/path/to/file'
LOAD DATA LOCAL INFILE '/path/to/file' REPLACE INTO TABLE hourly_usage_storage
avoids long locks
Archive old datathat’s not frequently used
Alternative solutionPartitioning
CREATE TABLE hourly_usage_storage ( ... )PARTITION BY RANGE (software_id) ( PARTITION p0 VALUES LESS THAN (...), PARTITION p1 VALUES LESS THAN (...), PARTITION p2 VALUES LESS THAN (...), PARTITION p3 VALUES LESS THAN (...), .... PARTITION pn VALUES LESS THAN MAXVALUE);
Didn’t work for us
Makes ‘WHERE software_id = ...’ very fast
Rails 2.3Apache
PassengerREE 1.8.7
MySQL 5.0Memcached
SphinxAmazon EC2
Other Techologies
Our REE settings
#!/bin/shRUBY_HEAP_MIN_SLOTS=500000RUBY_HEAP_SLOTS_INCREMENT=250000RUBY_HEAP_SLOTS_GROWTH_FACTOR=1RUBY_GC_MALLOC_LIMIT=50000000exec "/opt/ruby-enterprise/bin/ruby" "$@"
Questions?or suggestions?