Building High-Scalable Web Applications Using Q4M and Flare

Embed Size (px)


Q4MとFlareを使って スケーラブルなサービスを作る!Building High-Scalable Web ApplicationsUsing Q4M and Flare

Text of Building High-Scalable Web Applications Using Q4M and Flare

  • 1. Q4M FlareBuilding High-Scalable Web Applications Using Q4M and Flare Fillmoreadvisory,Inc.Yusuke Hata Fillmoreadvisory,Inc. 2009.9.5

2. Fillmore AdvisoryAbout me Yusuke Hata () blog: Committer in for PHP frameworkdevelopment DI container and AOP Framework: S2Container.PHP5 DAO (Data-Access-Object) ORM Framework: S2Dao.PHP5 Port to PHP5 from Java Working at Fillmore advisory since 2008 ( ) published from Shoeisha in Jan 2009Fillmoreadvisory,Inc. 2009.9.5 3. Fillmore Advisory vizooWe released vizooh t t p : / / w w w. v i s u a l z o o . c o m / Fillmoreadvisory,Inc. 2009.9.5 4. Fillmore AdvisoryAbout vizoo & YouTube PHP5.2.x + symfony Flash (flex) Fillmoreadvisory,Inc. 2009.9.5 5. Fillmore Advisory vizooQ4MFlare Talk about:Q4M and Flare Using in vizoo Fillmoreadvisory,Inc. 2009.9.5 6. Fillmore AdvisoryAgenda Q4MFlare About Q4M and Flare Web() How to improve: High-load to Web Applications Web () How to improve: Difficult to scaling in Web Applications Symfony Implementation in Symfony Q4MFlare (few) Advanced usage of Q4M and Flare Fillmoreadvisory,Inc. 2009.9.5 7. Fillmore AdvisoryQ4M Message Queue for MySQL MySQL MySQL Pluggable Storage Engine MySQL 5.1 Plugin API MySQL Fillmoreadvisory,Inc. 2009.9.5 8. Fillmore AdvisorySQL Queue Q4M SQL Table CREATE TABLEMessageCREATE TABLE phpcon_queue (param_a int unsigned not null,param_b varchar(200), Messagepriority tinyint not null) ENGINE=queue;MySQL Q4Mqueue Fillmoreadvisory,Inc.2009.9.5 9. Fillmore AdvisorySQLQueue QueueINSERT INSERT INTO phpcon_queue (enqueue param_a, param_b, priority) VALUES (param_aparam_bpriority 2009, hello, 92009 hello9);2009 world5INSERT INTO phpcon_queue ( param_a, param_b, priority) VALUES ( enqueue 2009, world, 5);Fillmoreadvisory,Inc.2009.9.5 10. Fillmore Advisory SQL Queue Queue Q4M API QueueSELECT SELECT queue_wait(phpcon_queue, 2); param_a param_b prioritySELECT * FROM phpcon_queue;2009hello 9queue_waitparam_aparam_bpriority2009world 52009 hello9 queue_endparam_a param_b prioritySELECT queue_end();2009world 5 Fillmoreadvisory,Inc.2009.9.5 11. Fillmore AdvisorySQL Queue queue_wait( )SELECT queue_wait(phpcon_queue, 2); queue_endSELECT queue_end(); queue_abort SELECT queue_abort(); Fillmoreadvisory,Inc. 2009.9.5 12. Fillmore AdvisoryQ4Minsert (enqueue)task1 insert (enqueue)task2Q4Mtask3 insert (enqueue)task4:: Fillmoreadvisory,Inc. 2009.9.5 13. Fillmore AdvisoryQ4M queue_waittask3 hit jobtask1 select task4 queue_end:Q4M : queue_waitjobtask2 queue_wait job Fillmoreadvisory,Inc. 2009.9.5 14. Fillmore Advisory MySQL 5.1 MySQL API API OKFillmoreadvisory,Inc. 2009.9.5 15. Fillmore Advisory Flare Memcached compatible KVS (key-value storage) Persistent storage (Tokyo Cabinet) Data Replication Dynamic re-construction Node monitoring GREE,Inc. 2009.9.5 16. Fillmore Advisorymaster serverindex server checking node status switch master node monitoring switch proxy telnetslave serversFillmoreadvisory,Inc. 2009.9.5 17. Fillmore AdvisoryMemcachedmemcache message setgetflaredappsFillmoreadvisory,Inc. 2009.9.5 18. Fillmore AdvisoryWeb Fillmoreadvisory,Inc. 2009.9.5 19. Fillmore Advisory Web Database read/writewritesreadsApp ServersDatabase Fillmoreadvisory,Inc. 2009.9.5 20. Fillmore Advisory Web Database read/writewrites readsDatabaseApp ServersFillmoreadvisory,Inc. 2009.9.5 21. Fillmore Advisory Web : Master / Slave MasterwritesreplicationreadsSlaveFillmoreadvisory,Inc. 2009.9.5 22. Fillmore Advisory Web DB requestwrite : : waits... : : responsereply synchronous!Fillmoreadvisory,Inc. 2009.9.5 23. Fillmore AdvisoryWeb : Q4M Q4Menqueueworkers write write asynchronous!Fillmoreadvisory,Inc.2009.9.5 24. Fillmore Advisory Web WebApp serverSMTP Fillmoreadvisory,Inc. 2009.9.5 25. Fillmore Advisory Web: database mail data app server every 5 min cron backendSMTP Fillmoreadvisory,Inc.2009.9.5 26. Fillmore AdvisoryWeb: event Q4M notify send mail backend request Q4Mevent notify excel download Q4M backend Fillmoreadvisory,Inc.2009.9.5 27. Fillmore Advisory Web SQLheavy queries...Slaveapp server Fillmoreadvisory,Inc. 2009.9.5 28. Fillmore Advisory Web: backend every 6 hours summary readreplication read readSlaveapp server Fillmoreadvisory,Inc. 2009.9.5 29. Fillmore Advisory Web: Q4M read only write only summary insert/update once row backendenqueue notify eventQ4Mapp serverFillmoreadvisory,Inc. 2009.9.5 30. Fillmore AdvisoryWeb Fillmoreadvisory,Inc. 2009.9.5 31. Fillmore AdvisoryWeb.... Session! Fillmoreadvisory,Inc. 2009.9.5 32. Fillmore AdvisoryWeb master/slavemaster slave DB read/write1 Fillmoreadvisory,Inc. 2009.9.5 33. Fillmore AdvisoryWeb Flareset xxxx Flare storage (protocol : memcache)Masterreplicationget xxxx Slave (protocol : memcache)Fillmoreadvisory,Inc. 2009.9.5 34. Fillmore AdvisoryWeb Flare flare-1000 memcache-1000db-1000db-1000 50000 100000 150000 200000Fillmoreadvisory,Inc. 2009.9.5 35. Fillmore AdvisoryWeb Flare(tx %) 100 1000 10000 (session) 10080604020 0 db memcache flare Fillmoreadvisory,Inc. 2009.9.5 36. Fillmore Advisory Symfony Fillmoreadvisory,Inc. 2009.9.5 37. Fillmore Advisory Q4M symfony task