Drupal Optimization

Оптимизируем DrupalAleksey Khodakovskiy

Adyax 38 rue N-D De Nazareth, 75003 Paris

Ligne directe : +33 1 76 77 34 65Standard : +33 1 76 77 34 60 Mobile : +33 6 37 47 45 83Email : [email protected]

Оптимизируем Drupal. Aleksey Khodakovskiy DrupalCamp Kyiv 2010Оптимизируем Drupal. Aleksey Khodakovskiy DrupalCamp Kyiv 2010

David Strauss

Predicting peak trafficTraffic over the day can be highly irregular. To plan for peak loads, design as if all traffic were as heavy as the peak hour of load in a typical month — and then plan for some growth.

Wed 2010-06-09

Оптимизируем Drupal. Aleksey Khodakovskiy DrupalCamp Kyiv 2010Оптимизируем Drupal. Aleksey Khodakovskiy DrupalCamp Kyiv 2010 Оптимизируем Drupal. Aleksey Khodakovskiy DrupalCamp Kyiv 2010

David Strauss

3%

10%

40%

50%

30%

100%

70%

20%

Analyzing hit distribution

Anony

mous

Authenticated

Dynamic

Pages

Static Content Human

Web

CrawlerNo Special Treatm

ent

7%

“Pay Wall” Bypass

Wed 2010-06-09


David Strauss

Throughput vs. Delivery MethodsGreen

(Static)

Yellow(Dynamic, Cacheable)

Red(Dynamic)

Content Delivery Network

Reverse Proxy Cache

PHP + APC + memcached

PHP + APC

PHP (No APC)

●●●●●●●●●● ✖ ✖

●●●●●●●● ●●●●●●● ✖

●●●● ●●● ●●●

●●●● ●● ●●

●●●● ● ●

1

Delivered by Apache without PHP

1

1

1

More dots = More throughput Some actually can do this.2

2

10 req/s

5000 req/s

Wed 2010-06-09


David Strauss

Layering: Less Traffic at Each Step

CDN

Load Balancer

Reverse Proxy Cache

Application Server

Database

Traffic

Your Datacenter

DNS Round Robin

Wed 2010-06-09


David Strauss

Offload from the master database

Search

Memory Cache

SlaveDatabase

Your master database is the single greatest limitation on scalability.

Application Server

MasterDatabase

Wed 2010-06-09


David Strauss

Relative importance

Processors/Cores Memory Disk Speed

Reverse Proxy Cache

Web Server

Database Server

Monitoring

●● ●●● ●●

●●●●● ●● ●

●●● ●●●● ●●●●

● ● ●

Wed 2010-06-09


David Strauss

All of your servers

‣ 64-bit: no excuse to use anything less in 2010

‣ RHEL/CentOS and Ubuntu have the broadest adoption for large-scale LAMP

‣ But pick one, and stick with it for development, staging, and production

‣ Some disk redundancy: rebuilding a serveris time-consuming unless you’re very automated

Wed 2010-06-09


David Strauss

Reverse proxy caches‣ Varnish and nginx have modern architecture and

broad adoption

‣ Sites often front Varnish with nginxfor gzip and/or SSL

‣ Squid and Traffic Server are clunkybut reliable alternatives

Save Your Money

CPU

1 GB base system+ 3 GB for caching+

MemorySlow

+ Small+ Redundant

Disk

+= 5000 req/s

Wed 2010-06-09


David Strauss

Web servers‣ Apache 2.2 + mod_php + memcached‣ FastCGI is a bad idea‣ Memory improvements are redundant w/ Varnish‣ Higher latency + less efficient with APC opcode

‣ Check the memory your app takes per process‣ Tune MaxClients to around 25 × cores

Max out cores

(but prefer fastcores to density)

CPU1 GB base system

+ 1 GB memcached+ 25 × cores × per-

process app memory+

MemorySlow

+ Small+ Redundant

Disk

+= 100 req/s

Wed 2010-06-09


David Strauss

Database servers‣ Insist on MySQL 5.1+ and InnoDB‣ Consider Percona builds and (eventually) MariaDB

‣ Every Apache process generally needs at least one connection available, and leave some headroom

‣ Tune the InnoDB buffer pool to at least half of RAM

No more than 8-12

cores

CPUAs much as you can

afford (even RAM not used by MySQL caches

disk content)+

MemoryFast

+ Large+ Redundant

Disk

+= 3000 queries/s

Wed 2010-06-09


David Strauss

Assembling the numbers‣ Start with an architecture providing redundancy.

‣ Two servers, each running the whole stack

‣ Increase the number of proxy caches based on anonymous and search engine traffic.

‣ Increase the number of web servers based on authenticated traffic.

‣ Databases are harder to predict, but large sites should run them on at least two separate boxes with replication.

Wed 2010-06-09


David Strauss

Offline processing‣ Gearman

‣ Primarily asynchronous job manager

‣ Hadoop

‣ MapReduce framework

‣ Traditional message queues

‣ ActiveMQ + Stomp is easy from PHP

‣ Allows you to build your own job manager

Wed 2010-06-09


Edge-side includes‣ Blocks of HTML are

integrated into the page at the edge layer.

‣ Non-primary page content often occupies >50% of PHP execution time.

‣ Decouples block and page cache lifetimes

<html><body>

<esi:include href=“http://drupal.org/block/views/3” /></body></html>

<div>My block HTML.

</div>

<html><body>

<div>My block HTML.

</div></body></html>

ESI Processor(Varnish, Akamai, other)

Wed 2010-06-09


David Strauss

HipHop PHP‣ Compiles PHP to a C++-based binary

‣ Integrated HTTP server

‣ Supports a subset of PHP and extensions

‣ Requires an organizational commitment to building, testing, and deploying on HipHop

‣ Scott MacVicar has a presentation on HipHop later today at 16:00.

Wed 2010-06-09


Оптимизируем Друпал (Pressflow):

https://launchpad.net/pressflowчто уже разработано и внедрено:* Lazy session creation (no sessions for anonymous users)* Fast access checks (access denied slow query)* Database repli cation (master/slave)* External page cache (reverse proxy support)* Fast path alias detection (drupal_lookup_path() optimization)* Native JSON encodingв скором будущем будет внедрено: https://blueprints.launchpad.net/pressflow* Path caching* Distributed menu tree generation and caching* HipHop PHP


Сравнение Pressflow с разными версиями Drupal:

https://wiki.fourkitchens.com/display/PF/Comparison+-+Pressflow+versus+Drupal


Caching: Modules that make Drupal scale:

http://groups.drupal.org/node/21897


Tuning:APC: https://wiki.fourkitchens.com/display/PF/Tuning+APC

DB:* Indexes: http://groups.drupal.org/node/57213via module: http://drupal.org/project/dbtuner* MySQL оптимизация: http://drupal.org/node/85768* MySQL for Drupal: http://drupal.org/node/51263* MyISAM -> InnoDB if SELECTs are blocked by INSERTs/UPDATEs* partitioning, memory tables


Front end optimisation modules:http://groups.drupal.org/node/57563http://drupal.org/node/266179 -> Comparison of CSS Preprocessors

Решения:Boost + Nginx (может быть использовам вместо Apache) для анонимного трафика

Authcach: + Memcached для авторизированного трафика + Boost для анонимного трафика


=================================All about optimization: http://drupal.org/node/326504=================================Drupal Group: High performance: http://groups.drupal.org/high-performance

при подготовке доклада использованы материалы, представленные David Strauss на DrupalCamp Stockholm (May 19, 2010).

--------------------------------------------------Aleksey KhodakovskiyAdyax 38 rue N-D De Nazareth, 75003 ParisLigne directe : +33 1 76 77 34 65Standard : +33 1 76 77 34 60 Mobile : +33 6 37 47 45 83Email : [email protected]

Technology

Drupal Optimization