Upload
philip-norton
View
631
Download
3
Embed Size (px)
Citation preview
Drupal PerformancePhilip Norton
Blog at #! code (www.hashbangcode.com)
@philipnorton42 on Twitter
Creator of Vlad
Technical Lead at
Philip Norton
Not a sysadmin!
Improving
Performance
Improving
Performance
Performance
Testing
“Performance enhancements without
benchmarks is little more than guesswork.”
Probably heard it at PHPNW...
Setting a benchmark.“How does the site perform?”
Identifying bottlenecks.“Where is the site slow?”
Comparing improvements with the benchmark.“Has that change improved things?”
Gauging the limits of resources.“If the current traffic doubles will the site fall over?”
1
2
3
4
Performance Testing Goals
Google Chrome
More than a web browser.
Lots of profiling tools available.
Measure all kinds of things like raw network speed, page rendering, and JavaScript performance.
YSlow / ShowSlow
YSlow
YSlow is a Firebug plugin developed by Yahoo! It measures factors that govern page speed and gives an overall score.
ShowSlow
ShowSlow can be used to store YSlow results and show difference of scores over time.
Drupal Devel Module
Devel module can be used to render statistics on the page load.
Shows all SQL queries run and how long they took to execute.
Queries highlighted in red are slow and should be addressed.
1
2
3
MySQL Process List
Provides a window on what the MySQL server is doing, and can pinpoint problems.
> SHOW PROCESSLIST;
Keep an eye out for sleeping or lengthy processes or processes that transmit lots of data.
Only provides a snapshot of processes.
MySQL Slow Query Log
Often overlooked but provides a valuable information on queries that are slowing things down.
Set a minimum execution time for queries to be entered into log.
Use EXPLAIN to figure out why queries are slow.
MySQL Slow Query Log
Add the following to my.cnf:
slow-query-log = 1slow-query-log-file = /var/log/mysql/mysql-slow.loglong_query_time = 2log-queries-not-using-indexes
Use mysqldumpslow to analyse the logs.
Xdebug Profiler
Part of the Xdebug tool.
Shows everything that happened during a page request.
Use tools like KCachegrind and Webgrind to view the information and identify bottlenecks.
Very resource heavy so run when needed (never in production!).
1
2
3
4
XHProf
Generates profiling data for PHP.
Performant enough to run on production environment.
Can profile 1 request in 100 or 1000 to get a window on performance.
Use XH Gui to view XHProf results and compare results over time.
1
2
3
4
Siege / JMeter / Loader.io
Benchmark a site by throwing traffic at it.
Find out how much traffic you can send before you start getting errors.
Siege and JMeter are a local tools so all traffic comes from one place.
Loader.io is a distributed service.
1
2
3
4
New Relic
Fully featured performance statistics package for front and back end monitoring.
Has knowledge of Drupal and can identify slow modules/views/hooks.
Lots of information available and shows changes over time.
Paid service and can be a little expensive.
1
2
3
4
Improving Performance
There is no magic bullet!Speed improvements are made through multiple
improvements and usually have diminishing returns.
Drupal Caching
Too obvious? Cache all the things!
Turn on anonymous page caching.
Use Block Cache Alter module to enable/force block level caches.
https://www.drupal.org/project/blockcache_alter
1
2
3
Drupal Caching
Turn on Views query and render caching.
Use Entity Cache module to cache entities.
https://www.drupal.org/project/entitycache
1
2
Boost Module
Creates a static cache of page content.
Will also gzip content, including HTML, CSS, JavaScript, XML.
https://www.drupal.org/project/boost
1
2
CSS / JavaScript Aggregation
Combines CSS and JavaScript files.
Less files being downloaded means:- Less strain on the server.- Quicker rendering times for users.
Drupal also removes comments from files which makes them smaller.
1
2
3
Advanced CSS / JS Aggregation
Easy to install module.
Further reduces JS/CSS footprint by minifying and compressing the files.
Not a substitute for slow JavaScript code.
https://www.drupal.org/project/advagg
1
2
3
Turn off database Logging
Drupal tends to log a lot of user activity.
PHP warnings and notices can bring a site down if this module is active.
(You should really be removing these errors where possible.)
Use syslog to log any errors to file based logs.
1
2
3
StatisticsWrites to the database on every page load.Use an analytics package instead.
PHP Input FilterBypasses most caches. No. Just no.
1
2
Other Bad Modules
Custom Code
Where problems mostly stem from?
Cache where possible.
Try to avoid potentially heavy Drupal hooks like hook_init() and hook_boot().
Profile and experiment, especially with lots of data. e.g. Test with 5,000 or 50,000 nodes instead of 5.
1
2
3
Queues
Doing more than one thing in a page request?
Stick that in a queue and process it later!
Built into Drupal 7, easily extendable.
Should form a key part of any batch process or APIservice you implement.
1
2
3
PHP Micro Optimisations
Single quotes '' vs double quotes "".
Printing stuff using echo vs print.
Pre increment (++$i) vs post increment ($i++).
Creating arrays from strings using explode() vs preg_split().
Micro optimizations are basically pointless*
*unless you are Facebook.
PHP Micro Optimisations
Performance analysis and benchmarking should feed into any code modifications.
Don’t optimise code this way unless you have evidence that you need to.
Don’t sacrifice code maintainability to increase site speed by 0.00001%.
1
2
3
Drupal Fields
Don’t overuse fields.
node_load() and user_load() will pull in all field data (plus revisions) into memory.
50 fields on a node means 50 delete and 50 insert statements on every update.
Try to use shared fields where possible.
1
2
3
4
Cron
Don’t get your users to run cron. Always do it via a crontab.
Use Elysia cron module to split the cron up.
Set cache clearing cron runs to hourly.
https://www.drupal.org/project/elysia_cron
1
2
3
PHP 5.5+
New versions of PHP are faster than ever.
Especially with objects.
You should be upgrading to take advantage of this.
Use PHP-FPM for high performance PHP.
1
2
3
4
PHP Opcode Cache
e.g. APC or Zend Cache.
Caches PHP scripts so they don’t have to be recompiled when run again.
Easy to install and gives a speed increase of up to 30%.
Make sure your cache size is big enough to accommodate the number of files!
1
2
3
4
Add indexes to tables. Slow query log will tell you where indexes are needed.
Add enough memory for MySQL to use (you don’t want lots of read/writes to disk).
Use a master/slave setup - Drupal will use slave server for expensive read only queries (eg. search).
MySQL Performance Tweaks
1
2
3
Apache Performance Tweaks
Turn off KeepAlive.
Set your max clients to a sensible value.- Too high and server crashes when busy.- Too low and you can’t serve traffic correctly.
Remove any modules that you don’t need.
Turn off AllowOverride and include .htaccess files in server configuration.
1
2
3
4
Solr Search
Moves search out of your database server.
Much more efficient and has a bunch of nice features (e.g. facets).
Hard to set up, but services exist to help you.
1
2
3
Memcache / Redis
Store your Drupal cache outside of a database.
Much faster than traditional database servers (but only for this kind of data).
Can be scaled easily.
1
2
3
Split Site Up
Complex sites can be slow. Breaking them into sub-sites allows for better experience.
Most users don’t care about the URL, as long as they get a coherent experience.
Other sites can be hosted elsewhere.
Bakery module provides single sign on.
1
2
3
4
Server Side Caching
i.e. Varnish.
Provides a reverse proxy that caches all page content.
Repeat requests never hit Drupal, Apache, or MySQL and are instead served by Varnish.
Huge speed increase for little effort.
1
2
3
4
Load Balancers
Balances the load across multiple web servers.
More complex to setup, but some providers have good solutions (Rackspace).
Stops Apache being the bottleneck.
Can have some negative effects on Drupal (e.g. file uploads available on all servers).
1
2
3
4
Content Delivery Network (CDN)
Move the files or even the entire site out of your hosting environment.
Pull vs push CDN.
CDN content nodes closer to users mean less distance to travel for request/response.
Make sure you allow authentication traffic through.
1
2
3
4
Throw money at it!
Throw money at it!
Buying monster servers does not necessarily mean performance improvements.
Can sometimes solve the wrong problem.
If you plaster over cracks, the cracks will still be there.
“We’ve spent all this money on servers and the site is still slow.” - An annoyed client
1
2
3
4
To Recap...
Server
PHP 5.5+ with APC
Optimised MySQL
Optimised Apache
Memcache/Redis
Solr
1
2
3
4
5
Drupal
Cache all the things
JS/CSS aggregation
Split complex sites up
Disable expensive modules
1
2
3
4
External
Varnish
Load balancers
CDN
1
2
3
Essentially...
“Making a Drupal site fast is mostly about removing
Drupal from the page request.”
Philip Norton
Failing GracefullyIf at first you don’t succeed, fail!
Drupal Log Levels
Set a level that won’t print error logs to screen.
Maintenance theme
Config item in Drupal’s settings.php file.
Create a nice looking error page that fits in with site theme.
Link out to social media channels.
Prevent Drupal from printing SQL error messages.
1
2
3
4
Varnish error page
If your web server doesn’t respond then Varnish can be made to issue an error page.
Or even redirect traffic to another (static) site.
1
2
Static pages
Don’t be afraid to present users with a static page.
“We’re busy, please come back later” is better than an error page.
With load balanced solutions you can give a percentage of your users a static page and serve the rest correctly.
1
2
3
Case StudyA quick case study about dealing with
peak traffic on awards nights
BAFTA: Peak traffic
High levels of traffic on major awards nights.
1,400% increase from normal traffic levels.
Needed a scalable hosting solution that could be ramped up on selected evenings.
Rackspace cloud chosen as the hosting platform.
1
2
3
4
BAFTA: Peak traffic
Film and TV awards
Games Awards
Nominees Announced
Children’s Awards
1
2
4
BAFTA: Peak traffic
Sites are hosted on a number of web nodes.
Rackspace Load Balancer provides unified front end.
Each web node consists of a Varnish/Apache setup.
1
2
3
BAFTA: Peak traffic
In the run-up to awards night new nodes are provisioned and added to the load balancer.
Caches on each web node are warmed and site is load tested using loader.io service.
After awards night the nodes are scaled down to normal service.
1
2
3
BAFTA: Peak traffic
Rackspace Load Balancer
Web Nodes
SolrNFSMySQL Master/Slave
BAFTA: Peak traffic
Varnish
Apache
Load BalancerEach web node setup as a Varnish/Apache pair.
Varnish configured to produce nice error page if Apache not available.
1
2
BAFTA: Peak traffic
Contingency plans put in place just in case things went wrong.
Error page templates put in place for:- Drupal- Varnish- Rackspace load balancer
Secure login module ready to disable member access to reduce load on database server.
1
2
3
BAFTA: Peak traffic
Team Access on call during the weekend in case of issues.
Rackspace also had a dedicated technician on hand.
Monitoring of servers during awards ceremony in order to adapt to any problems.
1
2
3
BAFTA: Peak traffic
Cloudflare chosen as a CDN.
Caches content externally for anonymous traffic.
Cloudflare module used to clear external caches from within Drupal.
>90% of all traffic served through Cloudflare.
1
2
3
4
BAFTA: Peak traffic
BAFTA: Results
No downtime.
Cloudflare served most of the traffic.
Site stayed responsive throughout the peak traffic levels.
Client happy.
1
2
3
4
BAFTA: Peak traffic
Fin.