Running a High Performance LAMP stack on a $20 Virtual

Server

Friday, July 20, 12

www.percona.com

Simplified Uptime

Started a side-business selling customized hosting to small e-commerce and other web sites

Spent a lot of time optimizing for RAM utilizationHad several sites appear in places like:

Digg -- Patterntap.comMakezine -- Simplifiedbuilding.com

Friday, July 20, 12

www.percona.com

The framework

Debian stable + backports.org + dotdebPuppetized configurationApache 2.2, PHP 4.x, MySQL 5.1.xx, Innodb pluginRedundant VPS providers/regions for Business

continuity goals

Friday, July 20, 12

www.percona.com

Tuning Goals

Allow users to connect with Keep-aliveServe static files efficientlyPrevent swapping on the serverTune the server to prevent self-DOSWork with most* open source PHP apps

Friday, July 20, 12

Serving Static Files

Friday, July 20, 12

www.percona.com

Standard forked Apache + mod_php

Single process per connection served

Expensive for memory utilizationPros:

“Just works” with most OSS.htaccess and php ini_set

ConsCan’t easily use keep-aliveNot going to scale

Our Server

MySQL

Clientsport 80

port 3306

ApacheApacheApache

Friday, July 20, 12

www.percona.com

Static Files in Front

End users connect to lighttpd or nginx

Match static files in config by filename (*.jpg|*.gif|*.png|*.cs|*.js) and serve those directly

Everything else is proxied to apache running on another port

Our Server

MySQL

Clientsport 80

port 3306

ApacheApacheApache

Lighttpd

port 8080

Friday, July 20, 12

www.percona.com

Static Files in Front - Pros / Cons

Pros:keep-aliveStatic files served uber-fast

Cons:Stupid regex -- inefficient

Files that appear static, but really are phpFiles that are static, but are served by a

*.phpMultiple webserver configs to

maintainCan confuse some apps, apps won’t

see client ips Our Server

MySQL

Clientsport 80

port 3306

ApacheApacheApache

Lighttpd

port 8080

Friday, July 20, 12

www.percona.com

Caching Server in front

Configured varnish to cache by Content-type

Pros:More accurate by-type cachingApplication can override cache with

headersCons:

Larger memory footprintVarnish config can be trickyApp still can’t see client ip and may

get confused about its hostname Our Server

MySQL

Clientsport 80

port 3306

ApacheApacheApache

Varnish

port 8080

Friday, July 20, 12

www.percona.com

Caching service in front

Run Apache on port 80Use an external caching service

(e.g., CloudFlare) and setup your domain name to point at them

Cloudflare caches static content for you, obeys cache headers, etc.

Our Server

MySQL

Clientsport 80

port 3306

ApacheApacheApache

port 80

CloudFlare

Friday, July 20, 12

www.percona.com

Caching service in front -- Pros / Cons

Pros:End-users use keep-alive, have geo-affinity with

Cloudflare’s proxiesNo extra daemon on your serverHTTP and HTTPS can be served by CloudflareApp server thinks it’s serving directly (and actually can)DOS attacks

Disadvantages:Dependency on external serviceNo transparency into caching methodologyStill can’t see client IP w/o extra config

Friday, July 20, 12

Serving Dynamic files

Friday, July 20, 12

www.percona.com

mod_php

Pros:Allows standard .htaccess files to

be used, including PHP ini_setDefault PHP, most things “just

work”Cons:

ties PHP with preforked Apache, problematic for serving a lot of connections with Apache

ApacheApacheApache + mod_php

Clients

Friday, July 20, 12

www.percona.com

fastCGI + php-fpm (or similar)

Pros:Allows PHP to be scaled

independently of concurrent connections on the webserver

Webserver automatically serves non-php content directly -- one config

Cons:One more knob to turnBreaks sites that rely on .htaccess

for php configSometimes breaks php in weird ways

ApacheApacheApache

ApacheApachePHP

Clients

Friday, July 20, 12

Tuning Apache

Friday, July 20, 12

www.percona.com

With mod_php

Must use prefork Apache because mod_php is not threadsafe*

Keep MaxClients LOW -- max 5-10

StartServers/MinSpareServers -- keep large enough to serve normal traffic

MaxSpareServers -- small enough to keep extra procs cleaned up

KeepAlive off!!

Apache + mod_php client

Apache + mod_php client

Apache + mod_php client

Apache + mod_php client

Friday, July 20, 12

www.percona.com

Apache mod_php configuration1. <IfModule mpm_prefork_module>2. StartServers 33. MinSpareServers 34. MaxSpareServers 45. MaxClients 106. MaxRequestsPerChild 10007. </IfModule>

9. KeepAlive Off

Friday, July 20, 12

www.percona.com

Without PHP

Use mpm_worker in ApacheRelationship between “Servers”

and “Threads”MaxClients - large, but not too

big (50-100?)ThreadsPerChild - not too small

to avoid stopping/starting Servers frequently

KeepAlive=ON, KeepAliveTimeout not too big

Apache mpm

client

clientclient

client

ApacheApachePHP

Apache mpm

client

clientclient

client

Friday, July 20, 12

www.percona.com

Apache config w/o PHP1. <IfModule mpm_worker_module>2. StartServers 23. MaxClients 754. ThreadsPerChild 255. ServerLimit 36. MinSpareThreads 257. MaxSpareThreads 508. MaxRequestsPerChild 10009. </IfModule>

11. KeepAlive On12. MaxKeepAliveRequests 10013. KeepAliveTimeout 10

Friday, July 20, 12

Tuning Everything Else

Friday, July 20, 12

www.percona.com

Tuning PHP

Keep your memory limits lowUse PHP cli for crons (not wget http://yoursite.com/

cron.php) and tune PHP cli with a higher memory limit

Use APC or similar!!!

Friday, July 20, 12

www.percona.com

Tuning MySQL

AVOID SWAPPINGInnodb:

innodb_buffer_pool_sizeinnodb_flush_method = O_DIRECT|ALL_O_DIRECTinnodb_flush_log_at_trx_commit = 2

Don’t use query cache, but try it in a pinchUnused cache sizes small or 0: key_buffer_size,

query_cache_size, etc.Keep various buffers as low as possible (and/or defaults):

*_buffer_size (sort, join, etc.)max_tmp_table + max_heap_table_size lowtable_cache, thread_cache_size, etc.

Friday, July 20, 12

www.percona.com

Progression of Architecture Choices

Lighttpd + Apache + mod_phpVarnish + Apache + mod_phpApache (threaded, keepalive) + php-fpmCloudflare + Apache + php-fpm

Best of breedGives you standard LAMP stack w/o custom configOSS .htaccess works as expectedCloudflare works pretty transparently*

Friday, July 20, 12

www.percona.com

Non-intuitive Steps

RAM is your most precious commodity, pinch every byte

Some swap usage is inevitable. VPS vendors won’t help you when you have swap

usageDon’t run daemons when you can run as cronjobs

puppetddenyhosts

Friday, July 20, 12

www.percona.com

RAM Tuning

Apache modules

Apache MaxClients

Varnish cacheAPC cachePHP processesInnodb buffer

poolExtra daemons

MyISAM key buffer

Query cacheMySQL per-

connection buffers

Innodb flush method

Binlog cache size

swappinessFilesystem

cache32-bit vs 64-bit

OSPersistent

connectionsThread cache

Friday, July 20, 12

www.percona.com

The Results

Friday, July 20, 12

www.percona.com

Conclusion

We vastly underutilize our serversProper Systems administration is balancing

resource utilization with good tuningGive the best customer experienceFor the lowest cost to your company

These principles apply if you have a single server or 100k

Friday, July 20, 12