Make Magento run like an antelope

Make Magento run like an antelope

01.58.56.60.80 / contact@nbs-system.com

BioAbout MagentoSpeed matters, a LOTFrom 10s to 3sBreaking the 1.5s wallWhat servers, what volumes140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Magento is oftenly considered like

Powerfull Solution of the futur Complicated to code So much fun !

And

Slow Ressources hungry CPU intensive Complicated to code

Well, this is true, most of it. 140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionNow why Magento can be more likeFirst consider the wins More sales More efficient More flexible Cost less to modify / admin Will last (22 M$ should help)

Then the drawbacks Ressource intensive A bit tricky to master

Then the solutionPut a better hosting environment to correct the fat boy feeling and only appreciate good points.140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionGoogle is now using speed as an important ranking factor (its official)

Graph (given by Google) show clearly the target: 1.5s Actually, 1,5s means youre part of the 75% fastest sites but More time and you are part of the red team in the red zone, called SLOW Guess what, this is no good for your ranking & SEO

BesidesRevelance is still the most important criteria among 200, but, http://ow.ly/1xfqt

Amazon loose 1% revenue every 0,1s latencyYour customers feel it, this is part of their experience on your site !

Our Goal is now clear : load in less than 1.5sSpeed maters, a LOT

140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901The right hardwareThe right OSAn efficient architectureCode like a God

The keysStepping down from 10+ to 3 seconds140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionHardware manufacturer have very different efficiency Intel and AMD processors doesnt behave the sameHyperthreading changes (almost) nothing in server performancesRAM make a HUGE difference on I/OHARDWARE

Stepping down from 10+ to 4 secondsXeon for front servers, 5670 if possibleAMD mono - hexacores for DBUse DDR3Use SAS controllers with cachePut a LOT of RAM in them (well see later why), 2 Go per core !Advices

140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901IRQ balancing problem

Network : no bottleneck, good peering, of course !Organisation de la productionBeing OS Agnostic, Im objectiveWindows consume too much ressources on PHP interpretation, weak TCP/IP stack, not optimized for *AMP environmentOpenSolaris gave poor results (30% less than Linux)NetBSD behave well, very fair competitorLinux is leading quite seriouslyOperating SystemStepping down from 10+ to 4 secondsAdvices

Debian LennyCentOSNetBSDRedhat(avoid the GUI based distro)140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Opensolaris slowness: overchecking => slow down

Organisation de la productionA Firewall + LoadbalancerA Reverse Proxy will offload the serversContent server for static files (CSS/JPG/GIF/JS)You can deliver satic content near to the user with CDNSeparated web & database servers (+33%)Create an efficient architectureStepping down from 10+ to 4 secondsAdvicesNetfilter for FWIPVS + keepalivedNGINX for the RPApache, Litespeed, Zeus, Nginx

Conservative ? Apache + SquidR&D guy ? Nginx + PHP FPM + Mango DB

140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionCode like a GodStepping down from 10+ to 4 secondsCode maters. A HUGE LOT.

Each little glitch is going to be amplified by the visitor numbers : Small site : dirty things wont be too much of a problem Big site : a miscoded loop can take down the servers

Be helped by communityFooman Speedster, Magento boosters (like delorums one)Read Forums, blogsBargento (to listen to our top notch talks)Apply the whitebooks and blog post optimisation tips

Be helped by Magento IncA bit of consulting by the pro, a lot of time won !140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionCode like a GodStepping down from 10+ to 4 secondsGo to a strong training courseUnderstand the block cachingTake care of the security ! (xss/csrf/sql-injection)Leaving a 404 on a picture create a PHP exec : bad !

Optimise the CSS selectors & the CSS calls (just needed one)Serialize CSS and then JS order in the downloads (not css/js/css/js)Minify your JS using JsminAvoid redirections, optimize pictures, and DNS queries

Be carefull while in a loop, think that your piece of code will sometimes be executed thousands of time a minute

140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionGet rid of the major bottleneck : I/ODown deep in the mineAdvanced TweaksAnalyse your traficUse cache the right wayMore ? a.k.a Where the fun startsBreaking the 1.5s wall !140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionThe I/O BottleneckBreaking the 1.5s wall !AdvicesPut sessions & cache on tmpfs ramdisksUse fast disks like 15Ktm SAS and a good controller with RAM to boost the writesHave your disks in RAID 10 if possibleDont log useless things (& dont log-resolve)Lot of RAM ? Put the whole site on !I/Os are a major problemWrite to disk, but with a good Controller, on a excellent diskRead from disk == loose time (except on VERY good SAS controllers)

ALL TO RAM ! Weve got plenty of it ! Faster and more reliable than a SSD Optimized by CPU & BUS Inexpansive Dawn fast

140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionSplit read & write to different DB

-> host it on a RAMDISK -> Host it on a Physical Disk

Safety + Performances

Use DRBD

DRBD is an opensource Blockdevice sharing over network.Mainline in Linux Kernel since 2.6.33 => safe

Simple, safe, efficient (and made for High avaibility environments). This give good results and can be used but its limited to only two servers for now.

MYSQLStepping down from 10+ to 4 seconds140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionAdvanced TweaksUse Sysfence to autokill the bombing crons or php processUse IRQ_BalancingMount your disks with noatime option (faster access to data)Lower socket Wait Timeout in /procTruncate (or archive) your log tables in MagentoCheck the best scheduler, iomapper, L3 (architecture dependant)Use GRsec/Pax (for security purposes)Tweak Zend server rulesOptimize your peering map (network)Put a QoS tag on your packets to speed them upCheck foreign includes response times and put them at the end of pagesUse pagespeed / Yslow / Gtmetrix to analyse your siteIntegrate Lucene SOLR on large catalog database (> 100 000 products)Use latest version of Magento and activate all caches in the BO (if possible)Breaking the 1.5s wall !140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionAnalyze your trafic (know your ennemy)AdvicesUse your mailing wisely, spread it on some hoursSame for all trafic generation, burst is your ennemy

Breaking the 1.5s wall !140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionUse Cache the right wayTips & TricksPHTML blocks should be cached it in order not to be generated each viewTo cache a block, you have to declare it explicitly, no auto mecanismDevelopers usually dont use the Block caching mecanism at all (80%)

Up to ten times faster loading when using cache properly !

The Enterprise Edition has a full page caching mecanism (for the final pages, not the individual blocs)

NBS will soon release a plugin that will automagically do it for you

Breaking the 1.5s wall !140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionFunkload & APDEX are the keys to realistics benchmarks

When you reach a 0,9 APDEX indice, your server has given enough power and your user still feel confortable.

4 levels for user experience (like your girlfriend, except for the timings) :

Satisfied (for exemple 3s)Tolerating (3x more, so less than 9s)Frustrated (more than 9s)Inacceptable (not served/errors)Benchmarking140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionBenchmarking

140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901Organisation de la productionWith 5670 dual quad core server8 Go of DDR3 1066 MhzOptimised settings & serversDatabase splited to another serverVery good site (optimized)80 000 productsMagento 1.4.0.1 + FC demostoreAPDEX Limited to 0,9

80 000 uniq visitor / day per front server650 Concurrent Users0,67s average load time on the home(its still a perfect case, count more on 50 KVU)What servers, what volumes ?140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901OThank you for your attention !NBS System140 bd haussmann75 008 ParisTel : +33.1.58.56.60.86Mail : contact@nbs-system.comWeb : www.nbs-system.com

01.58.56.60.80 / contact@nbs-system.comNeed (fast) hosting ?

8 november 2010, PARIS750 + visitors, 30 booths16 talks

140, Bd Haussmann 75008 Paris Standard +331.58.56.60.80 Fax: +331.58.56.60.81 Web: www.nbs-system.comSAS au capital de 203 905 R.C.S. Evry B 423410 901 N TVA: FR 61 423410901

InternetServeur Web