Introduction to Performance Tuning

Perl Web ApplicationsPerrin Harkins

Find someone to blame

Performance vs scalability

● Not really the same thing● Can look the same● We're mostly going to talk about

performance

Slowness

● Maybe it's the designer's fault!● Chrome Developer Tools

Network

PageSpeed

Narrow it down further

Create a repeatable test

● Measure progress● Metrics

○ Requests/second○ Time for n requests○ Concurrency

ab

ab -c1 -n100 http://127.0.0.1:8080/bugzilla/buglist.cgi

Benchmarking 127.0.0.1 (be patient).....done

Server Software: Apache/2.2.22

Server Hostname: 127.0.0.1

Server Port: 8080

Document Path: /bugzilla/buglist.cgi

Document Length: 14562 bytes

Concurrency Level: 1

Time taken for tests: 33.816 seconds

Complete requests: 100

Failed requests: 0

Write errors: 0

Total transferred: 1494100 bytes

HTML transferred: 1456200 bytes

Requests per second: 2.96 [#/sec] (mean)

Time per request: 338.163 [ms] (mean)

Time per request: 338.163 [ms] (mean, across all concurrent requests)

Transfer rate: 43.15 [Kbytes/sec] received

Connection Times (ms)

min mean[+/-sd] median max

Connect: 0 0 0.0 0 0

Processing: 313 338 20.7 333 448

Waiting: 312 337 20.6 332 448

Total: 313 338 20.7 333 448

Percentage of the requests served within a certain time (ms)

50% 333

66% 339

75% 343

80% 345

90% 357

95% 374

98% 430

99% 448

100% 448 (longest request)

httperf

httperf --wsess=10,5,2 --rate=1 \

--server=localhost --port=8080 \ --uri=/bugzilla/buglist.cgi

Total: connections 50 requests 90 replies 50 test-duration 21.680 s

Connection rate: 2.3 conn/s (433.6 ms/conn, <=11 concurrent connections)

Connection time [ms]: min 321.5 avg 2487.2 max 3884.7 median 2537.5 stddev 1002.1

Connection time [ms]: connect 0.1

Connection length [replies/conn]: 1.000

Request rate: 4.2 req/s (240.9 ms/req)

Request size [B]: 203.0

Reply rate [replies/s]: min 1.6 avg 2.4 max 3.0 stddev 0.6 (4 samples)

Reply time [ms]: response 883.0 transfer 3.4

Reply size [B]: header 388.0 content 14562.0 footer 2.0 (total 14952.0)

Reply status: 1xx=0 2xx=50 3xx=0 4xx=0 5xx=0

Session rate [sess/s]: min 0.00 avg 0.46 max 1.00 stddev 0.49 (10/10)

Session: avg 5.00 connections/session

Session lifetime [s]: 12.4

HTTP::Recorder

● Acts as HTTP proxy● Generates Mechanize script● HTTP::Recorder::Httperf

$agent->get('http://127.0.0.1:8080/bugzilla/');$agent->follow_link(text => 'Search', n => '1');$agent->form_name('queryform');$agent->field('bug_status', '__open__');$agent->field('content', '');$agent->field('product', 'TestProduct');$agent->click();

/bugzilla/ method=GET

/bugzilla/skins/standard/global.css method=GET

/bugzilla/skins/standard/index.css method=GET

/bugzilla/skins/contrib/Dusk/global.css method=GET

/bugzilla/skins/contrib/Dusk/index.css method=GET

/bugzilla/js/yui/yahoo-dom-event/yahoo-dom-event.js method=GET

/bugzilla/js/yui/cookie/cookie-min.js method=GET

/bugzilla/js/global.js method=GET

/bugzilla/skins/standard/index/file-a-bug.png method=GET

/bugzilla/skins/standard/index/search.png method=GET

/bugzilla/skins/standard/index/new-account.png method=GET

/bugzilla/query.cgi method=GET think=4

/bugzilla/skins/standard/search_form.css method=GET

/bugzilla/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=__open__&product=TestProduct&content= method=GET think=6

/bugzilla/js/yui/assets/skins/sam/autocomplete.css method=GET

/bugzilla/js/yui/assets/skins/sam/calendar.css method=GET

/bugzilla/skins/standard/buglist.css method=GET

/bugzilla/skins/contrib/Dusk/buglist.css method=GET

Autobench

Profile to find out where the time is going

● Devel::NYTProf● Wall clock time vs. CPU time● Use your real environment● Multiple runs and warmup avoid

skewed results● Let's look at one...

Ten bucks says it's your database

Sure, every now and then you find

● A bad regex● A string being parsed over and over● Massive object instantiation● Network operations● Disk reads

But mostly it's the database

● What’s slow in modern computers?● What does most of the I/O in a typical

web app?● Either fix your queries or cache them

Profiling DBI

● DBI_PROFILE=2/DBI::ProfileDumper::Apache

● Let's look at one...

Speeding up queries

● EXPLAIN plans○ pt-query-advisor

● SQL generation is not for everything● A little bit of database server tuning

○ pt-variable-advisor

Speeding up DBI

● Cache connections and statements● Manage commits● Use native bulk loading tools

The last resort: caching

● Cache management is a hard problem● Code complexity

○ invalidation calls○ dependency tracking

● Your content creators will hate it● When you do cache, use CHI

A brief word about runtime environments

● Webserver choice has a minimal effect on performance

● Persistent daemon: mod_perl, Plack, FastCGI

● Buffer your output● Size-limiting or auto restarts

Real-world pressures

Flailing

● Changing things based on guesses rather than data

● No QA● Lots of collateral damage● Emergency profiling

Benchmark::Stopwatch

my $stopwatch = Benchmark::Stopwatch->new->start;...$stopwatch->lap('load objects');...$stopwatch->lap('render template');...$stopwatch->lap('send response'); print $stopwatch->stop->summary;

Benchmark::Stopwatch

NAME TIME CUMULATIVE PERCENTAGE load objects 1.000 1.000 14.289% render template 3.000 4.001 42.853% send response 2.000 6.001 28.572% _stop_ 1.000 7.001 14.287%

Doubts about infrastructure

● “You're still using Foobar 1.5?! That’s so slow!”

● Keep your head● Get help: FAQ, Google, mailing list/IRC

Buying hardware

● Good idea!● A boatload of RAM hides a multitude of

sins● Make sure you know what the

bottleneck is

Further reading

● Tim Bunce's Advanced DBI slides on CPAN

● Percona Toolkit

Thank you!