Measuring the web with Boomerang (YUIConf 2010)

IntroductionHow does it work?Using boomerang

Data

Measuring the web with boomerang

Philip Tellis / [email protected]

YUIConf 2010 – 2010-11-08

YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

http://yuilibrary.com/yuiconf2010/



Data

$ finger philip

Philip [email protected]

@bluesmoongeek - paranoid - speedfreakyahoohttp://bluesmoon.info/


http://twitter.com/bluesmoon

http://bluesmoon.info/



Data

TimeThe adversaryMeasure twiceboomerang




Data





Data





Data


Less than 20% of page load time is something we can measureand fix during development




Data


It’s what we can’t control that bites us




Data


Too many variations

browsers




Data


Too many variations

plugins




Data


Too many variations

OSes




Data


Too many variations

viruses




Data


Too many variations

antiviruses




Data


Too many variations

microwaves




Data


Too many variations

baby monitors




Data


Too many variations

naughty neighbours




Data


Too many variations

file shares




Data


Too many variations

governments




Data


Too many variations

rodents




Data


Too many variations

Try simulating all that in the lab!




Data


We need to measure real end-user performance




Data


We need to measure real end-user performance from the realend-user’s box




Data





Data


While this might work, it isn’t necessarily representative




Data


What about JavaScript?




Data


boomerang is...

A piece of javascript that you add to your web page where itmeasures and beacons back to you, the end user’s perceivedperformance of your page




Data


How?

<script src="boomerang.js" type="text/javascript"></script><script type="text/javascript">BOOMR.init({

user_ip: "<user’s ip address>",beacon_url: "http://mysite.com/beacon.php"

});</script>




Data


And soon...

YUI.use(’gallery-boomr’, function(Y) {Y.BOOMR.init({


});});




Data


What does it do?

Measures user’s bandwidth and latency to your serverMeasures the current page’s load timeBeacons these results back to your server




Data


Let’s fork this talk

Do you want to:Lean how boomerang works internally?Learn how to use boomerang?




Data

LatencyBandwidthLoad timeAccuracy

How does boomerang work?




Data


Let’s take that one at a time




Data


Measuring latency

Download a 32 byte gif 10 times in sequenceMeasure the time to download eachDiscard the first measurement because it’s overpricedCalculate the arithmetic mean, standard deviation andmargin of error of the rest




Data


Wait, did you say overpriced?

The first image might require a DNS lookup and TCPhandshakeSlow start is not an issue since 32 bytes fits in 1 packet




Data


Measuring bandwidth

After the latency test is done, we download progressivelylarger imagesStop at the first image that times outRedownload that image a few more timesCalculate the median, standard deviation and margin oferror of the largest images




Data


Measuring latency before bandwidth helps here

Those 10 latency images do a lot to widen the TCPwindow sizeThe bandwidth images make much better use of bandwidthThe image we end with uses the most bandwidth




Data


How do we measure page load time?

In the onbeforeunload event, measure the time andstore it in a cookieIn the onload event, check the cookie, and measure thedifference with the current timeWe also make sure that the page that set the cookie is thereferrer of the current page




Data


What? Two pages?

Yes, this needs two pages and cookies. If those aren’tsupported, we try to use the WebTiming API.




Data


How accurate is it?

Latency measurements are very accurate (±1%)Bandwidth is to an order of magnitude. For badconnections can be ±30%

Page load time sometimes has outliers, you needpost-filteringThe margin of error tells you how good your data is




Data

BasicMulti-domainIn-page timersAJAX

Include it on your page

<script src="boomerang.js" type="text/javascript"></script><script type="text/javascript">BOOMR.init({


});</script>




Data


For most sites, that’s about it




Data


You probably want to do more




Data


Handle subdomains

Set the site_domain parameter:

BOOMR.init({user_ip: "<user’s ip address>",beacon_url: "http://mysite.com/beacon.php",site_domain: "mysite.com"

});




Data


This lets you measure transitions across *.mysite.com




Data


Measure more than just load time

<html><head><script>var t_pagestart=new Date().getTime();</script>...<script>var th=new Date().getTime();</script></head><body>...<script>var tj=new Date().getTime();</script>...




Data


...<script src="boomerang.js"></script>...var te=new Date().getTime();BOOMR.plugins.RT.setTimer("t_head", th-t_pagestart).

setTimer("t_body", te-th).setTimer("t_js", te-tj);

</script></body></html>




Data


This adds the t_head, t_body and t_js fields to the beacon




Data


Loading dynamic content

BOOMR.init({user_ip: "<user’s ip address>",beacon_url: "http://mysite.com/beacon.php",auto_run: false

});




Data



// Just before download startsBOOMR.plugins.RT.startTimer("t_done");




Data



// Just after download finishesBOOMR.plugins.RT.done();




Data


Much more

http://yahoo.github.com/boomerang/doc/howtos/


http://yahoo.github.com/boomerang/doc/howtos/



Data

shouldacouldawoulda

The beacon

A simple GET request to the beaconAll parameters passed in the query stringExtra timers are passed in as a comma separated list int_other

before_beacon JavaScript event fired just before thebeacon is sent




Data

shouldacouldawoulda

What should we do with the data?

Sanity checking to:Remove fake dataRemove abusive dataMaybe just rate limiting




Data

shouldacouldawoulda

What can we do with the data?

Statistical analysis to:Remove outliersAggregate based on bandwidth blocksMeasure trends over time and correlate them with codechanges




Data

shouldacouldawoulda

Bandwidth blocks

Ref: Analysing Bandwidth & Latency – YUI Blog


http://www.yuiblog.com/blog/2010/04/08/analyzing-bandwidth-and-latency/



Data

shouldacouldawoulda

Bandwidth blocks

Data points from some countries may require narrower bands




Data

shouldacouldawoulda

Geographic data

Looking at latency from different geographic locations can tellyou where your next mirror should be




Data

shouldacouldawoulda

ISPs

Grouping data by ISP can tell you who’s behaving badly




Data

shouldacouldawoulda

More data

Write plugins to get more performance dataWe already have a DNS pluginI’m thinking of an IPv6 v/s IPv4 pluginWhat about a full WebTiming plugin?Can we measure connection setup time?




Data

shouldacouldawoulda

You decide

Once you have the data, you can do anything with it




Data

Thank youhttp://github.com/yahoo/boomerang

http://yahoo.github.com/boomerang/doc/


http://www.github.com/yahoo/boomerang/

http://yahoo.github.com/boomerang/doc/



Data

Photo credits

flickr.com/photos/21233184@N02/4389412851


http://www.flickr.com/photos/21233184@N02/4389412851/



Data

Contact me

Philip [email protected]

@bluesmoongeek - paranoid - speedfreakyahoohttp://bluesmoon.info/slideshare.net/bluesmoon


http://twitter.com/bluesmoon

http://bluesmoon.info/

http://www.slideshare.net/bluesmoon



Data

References

github.com/yahoo/boomerangMore bandwidth doesn’t matter (much) – Mike BelsheAnalysing Bandwidth & Latency – YUI BlogIt’s the latency, stupid – Stuart CheshireThe statistics of web performance


http://github.com/yahoo/boomerang/

http://www.belshe.com/2010/05/24/more-bandwidth-doesnt-matter-much/

http://www.yuiblog.com/blog/2010/04/08/analyzing-bandwidth-and-latency/

http://www.stuartcheshire.org/rants/Latency.html

http://www.slideshare.net/bluesmoon/index-3441823


Technology

Measuring the web with Boomerang (YUIConf 2010)