Upload
catchpoint-systems
View
219
Download
0
Embed Size (px)
Citation preview
8/3/2019 Web Performance Insights 2011
1/47
1
WebPerformance
Insights 2011By Catchpoint Systems
Share this eBook
http://www.facebook.com/sharer/sharer.php?u=www.catchpoint.com/ebooks/web-performance-insights-2011.htmlhttps://twitter.com/intent/tweet?source=tweetbutton&text=Get+a+free+ebook+with+web+performance+insights+from+2011&via=catchpoint&hashtags=webperf,sysadmin,wpo&url=http://www.catchpoint.com/ebooks/web-performance-insights-2011.htmlhttp://www.linkedin.com/shareArticle?mini=true&url=http://www.catchpoint.com/ebooks/web-performance-insights-2011.html8/3/2019 Web Performance Insights 2011
2/47
2
Foreword
This is a collection of our most insightful and popular web performance blog posts from 2011. We
created this anthology as a way of sharing web performance knowledge.
Today, as the vox populidemands universal speed and reliability across the internet, it is more
important than ever for companies to meet end user expectations. Moreover, as businesses continue
to lease out their cyber real estate to 3rd Party Providers to increase revenue through advertising, track
and analyze user behavior, or save money in infrastructure costs, it is crucial for us to understand the
impact of each one to the overall performance of the website.
Enjoy the read andtake a look at our blogfor more insights on operating fast websites.
-- The Catchpoint Team
http://blog.catchpoint.com/http://blog.catchpoint.com/http://blog.catchpoint.com/http://blog.catchpoint.com/http://www.catchpoint.com/8/3/2019 Web Performance Insights 2011
3/47
3
Contents
Webpages turning into Airports without Traffic Controller! ...................................................................... 4The Biggest Misconception about Google Page Speed ......................................................................... 8Free DNS Can Hurt Web Performance! ............................................................................................... 13
Relying on Web Performance Monitoring to Discover Release Problems ............................................ 18Getting the Most Out of Performance Monitoring: Setting Alert Thresholds .......................................... 23Three Key Steps to Successful Web Performance Benchmarking ....................................................... 26A New Blind Spot for Web Performance Tools ..................................................................................... 33My 2 cents on the AWS Failure and lessons learned from the past ...................................................... 37Royal Wedding and the Internet ........................................................................................................... 40WPO Resolution for 2012! ................................................................................................................... 44
http://www.catchpoint.com/8/3/2019 Web Performance Insights 2011
4/47
4
Posted October 10, 2011
Webpages turning into Airportswithout Traffic Controller!
8/3/2019 Web Performance Insights 2011
5/47
--- Webpages turning into Airports without Traffic Controller! ---
5
he other day I tried to book a trip on my favorite airline,Virgin America. I love everything about
them: their planes, the service, the staff, the entire experience is just amazing. Plus I was
lucky enough to purchase some of their special vouchers onGilt(another company I love).
However, this time I faced frustration on their website as for some reasons I could not get the datepicker to work. The entire page was visible but I could not interact with it. No love back!
Annoyed, I started digging into what was getting on the way. Fired up the Chrome dev tools and also
ran tests onCatchpointto see what was being loaded and what was failing to load.
To loadhttp://www.VirginAmerica.com my browser had to:
Download 167 Objects (Images, CSS, JavaScript files)
Connect to 51 unique hosts
Download 1 million bytes.
What are those 51 hosts? Two of the hosts were required to load the content that mattered to me on
the page: static.virginamerica.com & www.virginamerica.com. The other 49 hosts did not deliver any
content required for me as a user to buy my tickets (I tracked SSL and Non SSL hosts as separate
hostnames). They were all tracking pixels for ads/marketing and analytics tags.
T
http://www.virginamerica.com/http://www.virginamerica.com/http://www.virginamerica.com/http://www.gilt.com/http://www.gilt.com/http://www.gilt.com/http://www.catchpoint.com/http://www.catchpoint.com/http://www.catchpoint.com/http://www.virginamerica.com/http://www.virginamerica.com/http://www.virginamerica.com/http://blog.catchpoint.com/2011/10/10/webpages_airports_no_traffic_controller/hosts/http://www.virginamerica.com/http://www.catchpoint.com/http://www.gilt.com/http://www.virginamerica.com/8/3/2019 Web Performance Insights 2011
6/47
--- Webpages turning into Airports without Traffic Controller! ---
6
By looking at the various requests, I discovered that a call to one of the analytics tags on the page was
hanging and made the page 100% unusable. After a few unsuccessful refreshes of the page I decided
to block the offending domain usingOpenDNSto get the site to work again and purchased my ticket!
I am a big believer in Online Advertising and in Analytics. Heck, I worked forDoubleClick&Googlefor11 Years and I know that many if not all these companies spend a great deal of money on monitoring
their performance.
However, lately I have observed an interesting trend: Webpages are becoming bloated with third party
tags which often cause user experience problems, all of this culminating in a battle between the IT and
Marketing teams on what actions should be taken.
For many companies the Marketing team has direct access to the site content and can place third party
tags live without proper testing or asking themselves: How is this tag going to impact my performance?What will the impact be on the experiences of my end users? Is the revenue generated by this tag
worth the user frustration?
The IT and web development teams are constantly trying to do more with less money or fighting battles
they know they will lose and they give up. I have also found out that for several companies the IT
operations teams ignore problems from third party tags (even when reported by third party monitoring
solutions like ours). Main reason is simple; they do not have the means to correct the problem. Yet end
users are impacted and action is not taken until someone else in IT notices performance numbers
creeping up or users complain on Twitter, Facebook, or Forums.
This is a dangerous path. There are several tools and techniques out there to make sure these 3rd
party tags do not impact your end user experience:
Ghostwriter
ControlJS
The BrightTag
Tagman
Outclip
Another important point to keep in mind is that even if you have optimized the tags so they do not
impact performance, all these JavaScript requests might not play nice with each other inside the
http://www.opendns.com/http://www.opendns.com/http://www.opendns.com/http://www.doubleclick.com/http://www.doubleclick.com/http://www.doubleclick.com/http://www.google.com/http://www.google.com/http://www.google.com/http://digital-fulcrum.com/solutions/ghostwriter-complete-control/http://digital-fulcrum.com/solutions/ghostwriter-complete-control/http://stevesouders.com/controljs/http://stevesouders.com/controljs/http://www.thebrighttag.com/http://www.thebrighttag.com/http://www.tagman.com/http://www.tagman.com/http://www.outclip.com/http://www.outclip.com/http://www.outclip.com/http://www.tagman.com/http://www.thebrighttag.com/http://stevesouders.com/controljs/http://digital-fulcrum.com/solutions/ghostwriter-complete-control/http://www.google.com/http://www.doubleclick.com/http://www.opendns.com/8/3/2019 Web Performance Insights 2011
7/47
--- Webpages turning into Airports without Traffic Controller! ---
7
browser. Hence, you might want to have some approval process which includes extensive testing of
the vendors tags in actual webpages.
The Virgin America page looked like a busy airport with 167 planes from 49 different airlines landing at
the same time and no Air Traffic Controller.
Safe travel and please do care about your End User Experience.
Photo Credit: Artist Ho-Yeol Ryu-Check out his amazing Gallery.
http://www.homato.com/http://www.homato.com/http://www.homato.com/http://www.homato.com/8/3/2019 Web Performance Insights 2011
8/47
8
Posted December 27, 2011
The Biggest Misconception aboutGoogle Page Speed
8/3/2019 Web Performance Insights 2011
9/47
--- The Biggest Misconception about Google Page Speed ---
9
n my conversations with customers and prospects we often talk about one of the biggest IT
problems: How can we make the website faster for end users.
Obviously Web Performance Optimization (WPO) techniques are key to faster webpage and
Google Page Speed score is a key tool on measuring how well they are applied. However, quite often I
hear comments about the score that make no sense like We have spent a lot of $, time and efforts
to get amazing Google Page Speed scores but our site is still slow. what did we
miss? or Competitor X has a score 50, we have 85, yet they load twice as fast!
These concepts clearly show a misconception of what Google Page Speed score does. Certain people
fall prey to the idea that pages with HIGH Page Speed scores should be faster than pages with LOW
scores. There are probably various causes to this misconception, from the name to how certain
individuals or companies sell products or services. Our goal with this post is to clarify why this belief is
false, and why you should not rely on it.
Google Page Speed is a score calculated based onWeb Optimization rulesthat would improve the
rendering and execution of the front end code (HTML, CSS, etc) on browsers. Per Googles definition:
These rules are general front-end best practices you can apply at any stage of web development. In
other words, the score does not look at your Infrastructure, Application Performance, DB Queries,
datacenter distribution, load handling, content loaded on the browser, etc. Most importantly it does not
include the actual speed of the page and therefore cannot be used a yard stick about who is faster
Page A or Page B.
To illustrate the point that there is no correlation between Page Speed Score and how fast a page loads
lets take a look at theperformance data we collected over Black Friday and Cyber Monday 2011 for
the top Internet Retailers. We measured several speed metrics and the Google Page Speed
scoreutilizing Catchpoint synthetic agents in major US cities. We will focus on two key metrics which
are most commonly used and that are the best gauges of webpage speed.
I
http://code.google.com/speed/page-speed/docs/rules_intro.htmlhttp://code.google.com/speed/page-speed/docs/rules_intro.htmlhttp://code.google.com/speed/page-speed/docs/rules_intro.htmlhttp://blog.catchpoint.com/2011/11/28/cybermonday-2011/http://blog.catchpoint.com/2011/11/28/cybermonday-2011/http://blog.catchpoint.com/2011/11/28/cybermonday-2011/http://blog.catchpoint.com/2011/11/28/cybermonday-2011/http://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://blog.catchpoint.com/2011/11/28/cybermonday-2011/http://blog.catchpoint.com/2011/11/28/cybermonday-2011/http://code.google.com/speed/page-speed/docs/rules_intro.html8/3/2019 Web Performance Insights 2011
10/47
--- The Biggest Misconception about Google Page Speed ---
10
Document Complete Time & Google Page Speed Score
As you can clearly see from the chart there is no relationship on the Document Complete and Google
Page Speed Score. The fastest site, J.C. Penny and the slowest site, Target, had an identical score of
84.
Web Page Response Time (fully loaded) & Google Page Speed Score
http://blog.catchpoint.com/2011/12/27/biggest_misconception_about_google_page_speed/webpageresponse-google-sort/http://blog.catchpoint.com/2011/12/27/biggest_misconception_about_google_page_speed/doccomplete-google/8/3/2019 Web Performance Insights 2011
11/47
--- The Biggest Misconception about Google Page Speed ---
11
Even when looking at the fully loaded page, we still see that there is no correlation with Google Page
Speed.
Google Page Speed score should be treated as a grade of how good of a job your front end developers
(or optimization solution) has done in making a page that renders as quickly as it can given the content
it needs to display. When you compare it with your competing sites, use it to compare how good of a
job they have done -do not use it as measuring stick for speed. To measure speed use metrics like
Response, Document Complete, Fully Loaded, etc. The same reasoning holds true for theYSlow score
from Yahoo.
So great we clarified the Page Speed misconception, your front end team does an awesome job and
gets the score in the 90s for the entire site but somehow the site is still under-performing competition
in speed. Well take a look at the rest of your system, analyze the data, review it carefully and I am sure
there is plenty you can do to make it faster:
Trim down bytes. You could have a well optimized page, but if you are loading 2mb+ of data it
will be slow for the average Joe. Do you really need that rather large SWF file? Should every
visual effect be an image?
Analyze application performance. Take a look at every page and process. Are there particular
queries taking too long? Is the mid-tier/backend code that needs optimization? Can you achieve
better caching rather than reading from disk for every pageview? Are certain processes
impacting performance?
Analyze how application does under user load. On a developers machine might be fast, but get
100 users on it and maybe there are problems related to code, hardware, etc.
Analyze your infrastructure. Take a look at Routers, Switches, Load Balancers, various
machines, are they over utilized? Are they misconfigured?
Evaluate your datacenter location and ISPs. Are your users mainly on West Coast, but serving
all content from East Coast? Maybe you need a new datacenter, or move it to Midwest so it has
better coverage of US.
Evaluate your third party vendors. If you are relying on CDNs, Adserving, etc ensure they arenot impacting your speed.
http://developer.yahoo.com/yslow/http://developer.yahoo.com/yslow/http://developer.yahoo.com/yslow/http://developer.yahoo.com/yslow/http://developer.yahoo.com/yslow/http://developer.yahoo.com/yslow/8/3/2019 Web Performance Insights 2011
12/47
--- The Biggest Misconception about Google Page Speed ---
12
In conclusion, the speed of your page is not dependent only on front end code or Page Speed score it
is dependent on your entire system and infrastructure. The engineering and operations team must talk
and work with each other and get a better understanding of what is impacting speed.
Everyone in an organization is responsible for a Fast Web Site / Application.
8/3/2019 Web Performance Insights 2011
13/47
13
Posted July 4, 2011
Free DNS Can Hurt WebPerformance!
8/3/2019 Web Performance Insights 2011
14/47
--- Free DNS Can Hurt Web Performance! ---
14
fter working with one of our clients earlier in August, I tweeted the following:
I am just amazed how many companies use theirregistrarsDNS as primary DNS not
GOOD!
In reply to the tweet I received several questions, and it became clear that registrar-
provided-DNS needed a discussion all of its own. (I have previously talked in our blog
aboutthe importance of DNS on web performance)
Usually a company buys adomainfrom a registrar, (such asGodaddy,Network Solutions,1and1, etc.)
Then they either delegate that domain to their ownDNS system, or rely a 3rd party service to manage it
(suchasDyn,Cotendo,Verisign,Nominum,Cloudfloor,UltraDNS,DNSmadeeasy, etc.), or rely on the
registrars DNS services.
Dont get me wrong the DNS services offered by a registrar are more than sufficient for the great
majority of the websites in the internet like blogs, personal sites, or sites with small presence. Even if
you are medium size website, a registrars DNS could work just fine if you rely on long TTLs and dont
need any advanced features like geographical load balancing or fast failovers capabilities.
On the other side, a registrars DNS might not be your best choice if you are a website with global
presence and web performance is key to your success, or you are a third party service that impacts the
performance of your clients (like adserving) and have SLAs. In addition if you rely on CDNs to serve the
static content, why rely on a registrar for the DNS entries pointing to the CDN? You are investing into
speed might as well invest on all the components impacting speed and DNS is the first one to
impact it.
Registrars offer their services for free and often the price reflects in their performance. Keep in mind not
all registrars are equal their level of investment in their infrastructure varies and so does their quality.
Either way, the most common reasons as to why the DNS performance of a registrar could be poor are:
Their DNS Servers are not well-distributed geographically and/or not relying on technologies
likeIP Anycastto route DNS queries to the closest servers.
Their ISP peering points might be limited.
Their DNS servers are not the fastest or not reliable. We have seen many timeouts as a direct result of
poor performance from registrar-provided DNS.
A
http://www.verisigninc.com/en_US/products-and-services/domain-name-services/domain-information-center/frequently-asked-questions/index.xhtml#q16http://www.verisigninc.com/en_US/products-and-services/domain-name-services/domain-information-center/frequently-asked-questions/index.xhtml#q16http://www.verisigninc.com/en_US/products-and-services/domain-name-services/domain-information-center/frequently-asked-questions/index.xhtml#q16http://blog.catchpoint.com/2010/11/16/dnsperfimpact/http://blog.catchpoint.com/2010/11/16/dnsperfimpact/http://blog.catchpoint.com/2010/11/16/dnsperfimpact/http://en.wikipedia.org/wiki/Domain_namehttp://en.wikipedia.org/wiki/Domain_namehttp://en.wikipedia.org/wiki/Domain_namehttp://www.godaddy.com/http://www.godaddy.com/http://www.networksolutions.com/http://www.networksolutions.com/http://www.networksolutions.com/http://www.1and1.com/http://www.1and1.com/http://www.1and1.com/http://en.wikipedia.org/wiki/Domain_Name_Systemhttp://en.wikipedia.org/wiki/Domain_Name_Systemhttp://en.wikipedia.org/wiki/Domain_Name_Systemhttp://www.dyn.com/http://www.dyn.com/http://www.cotendo.com/http://www.cotendo.com/http://www.cotendo.com/http://www.verisigninc.com/en_US/index.xhtmlhttp://www.verisigninc.com/en_US/index.xhtmlhttp://www.verisigninc.com/en_US/index.xhtmlhttp://www.nominum.com/what-we-do/hosted-network-serviceshttp://www.nominum.com/what-we-do/hosted-network-serviceshttp://www.nominum.com/what-we-do/hosted-network-serviceshttp://www.cloudfloor.com/http://www.cloudfloor.com/http://www.cloudfloor.com/http://www.ultradns.com/http://www.ultradns.com/http://www.ultradns.com/http://www.dnsmadeeasy.com/http://www.dnsmadeeasy.com/http://www.dnsmadeeasy.com/http://blog.patrickmeenan.com/2011/10/anycast-and-what-it-means-for-web.htmlhttp://blog.patrickmeenan.com/2011/10/anycast-and-what-it-means-for-web.htmlhttp://blog.patrickmeenan.com/2011/10/anycast-and-what-it-means-for-web.htmlhttp://blog.patrickmeenan.com/2011/10/anycast-and-what-it-means-for-web.htmlhttp://www.dnsmadeeasy.com/http://www.ultradns.com/http://www.cloudfloor.com/http://www.nominum.com/what-we-do/hosted-network-serviceshttp://www.verisigninc.com/en_US/index.xhtmlhttp://www.cotendo.com/http://www.dyn.com/http://en.wikipedia.org/wiki/Domain_Name_Systemhttp://www.1and1.com/http://www.networksolutions.com/http://www.godaddy.com/http://en.wikipedia.org/wiki/Domain_namehttp://blog.catchpoint.com/2010/11/16/dnsperfimpact/http://www.verisigninc.com/en_US/products-and-services/domain-name-services/domain-information-center/frequently-asked-questions/index.xhtml#q168/3/2019 Web Performance Insights 2011
15/47
--- Free DNS Can Hurt Web Performance! ---
15
At Catchpoint wemonitor the DNS performancefrom multiple geographical locations relying on three
distinct methods:
Measure DNS Resolution as part of a web performance monitoring. Relies on a DNS resolver
and it respect TTLs
Emulate a DNS Resolver (performs recursive queries to resolve the domain) with a clean cache.
Directly query a specific NS server, and measure the performance of that server.
To illustrate the performance problems, let me present two actual client cases we dealt with this year.
(To protect the privacy of our clients we are not making public who they are, the domains, or the
registrars):
Example 1: A Catchpoint client observed multiple DNS failures through our IE8 browser based
monitoring. The client relied on a registrar to host the CNAME to their CDN. We analyzed which NSservers involved in the domain resolution and ran a performance analysis for each server.
The following scatterplot displays the raw data collected on IE8 Agent on a 3 day period in
February/March 2011:
Each one of those red dots represent a failure to resolve DNS and they were all caused by a registrar
used.
http://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://blog.catchpoint.com/2011/10/05/free-dns-hurts-web-performance/image001/http://www.catchpoint.com/products.html8/3/2019 Web Performance Insights 2011
16/47
--- Free DNS Can Hurt Web Performance! ---
16
Example 2: An adserving company was relying on a Registrar for their DNS. They were experiencing
slow performance and had high impressions discrepancies with other adserving solutions. The
following chart shows the Response time of a simple ad call with the DNS resolution time.
AtWebperf meetupsI emphasize that when monitoring web performance it is vital to see the entire
picture, and that picture includes DNSDNS is the first, critical link between you and your
customers.
And finally, some of the recommendations we give regarding DNS handling:
Avoid Short TTLs where possible. (especially if you must rely on registrar DNS infrastructure) Avoid multiple CNAMEs.
Use distributed DNS infrastructures based on your user base, or use third party that specialize
in DNS resolution.
When hosting your own DNS infrastructure, make sure you have the capacity to handle DDOS
attacks & traffic surges.
Use Catchpoints tools to effectively and reliably monitor your complete DNS response paths.
Make sure to keep your internal LAN DNS records separate from your production DNS.
You can also make sure your CDNs and other 3rd parties rely on Anycast.Article from Patrick Meenan about the importance of Anycast and its impact on Web
performance.
http://www.nywebperformance.org/http://www.nywebperformance.org/http://www.nywebperformance.org/http://blog.patrickmeenan.com/2011/10/anycast-and-what-it-means-for-web.htmlhttp://blog.patrickmeenan.com/2011/10/anycast-and-what-it-means-for-web.htmlhttp://blog.patrickmeenan.com/2011/10/anycast-and-what-it-means-for-web.htmlhttp://blog.patrickmeenan.com/2011/10/anycast-and-what-it-means-for-web.htmlhttp://blog.catchpoint.com/2011/10/05/free-dns-hurts-web-performance/image003/http://blog.patrickmeenan.com/2011/10/anycast-and-what-it-means-for-web.htmlhttp://blog.patrickmeenan.com/2011/10/anycast-and-what-it-means-for-web.htmlhttp://www.nywebperformance.org/8/3/2019 Web Performance Insights 2011
17/47
--- Free DNS Can Hurt Web Performance! ---
17
In conclusion, make sure you rely on the right DNS service based on your needs. Just like any other
purchase, there is correlation between price, features and quality free or cheap services do not offer
the best speed and reliability and might lack some of the features you need. If speed is key to the
success of your company, invest money into a third party DNS service and make sure you configure it
right.
.
8/3/2019 Web Performance Insights 2011
18/47
18
Posted on March 25, 2011
Relying on Web PerformanceMonitoring to Discover Release
Problems
http://catchpointsystems.files.wordpress.com/2011/03/doc-server.jpg8/3/2019 Web Performance Insights 2011
19/47
--- Relying on Web Performance Monitoring to Discover Release Problems ---
19
n the 1990s the websites were quite simple, served by a single server talking to a single database,
JavaScript and Flash had just been introduced, AJAX was being developed, and HTTP 1.0 protocol
was prevalent across the World Wide Web. Now, years later and that same webpage has turned
into a complicated web of services, servers, and applications all working together to serve content to
the end-user.
Most websites rely on 2+ servers and services just to get the content of the base URL! Once the base
URL is loaded, its HTML has calls to even more internal and third party services like adservers, CDNs,
Content Personalization, Page Optimization, Tracking Pixels, Widgets, etc. The smallest mistake from
any of these services, internal or external, and the end user pays the price of bad experience and
frustration. The bad news for the company is that unlike in the 90s, when a user might not have a
choice to get the content elsewhere, today that same user can go to one of the 100s of competitors out
there in a blink of an eye.
Therefore optimizing the webpages and services for faster website performance and better fallback in
case of failure, has become very important, however, it is not enough.Continuous performance
monitoringof all the services involved in delivering you website has become a must for all companies.
Any un-expected performance degradations needs to be analyzed carefully and action taken before
there is any impact to business.
Case Study: New Website Release Impacts Web Performance for IE Users
We recently observed a major performance degradation with a very popular website in US, which we
were monitoring. The website performed a release on the night of March 22nd, during which time it was
down for about 2 hours. The day after the release the performance of the webpage slowed down by
100% going from 4.5 seconds to 9 seconds.
Response for the Base URL and the Webpage (Hourly Average)
I
http://www.catchpoint.com/http://www.catchpoint.com/http://www.catchpoint.com/http://www.catchpoint.com/http://blog.catchpoint.com/2011/03/25/relying_on_web_performance_monitoring_to_discover_release_problems/website-performance-monitoring-data/http://www.catchpoint.com/http://www.catchpoint.com/8/3/2019 Web Performance Insights 2011
20/47
--- Relying on Web Performance Monitoring to Discover Release Problems ---
20
Not only the response for entire webpage doubled, but also the base URL response slowed down by
80%. Looking at the requests and connections the webpage made, there was a jump in the number of
connections, however no increase in number of the items loaded on the page.
HTTP Connections and Hosts (Hourly Average)
Number of Items Requested (Hourly Average)
This was a clear sign that the hosts on the webpage were closing connections on every request. We
also confirmed the cause by looking at thewaterfall charts which showed 11 requests (including base
URL) utilized HTTP 1.0 and resulted in 11 different connections.
Number of Requests and Connections by Host
http://www.catchpoint.com/products.html#waterfallhttp://www.catchpoint.com/products.html#waterfallhttp://www.catchpoint.com/products.html#waterfallhttp://blog.catchpoint.com/2011/03/25/relying_on_web_performance_monitoring_to_discover_release_problems/connection-statistics-in-waterfall-chart/http://blog.catchpoint.com/2011/03/25/relying_on_web_performance_monitoring_to_discover_release_problems/website-performance-number-of-items-requested/http://blog.catchpoint.com/2011/03/25/relying_on_web_performance_monitoring_to_discover_release_problems/website-performance-number-of-http-connections/http://www.catchpoint.com/products.html#waterfall8/3/2019 Web Performance Insights 2011
21/47
--- Relying on Web Performance Monitoring to Discover Release Problems ---
21
The issue is also clear from the http headers of the request and of the response we can clearly see that
the site is utilizing HTTP 1.0 and closing the connection with the Connection: close HTTP header:
GET /layout/css/style-8194-1234.css?v=1234 HTTP/1.1Accept: */*Referer: https://www.SomeSite.com/
Accept-Language: en-usUser-Agent: Mozilla/5.0 (Windows; MSIE 9.0; Windows NT 6.1;Trident/5.0; BOIE9;ENUS)UA-CPU: x86Accept-Encoding: gzip, deflateHost: www.SomeSite.comConnection: Keep-AliveCookie: VisitorId=002.......
HTTP/1.0 200 OKDate: Fri, 25 Mar 2011 14:26:18 GMTServer: Apache-Coyote/1.1Last-Modified: Fri, 25 Mar 2011 08:13:57 GMTContent-Type: text/cssVary: Accept-EncodingContent-Encoding: gzipExpires: Thu, 15 Apr 2020 20:00:00 GMTCache-Control: privateConnection: close
The use of Connection: close had a bigger impact on the website performance, because the site was
utilizing HTTPS. As a result on every HTTP request the browser not only had to open a TCP
connection, but also had to establish an SSL handshake.
The other interesting fact we noticed was that the problem occurred only on Catchpoints Internet
Explorer agent, but not in the other agents we were testing from! The same requests were made by all
agents, however for IE the site used HTTP 1.0 while for the other browsers HTTP 1.1 .
We repeated the test on the IE agent and modified the user agent to exclude the MSIE string and
voil the server went back to using HTTP1.1 .
GET /layout/css/style-8194-1234.css?v=1234 HTTP/1.1Accept: */*Referer: www.SomeSite.comAccept-Language: en-usUser-Agent: Mozilla/5.0 (Windows; Windows NT 6.1;Trident/5.0; BOIE9;ENUS)UA-CPU: x86Accept-Encoding: gzip, deflateHost: www.SomeSite.comConnection: Keep-Alive
Cookie: VisitorId=002.......
HTTP/1.1 200 OKDate: Fri, 25 Mar 2011 14:35:10 GMTServer: Apache-Coyote/1.1Last-Modified: Fri, 25 Mar 2011 05:32:33 GMTContent-Type: text/cssVary: Accept-EncodingContent-Encoding: gzipExpires: Thu, 15 Apr 2020 20:00:00 GMTCache-Control: private
http://blogs.msdn.com/b/ieinternals/archive/2011/03/26/https-and-connection-close-is-your-apache-modssl-server-configuration-set-to-slow.aspxhttp://blogs.msdn.com/b/ieinternals/archive/2011/03/26/https-and-connection-close-is-your-apache-modssl-server-configuration-set-to-slow.aspxhttp://blogs.msdn.com/b/ieinternals/archive/2011/03/26/https-and-connection-close-is-your-apache-modssl-server-configuration-set-to-slow.aspxhttp://blogs.msdn.com/b/ieinternals/archive/2011/03/26/https-and-connection-close-is-your-apache-modssl-server-configuration-set-to-slow.aspxhttp://blogs.msdn.com/b/ieinternals/archive/2011/03/26/https-and-connection-close-is-your-apache-modssl-server-configuration-set-to-slow.aspx8/3/2019 Web Performance Insights 2011
22/47
--- Relying on Web Performance Monitoring to Discover Release Problems ---
22
Keep-Alive: timeout=15, max=94Connection: Keep-AliveTransfer-Encoding: chunked
The issue was caused by an old Apache configuration setting which by default forced HTTP 1.0 and
turned off Keep Alive for browser containing MSIE in the user agent string.
Summary
Websites have become more and more complicated relying on multiple services, servers, application
both managed by the website owner or outsourced to other third parties. These internal an external
dependencies have a direct impact on the web performance of the pages with varying impact.
Monitoring the web performance of the website continuously is key to ensuring its reliability.
http://newestindustry.org/2007/06/06/dear-apache-software-foundation-fix-the-msie-ssl-keepalive-settings/http://newestindustry.org/2007/06/06/dear-apache-software-foundation-fix-the-msie-ssl-keepalive-settings/http://newestindustry.org/2007/06/06/dear-apache-software-foundation-fix-the-msie-ssl-keepalive-settings/http://newestindustry.org/2007/06/06/dear-apache-software-foundation-fix-the-msie-ssl-keepalive-settings/http://newestindustry.org/2007/06/06/dear-apache-software-foundation-fix-the-msie-ssl-keepalive-settings/http://newestindustry.org/2007/06/06/dear-apache-software-foundation-fix-the-msie-ssl-keepalive-settings/8/3/2019 Web Performance Insights 2011
23/47
23
Posted July 4, 2011
Getting the Most Out ofPerformance Monitoring: Setting
Alert Thresholds
8/3/2019 Web Performance Insights 2011
24/47
--- Getting the Most Out of Performance Monitoring: Setting Alert Thresholds ---
24
common question with our customers is, Whats the best way to choose an alert threshold for
analyzing my webpage response time? It is a tricky question, and one whose answer varies
case by case. Set the threshold too low and youll be distracted by or worse, dismissive of
alerts as they fill up your inbox. But set it too high, and you might not know when some of your end
users are having an unacceptably slow site experience. Choosing response time alerts is very much a
balancing act.
To illustrate this point, lets look at a case from an actual Catchpoint customer who recently went
through the exercise of setting alert thresholds. First, they looked at their sites average response times
over the course of a week. A common practice is to take the average, add a little extra as a buffer, and
presto alerts are set!
For this customer, the average (Chart 1) was a little under 7 seconds 6,834 ms, to be exact. Adding
a little buffer, they set the alert threshold at 11 seconds. Unfortunately and unexpectedly the 11-
second threshold yielded about a gazillion alerts for our customer. So what happened?
The problem in this case has to do with variability of site usage and deviation from the mean. If you
look carefully at Chart 1, you will see that the valleys occur during off business hours, and the peaks
occur during the day. What the chart is not showing is that during business hours, there is significant
variability in response time. Looking at Chart 2, a scatterplot of the values measured over the same
period, you can see that the distribution of response times is far wider than Chart 1 would have youbelieve. In fact, the averages in Chart 1 never exceed 18,000 ms, whereas in Chart 2, we plainly see
that there are dozens of instances of response times in excess of 20,000 ms.
A
http://blog.catchpoint.com/2011/07/04/getting_the_most_out_of_performance_monitoring_setting_alert_thresholds/webresponseaverage-2/8/3/2019 Web Performance Insights 2011
25/47
--- Getting the Most Out of Performance Monitoring: Setting Alert Thresholds ---
25
Its obvious from Chart 2 that an 11 second alert threshold will trigger a lot of alerts. When youre using
simple average over a period time to set alerts, youre ignoring the fact that the average is only an
average. To set an alert you have to understand the data better and you need to dig deeper.
In Chart 3, we see the 95th percentile meaning that 5% of the samples had response times as slow or
slower. This is where you can look to get a better picture of a sites performance at worst-case
scenario. In the worst cases, the page is taking 24 seconds to load! So, what would you do? Would
you set the alert level at 24,000 ms? 20,000 ms? 15,000 ms? Its a balancing act.
An alternative to the 95% is to rely on moving average, which relies on a subset of data based on a
time frame. Catchpoint alerts support the ability to specify a dynamic threshold based on the average of
a previous set of time. For example alert if response is 50% above the last 15 minute average. This
solution allows you to take into consideration recent data to determine if the application performance
went down.
At the end of the day, its going to be a judgment call. Only you can decide what the proper level is for
alert threshold, but we can tell you one thing for sure: you wont find the answer by just looking at your
averages.
http://blog.catchpoint.com/2011/07/04/getting_the_most_out_of_performance_monitoring_setting_alert_thresholds/95th/http://blog.catchpoint.com/2011/07/04/getting_the_most_out_of_performance_monitoring_setting_alert_thresholds/scatter-3/8/3/2019 Web Performance Insights 2011
26/47
26
Posted May 26, 2011
Three Key Steps to SuccessfulWeb Performance Benchmarking
8/3/2019 Web Performance Insights 2011
27/47
--- Three Key Steps to Successful Web Performance Benchmarcking ---
27
ne of the frequent questions I receive from clients is on How do I benchmark my
performance against the competition?. There are different approaches to benchmarking,
some better than others. The key to a successful benchmark, is to plan it carefully and collect
the right data points.
I recommend companies to follow the following 3 steps:
1. Define the end goal of the benchmark. Ask yourself what will you do with this data? Are you
trying to improve your website, a webpage, or a process? Are you trying to build a business
case for a project or initiative?
2. Determine which areas of the site/application/system you will need to benchmark. If you
are benchmarking to figure out your infrastructure distribution, you might care more about DNS
and performance on the geographical areas of your end users. If you are planning on
redesigning/rebuilding the site, you might care about the full performance of key webpages or
key processes like a shopping cart.
3. Determine what tests to run, from what locations, and the frequency. Based on the
purpose of the benchmark, and benchmark areas, you can determine how and where to test
from. For example you might decide to benchmark DNS performance from key US states that
account for the majority of your end users. You might decide to run the tests every 10 minutes,
if you are planning major changes, or every 30 minutes if you are simply using the data for a
business case.
Over the years I have come across several benchmarks that failed for various reasons. Some of the
major pitfalls are:
Comparing Apples and Oranges. Sadly one of the biggest mistakes is not comparing the correct
items. If you arebenchmarking DNS performance, you cant simply average the DNS time of
HTTP requests. If you have a DNS TTL of 5 minutes, and your competitor has a TTL of 15
minute, the averages will lie.
Looking only at averages. If you are looking at an average across different cities, you might lose
sight of issues.
Looking at a metric without understanding what it means. Quite often people just pay attention
to the full load time of a webpage, and ignore the rest of the data. However, webpages are
different and the full time to load the page might not be the same across especially when
pages are dynamically modifying the content.
O
http://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.html8/3/2019 Web Performance Insights 2011
28/47
--- Three Key Steps to Successful Web Performance Benchmarcking ---
28
Looking only at one metric. You collected all this data, but looking only at one metric is not going
to help you. Dig deeper into the data so you can understand why others are better or worst.
Lear from your competitors success or failure, so you can improve.
Case Study: E-commerce Benchmark
Recently we assisted an e-commerce customer that hadcreated a benchmark in Catchpointto
compare how the homepages of key competitors ranked. The benchmark included the homepages
ofBestBuy,Amazon,Apple, andNewegg. The goal was to understand where their homepage ranked
relative to their competitors, and to determine the steps to improve their web performance.
Based on the data collected they came to the conclusion that the homepage of Apple.com was the
fastest. There are several factors on why Apples homepage is fast:
The Response of the url, the total time from issuing the request to receiving the entire HTML of
the page, was really fast.
Less Downloaded bytes, Apples homepage was 30-50% lighter.
Less Requests and Hosts on the page.
http://www.catchpoint.com/http://www.catchpoint.com/http://www.catchpoint.com/http://www.bestbuy.com/http://www.bestbuy.com/http://www.bestbuy.com/http://www.amazon.com/http://www.amazon.com/http://www.amazon.com/http://www.apple.com/http://www.apple.com/http://www.apple.com/http://www.newegg.com/http://www.newegg.com/http://www.newegg.com/http://blog.catchpoint.com/2011/05/26/benchmarks-comparisons/response1/http://www.newegg.com/http://www.apple.com/http://www.amazon.com/http://www.bestbuy.com/http://www.catchpoint.com/8/3/2019 Web Performance Insights 2011
29/47
--- Three Key Steps to Successful Web Performance Benchmarcking ---
29
This might seem like a successful benchmark, however, there was one little issue that made the
benchmark inaccurate.
The goal of the client was to compare the homepages of the competing ecommerce sites. But in the
case of Apple they were testing the corporate homepage, which had a different business goal and
therefore different design and implementation. The homepage of the e-commerce site for Apple
isstore.apple.com and notwww.apple.com.
When benchmarking to the correct e-commerce site of Apple, the picture changed. Apple was not much
faster than the rest of the stores. (I kept the home page of Apple to show the differences).
http://store.apple.com/http://store.apple.com/http://store.apple.com/http://www.apple.com/http://www.apple.com/http://www.apple.com/http://blog.catchpoint.com/2011/05/26/benchmarks-comparisons/con-exhosts-objects1/http://blog.catchpoint.com/2011/05/26/benchmarks-comparisons/size1/http://www.apple.com/http://store.apple.com/8/3/2019 Web Performance Insights 2011
30/47
--- Three Key Steps to Successful Web Performance Benchmarcking ---
30
http://blog.catchpoint.com/2011/05/26/benchmarks-comparisons/con-exhosts-objects2/http://blog.catchpoint.com/2011/05/26/benchmarks-comparisons/size2/http://blog.catchpoint.com/2011/05/26/benchmarks-comparisons/response2/8/3/2019 Web Performance Insights 2011
31/47
--- Three Key Steps to Successful Web Performance Benchmarcking ---
31
To get a better look at the impact at user experience, we also looked at other metrics liketime to
title andrender start time.
Visually, this is what it looked like loading those 5 sites from a node located in New York on the Verizon
Backbone (using a 400 ms timer, the blink of an eye is 400 ms).
http://blog.catchpoint.com/2010/12/30/dec2010release/http://blog.catchpoint.com/2010/12/30/dec2010release/http://blog.catchpoint.com/2010/12/30/dec2010release/http://blog.catchpoint.com/2010/09/20/catchpoint-september-release/http://blog.catchpoint.com/2010/09/20/catchpoint-september-release/http://blog.catchpoint.com/2010/09/20/catchpoint-september-release/http://blog.catchpoint.com/2011/05/26/benchmarks-comparisons/newyorkverizon/http://blog.catchpoint.com/2011/05/26/benchmarks-comparisons/response3-2/http://blog.catchpoint.com/2010/09/20/catchpoint-september-release/http://blog.catchpoint.com/2010/12/30/dec2010release/http://blog.catchpoint.com/2010/12/30/dec2010release/8/3/2019 Web Performance Insights 2011
32/47
--- Three Key Steps to Successful Web Performance Benchmarcking ---
32
We also implemented the use ofApdex, an excellent way to score and compare numbers from diverse
pages. Apdex normalizes the data based on target goals, which vary from webpage to webpage (as we
saw with Apple). For demonstration purposes I used an Apdex target response of 5,000 ms (5 seconds)
for all the tests above.
To sum it up, a successful benchmark depends on clear end goals, everything else depends on it.
Happy Benchmarking!
Methodology: all sites were measured using 26 US Nodes, every 10 minutes with Internet Explorer 8
http://apdex.org/http://apdex.org/http://apdex.org/http://blog.catchpoint.com/2011/05/26/benchmarks-comparisons/apdex/http://apdex.org/8/3/2019 Web Performance Insights 2011
33/47
--- A New Blind Spot for Web Performance Tools ---
33
Posted on March 2, 2011
A New Blind Spot for WebPerformance Tools
8/3/2019 Web Performance Insights 2011
34/47
--- A New Blind Spot for Web Performance Tools ---
34
ebperformance tools rely on actual browsers to capture the performance data of a
webpage and any requests made by a webpage. Monitoring from the browser, comes with
some limitations due to the complexity of the browser and the internet, causing at times
what we call Blind Spots. A blind spot occurs when the data provided by a tool lacks clarity.
The main blind spot with external monitoring is that you cannot always distinguish between network
andapplication performance. At Catchpoint we introducedCatchpoint Insight and other features to
remove this limitation and facilitate the understanding of the performance factors.
Recently we came across another blind spot related to monitoring tools built on top o f Internet
Explorer. We internally refer to it as Objects in IE might be faster than they appear.
It all started when a client of ours engaged us in a performance study regarding the impact of their
Iframe tag on the pages of one of their clients. Their client was observingnetwork timeson the Iframe
call that were quite high on an IE7 based performance monitoring service. We were able to reproduce
the problem on the webpage with various tools likeHTTPwatch,DynaTrace Ajax,Webpagetest,IE9
Developer tools, and even in the Catchpoint IE monitor.
The performance numbers we observed made no sense! The response content of the Iframe URL was
less than 1000 bytes (it fits in a single TCP packet), yet the tools were displaying 500ms+ for the time
from the1st byte to last byteof the HTTP Content. The only way this could happen is if there was
something wrong at the TCP level and packets were fragmented and lost.
To ensure it was not an issue at the TCP level, we utilizedWiresharkto capture the TCP packets as we
were monitoring the pages with the other tools and mapped the data from Wireshark to the metrics
displayed in the tools. The data confirmed that the URL content was delivered always in a single packet
and theURL responsewas less than 100ms. However, the monitoring tools built on top of IE still
showed that 1st to last byte was 500ms or more for the same request! Clearly a new blind spot with
IE!
Since we proved it was not the network, the only other possibility was that something happened during
the browser execution! We looked through the 20+ JavaScript files referenced on the webpage and we
determined that the page executed JavaScript code when theDOMContentLoadedevent was reached.
The event is not native in pre IE9 browsers, and the page relied on one of two
solutions:doScroll()orscript deferto approximate when the event has been reached. Once the
event was fired, JavaScript on the page made DOM modifications that were time consuming. However,
this JavaScript execution time was not being displayed on the tools as gap.
W
http://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://blog.catchpoint.com/2010/08/17/performance-picture-not-sharp-enough/http://blog.catchpoint.com/2010/08/17/performance-picture-not-sharp-enough/http://blog.catchpoint.com/2010/09/17/anatomyhttp/http://blog.catchpoint.com/2010/09/17/anatomyhttp/http://blog.catchpoint.com/2010/09/17/anatomyhttp/http://www.httpwatch.com/http://www.httpwatch.com/http://ajax.dynatrace.com/http://ajax.dynatrace.com/http://ajax.dynatrace.com/http://www.webpagetest.org/http://www.webpagetest.org/http://www.webpagetest.org/http://msdn.microsoft.com/en-us/library/gg130952(v=VS.85).aspxhttp://msdn.microsoft.com/en-us/library/gg130952(v=VS.85).aspxhttp://msdn.microsoft.com/en-us/library/gg130952(v=VS.85).aspxhttp://msdn.microsoft.com/en-us/library/gg130952(v=VS.85).aspxhttp://blog.catchpoint.com/2010/09/17/anatomyhttp/http://blog.catchpoint.com/2010/09/17/anatomyhttp/http://blog.catchpoint.com/2010/09/17/anatomyhttp/http://www.wireshark.org/http://www.wireshark.org/http://www.wireshark.org/http://blog.catchpoint.com/2010/09/17/anatomyhttp/http://blog.catchpoint.com/2010/09/17/anatomyhttp/http://blog.catchpoint.com/2010/09/17/anatomyhttp/https://developer.mozilla.org/en/Gecko-Specific_DOM_Eventshttps://developer.mozilla.org/en/Gecko-Specific_DOM_Eventshttps://developer.mozilla.org/en/Gecko-Specific_DOM_Eventshttp://javascript.nwbox.com/IEContentLoaded/http://javascript.nwbox.com/IEContentLoaded/http://javascript.nwbox.com/IEContentLoaded/http://dean.edwards.name/weblog/2006/06/again/http://dean.edwards.name/weblog/2006/06/again/http://dean.edwards.name/weblog/2006/06/again/http://dean.edwards.name/weblog/2006/06/again/http://javascript.nwbox.com/IEContentLoaded/https://developer.mozilla.org/en/Gecko-Specific_DOM_Eventshttp://blog.catchpoint.com/2010/09/17/anatomyhttp/http://www.wireshark.org/http://blog.catchpoint.com/2010/09/17/anatomyhttp/http://msdn.microsoft.com/en-us/library/gg130952(v=VS.85).aspxhttp://msdn.microsoft.com/en-us/library/gg130952(v=VS.85).aspxhttp://www.webpagetest.org/http://ajax.dynatrace.com/http://www.httpwatch.com/http://blog.catchpoint.com/2010/09/17/anatomyhttp/http://blog.catchpoint.com/2010/08/17/performance-picture-not-sharp-enough/http://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.html8/3/2019 Web Performance Insights 2011
35/47
--- A New Blind Spot for Web Performance Tools ---
35
To test what was happening, we created several simple pages that contained an Iframe pointing to
URL. The pages also contained a JavaScript that created 10,000 spans and appended them to a DIV
on the page. The JavaScript execution would vary on each page and rely on:
1.the doScroll() method to detect DOMContentLoaded and execute
2.the Script Defer method to detect DOMContentLoaded and execute
3.the native DOMContentLoaded for IE9 to execute the script
4.inline execution below the Iframe tag
In all four test cases we observed that all the tools, including IE9 developer tools, always included the
time of the JavaScript execution to the network time of the Iframe request! We replicated the test cases
with an image in place of the Iframe, and were unable to reproduce the same results. Interestingly the
issue did not occur on Firefox and Chrome on Windows but both clearly showed there was JavaScript
executing and delaying rendering of the Iframe content.
Dynatrace IE7 Inline
HTTPWatch IE7 Script Defer
http://www.catchpoint.com/blog_assets/new_blind_spot/doscroll.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/doscroll.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/doscroll.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/script_defer.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/script_defer.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/script_defer.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/domcontentloaded.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/domcontentloaded.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/inline.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/inline.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/inline.htmlhttp://catchpointsystems.files.wordpress.com/2011/03/httpwatch_ie7_script_defer1.gifhttp://catchpointsystems.files.wordpress.com/2011/03/dyna_ie7_inline1.gifhttp://www.catchpoint.com/blog_assets/new_blind_spot/inline.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/domcontentloaded.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/script_defer.htmlhttp://www.catchpoint.com/blog_assets/new_blind_spot/doscroll.html8/3/2019 Web Performance Insights 2011
36/47
--- A New Blind Spot for Web Performance Tools ---
36
IE9 Developer Toolbar IE9 DOMContentLoaded
Webpagtest IE8 doScroll()
We believe the problem occurs due to the fact that the browser executes JavaScript in a single
threaded mode and it takes precedence over the Iframe creation. The monitoring tools are relying on
the browser to tell them when the Iframe is complete, but the IE browser does not mark the Iframe
complete until the JavaScript execution is complete. Hence, the JavaScript execution time is included in
the Iframe response time!
This means that monitoring tools relying on Internet Explorer might append the time to execute
JavaScript to the Iframe request time, if the JavaScript executes right after the Iframe request starts.
This does not mean that the server serving the Iframe is slow and it does not mean that the Iframe
slowed down the page. It simply means the JavaScript time was incorrectly attached to the iframe
request. So the next time you see a very slow request in a monitoring tool, try the request standalone to
ensure it is the request and not something else on the page.
AtCatchpointwe understand such blind spots have an impact on ourusers, therefore we have
already started development work to address this issue on ourwaterfall charts. Our IE
based monitor will be able to clearly distinguish between the network request time, and the JavaScript
execution time.
- Catchpoint Team
http://www.catchpoint.com/http://www.catchpoint.com/http://www.catchpoint.com/http://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/products.htmlhttp://catchpointsystems.files.wordpress.com/2011/03/iedev_ie9_domloaded1.gifhttp://catchpointsystems.files.wordpress.com/2011/03/wp_ie8_doscroll1.gifhttp://www.catchpoint.com/products.htmlhttp://www.catchpoint.com/8/3/2019 Web Performance Insights 2011
37/47
37
Posted April 25, 2011
My 2 cents on the AWS Failure andlessons learned from the past
8/3/2019 Web Performance Insights 2011
38/47
--- My 2 cents on the AWS failure and lessons learned from the past ---
38
lot has been published already about AWS EC2 failure, I wanted to add my 2 cents on the
issue as it reminded me of a notorious event that happened toDoubleClick in August 2000.
What AWS and their customers experienced is unfortunate, but it will and it can happen to
anyone! In IT we are dealing with various complex systems hardware, software, and people things
are bound to break at some point. Failure is not limited to IT, human history is full of such failures with
automobile recalls,bank failures,nuclear disasters,collapsing bridges. What people should understand
is that failure is bound to happen, be ready for it, and learn from it to avoid it in the future.
Lets be real very few companies out there have the money and resources to have a redundant
transactional systems running in parallel which can act as back up. For most companies you just have
to fail nicely. You should have plans and processes to deal with everything from troubleshooting the
failure, recovering from failure, to notifying customers of the failure, and most importantly architect your
application and systems so they fail nicely and can recover from such failures.
Companies that have websites or web application must be able to redirect all requests to Service is
down webpage. Mobile or desktop applications relying on APIs might need to have special logic built-
in for such failures. However, if you are a company delivering services to other website via tags, like
adserving or widgets, things get a little more complicated. You cannot remove the tags from the
webpages, unless your clients build it in their pages. You need to ensure you can deliver from
another location enough to ensure your tags do not impact the web performance and usability of your
clients websites!
Back at DoubleClick we ran a fairly large infrastructure delivering billions of impressions, the DART tags
are present on almost every major website. One day in 2000 we had a really bad outage and our tags
stopped working because theadserving system experienced a catastrophic meltdown.Customers
were not happy, but they understood that technology fails sometimes, and they had SLAs to protect
them. What they were most unhappy about was that the DoubleClick ad tag had such an incredible
impact on the performance of their sites. Webpages came to a crawl or stopped loading, the user
experience was horrible! Our client couldnt recover from our failure some were able to remove the
tags via their Content Management Systems but others just had to suffer from our failure.
So we went back to the drawing board and built a complete secondary system capable of handling the
billions of ad calls but that will only deliver 11 pixels or empty JavaScript. So in case of a major outage
the ads would not work but at least would not take down the entire customers site with us and their
A
http://www.clickz.com/clickz/news/1699040/software-glitch-affects-doubleclicks-domestic-clientshttp://www.clickz.com/clickz/news/1699040/software-glitch-affects-doubleclicks-domestic-clientshttp://www.clickz.com/clickz/news/1699040/software-glitch-affects-doubleclicks-domestic-clientshttp://en.wikipedia.org/wiki/2010_Toyota_vehicle_recallshttp://en.wikipedia.org/wiki/2010_Toyota_vehicle_recallshttp://en.wikipedia.org/wiki/Lehman_Brothershttp://en.wikipedia.org/wiki/Lehman_Brothershttp://en.wikipedia.org/wiki/Lehman_Brothershttp://en.wikipedia.org/wiki/Three_Mile_Island_accidenthttp://en.wikipedia.org/wiki/Three_Mile_Island_accidenthttp://en.wikipedia.org/wiki/Three_Mile_Island_accidenthttp://en.wikipedia.org/wiki/Tacoma_Narrows_Bridgehttp://en.wikipedia.org/wiki/Tacoma_Narrows_Bridgehttp://en.wikipedia.org/wiki/Tacoma_Narrows_Bridgehttp://www.clickz.com/clickz/news/1699040/software-glitch-affects-doubleclicks-domestic-clientshttp://www.clickz.com/clickz/news/1699040/software-glitch-affects-doubleclicks-domestic-clientshttp://www.clickz.com/clickz/news/1699040/software-glitch-affects-doubleclicks-domestic-clientshttp://en.wikipedia.org/wiki/Tacoma_Narrows_Bridgehttp://en.wikipedia.org/wiki/Three_Mile_Island_accidenthttp://en.wikipedia.org/wiki/Lehman_Brothershttp://en.wikipedia.org/wiki/2010_Toyota_vehicle_recallshttp://www.clickz.com/clickz/news/1699040/software-glitch-affects-doubleclicks-domestic-clients8/3/2019 Web Performance Insights 2011
39/47
--- My 2 cents on the AWS failure and lessons learned from the past ---
39
user experience. That Dot system was never used in real life, but was always there in case we
needed it.
The first lesson for companies that provide services to other websites is to not rely on a single vendor
for hosting and spare a few hundred dollars and get a backup plan. So next time AWS or anyone else
goes down, you will not have impacted the user experience of the folks visiting your customers site.
And once you have that backup system in place, test it every frequently! Make sure the right folks know
when to pull the trigger and the system can handle it (capacity).
The second lesson is about diversification; do not put all your eggs in one basket. If you go with vendor
A for hosting, choose vendor B for DNS, choose vendor C for CDN
Lastly, if you are website relying on 3rd party vendors, make sure you monitor them. Also learn about
their technology and their vendors, who they are relying on for hosting their technology, who is their
DNS provider, and most importantly what are their back up plans in case that tag comes to a crawl!
The cloud is great, it is the future of IT -but do not drink too much of the kool-aid or cloud-aid, be
ready for outages and failures!
Mehdi - one of the guys who handled those angry customer phone calls in 2000.
For more about the AWS issue :The Big List of Articles on the Amazon Outage
http://highscalability.com/blog/2011/4/25/the-big-list-of-articles-on-the-amazon-outage.htmlhttp://highscalability.com/blog/2011/4/25/the-big-list-of-articles-on-the-amazon-outage.htmlhttp://highscalability.com/blog/2011/4/25/the-big-list-of-articles-on-the-amazon-outage.htmlhttp://highscalability.com/blog/2011/4/25/the-big-list-of-articles-on-the-amazon-outage.html8/3/2019 Web Performance Insights 2011
40/47
40
Posted April 29, 2011
Royal Wedding and the Internet
8/3/2019 Web Performance Insights 2011
41/47
8/3/2019 Web Performance Insights 2011
42/47
--- Royal Wedding & The Internet ---
42
Other Sites:
BBC Scatterplot view:
Yahoo Scatterplot view:
http://blog.catchpoint.com/2011/04/29/royal-wedding-and-web-performance-impact/yahooscatter/http://blog.catchpoint.com/2011/04/29/royal-wedding-and-web-performance-impact/bbc/http://blog.catchpoint.com/2011/04/29/royal-wedding-and-web-performance-impact/others-2/8/3/2019 Web Performance Insights 2011
43/47
--- Royal Wedding & The Internet ---
43
We also monitored the performance of the major CDNs
(Edgecast,Cotendo,Akamai,Limelight,CDnetworks) but the data did not reflect any major impact on
their performance.
URls monitored:
BBC:http://www.bbc.co.uk/news/uk-11767495
CNN:http://edition.cnn.com/SPECIALS/2011/royal.wedding/live/
MSNBC:http://windsorknot.today.com/
Facebook:http://www.facebook.com/event.php?eid=101946883225381
Twitter:http://twitter.com/#!/ClarenceHouse
Youtube:http://www.youtube.com/user/TheRoyalChannel
Official Site:http://www.officialroyalwedding2011.org/
Telegraph:http://www.telegraph.co.uk/news/uknews/royal-wedding/
Yahoo:http://royalwedding.yahoo.com/
Definitions:
Response Time: The time it takes from the request being issued by the browser, to the Last
Byte received from the server for the primary URL.
Web Page Response Time: The time it takes from the request being issued, to receiving the
Last Byte of the final element on the page. It reflects the impact of all the requests in the
webpage.
Wait Time: The time from the connection to the server established and the request sent, to the
First Byte of response from the server. It reflects the performance of the server in processingthe request.
http://www.edgecast.com/http://www.edgecast.com/http://www.edgecast.com/http://www.cotendo.com/http://www.cotendo.com/http://www.akamai.com/http://www.akamai.com/http://www.akamai.com/http://www.limelightnetworks.com/http://www.limelightnetworks.com/http://www.limelightnetworks.com/http://www.us.cdnetworks.com/http://www.us.cdnetworks.com/http://www.bbc.co.uk/news/uk-11767495http://www.bbc.co.uk/news/uk-11767495http://www.bbc.co.uk/news/uk-11767495http://edition.cnn.com/SPECIALS/2011/royal.wedding/live/http://edition.cnn.com/SPECIALS/2011/royal.wedding/live/http://edition.cnn.com/SPECIALS/2011/royal.wedding/live/http://windsorknot.today.com/http://windsorknot.today.com/http://windsorknot.today.com/http://www.facebook.com/event.php?eid=101946883225381http://www.facebook.com/event.php?eid=101946883225381http://www.facebook.com/event.php?eid=101946883225381http://twitter.com/#!/ClarenceHousehttp://twitter.com/#!/ClarenceHousehttp://twitter.com/#!/ClarenceHousehttp://www.youtube.com/user/TheRoyalChannelhttp://www.youtube.com/user/TheRoyalChannelhttp://www.youtube.com/user/TheRoyalChannelhttp://www.officialroyalwedding2011.org/http://www.officialroyalwedding2011.org/http://www.officialroyalwedding2011.org/http://www.telegraph.co.uk/news/uknews/royal-wedding/http://www.telegraph.co.uk/news/uknews/royal-wedding/http://www.telegraph.co.uk/news/uknews/royal-wedding/http://royalwedding.yahoo.com/http://royalwedding.yahoo.com/http://royalwedding.yahoo.com/http://royalwedding.yahoo.com/http://www.telegraph.co.uk/news/uknews/royal-wedding/http://www.officialroyalwedding2011.org/http://www.youtube.com/user/TheRoyalChannelhttp://twitter.com/#!/ClarenceHousehttp://www.facebook.com/event.php?eid=101946883225381http://windsorknot.today.com/http://edition.cnn.com/SPECIALS/2011/royal.wedding/live/http://www.bbc.co.uk/news/uk-11767495http://www.us.cdnetworks.com/http://www.limelightnetworks.com/http://www.akamai.com/http://www.cotendo.com/http://www.edgecast.com/8/3/2019 Web Performance Insights 2011
44/47
44
Posted December 19, 2011
WPO Resolution for 2012!
8/3/2019 Web Performance Insights 2011
45/47
--- WPO Resolution for 2012! ---
45
s I look back at the state of 2011 web operations, the thing that impressed me the most was
the success of the Web Performance Optimization (WPO) movement. Comparing it to recent
world events, I think this movement is the Arab Spring of the Web Development and
Operations community. Web Performance meetups launched everywhere around the world,Velocity
Conferencegot a passport and travelled to Europe and China, the number of people interested and
invested in this subject have exploded, and so have the number of companies and investments in this
field.
The success of this movement is mostly due to the hard work of several individuals and companies in
the last 5 yearsSteve Souder,Patrick Meenan,OReilly,GoogleSpeed initiatives,Joshua
Bixby,Sergey Chernyshev,Stoyan Stefanov,Alexander Podelko, and so many others. Thanks to them
WPO and Web Performance Monitoring is no longer reserved to the few 1% who can afford fast
servers and bright engineers. The techniques to speed web sites have been documented
(Yslow,Google Pagespeed), books have been published (High Performance Web Sites: Essential
Knowledge for Front-End Engineers by Steve Souders,Web Performance Tuning: Speeding Up the
Web,Building Scalable Web Sites: Building, Scaling, and Optimizing the Next Generation of Web
Applications,Even Faster Web Sites: Performance Best Practices for Web Developers), and automated
optimization tools have flourished likeaiCache,Strangeloops,Blaze.io, etc.
This movement has been amazing as it makes the web experience for end users faster and better.
Another positive development has been also sharing that is taking place in this industry thanks to sites
likePerfPlanet, twitter and engineerings teams are quick to share their latest experiments and results
likeWayfair,NetflixandEtsy.
But as 2011 is winding down I am still perplexed by the lack of implementation of 2 major WPO best
practices that give the most performance boost without any development efforts or new hardware:
HTTP Compression and Persistent HTTP Connections.
I am not sure if this is by choice or negligence or a combination of the two. In regards to compression,
when there is a CDN involved I think its mostly negligence because CDNs do not automatically turn on
compression. I have been on way too many calls where I have heard oh we forgot to turn that on.
Please compress HTTP on your own servers and ensure your CDN has compression enabled for your
account.
While we monitored the top 50 retailers on Cyber Monday, the Sony.com homepage downloaded
around 2.6 mb of data, of which 1.2 mb where un-compressed CSS and JS! In this case their CDN is
A
http://velocityconf.com/velocity2012http://velocityconf.com/velocity2012http://velocityconf.com/velocity2012http://velocityconf.com/velocity2012http://www.stevesouders.com/blog/http://www.stevesouders.com/blog/http://www.stevesouders.com/blog/http://blog.patrickmeenan.com/http://blog.patrickmeenan.com/http://velocityconf.com/velocity2012http://velocityconf.com/velocity2012http://velocityconf.com/velocity2012http://code.google.com/speed/http://code.google.com/speed/http://code.google.com/speed/http://www.webperformancetoday.com/http://www.webperformancetoday.com/http://www.webperformancetoday.com/http://www.webperformancetoday.com/http://www.sergeychernyshev.com/blog/http://www.sergeychernyshev.com/blog/http://www.sergeychernyshev.com/blog/http://www.phpied.com/http://www.phpied.com/http://www.phpied.com/http://www.alexanderpodelko.com/http://www.alexanderpodelko.com/http://www.alexanderpodelko.com/http://developer.yahoo.com/performance/rules.htmlhttp://developer.yahoo.com/performance/rules.htmlhttp://developer.yahoo.com/performance/rules.htmlhttp://code.google.com/speed/page-speed/http://code.google.com/speed/page-speed/http://code.google.com/speed/page-speed/http://www.amazon.com/exec/obidos/ASIN/0596529309/webperforinchttp://www.amazon.com/exec/obidos/ASIN/0596529309/webperforinchttp://www.amazon.com/exec/obidos/ASIN/0596529309/webperforinchttp://www.amazon.com/exec/obidos/ASIN/0596529309/webperforinchttp://www.amazon.com/exec/obidos/ASIN/1565923790/webperforinchttp://www.amazon.com/exec/obidos/ASIN/1565923790/webperforinchttp://www.amazon.com/exec/obidos/ASIN/1565923790/webperforinchttp://www.amazon.com/exec/obidos/ASIN/1565923790/webperforinchttp://www.amazon.com/Building-Scalable-Web-Sites-Applications/dp/0596102356/ref=pd_sim_b_2http://www.amazon.com/Building-Scalable-Web-Sites-Applications/dp/0596102356/ref=pd_sim_b_2http://www.amazon.com/Building-Scalable-Web-Sites-Applications/dp/0596102356/ref=pd_sim_b_2http://www.amazon.com/Building-Scalable-Web-Sites-Applications/dp/0596102356/ref=pd_sim_b_2http://www.amazon.com/Even-Faster-Web-Sites-Performance/dp/0596522304/ref=pd_bxgy_b_text_bhttp://www.amazon.com/Even-Faster-Web-Sites-Performance/dp/0596522304/ref=pd_bxgy_b_text_bhttp://www.amazon.com/Even-Faster-Web-Sites-Performance/dp/0596522304/ref=pd_bxgy_b_text_bhttp://aicache.com/http://aicache.com/http://aicache.com/http://www.strangeloopnetworks.com/http://www.strangeloopnetworks.com/http://www.strangeloopnetworks.com/http://www.blaze.io/http://www.blaze.io/http://www.blaze.io/http://www.perfplanet.com/http://www.perfplanet.com/http://www.perfplanet.com/http://engineering.wayfair.com/http://engineering.wayfair.com/http://engineering.wayfair.com/http://techblog.netflix.com/http://techblog.netflix.com/http://techblog.netflix.com/http://codeascraft.etsy.com/http://codeascraft.etsy.com/http://codeascraft.etsy.com/http://codeascraft.etsy.com/http://techblog.netflix.com/http://engineering.wayfair.com/http://www.perfplanet.com/http://www.blaze.io/http://www.strangeloopnetworks.com/http://aicache.com/http://www.amazon.com/Even-Faster-Web-Sites-Performance/dp/0596522304/ref=pd_bxgy_b_text_bhttp://www.amazon.com/Building-Scalable-Web-Sites-Applications/dp/0596102356/ref=pd_sim_b_2http://www.amazon.com/Building-Scalable-Web-Sites-Applications/dp/0596102356/ref=pd_sim_b_2http://www.amazon.com/exec/obidos/ASIN/1565923790/webperforinchttp://www.amazon.com/exec/obidos/ASIN/1565923790/webperforinchttp://www.amazon.com/exec/obidos/ASIN/0596529309/webperforinchttp://www.amazon.com/exec/obidos/ASIN/0596529309/webperforinchttp://code.google.com/speed/page-speed/http://developer.yahoo.com/performance/rules.htmlhttp://www.alexanderpodelko.com/http://www.phpied.com/http://www.sergeychernyshev.com/blog/http://www.webperformancetoday.com/http://www.webperformancetoday.com/http://code.google.com/speed/http://velocityconf.com/velocity2012http://blog.patrickmeenan.com/http://www.stevesouders.com/blog/http://velocityconf.com/velocity2012http://velocityconf.com/velocity20128/3/2019 Web Performance Insights 2011
46/47
--- WPO Resolution for 2012! ---
46
Akamai. (See link to HTTP ARCHIVE 11/15/2011). The compression at the CDN level must be ON by
default and not the other way around.
Persistent Connection or Keep Alive is a feature of HTTP 1.1 which allows a browser to re-utilize an
existing connection with the server. Today almost all web servers and browsers support HTTP 1.1
Keep Alive and there is no reason why so many sites still do not have it enabled. The biggest
advantage is that it eliminated the need to establish an HTTP connection for every request to the server
which can quickly add up.
So on the 2nd of January 2012, please take the time to make a call, send an email, fax, or even
send pigeons with these 2 instructions to your operations or devops and CDN account rep:
Turn on Compressions for all HTML, scripts, css and text resources; and make sure Keep Alive
is on!
You will save money on bandwidth, your end-user will be happier and your systems and network will be
less bloated!
http://httparchive.org/viewsite.php?u=http%3A%2F%2Fwww.sonystyle.com%2F&l=Nov%2015%202011http://httparchive.org/viewsite.php?u=http%3A%2F%2Fwww.sonystyle.com%2F&l=Nov%2015%202011http://httparchive.org/viewsite.php?u=http%3A%2F%2Fwww.sonystyle.com%2F&l=Nov%2015%202011http://en.wikipedia.org/wiki/HTTP_persistent_connectionhttp://en.wikipedia.org/wiki/HTTP_persistent_connectionhttp://en.wikipedia.org/wiki/HTTP_persistent_connectionhttp://en.wikipedia.org/wiki/HTTP_persistent_connectionhttp://httparchive.org/viewsite.php?u=http%3A%2F%2Fwww.sonystyle.com%2F&l=Nov%2015%2020118/3/2019 Web Performance Insights 2011
47/47
Make Users Happy,
Have a Fast Website.
Don't let slowness and poor performance impact your business.
Measure and monitor the performance of your website, to ensure
a fast and highly available website. Get a free demo and trial of
Catchpoint Web Performance solution.
www.catchpoint.com/trial
http://www.catchpoint.com/trial.htmlhttp://www.catchpoint.com/trial.htmlhttp://www.catchpoint.com/trialhttps://twitter.com/intent/tweet?source=tweetbutton&text=Get+a+free+ebook+with+web+performance+insights+from+2011&via=catchpoint&hashtags=webperf,sysadmin,wpo&url=http://www.catchpoint.com/ebooks/web-performance-insights-2011.htmlhttp://www.facebook.com/sharer/sharer.php?u=www.catchpoint.com/ebooks/web-performance-insights-2011.htmlhttp://www.linkedin.com/shareArticle?mini=true&url=http://www.catchpoint.com/ebooks/web-performance-insights-2011.htmlhttp://www.catchpoint.com/trialhttp://www.catchpoint.com/trial.htmlhttp://www.catchpoint.com/trial.html