Forrester Realistic Load Testing

Perform Realistic Web Testing To Ensure Blazing Fast Web Site PerformanceMike Gualtieri

Senior Analyst

Forrester Research

James Staten

Principal Analyst

Forrester Research

June 16, 2009

Realistic Web performance testing is

critical to achieving flawless user experiences.

4Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Agenda

• Why your customers won’t wait

• What is realistic performance testing?

• Recommendations

5Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Agenda

• Why your customers won’t wait

• What is realistic performance testing?

• Recommendations

6Entire contents © 2008  Forrester Research, Inc. All rights reserved.

• Raw page load speed Gomez Media Home Page BenchmarkApril 10 ’09 – May 10 ’09

0:00 - 24:00 EST

Performance means:

7Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Gomez Media Article Search Transaction

BenchmarkApril 10 ’09 – May 10 ’09

0:00 - 24:00 EST

• Customer workflow speed

Performance means:

1

2

3

Blazing fastWeb sites = customers

9Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Traditional Web performance testing doesn’t cut it• Traditional testing is expensive.

• Cost of licenses, hardware, and training

• Time-consuming to implement

• Idiosyncrasies of the software are often difficult to master.

• Traditional testing is not realistic.

• Scope is limited to internal, clean-room testing.

• Not focused on end-user point of view

• Misses too many pieces in the Web application delivery chain such as content delivery network (CDN), third-party components, geography, etc.

10Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Most developers are building Web apps“What types of new custom applications are your firm’s developers building?”

(Multiple responses accepted)

Base: 977 platform software decision-makers at North American and European enterprises and SMBsSource: Enterprise And SMB Software Survey, North America And Europe, Q4 2008

11Entire contents © 2008  Forrester Research, Inc. All rights reserved.

New trends make optimal user experiences even more important• The Millennial generation expects more.

– 18- to 27-year-olds

– The tech-savvy generation

• Web site redesigns

– Rich internet applications (RIA)

• More Web sites are composed of third-party components.

• Virtualization of infrastructure, including cloud

• Mobile apps availability is exploding.

12Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Agenda

• Why your customers won’t wait?

• What is realistic performance testing?

• Recommendations

Realistic testing means taking a walk in your customers’ shoes.

14Entire contents © 2008  Forrester Research, Inc. All rights reserved.

“Realistic testing” means testing from your customers’ point of view• Where do they access your site from?

– Bangkok, Cannes, Des Moines, Dubrovnik, Edmonton, LA, London, Miami, Osaka, Pusan

• What do they do?

– Key tasks, transactions, and business processes

• How do they do it?

– Browsers galore, mobile devices

• When do they do it (and how frequently)?

– Peak times, spikes

15Entire contents © 2008  Forrester Research, Inc. All rights reserved.

The user’s distance from the content matters

Source: Tom Leighton, “Improving Performance on the Internet,” Queue, October 2008

16Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Boston users of forrester.com have the fastest response time

Source: Gomez (www.gomez.com)

17Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Browsers render differently

Source: Gomez (http://www.gomez.com)

18Entire contents © 2008  Forrester Research, Inc. All rights reserved.

What features should a realistic Web performance testing platform have?

It should:

• Test from the end user’s point of view.

– Different locations

– Different browsers

– Different loads

– Different tasks

• Be easy for developers, testers, and businesspeople to use.

• Identify and resolve problems along the entire Web application delivery chain.

19Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Web sites are increasingly held hostage by third-party apps

Source: Gomez (www.gomez.com)

20Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Vendors offering performance measurement solutions

• Gomez

• Keynote Systems

• Cordiant Technologies

21Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Agenda

• What do customers want?

• What is realistic performance testing?

• Recommendations

22Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Recommendations• Design a test plan based on how real users interact with your Web application.

• Use testing tools that allow you to test from your user’s point of view:

– Where, what, how, and when

• Test the entire Web application.

• Democratize testing:

– Build a test team that includes anyone who has a stake in the outcome.

23Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Recommendations (cont.)• Test early to nip problems in the bud.

• Test often to maintain a consistently supreme user experience.

• Follow the “Blazing Fast Web Site Performance” best practices to keep your Web site humming.

Don’t leave user experience to chance.

25Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Perform realistic Web testing to achieve blazing fast Web

site performance.

26Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Thank you

Mike Gualtieri

+1 617.613.6145

mgualtieri@forrester.com

James Staten

+1 650.380.3733

jstaten@forrester.com

www.forrester.com

27Entire contents © 2008  Forrester Research, Inc. All rights reserved.

Selected Forrester research

• February 4, 2009, “Best Practices: Attaining And Maintaining Blazing Fast Web Site Performance”

• February 4, 2009, “Case Study: How Orbitz Takes You Places Faster”

• August 15, 2008, “Best Practices: Blazing Fast Web Site Infrastructure”

• August 15, 2008, “Case Study: How Msnbc.com Is First In News”