The Benefits and Practice of Early Lifecycle Performance Testing

8/2/2019 The Benefits and Practice of Early Lifecycle Performance Testing

1/15

A Creative Intellect Consulting Thought Leadership PaperCreative Intellect Consulting UK is constantly monitoring adoption of softwaretesting tools and best practices. In this paper we examine the benets and

challenges of early lifecycle testing for a cluster of technical test types,commonly known as performance testing. The aim is to bring managers up todate on the latest thinking on the subject and practice in the eld, with a viewto cutting away the myths and removing obstacles blocking adoption.

The Benets and Practice of EarlyLifecycle Performance Testing

An Opportunity for Optimizing

Software Delivery Processes

Paul Herzlich

Research Analyst,

Creative Intellect Consulting

EuroSTARSoftware TestingC o m m u n i t y


2/15

The Benets and Practice of Early Lifecycle Performance Testing

1

PA

GE

SummaryMessages

Early Testing Reserved forFunctional Testing

Early testing delivers business benets by

enabling IT organizations to deliver more

at lower cost. However, organizations have

concentrated on introducing early functional

testing and have largely ignored the potential

of early performance testing.

The dynamics of early testingapply equally or more toperformance testing

Early testing delivers benets in broadly

two ways:

It saves money on rework

It mitigates the risks of surprises late in a

project.

These dynamics are extremely relevant to

performance issues, which perhaps even

more than functional errors hold the potential

of sending a project back to the drawing

board if components or infrastructure dont

perform as required.

Practitioners believe there are twounsolvable challenges to earlyperformance testing

There are a whole host of potential difculties

to introducing early performance testing.

Most of them are the same as those that

impede the adoption of early functional

testing. They are largely soft, people issues,

centered around the roles and psychology

of software coders. There are fairly well

understood change management techniques

for dealing with them.

However, to most testers and QA managers,

there are two aspects of performance testing

that render it unsuitable for early testing. The

beliefs are:

You can only test performance on a fully

built system which doesnt exist early on.

You dont have suitable component level

performance requirements.

The solutions to these objections require

a ner look at what performance testing

consists of and a pragmatic approach to

achieving your goals without necessarilyachieving textbook perfection.

Introducing early performancetesting is a change managementproject

Starting down the road towards early

performance testing will involve developing

motivation among those who will be

responsible for doing it. You will need to

make controlled changes to all your existing

development processes, enlist the help of

specialists including your existing late cycle

performance testers and adopt tools that

give you the kind of automated support that

works best alongside existing development

tools. You should only undertake this within

the context of a properly conceived change

management project.

The BusinessImperative

Most software delivery organisations accept

the need for performance testing, but it is still

commonly done very late in the applicationlifecycle. Some even wait until after software

has been released in production.


3/15


2

PA

GE

We know from studies over the last 30 years

that testing early in the lifecycle yields big

benets. Although some resistance in the

eld still remains, the argument has been

largely won for early functional testing.

However, despite this general acceptance,

the practice lags far behind for performance

testing. Although QA managers at the leading

edge of process maturity are beginning to

adopt it, there is a widespread perception

that early performance testing, even if

desirable, is impractical or unfeasible.

The challenges to adopting early functionaltesting were soft issues mainly people

and process. Once the benets of early

functional testing were understood, it

became a matter of building the practice

into development processes and winning the

hearts and minds of developers to adopt it.

Developers traditionally resisted the notion

that formalised testing was part of their job.

Textbook unit testing was largely a myth.However, Agile methods, powerful IDEs and

unit test frameworks have been signicant

in bridging the gap between testing and

development activities. Early functional

testing increasingly has traction throughout

the industry.

Early performance testing is a practice

that presents not only the same set of soft

behavioural challenges as early functional

testing, but also a set of hard, technical

constraints. We know how to solve the soft

challenges, but the technical constraints

seem intractable to all but a few very

experienced specialists in test processes.

An example of a widely held view, stated

in the extreme, is that you can only test

performance meaningfully when you

have a fully assembled system in a nearlyproduction-like environment, things you

dont have early in the lifecycle. If that truly

were the case, early performance testing

would indeed be impossible. In fact, these

are not technical constraints at all but simply

intellectual obstacles that can be addressed.

Creative, innovative and forward-thinking

test practitioners have found solutions.

If early performance testing is so unintuitive

and counter to the received wisdom, then why

is it worth pursuing? The answer is that we

have an obligation to make software delivery

more responsive to the perennial business

needs to deliver more and cost less. Test,

QA and development managers aligned with

these business goals understand that early

performance testing is yet one more tool toachieving this.

How Early TestingDelivers Benets

There are two ways in which early testing

delivers benets:

cost and timescale reduction and

increased project predictability.

Both of these are extremely important in a

commercial environment where business

success depends on IT to minimize costs

and meet changing market demands quickly.

Businesses cannot afford cost overruns anddelayed product releases. The days when

an estimated 60% of all projects get canned

usually for not delivering the right systems

at the right time are over. Project budgets

must be controlled, timescales managed

and uncertainty kept to a minimum.

Rework reducing the costs of going

backwards

Many studies have been performed over

the years that conclude that testing early

has nancial benets. The National Institute


4/15


3

PA

GE

of Standards & Technology (NIST) study

from 2002 on The Economic Impacts of

Inadequate Infrastructure for Software

Testing is comprehensive and accessible

for understanding the whole picture. It

demonstrates that bugs are signicantly

more expensive to correct after code leaves

the developer for system and performance

testing. It states that, it is costlier to repair

a bug that is created in the unit stage in the

component or system development stage

than it is to remedy the same bug in the unit

stage when it was introduced.

The report estimates that a bug introduced in

the coding stage is 10 times more costly to

locate and correct in system testing phase,

20 times more costly in pre-production

or beta stages and 30 times more costly

post-release than it would have been at

the unit stage when the bug was created.

Programmers create the code and their code

contains bugs. Getting them to nd and x

them is not an unreasonable demand.

For bugs in the initial requirements the

picture is equally as dramatic. This is worthy

of attention since most bugs emanate from

badly captured or ill-dened requirements. If

the cost of nding a requirements bug during

the requirements capture stage is x, it costs

5x in coding and unit test stages, 10x in later

test stages, 15x in pre-production and 30x

in the released system.

If youre skeptical, think about the processes

required to locate and x a bug. When a bug

is found in a stage later than coding and unittest, it has to be documented and returned to

a developer. A developer has to establish an

environment to reproduce the bug. This may

entail re-creating test data and restoring an

earlier version of source code. Locating the

bug the error that is the source of a fault is

complicated by the fact youre now dealing

with a complex assembly of components

rather than simple units.

Attending to a bug report comes with an

immediate disruption to the developers other

work, which in turn has a cost to activities

wholly unrelated to this project. When

the bug is xed, code must be promoted

and retested at later stages, with all theheadaches that entails for source code and

build management.

1

10

20

30

Coding System Test Pre-producon Post-live

Chart 1: Relative cost of xing a coding bug at later lifecycle stages


5/15


4

PA

GE

It is clear that if there is any possibility of

producing and running a test while a unit

is in the original coding environment, any

bug that is found is signicantly cheaper to

locate and correct.

Predictability mitigating risks and

avoiding surprises

Although no one asserts that early lifecycle

testing would ever nd all bugs, when the

practice is deployed effectively, companies

benet from shorter project timescales and

lower project costs. Early lifecycle testinghas another benecial effect. It enhances the

predictability of software delivery, a benet

that managers cant quantify easily, but miss

sorely when they get unwelcome surprises

in the late stages of a project.

Early testing contributes to greater

predictability by providing better information

about the true state of a project in the earlystages. As work is passed from coding to

integration or system testing (or iteration to

iteration), there is a great deal more certainty

about the codes readiness when it has been

unit tested by developers (real unit testing,

not the mythical or ad hoc testing developers

have historically performed.) A manager has

a great deal more condence that coding is

truly complete for 10 programs handed over

with a full suite of passed unit tests than 10

programs without documented tests.

This condence works not only at handovers

within a project, but also outwards from the

project. Without completion evidenced by

passed tests, all but the most experienced

managers fall into the 95% complete

syndrome, and pass that misinformation to

their managers, the sponsor and end users.Such self-delusion means that everyone is

in for an unwelcome surprise not just a

surprise about the software quality but also

about the project progress.

Another way of understanding the non-

nancial benet of early testing is to consider

it as a risk mitigation technique. In fact, all

testing is about risk mitigation. The earlier you

can close down a risk determine that the

risk is mitigated the more condently you

can proceed to later stages and the better

you can focus your late cycle activities.

Early testing principles applied to

performance testing

The experience of those doing early functional

testing is that it brings clear business benetsand better project management. But are

there potential advantages from detecting

and correcting performance problems at an

early stage? You bet, and in spades.

Performance problems need to be treated as

bugs. Whether performance requirements

are explicit or implicit, if someone in

a project is capable of declaring thatperformance is inadequate, the discrepancy

is a potential bug just like any other gap

between requirement and implementation.

It needs the same scrutiny that a functional

failure undergoes. Ultimately there will be a

determination whether the failure is indeed

a bug.

When you nd a performance failure at

a late stage in the lifecycle, all the costs

of identifying the cause and reworking

the solution are magnied. Functional

code bugs (when they are not caused by

requirements errors) usually relate back

to individual, small sequences of code.

However performance bugs found late in

the lifecycle can call into question a whole

architecture or implementation. The source

of the bug could be a code sequence, or itcould be the conguration or environment.

It is hugely helpful if coding bugs that hit

performance have been eliminated by the

time you do a full-scale performance test on


6/15


5

PA

GE

a complete application. By doing this, you

enter your late cycle performance testing

with more certainty about code, giving you a

chance to concentrate on infrastructure and

conguration.

What We Mean ByPerformanceTesting

Up to now, we havent been precise

about what we mean when we talk about

performance testing. However in order

to uncover the possible approaches and

opportunities for early testing of code that

can impact on performance we need to be

more specic.

Performance testing is commonly used as

a term for several related test types whose

goals are to establish that an end user will

get a functionally correct response from

an application and that the timing of the

response will be acceptable. Some of the

constituent test types are load, volume,

stress, concurrency and soak. We need to

look more closely at the individual test types

to understand which are most suitable for

early testing.

Straddling the functional and performance

domains is concurrency testing. Running

more than one instance of a process

concurrently can raise functional problems

like inconsistencies from simultaneous

database updates and performance issues

like badly formed locks on resources, which

slow things down.

In the narrow sense, performance testing is

a measurement of response time or latency

and the resources consumed to achieve it. To

measure meaningfully requires tests under

conditions that represent, approximate

or can be mapped onto the production

environment. So software developers and

testers most commonly test for responsetime under production-like loading conditions

with production-size data volumes load

testing and volume testing, respectively.

Typical performance, load and volume

tests are measuring against targets based

on normal and peak workloads executed

over timescales of less than one working

day. When a test is run to ramp a system tobreaking point, it becomes a stress test and

when the test execution spans days, weeks

or even months, it becomes a soak test.

Its important to note that most performance

testing is impossible without tools. Tools

are needed for driving execution, measuring

activity and reporting results. Even crowd-

source approaches for driving tests

require tooling. Load and stress testing are

particularly tool intensive. But there are two

subtly different approaches to using a load

test tool.

Most commonly in load testing, testers

attempt to create realistic transaction

scenarios through a load test tool and

drive them through large numbers of virtual

users operating against production-likevolumes of data. During the course of the

test, measurements are gathered across

the software components and infrastructure

under test. This is a major logistical exercise

that characterizes what most software

engineers understand as performance

testing.

However, load can be used in a muchsimpler way, which well call background

load testing. Without attempting to mimic

realistic user activity, virtual users are used


7/15


6

PA

GE to create a background of infrastructure

activity disk access, CPU, network trafc that represents a particular normal loading

of the production environment. Then user

transactions are driven manually and

performance is measured of the manually

entered workload.

When we look at how to do early performance

testing, we need to keep in mind all the

various test types available and not getlocked into the image of the set-piece, large-

scale performance test.

The Case AgainstEarly PerformanceTesting

With so much to be gained, why is the

practice of early performance testing still

relatively rare the domain of a few visionary

test and QA managers plus specialist test

service vendors?

History and tradition in software testing

practice, and more widely in development,

have a lot to answer for. The Waterfall model

for development and, to a lesser extent, the

V model widely known among testers, both

convey a picture in which technical tests such

as performance are relegated to late in the

lifecycle. That is, at least, the most commoninterpretation, even if it was not strictly the

intention. Testing tool vendors have also

contributed. They have bought into the late

cycle testing vision for commercial reasons

and, in turn, the available tools have shaped

(or misshaped) current practice.

We alluded earlier to the soft issues

surrounding people and process that are

used to undermine early functional testing.

Those issues are still present for performance

testing: developers dont like testing and are

generally not accountable for it. Testing is

Chart 2: Relationships among various performance-family test types


8/15


7

PA

GE

considered a lesser technical skill (although

this is patently untrue for performance

or security testing), confers lower status,

is inherently less interesting, less sexy

and probably a bad career move. No onehas made a lm called The Social Testing

Network.

Agile processes have fewer such biases.

Cross-disciplinary teams are the norm, and

it is common for developers to perform

testing. With Agile processes and good

understanding of the benets of early

testing for functional testing, and with betterIDEs and unit test frameworks, it has been

possible to win the hearts and minds of

experienced developers (although newbies

may still need to fail once or twice before

they get the idea) to take some responsibility

for functional testing.

However, adoption of Agile does not totally

solve the problem. Early performance testing

poses two key intellectual challenges that

experienced developers on both Agile and

Waterfall projects have trouble seeing their

way through.

Challenge 1: You can only test performance

on a fully built system in a production-like

environment.

Challenge 2: It is impossible to set and you

almost never have meaningful performance

requirements at the level of individual

components.

It takes a highly motivated and creative

test or QA practitioner to see how these

challenges can be met.

PragmaticApproach To Early

PerformanceTesting

You need to have a pragmatic mindset to

make benecial use of early performance

testing. If you demand complete knowledge

for example of requirements or total

realism for example, of testing environment

you will never get started and never getthe benet of all the positive things you can

do early in the lifecycle to verify that your

software will perform as required. Your

objective is to close down as many doubts

about performance as you can with the

resources you have.

Were going to take a closer look at what

you can do to make early performance

testing a reality. There is no one-size-

ts-all prescription. Our aim is to open up

the possibilities and move away from the

intimidating complexities of the big event

late cycle load test.

Choosing what to test tackling the

risks

You could attempt to do performancetesting on every unit within the development

environment, but when that is not practical

(forget about possible) concentrate on the

riskiest components. Risky components

would include elements that are:

Resource intensive in bandwidth,

CPU, disk access, etc.

Time-critical latency within thecomponent is critical.

Database routines a perennial source

of performance bottlenecks.


9/15


8

PA

GE

The developer is the person most ideally

placed to evaluate the technical risks of the

components they are building.

Selecting test types

No one type of performance-family testing

is totally unsuitable for early testing. It

depends on the architecture of the system

under test and where the risks lie. However,

in developing a general early performance

testing process for a typical commercial

software delivery organisation, the following

types are arranged from the most to least

well suited.

Stress tests are unlikely candidates for early

performance testing. Load (as opposed to

background load) and soak are possibleprimarily as part of continuous integration. The

others are good candidates for developers

alongside functional unit testing.

Strategies for absent performance

requirements

Most performance requirements are cast in

business terms as end-to-end transaction

responses, database capacities, and

operational SLAs. Developers can easily

argue that performance testing of individual

components is impossible when they dont

have performance requirements at the

individual component, method or unit level.

The obvious solution is to analyse the high-

level performance requirements and create

requirements for components. If no one in

a team not even an architect or designer

is capable of doing that, you have to

conclude there is no certainty that the built

application will perform adequately when

the components are assembled into a whole

application. In which case, the need for

more granular performance requirements

and early testing is even greater.

Chart 3: Test types by suitability for early performance testing


10/15


9

PA

GE

In fact, we nd that adopters of early

performance testing can routinely

create reasonable working performance

requirements for components. They

sometimes do this in two distinct ways,

depending on whether they have a reference

point or not.

If there is no existing system or similar

components, they develop low-level

performance requirements in a simplistic,

uncomplicated way taking high-level

requirements and using crude averages

for, for example, numbers of methods perbusiness transaction and an assumption

about client-end and network latency.

If there are existing systems, they use

past performance as a baseline. Agile

developments have the advantage that

they quickly produce baselines from prior

iterations.

Crude as they are, these two approachesproduce reasonable results, especially

coupled with a developers experience and

insight into the code. The information you

get from such tests may not demonstrate

denitively that your codes performance is

adequate. But when a test like this fails, it is

a useful signal to a developer to investigate

more closely.

None of this is intended to remove theneed for good performance requirements

for risky components, but these methods

serve a useful purpose when you need to be

pragmatic.

Strategies for working with non-

production environments

As a rule, developers have neither a complete

application nor a production-like environmenton which to perform early performance

testing, but, actually, sometimes they do.

We tend to look at changes in software

delivery processes in the context of brand

new systems, but they represent the

minority of software delivery projects. When

enhancing existing systems, there is often

a complete application to test, one which

is built with one or more new components.

So the objection that you dont have a fullapplication at the early development stages

turns out to be less true than imagined.

The objection that the development

infrastructure is not sufciently close to

production is also surmountable. The

analysis to compare a development

systems capabilities to a production system

is feasible. Capacity planners do this sort of

calculation all the time.

Putting aside for the moment the process

management issues for maintaining a

performance testing environment for

developers, some organizations do indeed

have available test environments run as

labs with known relationships to production.

While these are generally utilized for late

cycle performance tests, they could be

made available for early cycle testing too.

Testing outside the coding environmentmay entail changes to source management

and build procedures, but these are not

insurmountable obstacles either.

In other cases, the capacity and scale of

the development environment compared

to the production environment are well

understood and can be used to calibrate

results. Once again, Agile iterations make

such comparisons easier.

In our pragmatic approach to early testing, we

can work with ballpark gures. Development

environments are less powerful than

production environments but they are also

less loaded. A component that performs

badly in an early cycle performance test is

an early warning, and a signal to investigate

further.

You still need the big event near the end

Just as you must still do system testing after

early functional testing, you must still expect

a full late cycle performance test. However,

you should enter the late test phase with


11/15


10

PA

GE

fewer uncertainties and more condence in

code performance. You will nd fewer code-

related performance bugs and save time

and costs from less rework.

Introducing EarlyPerformanceTesting Practices

Introducing early performance testing should

be treated like any change managementproject. Good communications and

sensitivity to people are always essential.

However, here are some specic points to

keep in mind.

Developing motivation

You need to convey two distinct messages:

that the benets of early cycle performance

testing are worth it and that it is feasible to

do.

The starting point has to be a clear

explanation of how nding defects early

has both a nancial benet to the business

and improves project manageability and

predictability. It must be made clear that

project delivery schedules will be adjusted

to take into account the additional up frontwork, and that any learning curve is built-in.

Whats in it for those who will adopt the

practice? It depends on your organisations

culture. The certain benet is that for an

additional effort in development, there will

be far fewer awkward disruptions later for

debugging and rework.

Beyond the benets, developers needto be reassured of the feasibility. For this

you will need to ensure that the change

management project includes a skilled

champion or evangelist someone who

instils condence.

In return, coders should be clear that

they become as accountable for early

performance tests whatever scope you

have chosen as they are for code and

functional unit tests.

Integration with existing processes

A software delivery organisations process

maturity matters. You need to be already

operating at a mature level of testing process

before you undertake early performancetesting.

We dont see any intrinsic unsuitability for

early performance testing within Waterfall

or Agile projects, but Agile does have some

characteristics that make introduction of a

new test type a little easier, for example:

The cross-disciplinary nature of Agileteams means they are used to coders

and testers working together

Agile iterations quickly begin to

provide the information needed to

dene component level requirements

and map development environment

performance onto production

performance

The Agile culture is generally more

dynamic and accepting of change and

uncertainty. Agile developers are more

pragmatic about incomplete

requirements.

We recommend that your process include

risk-based analysis for deciding whether

to do early performance testing on a

component, how much and what type (based

on the types we outlined earlier.)

There are several existing processes that are

likely to be touched by early performance

testing:


12/15


11

PA

GE

Coding stage testing

Build process and/or continuous

integration

Source code management.

We would expect to see processes for the

test types we identied earlier assigned

to the coding stage. Mature integration

and build processes, with integrated

functional unit test suites, can be extended

to included performance tests as well.

Those performance tests could include

any component performance tests, but

also further performance tests on the builtassembly of components. It is at this stage

that some early load testing becomes

feasible in addition to the early background

load testing you can do on components.

Changes to source code management

and build procedures may be needed if

performance tests are executed outside the

coding environment.

The pragmatic approach should be

encouraged across the board to ensure

exibility. It is essential that imperfect

knowledge of requirements and environmental

behaviour not be used as an excuse not to

test, where experience and common sense

could with the right attitude plug the

gaps.

Specialists

You will need the buy-in of those who are

usually involved in late cycle performance

testing, including any specialist performance

testers, architects, infrastructure experts

(network, DB, service orchestration). They

will need to change their own processes to

include support for the early performancetesting. This is sometimes viewed as an

opportunity because they have been at the

sharp end of the problems inherent in leaving

all the testing towards the end. However, as

the industry has developed with its bias

for late-cycle testing you may nd that

performance testing specialists may feel

threatened that their specialism is being

undermined.

Tools

Performance testing of all types requires

tools. For early performance testing, you

need tools that t in with coding and

integration environments and that dont

require all the scaffolding and preparation

of the set piece late cycle performance test.They need to support middleware and back-

end components as easily as they support

client-end scripts.

One commonly held idea is that functional

unit tests can form the basis of performance

tests. More often than not, this proves not to

be true, so dont rely on it.

It is worth pointing out that for background

load testing where you are just trying to

create a background of machine loading

against which you will run manual transactions

you dont need as ne a control of the

timing and ramp up of automated scripts

as you need for full load testing. Virtual user

scripts and orchestration by a virtual user

controller can be relatively simple. You are

not concerned about the performance of the

individual virtual user transactions; you are

interested in the load they place overall on

resources.

A key factor in the cost-benet equation

of early performance testing is the cost of

tools. When they are available in developers

IDEs the marginal cost is low. However,

the cost of virtual users for driving loadtests can become a signicant expense. It

is important to make a reasonable stab at

estimating how many are really required.

On one hand, there is a tendency to over-


13/15


12

PA

GE

specify the number, perhaps in the interests

of expediency. On the other hand, it can be

a mistake to scrimp and attempt to make up

for the shortage by distorting the content of

the scripts. The essential point is that you

want the right number of virtual users to

achieve your goals, whether that is 25, 1000

or more.

And one nal soft issue: you will need to

develop the skills of tool specialists who can

mentor others.

Conclusion

Early performance testing may not be

the rst test process improvement a

development organization chooses to make,

but there is no good reason why it should be

as overlooked as it is. Agile processes and

powerful development environments with

integrated testing capabilities are conducive

to a pragmatic approach. Determination to

obtain the benets of lower rework costs

and more manageable projects provides the

motivation.

Microsofts ALM Assessment has been

designed by expert ALM practitionersto help you understand how well your

organisation currently approaches

ALM and where and how you can make

improvements.To nd out more, click here.
http://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessment


14/15

Dont forget to join us for EuroSTAR 2012 in

Amsterdam from the 5 8 November.

Its our 20th anniversaryand we are going to

celebrate in style!

EuroSTARSoftware TestingC o n f e r e n c e
http://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessment


15/15

w w w . e u r o s t a r c o n f e r e n c e s . c o m

Join the EuroSTAR CommunityAccess the latest testing news! Our Community strives to provide testprofessionals with resources that prove benecial to their day-to-day roles.These resources include catalogues of free on-demand webinars, ebooks,

videos, presentations from past conferences and much more...

Follow us on Twitter @esconfs

Remember to use our hash tag #esconfs when tweetingabout EuroSTAR 2012!

Become a fan of EuroSTAR on Facebook

Join our LinkedIn Group

Contribute to the EuroSTAR Blog

Check out our free Webinar Archive

Download our latest ebooks

The EuroSTAR BlogWritten by Testers for Testers
http://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.microsoft.com/visualstudio/en-gb/strategies/almassessmenthttp://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/http://www.eurostarconferences.com/

Documents

The Benefits and Practice of Early Lifecycle Performance Testing