Tips and Tricks for successful

Risk Based Testing (RBT)

Satyanarayana Murthy Kotta Sep 2013

Context Setting

x This discussion is not around the theory part of what RBT is all about.

Focus is on the HOW part for a successful RBT implementation

2

Quick brush up on RBT

3

Tips

&

Tricks

4

Tip 1

Do not take to RBT just because ... the theory sounds really good and

fancy

Never pursue this until there is a commitment from Senior

Management

5

Trick 1

•Management to first understand the customization of

this concept to their context and then drive from front

•100% commitment from Senior management with a Sponsor to track fund allocation and progress of the initiative

•Highlight the benefits and encourage implementation of RBT in public forums like Town halls, Steering committee meetings, Rewards n Recognition (RnR) functions

Top down

approach for

Senior management

Communication

Bottoms up

approach for

Implementation

RBT will work best when

6

Tip 2

Just because … there is the word ‘Testing’ in RBT, do not assume that

it does not have any other stake holder involvement

This is never a ONE Team show

7

Trick 2

• Identify the stakeholders from BUSINESS, ARCHITECTURE

DEVELOPMENT and TESTING teams

• Define the RACI matrix for all the RBT activities.

• All the RBT activities – static testing, risk assessment, prioritization should be part of estimation and project plan

• Ensure to explicitly mention the RBT approach in the Test Strategy document that is signed off by all stakeholders

RBT will work best when

All stakeholders understand, agree and realize the benefits RBT brings to their teams and the project on the whole. Testing team acts a facilitator in the whole journey

8

Tip 3

Do not just get the external consultant/vendor to design a

generic framework for your organization

Framework designed without internal stakeholders involvement will always

FAIL

9

Trick 3

• Form a TMO office first.

• Workshop with relevant stakeholders (business, development and testing heads) in finalizing the framework. This will help in easy roll-out across teams

• Right framework should focus on 3 main aspects of RBT (static testing, risk assessment and execution based on prioritization)

• Proper flow detailing the tasks/ roles of stakeholders (like business criticality value from a business analyst and failure probability value from the coordinate effort of the development and testing teams)

• Tools available at each step.

RBT will work best

With a simple to follow framework. Customized in line with the Risk guidelines existing for the organization

10

Tip 4

Do not go for a Big Bang approach to roll out RBT

Improper deployment strategy and unstructured change management are

the biggest hurdles

11

Trick 4

•Identify at least 2 projects to pilot the RBT framework. Handhold them in implementation.

• Refine the framework and tools based on feedback.

•Help them analyse benefits in the scenarios of with / without RBT in place.

• Role of change management is critical for successful implementation. Coordinating with the right people through the right forums is essential

•Let the pilot projects do the talking to the other teams for a successful RBT across the organization

•Prepare the training plan on tools/process for each role in the project team

•Identify the right approaches (like identifying champions or portfolio managers) to conduct the trainings.

• An implementation tracker will be the key to success

RBT will work best

When asking for prioritization of tasks become a buzzword right from the project team-member level onward …

12

Tip 5

RBT is a methodology….

Does it mean there are no tools to assist this?

General feelings:

RBT is difficult and takes long time. No simple to use tools

Existing tools don’t support this

13

Trick 5

• Develop / Reuse simple to use tools that motivate the team and make their life easy

• 2 tools for static testing, 1 for risk assessment along with the right framework in place can do wonders

• Ambiguity checker – like a spell checker highlights possible ambiguous words and generates a quick report in less than a minute

• Testability scorecard - excel tool that verifies the testability risk of a requirement based on values given to parameters (like completeness, consistency, unambiguous, discrete, stable and traceability)

• Risk Assessment – excel tool to auto calculate Risk for chosen values for Business criticality and Failure probability

• Tools from HP, IBM come with in built features to support RBT. Explore to make effective use of this module

RBT will work best

By assisting the team with Simple and Effective tools that create a wilful pull to practice RBT than a resistive push

14

Tip 6

“What cannot be measured cannot be controlled” is a basic reality

The Notion:

RBT is more subjective and less objective with no defined metrics for

measurement

15

Trick 6

• Identify the right set of metrics that makes RBT objective

• Testability Risk Score: Indicates the testability risk at a requirement level in the early phase of the project - Good indicator of the issues that need upfront clarification.

•Risk Assessment: Indicates the number of requirements under high, medium and low categories - Good indicator for prioritization.

•High Severity Defect Removal Effectiveness (HDRE): Indicates the Sev1 and 2 defects in UAT and production against total number of defects. Good indicator of the success/failure of RBT and helps in refining the risk assessment guidelines.

• Risk-based test execution report: Indicates test progress against the risks associated with requirements.

• Good indicator of testing focus and residual risk at any point of time that assists business to take informed decision based on quantitative analysis than qualitative analysis.

RBT will work best

When we realize that this is no different from what we do in day to day life with a mentor in place

16

Thank You

17

About the Authors : Authors of this point-of-view have worked hand in hand with an Australian Banking client for

1.5 years from the definition to the successful implementation of Risk-based Testing (RBT)

Satyanarayana Murthy Kotta is a Principal Consultant at Infosys Technologies. With over 14 years of

rich IT testing experience, he has helped companies in Australia, Middle East and Europe to define n

implement testing efficiency improvement programs. Satya’s key strength is to provide customized practical

solutions that fit client needs.

Kamalahasan Madaswamy is a Senior Project Manager at Infosys Technologies. With over 15 years of

rich IT experience across verticals, he has involved in development, maintenance, production support and

testing projects. His vast experience and strong acumen to development life cycle has helped him to play the

role of testing consultant to define and implement Risk Based Testing and Metrics Governance program

successfully for a leading Australian bank

References for this article: Our own working experiences that taught us more lessons to share

Glossary RACI – Responsible,Accountable,Consulted,Informed

18