04 Kalyan Rao Konda Applabs

8/12/2019 04 Kalyan Rao Konda Applabs

1/24

Generating BusinessValue through TCoE


2/24

AppLabs Confidential 1

Testing Organization ModelsEvolutionary approach to building a TCOE is pragmatic. Slowly growing a Test COE yieldsopportunities to introduce mechanisms to overcome the pains of centralization.


3/24


TCoE services summary

Characteristics:

Common approach Ability to support frequent releases Common methodology and deliverables Manage key set of few metrics Centralisation of tasks / skills / functions

Benefits:

Consistent quality High performance Faster time to market Flexibility to support business needs Lower costs


4/24


Introduction TCOE

TCOE

Tools & AssetsProcess &TestMethodology

People &Organisation

Leverage a dedicated group of skilled testresources

Utilise offshore resources

Ensures appropriate skills are available

People & Organisation

Standardised approach and deliverableswith greater delivery consistency

Rapid adoption of best practice

Clear process accountability / responsibility

Disciplined test management

Process & Test Methodology

Effective leveraging of tools and automation

Assets made available to projects

Maintenance of multiple environments

Tools & Assets

Lower cost Consistent quality

Flexibility to supportbusiness needs

Control

Benefits

Higher quality

High performance

Speed to market

Benefits

Lower cost

Consistent quality

Speed

BenefitsCharacteristics:

Common Approach Ability to support frequent releases

Common methodology and deliverables

Management to key set of Few metrics

Centralisation of key testing tasks / skills /functions


5/24


AppLabs SCORE methodology

AppLabs proven proprietary methodology Maps processes and metrics to organization business goals Focuses on building organization wide test platform Core + Flexi resource model Cost goes down exponentially with each addition of product into TCoE


6/24


7/24


TCoE Framework

AppLabs Framework for Assessment

Clients Vision & Goals

AppLabs SCOREMethodology

Test Process Standardization Index Process Artifacts Creation / Modification

Plan Test Metrics & Measurement Roadmap Test Process Deployment Plan

TCoE Organization Model Resource Skill Matrix Capacity Planning and Utilization Report Proposed Knowledge Management

Framework

Infrastructure Inventory Report

Test Environments and Utilization Report Test Infrastructure Optimization Plan

Test Tool Inventory and Utilization Report Recommended Test Frameworks ROI Analysis of Tools Implemented Proposed Tool Optimization Plan

Understand Information gathered 360 degree Analysis Actionable Recommendations Establish baseline for Measuring

Improvements

Process Review

Infrastructure Review

Tools Review

Test Teams Review

Understand Analyze Recommend

Test Maturity Rating Test Spend Analysis Report


8/24


Centralization of Testing Services

Business / Product / Technology based frameworks

Resource management Types, skills and knowledge management

Process methodologiesReviews Review expertise Review techniques Review process Review database

Test management Test case Defect Test data Roles & Responsibilities

Risk management Plan Budget Reviews Prioritization

Measurement & Reporting Metrics framework MIS Escalation SLA

Testing infrastructure Inventory, scheduling and operational processes

Test engineering tools

Core testing

Functional andregression

Test strategy &approach

Methodology Templates

authoring & review Test suite selection

Certification(Compatibility)

Compatibilitymatrix

Acceptancecriteria

Performance

Test types Load, stress,etc

Tools Capacity

planning

Security

Testapproach

Analysis Tools Vulnerability

assessment Scorecard

Consulting

Test processreview

Baseline Audit Process

Improvement

Tools andAutomation

Test strategy Approach &

methodology Framework Tool feasibility Coverage


9/24


10/24


11/24


Business value of testing AppLabs has successfully demonstrated Return on Investment (ROI) for several customers in categories mentioned below. Metrics mentioned under each category are just a sample to showcase cost advantage, qualityimprovement and time-to-market

Test infrastructure Reduction in test tool licensing

costs Hardware infrastructure utilization Reduction in hardware servers Test data reusability

Test processes

Test processes standardization Process compliance Test maturity level

Test Engineering Defects in production Defect containment Reduction in test cycle duration Test case authoring productivity Test case execution productivity Test coverage

Effort to find a defect Test Automation

Test script authoring productivity Automation ROI


12/24


13/24

Leading InsuranceProvider in US


14/24


ROI / Efficiency GainsTest Authoring - Cost SavingsKey Initiatives:

Used All-Pairs techniques Participated in business requirements/review

meetings Reverse traceability to avoid redundant test

cases 2-Level formal review before Sign-off from

Business team (reduction in rework)

Test Execution - Cost SavesKey Initiatives::

Mapping defects to test objectives

Participation of test teams in early SDLC

cycles to gain application knowledge

Automating 40% of regression suite

Early Defect detection AFT/SIT/UAT

Releases

Releases


15/24


Post Production Root Cause Distribution

Analyzed the root causes for all the warranties and built awarenessacross all the groups to drive reduction in warranties

Root Cause for each production defect has been applied as

lessons learnt in the respective teams


16/24


Defect detection percentage has decreased for Functional, Regression and Warranty phase

Early defect detection ensured adherence to release schedules

Early Defect Detection

Integration System Regression Warranty

2007 2008 2009


17/24


Cost Saving$1.34m

50%

Cost Saving$1.99m

75%Baseline

Onsite/Offshore ratio Initial onsite: offshore = 80% Reduced onsite: offshore to 38%

Currently it is 15%

Key Initiatives:

Constant review and override of

onsite/offshore ratio

Maximized offshore capability using

effective governance

Onsite/Offshore rotation model

ROI / Efficiency Gains - Summary


18/24


Executive dashboard


19/24


Oct 07 Agent dashboard (functional, automation, regression)Aug 08 Agent commissioning system (functional)

Sep 08 Sales mgmt (functional, regression, maintenance, compatibility, integration)

Sep 08 Contact management system (functional, regression, automation)

Nov 08 Policy processing system (automation QTP/Certify )

Dec 08 Helpdesk application (functional, regression, automation)

Jan 09 Fast Quote 2 (rate validation)

Apr 09 Fast Quote 3 (rate validation)

Mar 09 Service management system (vulnerabil ity)

Phase I Discovery(Oct 07 Mar 08)

Phase II - Steady State & Transformation (Apr 08 Mar 10)

Phase III - Transformation(Apr10 )

Acquired knowledge onbusiness processes

Understood technology andarchitecture of applications

Reviewed current testingprocesses

Devised testing strategies Defined qualitative and

quantitative goals Institutionalize Insurance

certification program

Common practice and methodology for testing Revamp testing and implementation life cycle for current

applications Enforce discipline in SDLC by adherence to defined milestones Increase the rigor in usage of automation tools Focused groups for various test events Rollout and implementation of QA practices for other applications Rollout of QA practices and delivery of QA services for other

Lines of Business and other divisions Develop and roll out infrastructure optimization and non

functional testing services

Continuous improvement Annual review of core team Quarterly review of flexi team

Multi LinesPersonal Lines

Dedicated Team size 240 (15 onsite +225 offshore)

Dec 09 FAB Regression

Aug 09 Claims (functional, automation, performance)

Journey

Feb 09 Service management system (integration)


20/24

Largest e-LearningPlayer


21/24


Largest e-Learning Player Set up(Nov03 -Oct04)

Transform(Nov04 - Dec06)

Deliver (Jan07 - onwards)

Benchmarking Test scope Testing effort Test approach

Standardization of QA processes

Basic reporting functions for Defect life-cycle Resolution management Test case authoring / execution

Established basic communication model

Methodology

Adopted agile practices for QA

Quarterly audits and reviews

Introduced quality metrics

Service delivery performance

Monitored effort and schedule Resource utilization

Scope change Monitored defect detection and defect removal efficiency

Built local test lab Isolation server and QA environment

Trusted partner for QA

Manage the entire quality function of

customer with minimal supervision

Integrated AppLabs into client

business process and release

management

Nov 2003 Manual (Functional, Compatibility, Multi Lingual, Integration)

Nov 2004 Test Automation (Functional, Integration)

Oct 2005 Performance

Dec 2006 Certification

Jan 2008 Engineering (L2 & L3)

Team size 150Oct 2009 Professional Services


22/24


Defect Detection Cost Savings Effort to find defect reduced by 4 times

for core release

Effort to find defect reduced by 30% for

maintenance releases

Cost savings of $2.58 Million for core

releases

Cost savings of $191 K for maintenance

releases

Cost to find a defect

Drivers

Enhanced product knowledge through AppLabs Knowledge Management Framework

Better test coverage by using Bi-directional requirements traceability matrix

Formalized Statistical analysis and impact analysis to find vulnerable test areas

Defect Taxonomies used to drive testing effort


23/24


Test Engineering Efficiencies

Better product quality & decrease in test cyclehours due to early defect detection

Volume of defects found during sprints is50% higher

Reduction in overall testing efforts by 20%(appr. 10,000 hrs)

Effort variance reduced YOY due to better TestPlanning

Defects acceptance ratio improved by 7%because of defect triage and SME knowledge

Cost saving due to test engineering efficiencies is$152 K

Defects in Sprints

Effort Variance

D e f e c t s

Defects Acceptance Ratio


24/24

Documents

04 Kalyan Rao Konda Applabs