Digital Assurance: Develop a Comprehensive Testing Strategy for Digital Transformations

Shamim Ahmed, AVP, Quality Engineering and Assurance

Application Economy: What’s Next

Cognizant

AET02S

Vikul Gupta, Director DevOps, Quality Engineering and Assurance

2 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD

© 2015 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.

The content provided in this CA World 2015 presentation is intended for informational purposes only and does not form any type

of warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.

For Informational Purposes Only Terms of this Presentation

3 © 2015

Digital Assurance in the context of digital business challenges

Technology Assurance

Agility Assurance

Customer Experience Assurance

Information Assurance

CTO/ CIO

CSO / CDO (data)

CMO /

CDO

CXO / Business

How do I enhance the digital

customer experience?

How do I deliver speed in a

digitally assured business?

How do I secure the hi-performance

digital enterprise?

How do I maximize the

digital value chain?

Assuring Compliance, security and

performance to customers and

organization across geos

Challenges in integrating legacy

systems with digital ecosystems

Assuring “first time right” amidst

increased ecosystem complexity,

shrinking timelines

Test like a customer would – any

device/anywhere/anytime

T h e C h a l l e n g e Fa c e d b y S o l u t i o n

4 © 2015

Application level testing Digital product/system/value chain assurance

Guardian for functionality, performance, security Guardian for customer experience & brand

Descriptive metrics Predictive metrics, Quality Intelligence

What’s different in Digital Assurance vs Traditional QA

Tr a d i t i o n a l QA D i g i t a l A s s u r a n c e

Test automationLifecycle automation, early & continuous automation

Robotic process automation

Dev->Test->OpsAgile, Continuous Delivery, DevOps; QA->QE->SET->DET

QA shift left, right, up, down/deep

5 © 2015

Digital Technology and

Value Chain AssuranceAgility Assurance Customer Experience Assurance

Security, Privacy, Omni-Channel, UX, Quality Intelligence, Compliance, Reliability Assurance, I18N,Performance

Digital assets, SMAC+

Value Chain TestingDevTOps

Customer Experience

Testing

Information Assurance

Cognizant Digital Assurance Focus Areas

6 © 2015

Example: Digital Healthcare Transformation

Limited-Engagement

Highly Engaged, Empowered

Inaccessible

Convenient and 24/7

Transactional, Isolating

Care Team Managed

Office Hours

Virtually Mobile, Anytime

Physician-Centered

Patient-Focused

Isolated Individual

Socially Connected

Sick- care

Health and Well-Being

Limited Consequence

Financial Rewards, Incentives

CHANGING INDUSTRY LANDSPACE

7 © 2015

Example: Code Halos in Digital Healthcare

Use

gamification,

apps,and

appropriate

literacy levels

to learn

about

the disease

state

Doctor sends

an email

and also

recommends

an app

Real-time

Patient

coachWearables

help to remind

Inspire, track,

and communicate

Visually

Elegant

Patient

dashboards

show key

measures

that matter

OMG,I HAVE

A SERIOUS

DISEASE

WHAT DO I

DO NOW?

CAN I DO

THIS MYSELF?

HOW DO I

KEEP GOING?

HOW AM

I DOING?

8 © 2015

Customer

Service

Individual or Group policy

selectionSales business process

(Prospect and Quote) Member

Enrolment

Claim Management (Claim

Entry and Claim adjudication

and Claim Payment)

Receivables /

Payment

Social Media Big Data Internet of Things Digital Hub Mobile & Smart DevicesCloud Computing

PBM, Payer provider

pharmacy benefits

TPAs

HEALTHCARE VALUE CHAIN

IDENTIFY

SYMPTOMS

CAUSES of

symptoms

PR

OV

IDER

AP

PLIC

AT

ION

CLOUD

MEDICAL

PROCEDURES

PA

YER

AP

PLIC

AT

ION

APPROX.

COST

Suggested

Prescriptions

Liability

Collects

data of

symptoms

PATIENT

SC

EN

AR

IO

Social, Mobile, IoT

& Wearables

Testing, Omni-

Channel testing

CX Testing

Cloud + SOA Testing

(API/Func/Sec/Perf)

Big Data &

Analytics Testing

Omni-channel

API testing

SOA testing

Example: DA Solutions Mapped to Digital HC use cases

Agility Assurance

9 © 2015

SMAC Testing

Digital Assurance

Platform

Core Testing

Multi-channel &

IoT Testing

Hybrid Cloud Apps

Testing

Digital Content/Media

Testing, Digital Asset

Management Testing

Deployment Testing

Infra Testing

Business Process

Validation Testing

Customer sentiment Analysis

A/B Testing

DRM Testing

Creation

Aggregation

Distribution

Digital Value Chain Assurance

10 © 2015

The impact of Analytics on QA: Digital Assurance Platform

Plan Define Develop Build Test Release OperateProcess

IT Lifecycle Intelligence using Continuous Analytics

Biz

Analytics

Dev

Analytics

Test

Analytics

Ops

Analytics

Cust

Analytics

Analytics

Types

Examples of

Smart QA

based on

Analytics

Feature

ValidationCode-quality

based QA

Schedule &

defect

prediction

A/B Releases,

failure

prediction

Customer

Sentiment

and CX

11 © 2015

DevOps

Plan

Define

Develop

Build

Test

Deploy

Release

Operate

Develop

Deploy Operate

Learn

QE QA AA

Requirements

Bugs and feedback

Product

backlog

Ops

backlog

Working

software

Sprint

Monitor

Dev Team OPS Team

QA Shift RightQA Shift Left

QA Sits in the middle

connecting Biz, Dev

and Ops

QA

Sh

ift Up

QA

Sh

ift

Do

wn

Focus on Continuous Quality

Focus on Frictionless Testing

Focus on “Build to Run/Deploy/Perform”, vs “Test…”

QA’s role is changing in a world of DevOps…

12 © 2015

DEV-OPS implications for

QA

Quantify impact and test what

matters

Integrate test with Dev. and Ops

• System teams (with Dev and Ops)

• QA tools integrations w/ Dev and Ops

• Code quality-based tests

• CI/CD process integrations

• Automated workflows

• Code change impact testing

• Traceability to environments

• CX focused testing

• Monitoring during tests

• Apps lifecycle analytics & intelligence

Automate aggressively

Virtualize

• Service/App Virtualization

• Network Emulation

• Data Virtualization

• Test Environment

• Device Virtualization

• Test environment

• Test generation & execution

• Test automation

• Test data

• Process and analysis

DevOps Implications for QA

QA Skills Transformation: QA->QE->SET->DET

Velocity

Quality

13 © 2015

Testing framework for end to end Digital Stack with IoT

THINGS and PEOPLE

Devices/Sensors

External conditions

External (test) data

Scaling

Simulators and Emulators

Robotic Testing

Non-conventional interactions

Scaling

Omni-channel

Usability

Privacy

NETWORK

Network Virtualization

N/W protocols

N/W conditions

Fault injection

DATACENTER/ CLOUD

APPLICATIONS

Service Virtualization Performance,

Capacity, Scalability, Failover

Security

Omni-Channel delivery

Functionality

Data Integrity

Analytics

CTS

Bluelink

Robowear

14 © 2015

Social Driven QA

Earlier if there is a problem with production system, the testing team would be likely to hear through conventional channel such as customer

support, emails or satisfaction survey. Fast-forward to the Digital era

Production Issue User affected Shares experience on Social Media

Before action is taken. Goes viral , reputation damaged

Approach

Social driven testing harvests Social media data , analyze it to exact trends to draw inferences on focus areas to improve product quality/additional validations for QA

Listen Using social media web services (Facebook, Twitter, Foursquare, YouTube, blogs, emails, etc.)

crawls through social media websites

IntegrateHouses social media data and associated BI/analytics structures. Integrates with the

organization’s data warehouse and data mart to bring 360 degree view

AnalyzeMeasures brand’s sentiment on social media sites using Cognizant’s proprietary sentiment

analysis algorithm

TrackMobile Bl ready dashboards and reports that measure brand sentiment analytics, competitive

intelligence & customer insights, social media ROI calculation and marketing campaign

effectiveness

Act Make informed decision, take action and track the results

15 © 2015

Case Study: Leading Apparel Manufacturer

Tweets/posts on

products Analyzed

Info extracted in

JSON format using

GNIP

Load Data from

HDFS into Hive for

Querying

Analyze Twitter data

using Tableau reporting

tool

Device

Multiple JSON files stored in HDFS

Trophy

Battery Sync40,000 Tweets on user issues

specific to customer extracted

Validated with test teams,

9 Product Insights/Additional

Test Scenarios on

70 Validations/Scenarios

to improve Product Testing

1.7 MILLION

16 © 2015

Key tenets for a successful Customer Experience Testing strategy

Emotion is a key driver Omni Channel Experience

Understanding Customer Intent Multi platform & devicesKey

Tenets

CX is a Continuous Journey Persona Based Test Strategy

17 © 2015

The importance and criticality of the Quality Assurance across the User Experience life cycle

UI architecture, content and navigation

Create Story boards & screen flows

Visual design/branding

Design & Build

• Wireframes and mockups to

prototypes through design Iterations

• UI feedback and usability issue list

• High-level UI specifications

RWD design, Scenario and task creation,

consent forms, and focus Group study

Usability, accessibility & compliance

Formative & summative testing

Experience Tracking

Monitoring and analytics

Dashboard and reporting

Design & feature enhancements

Post Production

• Channel integration

• Channel innovation

CET, Usability & comparative

benchmarking

Continuous assessment and testing

Use of production data for further testing

Deploy & Release

Page templates and UI pattern creation

Screen design - modules/use cases

Feedback, messages and error handling

RWD Testing, Multivariate, A/B testing,

Testers as users

UI compliance test against guidelines and

best practices

Issue tracking and fixing

Strategize & Plan

Find customer segments & needs

As-Is assessment

Assess Competitive landscape

Test Plan/Tester Profiles

• Customer insights & user profiles

• Customer Journey maps

• Design blueprint

• KPIs and scorecard

QA involvement in requirement and

customer analysis to build scenarios

User analysis and usability base lining

Use of production scenarios and data

• UI standards, templates and

reusable UI components

• Detailed design specifications

• Visual Design Library

Test Scenario, Test Case, Test Reports, Metrics

CX Testing across the Apps Lifecycle

18 © 2015

Application level testing Digital product/system/value chain assurance

Guardian for functionality, performance, security Guardian for customer experience & brand

Descriptive metrics Predictive metrics, Quality Intelligence

Summary: Key Takeways

Tr a d i t i o n a l QA D i g i t a l A s s u r a n c e

Test automationLifecycle automation, early & continuous automation

Robotic process automation

Dev->Test->OpsAgile, Continuous Delivery, DevOps; QA->QE->SET->DET

QA shift left, right, up, down/deep

For More Information

To learn more, please visit:

http://cainc.to/Nv2VOe

CA World ’15