How Verizon Uses Automation to Accelerate SAP Projects

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

How Verizon Uses Test Automation to Accelerate SAP Implementations

Nick Germanotta, Verizon

Bala Thangaraj, IBM

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 2

Agenda

• Overview

• Uni-Fi Program Overview

• Automation Testing Objectives

• Accelerated Implementation Approach

• Execution Strategy

• Benefits Summary

• Next Steps

• Questions


Overview

This session explains how Verizon used Worksoft test automation suite to accelerate two successive SAP convergence projects with a combination of IBM methods and Worksoft best practices

Learn how we executed end-to-end business process testing to reduce cycle times by 35%, save time with streamlined / reusable test scenarios, and improve business process quality and test coverage.

Three main takeaways:

• Explore how Verizon used automated testing to effectively build & execute end-to-end testing early in the project. This significantly reduced the traditional three testing cycles to two and enabled early detection of defects

45% of overall testing scope was automated in less than 90 days

• Learn how to reduce manual effort and improve staff efficiency and how to apply the efficiency model to future releases.

Executed automated test scenarios in 1.5 hours vs. 40 hours when done manually

• Develop plans to maintain high quality business process execution during a major SAP implementation to ensure business operations and users are not adversely affected during a consolidation, migration, or upgrade.

Worksoft enabled Team without increasing the project resource pool


Uni-Fi Program OverviewUni-Fi, a financial systems unification program was approved by Verizon executive management and is sponsored by Verizon’s Corporate Executive Vice President and CFO. Its objective is the consolidation of several legacy systems and acquisitions into one of the select ERP systems that best fit the existing standard Verizon business processes.

By consolidating and standardizing systems and processes, Uni-Fi promotes:

Corporate

Agility

Improved Financial

Controls and Reporting

Rapid Acquisitions

and Divestitures

Uni-Fi will consolidate to select SAP and PeopleSoft ERP systems

Consolidated System

Architecture


Uni-Fi Program Overview

Challenges:• Inconsistent policies, processes and

metrics, and lack of controls• Aging legacy systems and increased

maintenance costs

• Multiple reporting sources and increased regulatory and reporting requirements

• Multiple transactional systems and support strategies

Priority:


Uni-Fi Program OverviewERP System Consolidation: 2012 – 2017*

• Schedule and plan pending funding allocation and scope determination

V1V

VNIC/VSSI/VZEDI

EdgeCast

Dispatch1

Corp. & VDSL

ECS

Uplynk

Hughes

VDSI

nSAP

vSAP

PS 9.1

CostPoint VFI

Terremark

OnCue

TBD

Legend

Select System

System / Org.

Completed

Underway


Uni-Fi Program Overview

Release 3NPRS & D1 to nSAP

Release 3.1Edgecast to nSAP

Release 4 Corporate & Domestic IT to PeopleSoft 9.1

Release 4.1*VDSI to PeopleSoft 9.1

Release 5*Empire City Subway to vSAP

Hughes ConversionRelease # and target system TBD

2014 2015 2016 2017

Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1

A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M

NRPS & D18/1/2014

Uplynk & OnCue went live in 1Q2014

*Go-Live date confirmation pending scope lock

TODAY

Edgecast10/1/2014

ICE & VDS8/3/2015

VDSI2016 TBD

ECS7/1/2015 (TBC)

Scheduled/Completed

Targeted/Go Live TBD

Under Assessment

HughesBeing Assessed


Automation Testing Objectives

• Testing is said to be one of the biggest pain-points of SAP Implementation and Upgrades

“For SAP implementations more than a third of total project effort is used for “Quality & Testing”

• Experience from large IBM SAP projects and projects in general show that level of automation is ”too” low & this trend is changing

Improved Test Coverage Reduced TCO Accelerated Implementations

Objectives to be achieved

• Accelerate Uni-Fi project releases by reducing test cycle time• Free-up business critical resources to drive strategic important projects without increasing budget• Increase efficiency by reducing manual testing effort, email hand-offs and wait times• Create an Automation framework that can be adopted for future Uni-Fi releases • Early detection of defects – faster execution of tests. Improve time to react.• Minimize the creation of automated test scripts by using re-use Test Libraries – create once, use many times• Use well documented business processes for user training and regulatory requirements

IBM’s experience at large SAP clients shows that Worksoft automation tool is 30-45% faster in test build and execution than previously deployed toolsTTQ


Accelerated Implementation:Blueprint Phase

Scenario Scenario

ProcessProcess

StepsSteps

Confirm the Business Process Hierarchy (BPH) in Solution Manager as soon as the Business Blueprint phase is completed.

Design the Business Process Hierarchy in SOLMAN

BPH defines the “Scenario Level” scope of Testing

Design the Business Process Hierarchy in SOLMAN

BPH defines the “Scenario Level” scope of Testing

Expand the Scenarios in BPH to identify the Data Variant Scenarios to define the Overall Test Scope

Ex: Procure to Pay – Procure to Pay for Capital Projects

Procure to Pay for Expense Projects

Procure to Pay for Limit Item Services

Expand the Scenarios in BPH to identify the Data Variant Scenarios to define the Overall Test Scope

Ex: Procure to Pay – Procure to Pay for Capital Projects

Procure to Pay for Expense Projects

Procure to Pay for Limit Item Services

Classify the Scenarios under “Quad A” methodology “Adopt” – will use process & system as designed

“Adapt” – some changes to accommodate requirements

“Add” – new process or functionality currently not available

“Abstain” – Process not applicable for this release

Classify the Scenarios under “Quad A” methodology “Adopt” – will use process & system as designed

“Adapt” – some changes to accommodate requirements

“Add” – new process or functionality currently not available

“Abstain” – Process not applicable for this release


Scope of Business Process Validation

SRM

Leasing

Supply Chain

Project & Assets

Order to Cash

Accounts Payable,

T&E

nSAP Finance

In Scope

Out of Scope

Procure to Pay Scenarios

[SRM, Portal & SAP ECC]

Procure to Pay Scenarios

[SRM, Portal & SAP ECC]

Modules Types of Scenarios

Order to Cash ScenariosOrder to Cash Scenarios

Project Creation to SettlementProject Creation to Settlement

Asset MaintenanceAsset Maintenance

Travel & Expense ProcessesTravel & Expense Processes

AP Invoicing & Payment ProcessesAP Invoicing & Payment Processes

Tax

&

Fin

anci

al

Val

idat

ion

sTa

x &

F

inan

cial

V

alid

atio

ns


Accelerated Implementation:Realization Build Phase

Define the selection criteria for automation and identify the Scope of Automation Testing in early Realization Phase

SAP Projects of such scale will use the following criteria to decide the Business Scenarios that fall under Automation

• Business Critical Test Cases – Global Scenarios that have broader System and release Impact• Data Driven Test Cases - Business Scenarios that use multiple data values for the same functions• Complex and Time Consuming Business Cases - Multiple Hand-offs across different Functional Areas• Business Scenarios that requiring great deal of precision on Qty or Dollars [High Financial Impacts]• Testing Needed on Multiple Systems / SAP Clients or Portal • Simple Test Cases involving repetitive tests or several iterations • And those that can be Automated using the Tool technology

SAP Projects of such scale will use the following criteria to decide the Business Scenarios that fall under Automation

• Business Critical Test Cases – Global Scenarios that have broader System and release Impact• Data Driven Test Cases - Business Scenarios that use multiple data values for the same functions• Complex and Time Consuming Business Cases - Multiple Hand-offs across different Functional Areas• Business Scenarios that requiring great deal of precision on Qty or Dollars [High Financial Impacts]• Testing Needed on Multiple Systems / SAP Clients or Portal • Simple Test Cases involving repetitive tests or several iterations • And those that can be Automated using the Tool technology

The following test cases will not fall under Automation

• One time / Ad-Hoc Testing• Back End Testing that runs behind GUI applications• Usability – If the Business Process is easy to execute and Test• Nature of the Business Process does not affect future changes and releases to a broader degree• Localized Business Processes

The following test cases will not fall under Automation

• One time / Ad-Hoc Testing• Back End Testing that runs behind GUI applications• Usability – If the Business Process is easy to execute and Test• Nature of the Business Process does not affect future changes and releases to a broader degree• Localized Business Processes


Accelerated Implementation:Realization Test Phase:

Categorizing processes using this framework helped manage scope and drive efficient work.

Functional Unit Test Cycle 1: End-to-End / UAT Dry Run

Cycle 2: End-to-End /User Acceptance Test

‘Adopt’• Build Automation Component Transactions

needed for E2E Scenarios that most regions will likely adopt

• Execute automated end-to-end scenarios using organizational elements during Week 1 of the cycle

• Execute automated end-to-end scenarios using organizational elements during week 1

‘Adapt’• Adjust the Automation Components

Transactions needed for E2E Scenarios• Execute automated end-to-end scenarios using to

accommodate regional and/or site relevant adjustments

• Execute automated end-to-end scenarios using to accommodate regional and/or site relevant adjustments

‘Add’• Manually Script and test all core SAP

transactions and development objects during Global Build in HPQC

• Script and validate any additions to end-to-end tests in HPQC.

• Manually execute E2E scripts using converted data & security roles

• Manually execute approved end-to-end scripts in HPQC using new security roles and converted data.

‘Abstain’ • Identify and communicate transactions

and development that are out of scope for a site

• Identify scenarios that are low risk or not applicable • Refrain from executing low risk scenarios that passed during informal testing.

Build Automation TestsBuild Automation Tests Execute Automated Tests as early as Week 1 of each testing cycleExecute Automated Tests as early as Week 1 of each testing cycle


Functional

Design

Baseline Configur

ation

Functional Unit

Testing (FUT)

Move to QA

End User Role

Profiles

Integration

Cycle 1

UAT Dry Run

Integration

Cycle 2

UAT

Final Regressio

n Test

Risk Assessment

Go / No-Go

Decision

Realization Phase: Milestones

PA

SC

FICO

AP

LE

Automation Test “BUILD”

Automation Test “EXECUTION”Proj

ects

GL

P2P

- 45% of the Total Test Scope was automated in less than 90 days

- 70 Core End to End Scenarios (~237 Scripts) were built versus the original plan of 42 scenarios

- “Adopt” & “Adapt” Scenarios were automated with the dual intent of utilizing them for Integration as well as regression testing

- 45% of the Total Test Scope was automated in less than 90 days

- 70 Core End to End Scenarios (~237 Scripts) were built versus the original plan of 42 scenarios

- “Adopt” & “Adapt” Scenarios were automated with the dual intent of utilizing them for Integration as well as regression testing

- 40% of the Test Scope was executed during week 1 of each integration test cycle.

- Fully validated implementation with 2 integration test cycles in contrast to 3 cycle approach

- Reduced the overall test execution time for each E2E scenario from ~35 Hrs to 1.5 Hrs and increased the number of Data Variants per scenario

- 40% of the Test Scope was executed during week 1 of each integration test cycle.

- Fully validated implementation with 2 integration test cycles in contrast to 3 cycle approach

- Reduced the overall test execution time for each E2E scenario from ~35 Hrs to 1.5 Hrs and increased the number of Data Variants per scenario


Worksoft Enabled Team

• Subject Matter Experts (drawn from Business Team)

– Responsible for recording the business process using Certify Capture, typically from Client’s Business Team

– Provides guidance to the Automation Test Engineers on the Business Process

– Validate, Reviews & Approves automation Tests Build– 2 to 4 Business SMEs are typically engaged

• Automation Test Engineers – Responsible for enhancing the Business Process

recorded by SMEs, typically fits a Consultant Profile– Defining Variables for use in the Test and setup

RecordSets– Build continuity in E2E Scenarios by linking several

processes– Build centralized processes for use in cross

functional integration scenarios– 2 to 3 Automation engineers per process area

• Central Test Management– Responsible for Administration of Worksoft & User

Access Management and HPQC– Establishes Best Practices, Naming Conventions and

Project Structures for use– Coordinates the Test Planning & Execution

Responsible for management of issues, Risks, Scope and Schedule

– One Test Lead and 2 Test Coordinators

• Worksoft Support Consultant– Responsible for Installation & Setup of Worksoft– Provides Training & Guidance to the Automation Test

Team during Planning, Build & Execution– Resolves complex Test Requirements, Collaborate,

mentor and assist in Build Process– 1 Worksoft Support Engineer was onsite for the

duration of Build.

Business SMEs and Automation Test Engineers Commit 25% of their time towards the Automation Test Build Efforts during the Realization Phase in parallel to the other Project Tasks to which they have been assigned


Benefits Summary

15

Key Project Approach Drivers Minimal Verizon staff to support project & better utilization of

Consulting resources

Significant integration requirements to 3rd party and legacy systems

Multiple sites for manufacturing and distribution

Highly successful operations cannot be impacted

Project Results 45 % of the Total Test scope was automated

70 E2E Processes [~ 237 Scripts] reduced Testing Cycle time by 35%

Fully validated implementation with 2 Integration test cycles in contrast to traditional 3 cycle testing approach.

Reduced overall test execution time for each E2E scenarios from `35 Hrs to 1.5 Hrs and increased the number of data variants per scenario

Company Overview / Background SAP Users: 99,000 in 55 Countries

The Uni-Fi Convergence program was approved by Verizon executive management. Its objective is the consolidation of all wireline ERP processes into 2 instances of SAP using the current nSAP system and processes as the mandatory go-forward solution.

164 Core Test Scenarios with ~ 500 Data Variants involving 137 Testers positioned globally

Scope of Services ERP Implementation and Integration

SAP ECC 6.0 / SRM / BW / Legacy

FICO

SD

MM

WM

PP-PI

Project Systems

Asset Management

Business Intelligence

Key Success Factors

45% E2E Automated Tests executed during Week 1 of each test cycle. Relentless focus on accomplishing milestones on schedule

Early test execution reduced the risk with late defect detection

Eliminated the needed for Test Script Programming specialists. Test build engineers drawn from existing functional team

Management support for adherence to the Worksoft Test Automation tool & IBM’s “Quad A” methodology

Quick Adoption, Team Training & Ownership

Automation Test foundation laid for future releases & benefits are being realized


Next Steps

• The initial Worksoft capabilities built for Release 3 was leveraged significantly for a follow-on EdgeCast Release, a new Verizon acquisition onto nSAP.

• Expand the automation framework to the vSAP ERP platform

• Increase automation capabilities in the SRM, Portal and Workflow notifications which

constantly changes due to policy updates and technical upgrades.

• Considering deployment of “Lights-out” Testing and Certify Impact for future implementations


References

Verizon: Nick Germanotta, Uni-Fi Program Director [email protected]

IBM:

Bala Thangaraj, Global Business Services [email protected]

Armando Ortiz, Partner Comms Sector [email protected]

Michael Thelen, Delivery Partner [email protected]

Worksoft:

Harrison Schultz, Partner [email protected]

Chris Lynch, Partner [email protected]

mailto:[email protected]






Software

How Verizon Uses Automation to Accelerate SAP Projects