How Metso uses robotic software testing in a global SAP/Hybris … Success... · 2019-11-08 · How...

How Metso uses robotic software testing in a global SAP/Hybris implementation.

How Qentinel Pace robots do regression tests of business processes in a global SAP/Hybris implementation.

• Case background

• What has been done?

• Qentinel Pace™

• Qentinel QA solution

• Environment architecture

• Manual vs robotic software testing

• Typical findings from automated testing

• Qentinel Pace – Metso ; facts, insights

• How Metso has seen the service? What are the benefits seen already?

2

Agenda

Metso highlights

Mining Aggregates

Process industries Recycling

12,000

employees

worldwide

FY2017

Orders received EUR 3,0 billion

Sales EUR 2,7 billion

Adjusted EBITA 9,0%

35% equipment, 65% services

PUBLIC

51% 26%

19% 4%

3

my.metso.com –an improved digital customer journey for the entire equipment lifecycle

4

Complexities across the online sales process

5

Testing various variables and combinations critical

• Distributor-specific catalogues (items vary)• Several delivering plants – ’saleability’ defined

by the default plants• Price & availability specific for customer, plant &

material combinations

• Different checkout parameters for different plants(= order / quotation types)

• Different currencies• Different quotation/order types for printouts and

order history

Metso needs confidence in digital business processes

Target: Improving online business with modern easy-to-use online tools

• Over 100 Aggregates distributors, who need to quote and order on Metso offering based on real-time supply chain data from SAP

• 100 000+ parts and 100+ capital equipment provided online

Business processes must work end-to-end• Data integrity is a major issue: must test the whole business process with data

• Number of combinations with distributors and parts, equipment: testing is frequent and laborious

• Regression testing important – core ordering processes have to be robust!

• Equipment are possible to configure based on customer needs; testing needed that configuration is correct and saved in SAP

• Agile development: need fast feedback to developers

6

• Preparations to the first pilot distributors

• Roll-outs for first group of users (EMEA)

• Manual quality validation

• Discussion begun about automatic validation –because the amount of distributors was seen to grow fast

• More distributors added to production

• Qentinel Pace taken into use in sprint mode

• Focus in validating the purchase flow end-to-end (SAP automation excluded)

• Multibrowser and multienvironment support implemented

• Automation to cover SAP (QVision)

7

• Testing coverage increased also to capital equipment side

• Ensuring the capital readiness for pilot

• Readiness for Qentinel Pace achieved to SAP back-end verification

What has been done?

1Q2018 ? 2Q2018? 3Q2018

Qentinel Pace™Robotic software testing

https://qentinel.com/software-testing/

Test automation• Cloud-based• Keyword-based tests require

no programming skills• Integration with DevOps tools

Business impact• Cause-effect model

Quality reporting• Any number of KPIs• From any source• In a hierarchy

Usage-based pricing

8

Qentinel QA solution

Qentinel Pace™ - Robotic Software Testing

End-to-end testing of the purchase flow

• Ensuring that system operates as it is planned

• Both in the parts’ and the capital equipment side

• Main features and data are verified

Qentinel helps in user validation

• Assisting Metso in validating featuresMain features are first tested manually then automatized

• Coordinating business testing activities at Metso

9

Simplified environment architecture

SAPMMMaterials

Management

Hybris Cloud

Shop front-endmy.metso.com

Refresh

Development environments

STG

STG

RMMQMM

QA

QA

DMM

Dev

Dev

PMM

Prod

Prod

Production

QentinelPace

Priority

10

Manual vs. Robotic software testing

• Robot won’t take breaks.

• Between test runs some maintenance for scripts is needed.

• Manual testing can focus on possible problematic areas.

Qentinel Pace automation:Execution time for 1 distributor and 11 casesfor Parts: 8min 25sec

We support ~110 distributors in Qentinel PaceManually ~ 1 week (37 hours)

With automation ~ 7 hours

Manual testing:Execution time for 1 distributor and 11 cases for Parts: 20min 50sec

11

Typical findings from automated testing

• Scope related changes

• Configuration issues

• System ”Oops errors” in various situations.

- Changing the sales area of same distributor, corrupted cart

• Functionality errors

• Problems in SAP environment reflects to front-end

• Inconsistency errors in:

- Format how to present different measures, decimal digits etc.

12

13

CONFIDENTIAL12

Insights• Dashboard of KPI’s and metrics which are important

to project(Data source can be any relevant system e.g. JIRA, Azure DevOps etc.)

• Quality Index provides real time status of overall business benefits

• Automated testing creates confidence and pushes quality

• Monthly deployments cause pressure on continuous quality assurance

• Regression testing is critical – core ordering processes cannot tolerate errors

• Quality assurance feedback for go-live decisions always reviewed

• Larger adoption of test automation among the developers and testers progressing

14

How Metso has seen the use of robotic software testing? What are the benefits seen already?

15

Qentinelwww.qentinel.cominfo@qentinel.com

Thank you !