Embed Size (px)
Citation preview
How Metso uses robotic software testing in a global SAP/Hybris implementation.
How Qentinel Pace robots do regression tests of business processes in a global SAP/Hybris implementation.
• Case background
• What has been done?
• Qentinel Pace™
• Qentinel QA solution
• Environment architecture
• Manual vs robotic software testing
• Typical findings from automated testing
• Qentinel Pace – Metso ; facts, insights
• How Metso has seen the service? What are the benefits seen already?
2
Agenda
Metso highlights
Mining Aggregates
Process industries Recycling
12,000
employees
worldwide
FY2017
Orders received EUR 3,0 billion
Sales EUR 2,7 billion
Adjusted EBITA 9,0%
35% equipment, 65% services
PUBLIC
51% 26%
19% 4%
3
my.metso.com –an improved digital customer journey for the entire equipment lifecycle
4
Complexities across the online sales process
5
Testing various variables and combinations critical
• Distributor-specific catalogues (items vary)• Several delivering plants – ’saleability’ defined
by the default plants• Price & availability specific for customer, plant &
material combinations
• Different checkout parameters for different plants(= order / quotation types)
• Different currencies• Different quotation/order types for printouts and
order history
Metso needs confidence in digital business processes
Target: Improving online business with modern easy-to-use online tools
• Over 100 Aggregates distributors, who need to quote and order on Metso offering based on real-time supply chain data from SAP
• 100 000+ parts and 100+ capital equipment provided online
Business processes must work end-to-end• Data integrity is a major issue: must test the whole business process with data
• Number of combinations with distributors and parts, equipment: testing is frequent and laborious
• Regression testing important – core ordering processes have to be robust!
• Equipment are possible to configure based on customer needs; testing needed that configuration is correct and saved in SAP
• Agile development: need fast feedback to developers
6
• Preparations to the first pilot distributors
• Roll-outs for first group of users (EMEA)
• Manual quality validation
• Discussion begun about automatic validation –because the amount of distributors was seen to grow fast
• More distributors added to production
• Qentinel Pace taken into use in sprint mode
• Focus in validating the purchase flow end-to-end (SAP automation excluded)
• Multibrowser and multienvironment support implemented
• Automation to cover SAP (QVision)
7
• Testing coverage increased also to capital equipment side
• Ensuring the capital readiness for pilot
• Readiness for Qentinel Pace achieved to SAP back-end verification
What has been done?
1Q2018 ? 2Q2018? 3Q2018
Qentinel Pace™Robotic software testing
https://qentinel.com/software-testing/
Test automation• Cloud-based• Keyword-based tests require
no programming skills• Integration with DevOps tools
Business impact• Cause-effect model
Quality reporting• Any number of KPIs• From any source• In a hierarchy
Usage-based pricing
8
Qentinel QA solution
Qentinel Pace™ - Robotic Software Testing
End-to-end testing of the purchase flow
• Ensuring that system operates as it is planned
• Both in the parts’ and the capital equipment side
• Main features and data are verified
Qentinel helps in user validation
• Assisting Metso in validating featuresMain features are first tested manually then automatized
• Coordinating business testing activities at Metso
9
Simplified environment architecture
SAPMMMaterials
Management
Hybris Cloud
Shop front-endmy.metso.com
Refresh
Development environments
STG
STG
RMMQMM
QA
QA
DMM
Dev
Dev
PMM
Prod
Prod
Production
QentinelPace
Priority
10
Manual vs. Robotic software testing
• Robot won’t take breaks.
• Between test runs some maintenance for scripts is needed.
• Manual testing can focus on possible problematic areas.
Qentinel Pace automation:Execution time for 1 distributor and 11 casesfor Parts: 8min 25sec
We support ~110 distributors in Qentinel PaceManually ~ 1 week (37 hours)
With automation ~ 7 hours
Manual testing:Execution time for 1 distributor and 11 cases for Parts: 20min 50sec
11
Typical findings from automated testing
• Scope related changes
• Configuration issues
• System ”Oops errors” in various situations.
- Changing the sales area of same distributor, corrupted cart
• Functionality errors
• Problems in SAP environment reflects to front-end
• Inconsistency errors in:
- Format how to present different measures, decimal digits etc.
12
13
CONFIDENTIAL12
Insights• Dashboard of KPI’s and metrics which are important
to project(Data source can be any relevant system e.g. JIRA, Azure DevOps etc.)
• Quality Index provides real time status of overall business benefits
• Automated testing creates confidence and pushes quality
• Monthly deployments cause pressure on continuous quality assurance
• Regression testing is critical – core ordering processes cannot tolerate errors
• Quality assurance feedback for go-live decisions always reviewed
• Larger adoption of test automation among the developers and testers progressing
14
How Metso has seen the use of robotic software testing? What are the benefits seen already?
