ManuAl Testing Best Practic

7/23/2019 ManuAl Testing Best Practic

http://slidepdf.com/reader/full/manual-testing-best-practic 1/122

Marcus Wefers, ALM Solution Management, AGS, SAP AG

Best PracticeManual Testing of E2E processes using SAP SolManager 7.1 - Test Option 1



© 2014 SAP AG. All rights reserved.

Agenda

1. Approaches for manual testing of E2E Business Processes using SAP Sol

2. Approach A: one Test Case covering the entire E2E Business Process

3. Approach B: individual Test Cases by business transaction composed into

4. Comparison Approach A and B

5. Appendix

Business Process Documentation: Process Step Library and composition of E2E Busin

Test Data Container for business process „Order to Cash“

SAP Note and ABAP report to create access link for Test Data Container

Hybrid Approach C as combination of Approach A and B for very long E2E processes

Document types for Test Package Attachments

Executable Variants



Alternative approaches for manual testing o

Business Processes using SAP Solution




Test ManagementCapabilities of test option 1 with SAP Solution Manager 7.1 SP07 or higher

Business Blueprint

Business Process

Change Analyzer

(BPCA)

WoC Test Management

• Manual Tests

• Automated Tests

SAP TDMS

Test Planning

eCATT

WoC Test Manag

Test Workben

BW Reportin

Partner Logs and R

Test Status Repor

Test script creation

Test system assignment

Test data assignment

Test Automation Framework

Test Case Assignment

other 3rd party test automation tools

Test Scope Optimization

Test Execution

HP QTP

SAP Solution Manager 7.1

Lean test system setup

CBTA (Component based Test Automation)

Business Blueprint

Test Planning

Test System SetupTest Execution and Reporting

Change Impact Analysis

WS Certify

WoC Test Management

• Test Plan generation

• Tester assignment

This is the current state of planning and may be


http://slidepdf.com/reader/full/manual-testing-best-practic 5/122© 2014 SAP AG. All rights reserved.

Manual TestingE2E Business Processes and involved testers

End to End Business Processes

Sales OrderNumber

Sales

OrderDelivery

Goo

IssQuotation

Transfer

Order

DeliveryNumber

QuotationNumber

DeliveryNumber

Tester 1 Tester 2

Purchase

Order

Goods

ReceiptPay

Purchase

Requistion

Invoice

Receipt

Order to Cash

Procure to Pay

Demand to Supply … … …

http://localhost/var/www/apps/conversion/tmp/900_Rollout_CDs/mySAP_ERP_CGV_Rollout_CD_intern/1_Corporate_Governance/MIC/P2P_4_Purchase_Order_Processing.ppt

http://localhost/var/www/apps/conversion/tmp/900_Rollout_CDs/mySAP_ERP_CGV_Rollout_CD_intern/1_Corporate_Governance/MIC/P2P_5_Goods_Receipt.ppt




















Manual TestingTest Types and Test Scope Identification

Test Type Test Scope Identification Shared test

infrast

Regression Test BPCA – Test Scope

Optimization (TSO)

• Business B

• Test Manag

• Manual test

assigned to

business pr

• Automated • BPCA TBO

• BPCA TSO

• Test Data p

Test Data C

• …

User Acceptance Test Mostly manual selection of

test cases for new /adjustedfunctionality

Integration Test BPCA TSO or full test scope

based on Blueprint attributes

or Test Case Keywords



Manual TestingMotivation and Alternatives

Approach B

For each process step of an E2E

business process a single manualtest case is provided

Each manual test case is assigned

at the relevant process step of the

E2E business process in the

Business Blueprint

Motivation for Manual TestingMotivation

Test Managers require business process experts (BPx) to perform functional tests of E2E bu

Although automated tests can cover wide ranges of business processes, additional manual t

for areas not covered by automated tests and for User Acceptance Tests (UAT).

Approach A

One manual test case for the entire

E2E business process The test case is assigned at the

business process level of the

Business Blueprint

Recommended approach

Additional a

Test Scope O

change impa Test executio

Process Expe

BPCA TBOM

Handling of te

tests



Manual Testing of E2E Business Processes

SAP Solution Manager - Business Blueprint with E2E Business Process: Order to C

Manual Testing - Approach A

1 E2E test case assigned at

business process level

Manual Test

6 test cases

process ste

E2E busine



Approach A

1 E2E Test Case covering the entire Business Pro



Environment used in SAP internal test system Approach A

SAP Solman SI7 / 200

SAP ERP XUQ / 800

Project AGS_MW_01 (logical component Z_XUQ, System Role: Q)

Blueprint Business Scenario: ERP (XUQ) manual E2E tests - A

Business Process: Financials

Business Process: Procure-to-Pay V1

Business Process: Order-to-Cash V1 (Standard Order)

Business Process: Order-to-Cash V2 (Rush Order)

Test Data Container TDC: Z_ORDER_TO_CASH

BPCA Analysis BPCA Result ID: 383 (change event: Transport: SAPK-604DHINSAPAPPLOptimization Approach: OA_5 „Key Word filter for test cases using value R

Test Plan / Package Created via BPCA:

• Test Plan: Regression tests for E2E Bus.Processes using E2E proc

• Test Packages: O2C, P2P, FIN

Created via WoC Test Mgmt – Test Plan Management

• Test Plan: Regression Test Plan E2E tests generated via Keywords

• no Test Packages created until now





SAP Solution Manager - Business Blueprint with E2E Business Process: Order to C


1 E2E test case assigned at

business process level

Manual Test

6 test cases

process ste

E2E busine




Solution Documentation: Composition of E2E Business

From Process Step Library ….

… to E2E Business Processes

Automatic generation

Manual composition of E

Decision on how to setup variants of Business Processes0

For more details see Appendix: Process Step Library and composition of E2E Business Pro






Blueprint:

Business Process levelTransaction to start the entire

For more alternatives see Appendix: Executable Variants




Manual Testing – Approach A1 E2E Test Case covering the entire Business Process

Manual test case (docx, xlsx, …) describing test activities for all transaction of E2E1





Manual E2E Test Case assigned to Business Process level (SOLAR02)

• Example business process Order to Cash with 2 process variants

• Assignment of one E2E Test Case – covering all executables of the BP variant - to each Business Proc

2

BP variant 1




Manual Testing – Approach A

1 E2E Test Case covering the entire Business Process

Key Word assignment for manual tests (SOLAR02)3

Op

Sele

(Key

SOL

Tab

Sub

As






BPCA TBOM Generation

• Purpose: change impact analysis and test scope identification

4

Semi-dynamic TBOM generation

• Automatic TBOM generation for majority of

executables of process steps

• Available with SAP Solution Manager 7.1

SP10

• Prerequisite: UPL activated in managedsystem

• Scope: for all process steps and

executables that can be tested individually,

thus E2E tests not required.

Approximately > 30% of your blueprint

Dynamic TBOM creation

• Manual TBOM generation at

(not process step) level by Bu

Experts (BPx)

• BPx executes all executables

the E2E Business Process wh

generated in the background.

• Note: process execution can

other using „TBOM extentions

• Scope: for all executables tha

as E2E business process. Ex






BPCA TBOM Generation. Step 1: semi-dynamic TBOM generation via backgrou4

Workcenter Test Management

View Admininistration

TBOM Utilities

BPCA TBOM Utilities

Setting of parameters for background job fo

Result: semi-dynamic TBOM for each executable (e.g. T-Code) assigned to Busines






BPCA TBOM Generation. Step 2: Dynamic TBOM generation via manual proce4

Step 1: select Business Process Variant and „Start“ transaction

Step 3: create dyn.TBO

Step 4 to n: continue wi

transactions of the Busi

entire flow of Business P

Note: you can pause an

continue with TBOM ge

using TBOM Enhancem

Step 2: Button „Attribute“

Result: dynamic TBOM for each E2E Business Process and assigned E2E test case





Test Data provisioning via Test Data Container (TDC)

• TDC definition: by Test Engineer or Business Process Expert (BPx)

• Test data provisioning: by BPx (manual planning in TDC or xls upload to TDC

• Access: Work Center Test Management – View Extended Test Automation (T

• SAP recommendation: 1 TDC for the entire business process (see Appendix

5





Include link to test data within test script for smooth access for your users5

1. Run ABAP report AGS_SMT_BUILD_T

Test Data Container (TDC) that includes

2. Result of ABAP report: URL which allow

script to test data of specified TDC

3. Use SAP Note 1927389 to create the A

Solution Manager 7.1 when current SP

4. Paste link into your test scripts (see Ap

Advantages

• Central maintenance of test data for all c

cycles – no need to use xlsx files that co

only 1 test cycle

• Smooth access for tester directly from te

which opens a second window during te

• Urgent changes to test data during the c

done centrally - no need to distribute xls






Test Plan Generation - Alternatives6

Test Cases assigned at Process Step level of Business Bluepri

2. TP Gen + Key Word Filter

Automatic selection of all

test cases with the relevant

Key Word value

3. TP Gen + BB Attribute Filter

Automatic selection of all test

cases assigned to process

steps with the relevant attribute

value

1. BPCA TSO + Key Word Filter

Automatic selection of E2E test

cases for Business Processes

impacted by change events plus

filter Key Word of test cases

4. TP G

Test

ca

visu

Bluep

WoC Test Mgmt BPCA

BPCA Test Scope OptimizationOptimization Approach with filter:

Test Case – Key Word, e.g. RT1a

BPCA popup: TP Generation

WoC Test Mgmt

Test Plan Management

Create Test Plan

Test Plan filter:Test Case Key Words with

value assigned to test

cases, e.g. RT1a

WoC Test Mgmt Test Plan Management

Create Test Plan

Test Plan filter:

Business Blueprint Attributes,

e.g. custom attribute „Business

Process Prio“ with value 1

T

Man






Test Plan Generation via BPCA Test Scope Optimization (TSO) 6.1

Step 1: Run BPCA for change event Step 2: continue with BPCA

select BPCA result ID and button „Op

M l T ti A h A







Step 3: Activate filter using Test Case Key Words Step 4: BPCA TSO - Creat

Tip: Process Steps of E2E Business

Process w/o test cases will be excluded

M l T ti A h A






Test Plan Generation with filter Key Words assigned to test cases6.2

Workcenter Test Management Test Plan Management

Filter: Key Word assigned to test c

Value: Regression Test Priority 1 (R

M l T ti A h A






Test Package creation for selected Test Plan7

Goal: set of test cases suitable for one tester or group of testers which execute the business proces

Path: Workcenter Test Management Test Plan Management Test Package Management

1.select Test Plan

2.generate Test Package



M l T ti A h A





Test Execution: Step 1: Workcenter Test Managment – View: Tester Work8

Tester 1

select Test Package

… and starts E2E

Test Case via „Run“

M l T ti A h A





Test Execution: Step 2: Read Test Case Description8

M l T ti A h A





Test Execution: Step 3: view test data centrally stored in Test Data Contai8

Second window with view of available test data stays open during test executio

TDC with test data for business process „



Manual Testing Approach A





Test Execution: Step 5: test execution and test data provisioning8

Screen 2: Test Data Container for business process O2C

Screen 3: Transaction „Create Quotation“ launched in test system






Test Execution: Step 6 after test execution:

• change to Edit mode in Test Data Container

• add Quotation Number to Test Data Container – as handover information fo

8

Tip: define additional columns in TDC for test results – see Appendix for details






Test Execution: step 7 (optional): documentation of test results

• create Test Note via „Copy Test Document“

8






Test Execution: Step 7 (optional): Test Note8

Optional activity:

tester adds test results, suc

or document number create

execution.






Test Execution: Step 8: Test status and results8






Test Execution: Step 9: Tester Worklist - end of test case 1 execution8

„Comment“ used as handover information for







Manual Testing Approach A1 E2E Test Case covering the entire Business Process

Test Execution: Step 11: Reads test case description8






Test Execution: Step 11: access Test Data Container 8

Test Data Container for O2C

Get result from previous test step.

Here: Quotation Number 20000138

Get test data fo

Here: Sales doc






Test Execution: Step 12: access System under Test (SUT) and execute tra8

Screen 1: test script – start test execution Screen 2: TDC test data

Screen 3: automatic launch of trans

be tested by user in pre-selected te

Test execution

Note: u

manua

transac

since o

transac

the E2









g pp1 E2E Test Case covering the entire Business Process

Test Execution: Step 15: Test status, comment and results8





g pp1 E2E Test Case covering the entire Business Process

Test Execution: Step 16: (optional) Test result documentation via Test No8






Test Execution: Step 17: Second test case execution finished8

Tester comment: „Sales Order ok – next te






Approach A - Advantages

1. Smaller number of test cases since test cases are defined at BP variant level for areas like Logis

step level (less creation and maintenance effort).

2. No need to define test sequences – Test Packages much easier to maintain (less effort)

3. Automatic Test Plan generation from BPCA Test Scope Optimization (TSO): correct E2E tests fo

processes and their variants can be detected automatically by BPCA. No additional selection o

to include all required test cases of the E2E business process (less effort, less error prone)

4. Easier handover of test results (Test Note visible to all testers, Tab Results including screensho

5. No Blueprint shortcut problem, since test cases are defined at Business Process level. (Test Pla

Keywords assigned to test cases, can‘t be used when process steps in E2E Business Processes

which is best practice)

Approach A - Challenges

1. No email notification for inform next tester of a test sequence. No icon „ready to test“ at Test Pac

2. Automatic launch of transaction in test systems only works for first transaction of the E2E busine



Environment used in SAP internal test system




Approach B


SAP ERP XUQ / 800

Project AGS_MW_01B (logical component Z_XUQ, System Role: Q)

Blueprint Business Scenario: ERP (XUQ) manual E2E tests - B

Business Process: Financials

Business Process: Procure-to-Pay V1

Business Process: Order-to-Cash V1 (Standard Order)

Business Process: Order-to-Cash V2 (Rush Order)

Test Data Container TDC: Z_ORDER_TO_CASH

BPCA Analysis BPCA Result ID: 440 (change event: Transport: SAPK-604DHINSAPAPPL

Optimization Approach: OA_5B „Key Word filter for test cases using value

Test Plan / Package Created via BPCA:

Test Plan: Regression Test Plan: multiple single test cases for E2E Bu

• Test Packages: O2C Variant 1, P2P Variant 1, FIN

Created via WoC Test Mgmt – Test Plan Management

• Test Plan: not yet (see Appendix: available for project AGS_BS_IMP)

• Test Packages: not yet (see Appendix: available for project AGS_BS_I






SAP Solution Manager – Business Blueprint with E2E Business Process: Order to


1 E2E test case assigned

at business process level

Manual Testing

6 test cases as

process step le

E2E business p





p

From Process Step Library ….


Automatic generation



For more details see Appendix: Process Step Library and composition of E2E Business Pro



Manual Testing – Approach BIndividual Test Cases by business transaction composed into E2E Test Package




Individual Test Cases by business transaction composed into E2E Test Package

Manual test case with description of test activities for 1 business transaction (1






Manual test case assigned to process step level of the E2E business process

• Example process „Order to Cash“: six test cases are assigned – each at the process s

• SAP recommendation: assign Test Object (e.g. transaction code) to the test case

• SAP recommendation: assignment of Key Word for test case, e.g. „Regression Test P

2






Key Word assignment for manual tests (SOLAR02)3

Op

Sele

(Key

SOL

Tab

Sub

As






BPCA TBOM Generation (automated)

• Purpose: change impact analysis and test scope

identification

• SAP Solution Manager 7.1 SP10 (Nov 2013)

• For majority of the Business Blueprint: automatic

generation of semi-dynamic TBOMs

• for multi-purpose transactions embedded in E2E

process with high degree of variation:

• SOLAR01: Blueprint setup: create Executable Variantsto describe BP variants

• SOLAR01: BPCA TBOM: manual creation of dynamic

TBOM at process step level to capture process variants,

such as different document types

• Result: TBOM for each process step

4













y p g


Step 1: Run BPCA for change event Step 2: continue with BPCA

select BPCA result ID and button „Opti













Test Plan Generation with filter Key Words assigned to test cases6.2

Workcenter Test Management Test Plan Management

Filter: Key Word assigned to test c

Value: „RT1“ Regression tests with

Remark: Filter does not work for process steps using shortcuts - SAP Note 1816801






© 2014 SAP AG All rights reserved

Test Package creation for selected Test Plan7

Goal: set of test cases suitable for one tester or group of testers which execute the business proces

Path: Workcenter Test Management Test Plan Management Test Package Management

1.sele

2.gene









Test Execution: Step 1

Tester #1 opens Work Center „Test Management“ - view „Tester Worklist“ and

assigned Test Packages include test cases which are in status „ready to test“ fo

8

Tester 1 Column indicates

Tester 1 in status







Test Execution: Step 3: Read Test Case Description8













Test Execution: step 8 (optional): documentation of test results

• create Test Note via „Copy Test Document“

8









Test Execution: Step 11: Tester Worklist - end of test case 1 execution8











Test Execution: Step 15: access System under Test (SUT) and execute tra8

Screen 1: test script – start test execution Screen 2: TDC test data

Screen 3: automatic launch of trans

be tested by user in pre-selected te

Test execution

Note: u

manua

transac

since o

transac

the E2




T t E ti St 17 t t lt (S l O d b ) t d i TDC f8




Test Execution: Step 17: test result (Sales Order number) stored in TDC fo8


T t E ti St 18 T t t t t d lt8




Test Execution: Step 18: Test status, comment and results8






Alternative B - Advantages




Alternative B Advantages

1. Email notification for inform next tester of a test sequence. Icon „ready to test“ at Test Package l

2. Automatic launch of transaction in test systems only works for each transaction of the E2E busin

Alternative B - Challenges

1. Higher number of test cases

2. Definition of sequences required which repeates the information altready provided in the E2E Bu

(duplication, higher maintenance effort during initial creation and ongoing maintenance)

3. Test Plan generation using BPCA Test Scope Optimization: user needs to manually add process

the remaining E2E process steps

4. Semi-automatic Test Plan generation using Keywords in combination with use of Blueprint Short

5. Handover of test results: information provided for individual test case at process step not visible







Appendix

Business Process Documentation: Process Step

and composition of E2E Business Processes

Solution Documentation: Process Step Library

Process Step Library




Process Step Library

• List of Process Steps by any grouping, e.g. by SAP Modules

• Assigned entities like Executables (e.g. transaction codes) and documentation

Example:

Process Steps and Transactions for SD

Automatic generation of Process Step Library

• Programm Z_Utility_Blueprint_Generation for selected customers provided by PM, RIG, C

• See next pages for details



Solution Documentation: automatic generation of Process Step LSAP Utility Program to generate SAP Solman Project and SAP-Module oriented Blue

Purpose




• Auto-generation of SAP-Module oriented Blueprint based

on execution history (workload statistics)

Program name

• Z_Utility_Blueprint_Generation (not SAP standard)

Selection Screen

• Enter managed system / client

• …

• SAP SolMan projects that include BB and transactions /

reports to be excluded

Usage History

• Alt 1: read from managed system

• Alt 2: read from SAP SolMan InfoCube

Availability

For selected customers via PM, RIG, COE

Available for SAP SolMan 7.1 SP09 and SP10

BPCA Prerequisite No 1: Business BlueprintSAP Utility Program to generate SAP Solman Project and SAP-Module oriented Blue

Result




• 3 level Blueprint hierarchy

• Level 1: SAP Module

• Level 2: ACH L2

• Level 3: Transaction or

Report

• Tab „Transaction“ with

assigned executable

Custom Code

• Node „Customer“

• Custom Code transactions

and reports are grouped by

development class



Section 2 of Business Blueprint: E2E Business ProcessStep 1: create business process




Create manually the business proc

Section 2 of Business Blueprint: E2E Business ProcessStep 2: Copy business process step from Process Step Library to E2E Business Pro

Copy business process step




Copy business process step

• from Process Step Library

• to E2E Business Process

Process Step in Process Step Library

• Right mouse – „Copy“

Section 2 of Business Blueprint: E2E Business ProcessStep 3: Navigate to process area and insert Process Step to E2E Business Process

Copy Process Step from Process Step Library Navigate to E2E Business Proces




g

and insert as Copy (Alt 1) or inse

Result



Section 2 of Business Blueprint: E2E Business ProcessResult after disbanding all shortcuts

From Process Step Library …. Automatic generation








Appendix

Test Data Container for entire Order to Cash Proc







Appendix

Link from manual test script to Test Data Contain

Generation of link to access Test Data Container (TDC)Purpose: access from manual test script to Test Data Container

Report AGS_SMT_BUILD_TDC_NAV_URL




• SAP Note 1927389 includes the code in case your SAP Solman system does not inlcude th

• Input parameter: Test Data Container – select via value help (F4)

• Run report in SAP Solution Manager system which creates 2 snippets of the required link

Result

Alternative 1:

use buffered URL and

include in test scipts

via CTRL V





Appendix

Approach C – combination of Approach A and B

Approach CCombination of Approach A and B for E2E processes with very high number of proce

T t Pl

E2E Business Process grouped into 3 processes containing multiple process

E d t d S i St t




Approach C

• E2E business process separated into multiple business processes –each containing multiple process

• Suitable when using very long E2E Business Processes which cannot be covered by 1 tester group

• Test sequence used as handover from one business process test to next business process

• Each Test Package includes multiple tests which can be handled by one test team

Test PlanTest PackageTest PackageTest Package: Order To Cash

Sales Order

Logistics Execution

Billing

End-to-end Scenario Structure

Test

Sequence



Appendix

Definition of document types used during test ex

store document numbers via tab “Test Package

Attachment”

Definition of new Document TypesTransaction: SOLAR_PROJECT_ADMIN

Path: Tab: „Project Standards“ Tab „Documentation Types“




Define new Documentation

„Test Results via Test Pa





Define D





•

•

•

•




Decision on how to setup variants of Business Processes

0

Alt 1: multiple Business Processes Alt 2: Executable Variants*




Alt 1: multiple Business Processes Alt 2: Executable Variants

Example: Example:

SalesOrder-to-Cash Variant a

Process Step 1a (transaction 1)

Process Step 2a (transaction 2)

….

Order-to-Cash Variant b

Process Step 1b (transaction 1)

Process Step 2b (transaction 2)

ProcurementProcure-to-Pay Variant a

…

Procure-tp-Pay Variant b

…

SalesOrder-to-Cash

Process Step 1 (transaction 1*, 1**)

Process Step 2 (transaction 2*, 2**)

Procurement

Procure-to-Pay

Process Step 1 (…)

Process Step 2 (…)

Remark:

executable * and ** might be the same transac

different document types or other types of diffe

navigation through the transaction.

* available with S


Decision: documentation of Business Processes variants – Alt 2

0




Blueprint:

Business Process level

Button to c

Transaction to start the entire business proc


BPCA TBOM Generation: Example: manual creation of TBOM for each Business P

4

Step 1: select BP and Executable Variant #1 Step 2: button „Attributes“




Step 3: create dyn.TBO

Step 4 to n: continue wi

transactions of the Busientire flow of BP variant

Note: you can pause an

continue with TBOM ge

using TBOM extentions



Environment used in SAP internal test system Approach B using SAP Solman Project AGS_BS_IMP


SAP ERP XUQ / 800




SAP ERP XUQ / 800

Project AGS_BS_IMP

Blueprint Business Scenario: ERP Logistics O2C (XUQ) / Process: 0_

Test Data Container TDC: Z_ORDER_TO_CASH_1

BPCA Analysis BPCA Result ID: 275

Optimization Approach: OA_4

Test Plan / Package Created via BPCA:• Test Plan: Test Plan from BPCA using Key Word Value RT1

• Test package: no test package createdCreated via test plan generation using Test Case Keywords• Test Plan: Test Plan E2E Process via Keywords

• Test Package: E2E Test Package O2C

• Sequence: SQ1

Documents

ManuAl Testing Best Practic