Automatic Test of Queries

Automatic Test of Queries

with RS Trace Tool

Tobias Kaufmann

SAP AG, January 2013

© 2011 SAP AG. All rights reserved. 2

Agenda

1. Overview

2. Prerequisites

3. Trace

4. CATT Trace

5. Test Package

6. Test Job

7. Test Result

8. Error Analysis

9. Summary


Motivation

Current Situation

SAP ensures correctness of standard test cases

Standard test cases do not cover all customer test cases

Customer test cases require testing in customer system

Proposed Situation

Trace Tool is used to record navigation steps and data of customer-specific

Queries

Trace Tool is used to compare actual data with recorded reference data

Reduce manual test effort for customer


Use cases

After applying Notes or Support Packages

Ensure runnable Queries (no syntax error)

Ensure correctness of Query results

After loading Data (InfoProvider and Master Data)

Ensure correctness of cache and/or BI Accelerator

Warm-up OLAP cache


Automatic Test of Queries

General Procedure

1. Prepare Test by recording navigation steps and reference data

2. Apply Notes or Support Packages

3. Replay Test

Transaction RS Trace Tool (transaction code RSTT)

Definitions

Trace is a sequence of navigation steps with Checkable Program Objects

CATT Trace is a Trace with recorded Reference Data

Test Package bundles several CATT Traces

Test Job is a scheduled job with Test Package

Test Package

Test Job

CATT Trace

Trace


Reference Data and Focus of Test

Comparison of Reference Data and actual Data

Fix Master and InfoProvider Data is required

Fix Metadata (definition of Query, InfoObject, etc.) is required

Restrict to specific date (e.g. restrict to 2006 or 2005)

Restrict to specific characteristic values (master data)

Scope of Test

OLAP functionality (incl. figures and ABAP syntax)

Productive or test system

No BEx Web (ABAP and Java Runtime) functionality

No BEx Analyzer (Excel) functionality

Use transaction RSRT2 (HTML view) to record navigation steps


Detailed Procedure (1)

Query 2. Record navigation steps

CATT Trace

3. Create CATT Trace

based on Trace

Test Package 5. Bundle CATT

Traces to Test Package

4. Record

Reference

Data

Test Job

6. Schedule Test

Package in Test Job

Notes

Support Packages

8. Apply Notes or

Support Packages

10. Analyze Log of

Test Job

11. Identify erroneous

CATT Trace

12. Check erroneous

CATT Trace manually

Trace

1. Activate User for Tracing

9. Running

Test Job

InfoProvider /

Master Data

7. Load InfoProvider

or Master Data


Detailed Procedure (2)

Prepare Test

1. Activate User for Tracing using transaction RSTT

2. Record navigation steps in query as Trace

3. Create CATT Trace based on Trace

4. Record Reference Data of CATT Trace

5. Bundle CATT Traces to Test Package

6. Schedule Test Package in Test Job

Change System

7. Load InfoProvider or Master Data

8. Apply Notes or Support Packages

Execute Test

8. Check CATT Traces by running Test Job

9. Analyze Log of Test Job

10. Identify erroneous CATT Trace

with log and reporting

11. Check erroneous CATT Trace manually

Test Package

Test Job

CATT Trace

Trace

Test Job

CATT Trace

Query

Data

Support Packages


Agenda

1. Overview

2. Prerequisites

3. Trace

4. CATT Trace

5. Test Package

6. Test Job

7. Test Result

8. Error Analysis

9. Summary


Prerequisites

Technical Content

Used to stored Test Results

InfoCube 0RSTT_C01

Query 0RSTT_C01_Q002

These objects are activated automatically

Run report RSTT_CONFIG once (see note 1548961)

Basic Settings

Start transaction RSTT for one time initialization

1.Press Basis Settings

2.Choose Reporting, Planning and OLAP Technology as Application Area

3.Check on Trace Execution in New Roll Area

2

3

1


Agenda

1. Overview

2. Prerequisites

3. Trace

4. CATT Trace

5. Test Package

6. Test Job

7. Test Result

8. Error Analysis

9. Summary


User Activation with Transaction RSTT

Trace is a sequence of

navigation steps

1. Press Trace Tool

2. Check on Activation for

Test Generation

3. Press Activate

4. User will appear in list

of Trace Users

Test Package

Test Job

CATT Trace

Trace

2

3

4

1

Without

activating this

checkbox, no

test could be

created


Record Navigation Step of Query as Trace (1)

1. Start transaction

RSRT2

2. Enter Query

3. Choose HTML as

Query Display

4. Press Execute

Using Value help creates

additional Trace with

own Trace ID

1

Test Package

Test Job

CATT Trace

Trace


Record Navigation Step of Query as Trace (2)

1. Navigate

2. Drill down

3. Filter

4. …

Test Package

Test Job

CATT Trace

Trace


View Trace (1)

1. Select Traces

2. New Trace appears in

History

3. Press Display to view

details

Traces shows History of

current user

Trace Collection shows

complete list of all

Traces

2

3

1

Test Package

Test Job

CATT Trace

Trace


View Trace (2)

Traces contain a sequence

of navigation steps

Each navigation step is

represented by one or

more Checkable

Program Objects

Traces without Checkable

Program Objects could

not be used for testing

(e.g. RRW3_WEBRFC)

Test Package

Test Job

CATT Trace

Trace


Agenda

1. Overview

2. Prerequisites

3. Trace

4. CATT Trace

5. Test Package

6. Test Job

7. Test Result

8. Error Analysis

9. Summary


Create CATT Trace using Wizard

CATT Trace is a Trace with

recorded Reference

Data

1. Press CATT Tool

2. Click on Wizard

3. Select Trace

4. Press Next step

Test Package

Test Job

CATT Trace

Trace

3

4

1

2


Specify Standard Attributes

1. Enter Trace ID (*)

2. Enter Description

3. Choose BI Business

Explorer as Application

Layer

4. Choose OLAP Trace as

Trace type

(required to specify Test

Context; see next step)

5. Press Next step

Test Package

Test Job

CATT Trace

Trace

5

4

1

3 2

(*) Own Trace ID helps to work with wildcards (see Test Package)


Specify Test Context (optional)

1. Choose Topic of Test

and related

Functionality

2. Press Insert

3. Press Next step

Test Package

Test Job

CATT Trace

Trace

2

3

1

Optional


Select Data Object (optional)

Usage of default setting is

recommended

1. Select Program Object

2. Press Display

3. View Data Objects

4. Press Next step

Test Package

Test Job

CATT Trace

Trace

2

4

1

Default setting –

no action

required

3


Record Reference Data

1. Press Record

2. Status will change to green

and Program Objects will

have content

3. Press icon to view

Reference Data

(see next slide)

4. Press Next step

Test Package

Test Job

CATT Trace

Trace

1

2

3

3

4


View Reference Data

1. Choose Previous or Next

Check Position

2. Press Back to return to

Wizard

Test Package

Test Job

CATT Trace

Trace

2

1


Save CATT Trace

1. View summary of settings

2. Press Save

Test Package

Test Job

CATT Trace

Trace

1

2


Working with CATT Traces

1. Click on CATT Traces

2. Press button to perform

action

Check executes CATT Trace in

Check Mode (run query and

compare to Reference Data)

Useful for problem analysis

Change allows recording of

Reference Data again

Accept new state

Test Package

Test Job

CATT Trace

Trace

1

2


Agenda

1. Overview

2. Prerequisites

3. Trace

4. CATT Trace

5. Test Package

6. Test Job

7. Test Result

8. Error Analysis

9. Summary


Create Test Package

Test Package bundles several

CATT Traces

1. Press CATT Tool

2. Click on Test Packages

3. Enter name of Test Package

4. Press button Create

Test Package

Test Job

CATT Trace

Trace

3 4

2

1


Trace Selection


2. Enter Trace ID or…

3. Use Multiple Selection to

choose several Trace IDs

Wildcards are possible and

resolved into several Trace

IDs at execution time, i.e.

new Trace IDs will be added

automatically

See register Trace List

Test Package

Test Job

CATT Trace

Trace

2 3

1


Play Mode Selection

1. Click on register Play Mode

Selection

2. Choose one or more

combination of Read,

Cache and Aggregate

Mode

Play Mode determinate under

which conditions a Trace is

performed

If fields left empty, all modes

are executed (i.e., 3 * 3 * 3

= 27 combinations are

executed)

Test Package

Test Job

CATT Trace

Trace

2

1

A Read All

X Read During Navigation

H Read During

Hierarchy/Navigation

0 No Optimized Operations in SAP

HANA/No BWA

2 Individual Access per

InfoProvider (as in Release 7.0)

3 Standard

6 Exception Aggregation

Q Query Setting

0 OLAP Cache Inactive

1 Main Memory Cache Without

Swapping

2 Main Memory Cache with

Swapping (Flat File)


Swapping (Cluster)


Swapping (BLOB)

5 Persistent Per Application Server

(Flat File)


(Cluster)


(BLOB)

8 Persistent Global (Flat File)

9 Persistent Global (Cluster)

A Persistent Global (BLOB)

B BLOB/Cluster Enhanced


Trace List

1. Click on register Trace List

List of all Trace IDs

appears that fit to the

selection under Trace

Select

See register Trace Select

2. Press button Save to store

Test Package

Test Package

Test Job

CATT Trace

Trace

1

2


Agenda

1. Overview

2. Prerequisites

3. Trace

4. CATT Trace

5. Test Package

6. Test Job

7. Test Result

8. Error Analysis

9. Summary


Create Test Job

Test Job is a scheduled job

performing CATT-Traces in

one Test Package

1. Press CATT Tool

2. Click on Test Jobs

3. Enter name of Test Job

4. Press button Create

Test Package

CATT Trace

Trace

3 4

2

1

Test Job


Parameters


2. Select Test Package

3. Select Process Mode

Check Mode is recommended

Play Mode is only useful for

ABAP syntax check

4. Check on Execute Traces

in Parallel to accelerate

execution

Test Package

Test Job

CATT Trace

Trace

2

3

1

4


Time Settings

1. Click on register Time

Settings

2. Choose Execution Mode

Use Execute Immediately to

check Test Job

3. Press button Save

Test Package

Test Job

CATT Trace

Trace

2

1

3


Logs

1. Click on register Logs

2. Press on button Job Log to

view details of Job (e.g.

progress of execution)

3. Column Current shows

actual step of job (press

Refresh)

See next section how to

view Test Log Results

Test Package

Test Job

CATT Trace

Trace

2

1

3


Execute Test Job

1. Choose existing Test Job

2. Press button Execute

Job will be instantly

released for batch

processing

Test Package

Test Job

CATT Trace

Trace

2

1


Agenda

1. Overview

2. Prerequisites

3. Trace

4. CATT Trace

5. Test Package

6. Test Job

7. Test Result

8. Error Analysis

9. Summary


System Changes

Two kind of changes to system could cause errors...

1. Data Changes

Load InfoProvider Data Set filter to time period

Load Master Data Set filter to characteristic values

Change of Meta Data Record Reference Data again

To avoid errors cause by changed data…

Define Traces with fix filter to be independent of InfoProvider and Master Data

load or

Record Reference Data again before applying coding changes

2. Coding Changes

Apply Notes or Support Packages

Traces must be without error before to identify a coding error


View Test Log Results (1)

Display Test Job

1. Click on register Logs

2. Use keyboard to scroll to

column Reporting

3. Press icon

Test Package

Test Job

CATT Trace

Trace

2

3

1


View Test Log Results (2)

• Test log results are stored

into InfoCube 0RSTT_C01

• Test log results can be

viewed with Query

0RSTT_C01_Q002

1. Use navigation to drill

down to Trace ID, etc.

2. Remove filtered Log GUID

to view all results

Test Package

Test Job

CATT Trace

Trace

1

2


Analyze Test Log Result (1)

1

2

3

4

5

Test Package

Test Job

CATT Trace

Trace

In case of an error... 1. Click on CATT Traces

2. Enter Trace ID retrieved by

Test Log Result

3. Press button Check

4. Choose Check Mode as

Process Mode


… (see next slide)


Analyze Test Log Result (2)

6. View Overview

7. Click on icon

Analysis

8. Compare CATT

Data (actual) with

Reference Data

Test Packages

Test Jobs

CATT Trace

Trace 6

7

8


Agenda

1. Overview

2. Prerequisites

3. Trace

4. CATT Trace

5. Test Package

6. Test Job

7. Test Result

8. Error Analysis

9. Summary


Error Analysis

1.Use RSTT Reporting to detect traces with errors

2.Drilldown by Trace ID

3.Enter Trace ID under CATT Traces

4.Press button Check

5.Set Play Settings

6.Press button Execute

7.Press icon Analysis

8.View highlighted comparison


1. Use RSTT Reporting to detect traces with errors

2. Drilldown by Trace ID


3. Enter Trace ID under CATT Traces

4. Press button Check


5. Set Play Settings



7. Press icon Analysis


8. View highlighted comparison

Reference Data is recorded

with additional attribute

Product Group

Columns 2 and 3 are shifted


Agenda

1. Overview

2. Prerequisites

3. Trace

4. CATT Trace

5. Test Package

6. Test Job

7. Test Result

8. Error Analysis

9. Summary


Summary

RS Trace Tool allows…

Checking correctness of Query results

After applying Notes or Support Packages

Additional help

Documentation

http://help.sap.com/saphelp_nw73/helpdata/en/42/ed1942bcb35433e10000000a155106/frameset.htm

© 2013 SAP AG. All rights reserved.

© 2013 SAP AG. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

Microsoft, Windows, Excel, Outlook, PowerPoint, Silverlight, and Visual Studio are registered trademarks of Microsoft Corporation.

IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, z10, z/VM, z/OS, OS/390, zEnterprise, PowerVM, Power Architecture, Power Systems, POWER7, POWER6+, POWER6, POWER, PowerHA, pureScale, PowerPC, BladeCenter, System Storage, Storwize, XIV, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, AIX, Intelligent Miner, WebSphere, Tivoli, Informix, and Smarter Planet are trademarks or registered trademarks of IBM Corporation.

Linux is the registered trademark of Linus Torvalds in the United States and other countries.

Adobe, the Adobe logo, Acrobat, PostScript, and Reader are trademarks or registered trademarks of Adobe Systems Incorporated in the United States and other countries.

Oracle and Java are registered trademarks of Oracle and its affiliates.

UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.

Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems Inc.

HTML, XML, XHTML, and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.

Apple, App Store, iBooks, iPad, iPhone, iPhoto, iPod, iTunes, Multi-Touch, Objective-C, Retina, Safari, Siri, and Xcode are trademarks or registered trademarks of Apple Inc.

IOS is a registered trademark of Cisco Systems Inc.

RIM, BlackBerry, BBM, BlackBerry Curve, BlackBerry Bold, BlackBerry Pearl, BlackBerry Torch, BlackBerry Storm, BlackBerry Storm2, BlackBerry PlayBook, and BlackBerry App World are trademarks or registered trademarks of Research in Motion Limited.

Google App Engine, Google Apps, Google Checkout, Google Data API, Google Maps, Google Mobile Ads, Google Mobile Updater, Google Mobile, Google Store, Google Sync, Google Updater, Google Voice, Google Mail, Gmail, YouTube, Dalvik and Android are trademarks or registered trademarks of Google Inc.

INTERMEC is a registered trademark of Intermec Technologies Corporation.

Wi-Fi is a registered trademark of Wi-Fi Alliance.

Bluetooth is a registered trademark of Bluetooth SIG Inc.

Motorola is a registered trademark of Motorola Trademark Holdings LLC.

Computop is a registered trademark of Computop Wirtschaftsinformatik GmbH.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.

Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company.

Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase Inc. Sybase is an SAP company.

Crossgate, m@gic EDDY, B2B 360°, and B2B 360° Services are registered trademarks of Crossgate AG in Germany and other countries. Crossgate is an SAP company.

All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG.

Documents

Automatic Test of Queries