STRATEGIC APPROACH FOR A CREDITABLE AND …conference.qaiglobalservices.com › stc2013 › PDFs › Sindhu_Prabhu.pdfSanity testing required to be performed on daily basis to prepare

Strategic Approach for a Creditable and Approbative Testing (SACAT) Of Oracle Apps in Association with a Third Party Application

STC 2013

A Paper by:

Sindhu Prabhu E-mail: [email protected]

[email protected]

Century Link, Bangalore, India.

www.centurylink.com

mailto:[email protected]

mailto:[email protected]

http://www.centurylink.com/

Page 2 of 13

SACAT of Oracle Apps with 3rd Party App

ABSTRACT This paper specifically discusses the strategies to handle the testing of oracle apps interacting with a third

party application in an undeviating manner by rightly tackling the unhappy paths encountered. Implementation of

the appropriate tools and methodologies during the testing phase, employing right KPI measurement parameters,

identification of performance targets and estimation, development of performance plan, helps accomplish a well

organized and a successful release and thereby customer satisfaction.

INTRODUCTION

The Oracle E-Business Suite (also known as Oracle Applications/Apps) manages important business

processes across an organization. This is a suite of several complex business applications. Since the Oracle

Applications are designed to authorize the business user to take accountability of the business process and resulting

data, employees using this Oracle E-Business Applications perform a dual role. They:

understand the business process and

understand the underlying supporting application tasks.

Many times these application tasks are critical to the business process, even though usually cumbersome and manual

in nature. An ordering application which is a GUI interacts with Oracle Apps by initiating an order which then

reflects in the Apps.The having a variant environment requires daily build deployment.

The discussion here is on the best way to efficiently handle various challenges encountered when the

Oracle Application, interacting with such an ordering application in a vacillating environment is subjected to several

vital levels and different types of testing.

The observation in the Case Study considered in this paper depicts the outstanding outcome obtained by the

implementation of the static testing tool, concept of CRP, and accredited amount of interaction with the business and

well planned resource utilization.

The bottom line: In case of Oracle Apps and 3rd

party application testing, practicing the right communication, a

well defined ethical way of resource planning and an effective use of suitable tools helps meet the Cost, performance

and functionality trade-offs.

CHALLENGES Challenges of PACKAGED APP(Oracle Apps 11i)

Application complexity Enterprise scope: Used by & affects many users, distributed operations. Extra attention required during

testing. Many modules: Customized set of modules involving CRM and SCM … Unique configuration: Implemented to support your company’s processes, initialization data, reporting

needs Integration between modules, many interfaces to external systems.

Application Quality Codes have numerous bugs. Patches/upgrades can number into the several per quarter.

Page 3 of 13


Test Challenges Test Environment:

Daily build deployment required. GUI environment highly fluctuating. Sanity testing required to be performed on daily basis to

prepare environment it for testing.

Impact Analysis and Requirement Analysis

There may be potential static defects which would cost dearly if not identified during initial

phases. High chances of unclear requirements having some latent requirement gaps.

Test Execution High reliance on business users for manual testing. Fulfilling the requirement of appropriate orders for diverse scenarios during intersystem and

regression testing.

SOLUTION DESCRIPTION The description of Solution is given considering a Case Study as an Instance. Oracle Application (QOracle) working in association with with a Web application (named QCentral)

Solution discovered and best practices followed: Handling PACKAGED APP(Oracle Apps 11i) An assured best choice

Define key performance indicators in order to track your organization’s progress and achievement of

measurable business benefits once this new software is implemented.

During ERP implementation, software should be designed, configured and tested to ensure alignment

with corporate strategy.

Where there is ERP there must be CRP

It is crucial to gain an understanding how it is working with regards to the organization’s specific

business processes. The testing should never be confined just to validate the working of the ERP software

at technical level. This demands several rounds of functional and integration testing to be executed.

The wisdom lies in realizing that testing should begin testing during the ERP selection process. Conference

Room Pilots (CRP) are instituted and these allow the Business tasks to be performed by the users with the

new software. CRPs give organizations true insight into how end-users will work within the system, and

how their business will function once the system is in place. The results of CRPs can provide a framework

for the organization’s operations moving forward.

Page 4 of 13


Handling TEST challenges: Test Basis and Static Testing

It all starts with accumulating the right test basis. Right Technical Operational Environment

Specification and Order Data Mapping between the Oracle Applications and the third party application(Web

application).

Best Test Oracle contributes to test Oracle Applications:

Oracle-based testing, involves comparison of the behavior of the program under test to the

behavior of a source we consider accurate (an oracle).One of the common early tasks when testing a program is a

survey of the program's capabilities. You walk through the entire product, trying out each feature to see what the

product can do, what it does well, what seems awkward, and what seems obviously unstable. The tester constantly

evaluates the program: Is this behavior reasonable? Correct? In line with user expectations? This is done in CRP.

Thus, CPR is good enough as the generator of the Test Oracle. The requirements documents explaining the data and

control flow fit in as the comparator and unit tests along with the expected results become the evaluators.

Static testing is an important phase of the testing.

A static testing Tool always helps track the static defects with all the details.

Benefits: This early defect detection in the primitive phase makes the fix highly economic.

Total Efforts saved is calculated based on the severity and the status of defect.

Detailed explanation depicting which part of requirement or HLD or LLD is unclear or ambiguous

or faulty

Prevents the fear of defect rejection.

Increases the probability of finding business gaps and enhancements.

An end product is a clear, unambiguous HLD (High Level Document), LLD (Low Level

Document with the better code.

Static Test Tool built In-House (Fig 1a and Fig 1b) Images (Fig 1a,Fig1b) show the static testing tool developed in–house. This was used to track the static

defects found in the below mentioned test basis with all the details:

i) Requirements (mentioned in Caliber)

ii)High Level Documents

ii)Low Level Documents

Page 5 of 13


Fig 1a

Fig 1b

Page 6 of 13


A collaborative and cooperative approach of testing including the apt techniques and

methodologies and tools. A positive and collaborative way of engaging the end-users helps create realistic expectations for

everyone involved. Ensure enough time is invested and people are involved in early testing of the project,

to harness all of the value CRPs.

o Reviews: The prominent chapter of Testing. Reviews form the best part of testing as they help resolve half the set of issues and get maximum

clarifications.

The Peer Review enables the peers to know the requirement and share their opinions of raise

any questions to be answered.

The Review with IT and business assisted the testers to be in sync with what the business

exactly wanted.

It is a best practice to use a Test Case Review Check List. A checklist is intended to be a job aid to

complement the process of test case reviews.

A sample review checklist:

Fig 1c

Used in

Review By

No Checklist Item Peer to Peer Analyst Development PM/Business

1 <Points on Purpose and on Scope >

2 <Standard and Conventions>

3 <Other requirement related points>

Expected

Results

Grammar

1 Look for grammatical errors

Coverage

2 Verify the Coverage of positive and

negative scenarios

Reviewer

Comments

Approved by Date

Page 7 of 13


o Sanity: The early bird catching the bug in the build Sanity test is a must in the project requiring daily build deployment. This prepares the

environment all set.

One of the testers in the test team check in early and perform sanity by verifying the basic

functionalities in the application, immediately after the build is deployed. The environmental issues

found can be resolved as soon as possible through active communication and make the environment

ready to start system, intersystem or regression testing.

A SWAT(Swift Action Team) call on time helps make the Environment works fine

If sanity fails due to issue the concerned team is immediately communicated. If the e-mail

exchange consumes more than 15-20 minutes it is a best practice to initiate a swat call and

involve all the support team to get the issue resolved.

Benefits: Early sanity helps find the issues in the primitive phase.

SWAT concept helps quicker resolution of issues and saves time.

The issue with the resolution tracked acts a reference to get the issue resolved, if any similar

issue repeats the next time.

The downtime tracked in the environment accounts for reduced productivity for the day.

o System Test: Watch out for those clustered defects!

System Test is performed bearing defect clustering in mind. Defect Clustering in Software Testing is based on the Pareto principle, also known as the 80-20

rule, where it is stated that approximately 80% of the problems are caused by 20% of the

modules. Make an attempt to identify the section of the application where the defects remain

centralized. In the case discussed the Service Contract module and the MACD were the highly

vulnerable.

o Inter System Test: A Phase of the P4 Evaluation Phase

Proactive, Productive and Problem Solving Skills and Persistent communication

with multiple teams.

In this phase of testing following are to be ensured:

Flow through test cases is involved.

Accurate documentation for involved system. Documents include:

Proper parameters and data are correctly passed between the applications IST Progress Report Table

SP (Solution Repository) containing the issues faced in various environments and the

resolutions for the same.

Prioritize the test cases.

Verify the new changes of the parameters in the system, which are being tested, update

and maintain the document.

Page 8 of 13


o Automation of Regression Test Cases: A separate database for automated regression testing is set up.

A separate test team is identified to focus on automated testing.

Automated regression testing is taken up once the function is stable, i.e. most faults have

been identified and fixed.

Over time, a “lifetime” test script is developed for all transactions, i.e. an automated test

script to be used as an installation test at customer’s site.

o Configuration Management: Discipline defines Success Right Configuration Management is a principal necessity for best testing.

Functional audit: Verifying that its configuration items' actual functionality and

performance is consistent with the relevant requirement specification. Held prior to

software delivery to verify that all requirements specified in Software Requirements

Specification have been met.

Physical audit: Provides an independent evaluation of a software product's configuration

items to confirm that all components in the as-built version map to their specifications.

Specifically, held to verify that the software and its documentation are internally

consistent

In addition to the existing documents generally maintained, two documents adapted assist

the team to be abreast with the happenings in the project and the action to be taken for the

improvement. They are:

Scope and Schedule of every release: This is a ready reckoner to track the release

of each ITR.This is easy and precise way of maintaining the time table than just

relying on the calendar and the list of dates provided.

Past releases NC list: This forms a reference document to as a warning to avoid the

mistakes done in the previous audit and a guide for improvement in the upcoming

release.

Benefits: Ability to deliver revisions, updates and cross-platform versions, faster

Improved customer satisfaction

Less time wasted fixing old code

Confidence that each release addresses all the requested change.

Page 9 of 13


o ROI/ Outcomes after implementation of the mentioned strategies in brief: The results obtained after implementing the discussed practices in the Quarterly release are shared with the

details in the figures: Fig 2a, Fig 2b, Fig 2c, and Fig 2d.

Project Metrics Analysis

Project Metrics - Test Org Goal Project Goal Q1_Rel_Result

Resource Utilization 95 95 95.8

Quality - Defect Leakage <0.5 <0.5 0

Productivity - TC Created / PM 225 225 225.79

Productivity - TC Executed / PM 95 27 17

Efficiency (Effort Variance) +/- 5% +/- 5% -2.78%

Fig2a Defects Root Cause Analysis-Test

Rel Defects - root cause

Root Cause Total

IT Req - Gap or Change 5

Business Req - Gap or Not met 20

Development - Architectural Design 2

Development - Build defect 10

Development - Code defect 63

Environment - Configuration 34

Environment - Database 5

Environment - Network 15 Grand Total 154 Total Valid Defects 84% Total Environmanet Defects 26% Total 100%

Fig2b

Page 10 of 13


System Test – Defect Rejection Ratio analysis

Fig2c

Risk Analysis:

Risk & Issues ( from

Current Milestone) Mitigation/ Contingency actions

taken (Q1) Effectiveness of

actionsin (Q2) Risk Occurred

Yes/No

Late Code Delivery Following up with the

Development Yes

Environment Sability Sanity performed by the testers

checking in early in rotational

shifts.

Early initiation of

SWAT calls to resolve

critical issues in the

environment

Yes

Automation in different

environments Scripts updated to handle

multiple data set options No

Fig2d

Page 11 of 13


CONCLUSION Smart leaders understand that testing is an investment in quality and thus set aside reasonable amount from the overall budget for assessing the system and resolving the bugs which

involves CRPs, sanity and ISTs . Managing a wise investment on test :

Produces a positive return, Fits within the overall project schedule, Has a quantifiable finding, which results as a definite contributor to the project.

A little bit goes a long way: Picking the right tests, managing the appropriate quality risks, using the proper tools and the right

techniques used and the apt modus operandi of driving testing throughout the organization will result in delivery of

a high quality product that in turn helps effectuate customer satisfaction.

Page 12 of 13


References:

1. B. Beizer, Black Box Testing: Techniques for Functional Testing of Software Systems 2. Øvstedal, E. Ø. and Stålhane, Tor “A goal oriented approach to software testing”, Reliability Engineering

and System Safety. 3. http://www.oracle.com/us/education 4. Dorothy Graham and Mark Fewster, "Automating Software Testing", 1999.

Author’s biography: The Author,Sindhu Prabhu, has 3+ years of experience as a Software Quality Assurance Engineer, Century

Link India, a telecom domain company. Worked for projects delivering complete testing solutions with having

executed business critical testing assignments, which include detailed test planning, creation and execution of QA,

testing processes and test strategies. Also possess experience as a Configuration Controller and functional point

practitioner for projects.

Involved in COE s and testing Oracle Application and being part of an independent verification and

validation group driven by the theme "We are Custodians of End Users' Interests".

Provided service on CenturyLink’s OSS/ BSS layer. On the applications ranging from Ordering Management, Order

Validation, Provisioning, Service Assurance.

Education: A B.E graduate pursuing a M.Tech on Software Design and Engineering from MIT, Manipal

University.

Page 13 of 13


Appendix:

Acronyms/Terms Expansion/Description

Evaluator

part of test oracle determines whether the comparison

results are sufficiently

close to be a pass

Comparator part of test oracle compares predicted and obtained results

CRP Conference Room Piloting

Generator

Part of test oracle that provides predicted or expected

results for each test.

ITR IT Requirement

KPI Key Performance Indicator

SACAT

Strategic Approach for a Creditable and Approbative

Testing

SP Solution Repository

Test Oracle

Documents related to the requirements and also the

detailed materials containing the expected results

****************************************