#96600 – Using Oracle Application Testing Suite (OATS) · PDF fileOLT provides an easy and accurate way to test scalability of your applications. ... SQL Server, EnterpriseOne

#96600 – Using Oracle Application Testing Suite (OATS): A JD Edwards EnterpriseOne

Load and Functional Testing Case Study

David Scheeff Bob Lynch Krisztina Scheeff Jim Mackowiak

KDS Technologies

We are an ERP consulting and implementation company focused on Oracle’s JD Edwards application suite.

We provide technical solutions to companies running Oracle's JD Edwards EnterpriseOne© software.

We also offer a full range of software testing solutions.

Edwards Lifesciences

Edwards Lifesciences is the global leader in the science of heart valves and hemodynamic monitoring.

Supplying products and technologies to clinicians in nearly 100 countries around the world.

Global company, 7000 employees, 500 concurrent JDE users in 6 regions.

ABOUT US

Introduce OATS (Oracle Application Testing Suite)   Features & Functions

How to get started (beginners welcome!)   Script creation – demonstration

Case study (JDE load test / Japan network test)   Requirements and scenarios   Load test results

Tips and lessons learned; Advanced Topics; Q&A

GOALS OF OUR PRESENTATION

session ID 96600 �

Functional test - does the application meets end user business requirements?   OpenScript is Oracle’s scripting platform to create automated test scripts

Load / stress test - can the system handle the number of users expected?   Stress test. When / how big of a load causes the system to fail   In OATS load tests are run by OLT (Oracle Load Testing)

Test management - this is where you keep all your “stuff”. Integrated modules for total test management.

  OTM (Oracle Test Manager)   Test cases, scripts, requirements, bugs, and reports

TESTING CONCEPTS AND OATS


OATS ARCHITECTURE

OpenScript WINDOWS SERVICES

OATS Agent Service OATS Application Service

OATS Helper Service

Repository C:\OracleATS\OFT DATABASE

Oracle XE

WEB LOGIC

OATS Administrator Oracle Load Testing (OLT) Oracle Test Manager (OTM)

OATS Windows Server

A script is a sequence of actions and tests performed on a website or application.   Eclipse based scripting tool for OATS

  For JD Edwards create a “Web” script

  Automated functional and load scripts

  Run scripts from OLT and Test Manager

  OATS doesn’t know about JD Edwards...yet!

OPENSCRIPT – Create Scripts

Tree View shows the script navigations and data as nodes in a tree view.

  Step groups

  Recorded actions (clicks, typing, delays)

  Modifications

  Text match

  Object test

OPENSCRIPT

Java Code View shows the script navigations and data as Java programing code.

  Java Code view corresponds to the

Tree View.

  Modify OATS generated code

  Add custom functions

OPENSCRIPT

Initialize – script actions executed once at beginning of the script.

•  ie. Login & Setup

Run – script actions to perform one or more times during playback.

Finish – script actions executed once at the end of the script.

•  ie. Close & Signout

OPENSCRIPT


VIDEO Recording a script in Open Script


OLT provides an easy and accurate way to test scalability of your applications. It can emulate thousands of users accessing your site simultaneously.

  Reusable scripts – uses the scripts you created in OpenScript   Distributed agents   Scenario manager and autopilot   Real time graphs and reports   Post – run analysis   Server side monitoring with ServerStats

LOAD TEST – ORACLE LOAD TESTING (OLT)


LOAD TESTING – OLT SETUP

Setup -  VUs -  System -  Iteration -  Think Time -  Other

Settings

LOAD TESTING – OLT SETUP

Autopilot -  Start & Stop -  Ramp-Up -  Server Stats

LOAD TESTING – OLT REPORTS Reports

-  Run Graphs -  VU Grid -  Saved Scenario -  Graphs -  Reports (text)

Load Test Video

RECORDED A SCRIPT. NOW WHAT?

Test Manager

  Integrated modules for total test process management – requirements, test cases, bugs related to it

  Run functional test cases from Test Manager, automatically save results

  Web based interface

  Manual and automated tests

  User defined roles and fields

  Reports on results etc by user, test cases failed, passed, time stamp, area etc.



CASE STUDY: OATS AT EDWARDS LIFESCIENCES

Oracle / JDE Load Test - load test prior to July 25th, 2011 go-live   New infrastructure: hardware refresh & software upgrades   Moving from Citrix (JDE 8.10) to Web (JDE 9.0 on 8.98.4.3 tools release)   VmWare: SQL Server, EnterpriseOne & WebLogic on Windows 2008 R2   5 different scripts

Japan Network Test - Japan user base migrating from World to EnterpriseOne

  Network impact: user traffic moved from local systems to corporate system   Project goal: use load test to verify network infrastructure   Plan: re-run load test after infrastructure upgrade to compare with baseline   13 different scripts session ID 96600 �

JDE Environment

at Edwards Lifesciences


ORACLE JDE LOAD TEST   450 Users on one machine and two agents   30 min ramp up, run for 1.25 hours   Verify that new hardware can support the business   New software upgrade   5 scripts

ORACLE JDE LOAD TEST - RESULTS

Users vs Time



KBs per sec, Users vs Time




ORACLE JDE LOAD TEST - CONCLUSION

  Infrastructure problem identified before go-live

  Intermittent response time issue in JDE

  Found problem only occurred when Data Warehouse was in use

  Problem: shared SAN (per vendors recommendation)

  Resolution: added SAN controller and disks to separate data

  Final result: System can handle 450 users

  “OATS paid for itself by helping to identify this issue”

JAPAN NETWORK LOAD TEST

GOAL: load system to test bandwidth

  The test scenario designed for this run was to ramp up a concurrent user load to max out at 150 users from 4 different Locations in Japan

  OATS agents in 4 machines located in Tokyo, Sapporo, Osaka, & Fukoka

  25 users per location with an additional 50 users in the Tokyo data center

  Different activities while emulating users on the system

  Iteration delay to allow for user think time

  13 scripts recorded and played back in Kanji

JAPAN NETWORK LOAD TEST - SETUP

JAPAN NETWORK LOAD TEST - SETUP

  Recorded vs Random

  Ramp up 30 min

  Run for 45 minutes

  One script failed

  Data preparation

JAPAN NETWORK LOAD TEST - RESULTS

Users vs Time (per script)



Users vs Time (per script and

total users)


JAPAN NETWORK LOAD TEST - RESULTS Number of Users / HitsPerSec/ KB RecSent

Traffic from Tokyo


Performance vs

Time






JAPAN NETWORK LOAD TEST - CONCLUSION

  OATS generated load of 150 “fast paced” users

  Ramp up 30 minutes, followed by 45 minutes run

  Only 100 “normal” users are expected concurrently from Japan

  Network handled the load without issue

  Upgraded network should be even better


  New Users: keep it simple when getting started; re-record!

  Subject matter experts *must* participate when creating custom scripts

  Planning takes time - scripts and data

  Load and Functional scripts must be recorded separately

  Learning curve – for more advanced scripts

  Restart OATS services

TIPS & LESSONS LEARNED


Parameterization - alter scripts to use data from a DataBank Correlation - replace dynamic data with variables (ie: session ID) ServerStats - record server performance information in OLT

  OLT collects: scripts, # users, time to complete each script   ServerStats adds server data: CPU, Memory, JVM Heap usage, etc

Command Line Interface - run OpenScript from a command line

  Create Windows batch files to run scripts; batch files can be scheduled

ADVANCED TOPICS


Error Handling   Error types: Fail, Warn, Ignore, ReportAndContinue   Default is to “Fail” for any error   Different types of errors can be independently configured

Concurrency –two users might select the same order at the same time

  Business Object Reservation ensures mutually exclusive access   How do you handle that type of situation?

- Script Synchronization - Detect “Reserved By”

ADVANCED TOPICS


NEXT STEPS

  Download OATS (30 day trial)

  Getting Started Guide

  OTN: QA/Testing Forum

  Coming soon: Oracle Functional sample scripts


Email us [email protected]

Download presentation at www.kdstechnologies.com

Find us at the 80’s party tomorrow!

QUESTIONS?


Documents

#96600 – Using Oracle Application Testing Suite (OATS) · PDF fileOLT provides an easy and accurate way to test scalability of your applications. ... SQL Server, EnterpriseOne