Orthogonal Arrays Testing Strategy (OATS) Mohamed Fathy QA Engineer Arx ICT September 4, 2009

Orthogonal Orthogonal ArraysArrays TestingTesting StrategyStrategy(OATS)(OATS)

Mohamed FathyMohamed FathyQA Engineer

Arx ICTSeptember 4, 2009

AgendaAgenda

• OATS Overview.• How to use Orthogonal Arrays?• OATS Examples.• Case Study.• Where OATS could be applied?• OATS Benefits.• References & Resources.• Questions?

OATS Overview

• Taguchi's orthogonal arrays are highly fractional orthogonal designs proposed by Dr. Genichi Taguchi.

• OATS is one such combinatorial testing method.

• OATS is a systematic, statistical way of testing pair-wise interactions.

OATS Overview (Cont’d)

• It addresses generation of test cases for problems involving multiple parameters and combinations.

• It can be used to reduce the number of combinations and provide maximum coverage with a minimum number of test cases.

How to use Orthogonal Arrays?

Lruns(Levels^Factors) or Ln(s^k): Orthogonal array of N runs where k factors take on any value from a set of s symbols/level.

• Runs: The number of rows or number of test cases in the array that will be

generated by Orthogonal Array technique. Each row represents a test case.

• Factors: The number of columns or the number of parameters/variables in an array that need to be tested in the System.

• Levels: The maximum number of values in an Orthogonal Array that can be taken on by any single factor.

OATS Examples

Example 1

OATS Examples (Cont’d)

The borders and shading dialog box :• 5 settings• 5 styles (showing)• 9 colors• 9 widths

5 x 5 x 9 x 9 = 2025 combinations!L81(94) array will suffice

OATS Examples (Cont’d)

Example 2

OATS Examples (Cont’d)

• 3 toolbars (pictures, text, or both).• 3 choices for launch on startup (browser, mail or news).• 3 choices for startup page (blank home page, home page

names a valid file, or home page name has a syntax error).• 2 choices for underlining of links (don’t underline, or

underline).• 2 choices for expiration of links (never expire, expire after 30

days).

3 x 3 x 3 x 2 x 2 = 108 combinations!

OATS Examples (Cont’d)

OATS Examples (Cont’d)

OATS Examples (Cont’d)

OATS Examples (Cont’d)

• Example 3:

Factors Levels

terminal_capability 1 - 2 - 3

pos_entry_mode SWIPED - KEYED

card_present Yes - No

card_type Visa – MC

OATS Examples (Cont’d)

• Orthogonal array to be used: L9000001120221101111201202202221012210

Case Study

Optimization

• Remove invalid runs.• Optimization need not necessarily mean the

reduction of tests. You can add some combinations that are critical for testing, but missed by OATS.

Where OATS could be applied?

OATS could be applied in: User interface testing. System testing. Regression testing. Configuration testing. Performance testing. This method is extremely valuable for testing complex

applications. Useful for testing products with a large number of

configuration possibilities.

Benefits of OATS

• Provides uniformly distributed coverage of the test domain.

• Concise test set with fewer test cases is created.

• All pair-wise comibnations of test set created.• Arrives at complex combinations of all the

variables.

Benefits of OATS (Cont’d)

• Simpler to generate and less error prone than test sets created manually.

• Reduces testing cycle time.

References & Resources

• ReferencesReferences::1- R Krishnan, S Murali Krishna, P Siva Nandhan - Motorola:

Combinatorial Testing: Learnings from our Experience .2- Madhav S. Phadke , Quality Engineering Using Robust Design.

• Resources:Resources:1- http://www.research.att.com/~njas/oadir/index.html2- http://www.mathdox.com/nguyen/orthogonal_array.jsp

Questions?

Thank YouFeedback:

mfathy@arxict.com