The Role of the Quality Analyst The Role of the Quality Analyst in Software Developmentin Software Development

Presented by Presented by Jim Coleman – HJIP QA AnalystJim Coleman – HJIP QA Analyst

A little bit about meA little bit about me

Over 9 years of experienceOver 9 years of experience

Certified in Software TestingCertified in Software Testing

Degrees in IS and ITDegrees in IS and IT

Stereotypical Software TesterStereotypical Software Tester

Isn’t quality everyone’s job?Isn’t quality everyone’s job?

Quality must be an Quality must be an independentindependent processprocess

Quality means more than finding errorsQuality means more than finding errors

Quality requires measurementQuality requires measurement

Quality must be consistent & repeatableQuality must be consistent & repeatable

Quality may be viewed differently by Quality may be viewed differently by business and developmentbusiness and development

Quality as a matter of Quality as a matter of perspectiveperspective

How do you define Quality?How do you define Quality?

TranscendentTranscendent - I know it when I see it - I know it when I see it Product-BasedProduct-Based - Possesses desired features - Possesses desired features User-BasedUser-Based - Fitness for use - Fitness for use Development and Manufacturing BasedDevelopment and Manufacturing Based - Conforms to - Conforms to

requirementsrequirements Value-BasedValue-Based - At an acceptable cost - At an acceptable cost

Quality Control Quality Control Vs Vs

Quality AssuranceQuality Assurance

Cost of QualityCost of Quality

Prevention costsPrevention costs

Appraisal CostsAppraisal Costs

Failure CostsFailure Costs

External failure External failure costscosts

VerificationVerificationVVs s

Validation Validation

Verification answers the question, Verification answers the question, “Did we build the right system?”“Did we build the right system?”

Validations answers the question, Validations answers the question, “Did we build the system right?”“Did we build the system right?”

Focuses on testing the function of the Focuses on testing the function of the program or application against its program or application against its specification. Determining what and how specification. Determining what and how much to test is criticalmuch to test is critical

Equivalence PartitioningEquivalence PartitioningA technique for testing equivalence classes rather than undertaking A technique for testing equivalence classes rather than undertaking exhaustive testing of each value of the larger class. For example, a exhaustive testing of each value of the larger class. For example, a program which edits credit limits within a given range ($10,000 - program which edits credit limits within a given range ($10,000 - $15,000) would have three equivalence classes:$15,000) would have three equivalence classes:

< $10,000 (invalid)< $10,000 (invalid) Between $10,000 and $15,000 (valid)Between $10,000 and $15,000 (valid)

> $15,000 (invalid> $15,000 (invalid))

Boundary AnalysisBoundary AnalysisA technique that consists of developing test cases and data that focus on A technique that consists of developing test cases and data that focus on the input and output boundaries of a given function. In same credit limit the input and output boundaries of a given function. In same credit limit example, boundary analysis would test:example, boundary analysis would test:

Low boundary +/- one ($9,999 and $10,001)Low boundary +/- one ($9,999 and $10,001) On the boundary ($10,000 and $15,000)On the boundary ($10,000 and $15,000) Upper boundary +/- one ($14,999 and $15,001)Upper boundary +/- one ($14,999 and $15,001)

Error GuessingError GuessingTest cases can be developed based upon the intuition and experience of Test cases can be developed based upon the intuition and experience of the tester. For example, in an example where one of the inputs is the the tester. For example, in an example where one of the inputs is the date, a tester may try February 29, 2000.date, a tester may try February 29, 2000.

Black Box Black Box TestingTesting

White Box White Box TestingTesting

Assumes that the path of logic in a unit or Assumes that the path of logic in a unit or program is known. White-box testing program is known. White-box testing consists of testing paths, branch by branch, consists of testing paths, branch by branch, to produce predictable results. to produce predictable results.

Statement CoverageStatement CoverageExecute all statements at least once.Execute all statements at least once.

Decision CoverageDecision CoverageExecute each decision direction at least once.Execute each decision direction at least once.

Condition CoverageCondition CoverageExecute each decision with all possible outcomes at Execute each decision with all possible outcomes at least once.least once.

Decision/Condition CoverageDecision/Condition CoverageExecute all possible combinations of condition Execute all possible combinations of condition outcomes in each decision.outcomes in each decision.

Multiple Condition CoverageMultiple Condition CoverageInvoke each point of entry at least once.Invoke each point of entry at least once.

Structural TestingStructural Testing

Technique Description Example

Stress Determine system performs withexpected volumes

• Sufficient disk space allocated• Communication lines adequate

Execution System achieves desired level ofproficiency

• Transaction turnaround time adequate

• Software/hardware use optimized

Recovery System can be returned to an operational status after a failure

• Induce failure• Evaluate adequacy of backup data

Operations System can be executed in a normal operational status

• Determine systems can run using document

• JCL adequate

Compliance (to process)

System is developed in accordance with standards and procedures

• Standards followed• Documentation complete

Security System is protected in accordance with importance to organization

• Access denied• Procedures in place

Functional TestingFunctional Testing

Technique Description Example

Requirements System performs as specified • Prove system requirements• Compliance to policies, regulations

Regression Verifies that anything unchanged still performs correctly

• Unchanged system segments function• Unchanged manual procedures correct

Error Handling

Errors can be prevented or detected, and then corrected

• Error introduced into test• Errors re-entered

Manual Support

The people-computer interaction works

• Manual procedures developed• People trained

Integration Data is correctly passed from system to system

• Intersystem parameters changed• Intersystem documentation updated

Control Controls reduce system risk to an acceptable level

• File reconciliation procedures work• Manual controls in place

Parallel Old system and new system are run and the results compared to detect unplanned differences

• Old and new systems can reconcile• Operational status of old system maintained

Quality MetricsQuality Metrics

Technical MetricsTechnical MetricsMade throughout the Made throughout the development processdevelopment process

““Is the product ready for Is the product ready for production?”production?”

After-the-fact MetricsAfter-the-fact MetricsMade after product is in Made after product is in developmentdevelopment

““What should we What should we differently next time?”differently next time?”

References and ResourcesReferences and Resources

Quality Assurance Institute (QAI)Quality Assurance Institute (QAI)qaiworldwide.orgqaiworldwide.org

American Society for Quality (ASQ)American Society for Quality (ASQ)asq.orgasq.org

Sticky MindsSticky MindsStickyminds.comStickyminds.com

Benchmark QA – Sheila ConwayBenchmark QA – Sheila Conwaybenchmarkqa.combenchmarkqa.com

Twin Cities Quality Assurance Assc.Twin Cities Quality Assurance Assc.tcqaa.orgtcqaa.org