Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Computer Science and Software Engineering
University of Wisconsin - Platteville
Note 4. Organization
Structure and Test
DocumentsYan Shi
Lecture Notes for SE 3730 / CS 5730
Outline
QA organizational structure
Test Documents
— Test plan
— Test specifications
— Context driven test planning
Good requirements to test
http://bobcesca.thedailybanter.com/files/2013/08/fox_henhouse_12.jpg
“The guardians of the hen house can’t be in the employment of the fox.”
Independent QA Organization
http://ibagroupit.com/about/structure/
Independent QA Organization
QA organizations are generally separate from organizations such as Development, Marketing, and Sales and report to different VPs.
This maintains independence and allows specialization of skills.
This independence supports:
— Decision making that primarily reflect a company’s software quality goals rather than development, financial, marketing, … goals.
QA Roles in the Lifecycle (1)
Process cops: — Is the process followed? PDCA?
Requirements and/or use cases:— current, validated and formally documented?
Test Cases: — generated and reviewed after requirements are
confirmed!
Configuration Management:— QA owns the code once it’s baselined!
Build Control:— clean and controlled process and environment.— documented and repeatable?
QA Roles in the Lifecycle (2)
During the testing process:— Unit testing (white-box): done by developers,
audited by QA
— Integration testing
— System testing
— Acceptance testing: written by the customer
— TDD: assist customer to produce automated acceptance tests.
— Simulators/Test Harness: build and maintain.
— STE(special test equipment).
Test Document Types
Test plan— Test goals, overall plans, schedules, special equipment and resources— But not the test cases! — Completed during INCEPTION or early ELABORATION phase
Test specification— Detailed test cases— Created during ELABORATION phase after requirement is detailed.— Completed during early CONSTRUCTION phase.— Evolve during CONSTRUCTION and early TRANSITION phases.
Test plan and specification are often merged into one test document.
IEEE Std 829-2008: Standard for Software Test Documentation
Test Document Styles
Inclusive test documents: — chapters for everything from unit tests to acceptance
tests.
Individual test documents:— Each software phase has its own test plan and
specification— Perhaps an overview document to link all phases
together
Unit test documents:— Sometimes it is a part of a Detailed Design
document.
Test Document Template
Introduction:— Purpose— Test approach— Definition of test environment— Definition of the build environment and version control
Test cases— Test setup and preconditions— Test inputs and steps— Expected result— Pass/Fail— Estimated and actual test time— Teardown procedure: cleanup after testing so that the running of
this test does not contaminate subsequent tests
Traceability to requirements Traceability to software
Test Document for Phone Calling
Context Driven Test Planning
• Monitor major test planning challenges
• Clarify your mission
• Analyze the product
• Analyze product risk
• Design the test strategy
• Plan logistics
• Share the plan
Seven task
themes to assist with test planning:
Kaner, Bach and Pettichord, “Lessons Learned in Software Testing”
How to Evolve a Context Driven Development Plan
Good Requirements to Test
Good requirements are essential for producing good test cases!
Forms of Requirement Spec.
Raw individual statements
Requirement Specifications:— Hierarchical structure starting from the top:
system subsystems major components
Use Cases:— Specify a “sequence of transitions performed by a
system, which yields an observable result or value for a particular actor”
— More natural; easier to understand by all parties
— Easier to validate, to write test cases for
Examples of Requirements
A1: The system shall have a wonderful users’ interface. A2: The user interface shall require four or fewer mouse clicks to initiate any transaction and follow Company XYZ’s user interface standards.A3: It would be nice to have white lettering on a blue background.
B1: The system shall deliver rapid responses to user queries.B2: The system shall deliver 98% of all users queries within 3 seconds while sustaining transaction rates of 10,000 transactions per second on the target hardware. B3: The system shall maintain a maximum transaction rate of 9,000 transactions per second.
C1: The control mechanism shall use E.W.M.A. (with a lambda of 0.3), and L.C.L and U.C.L. of 99.98 and 101.23, respectively.
SMART Requirement
SMART Requirements— Specific
Valid – meets user expectations and needs
Complete and non-ambiguous – where interpretation is necessary, there will be different interpretations.
• “I saw the man in the park with a telescope.”
Technical level –User technical terms, algorithms, business logic need to be clearly defined (generally in a glossary). C1
— Measurable Testable – the system can test and have very little uncertainty whether the
requirement was satisfied or not satisfied. (A1 or A2) and (B1 or B2)
— Attainable (Achievable, Actionable, Appropriate) Non-conflicting – one requirement should not conflict with another requirement.
(B2 or B3) Actually needed – not just wishes
— Realistic— Time-bound (Timely, Traceable)
Well organized – cross reference related requirement.
Summary
Independent QA organization QA roles in the software life cycle Test Documents:
— Test plan— Test specification— Other styles— IEEE 829-1998 standard— Context driven test planning
Good requirements to test— Valid, testable, non-conflicting, non-ambiguous, low
technical level, necessary, cross-reference