ISO/IEC 29119 Part 1, Part 2, Part 3, Part 4 Testauksen prosessit, dokumentointi ja menetelmät

ISO/IEC 29119 Part 1, Part 2, Part 3, Part 4

Testauksen prosessit, dokumentointi ja menetelmät

Heikki Uusitalo

FiSMA r.y.

2Testauksen standardi ISO/IEC 29119, yleisesittely

Content

ISO/IEC 29119 Testing standard Part 1 Concepts and Definitions Part 2 Testing Process Part 3 Test Documentation Part 4 Test Techniques


Tietotekniikan standardoinnista

Kattavin standardointitaho tietotekniikan alueella on ISO eli International Organisation for Standardisation. Elektroniikan alueella standardoinnista huolehtii maailmanlaajuinen järjestö IEC eli International Electrotechnical Committee.

Näillä on yhteinen standardointikomitea JTC1. Ohjelmistotuotannon ja järjestelmäkehityksen standardointi on tämän komitean alainen työryhmä SC7.

Suomessa tietotekniikan standardoinnin yleisvastuu kuuluu ISO-jäsenyyttä hoitavalle Suomen Standardisoimisliitolle (SFS).

Politiikkansa mukaisesti SFS on delegoinut SC7-standardoinnin käytännön työn FiSMA-yhteisölle (Finnish Software Measurement Association ry).

SC7

Secrétariat

WG26

Software Testing

JTC1 SC7 working groups (WG)

Life Cycle ProcessHarmonization Advisory

Group

LCPHAG


Standardin synty

ISO/IEC pitää vuosittain Plenary ja Interim –kokoukset, joissa työryhmien ehdotuksia käsitellään.

Standardit valmistellaan työryhmissä ja hyväksytään äänestysmenettelyillä.

SC7 –alakomitea työryhmä WG26 testaus. Suomen edustajana WG26 ryhmässä on TT Ossi Taipale LUT


Preliminary Work Item (PWI)

New Work Item Proposal (NP)

Working Draft(s) (WD)

Committee Drafts (CD)Final Committee Draft (FCD)

Final Draft International Standard (FDIS)Draft International Standard (DIS)

Stages of Development: Standards

12 months

33 months

International Standard (ISO/IEC)36 months

Typical TimeframesNominal schedule durations are the

typical cumulative number of months following Approval of work item.

Duration of stages may be shortened or lengthened depending on time To reach consensus to proceed to the next Stage..

New Work Item Proposal (NP) is a proposal for:• a new standard• a new part of an existing ISO standard• revision of an existing ISO standard or part• an amendment to an existing ISO standard or part• a Technical Specification or a Publicly Available Specification

Stabilized StandardA stabilized standard is one that has

ongoing validity and effectiveness but is mature and insofar as can be

determined will not require further maintenance of any sort. While a

standard is in stabilized status it will no longer be subject to periodic

maintenance but will be retained to provide for the continued viability of

existing products or servicing of equipment that is expected to have a

long working life.

Stabilized Standard

~15-20 years

Committee Doc

JTC 1 Doc

ISO/IEC ITTF Doc

Preparatory Stage 2

Proposal Stage 1

Preliminary Stage 0

Committee Stage 3

Approval Stage 4

Publication Stage 5

WorkingGroup Doc

WorkingGroup Doc


FiSMA r.y.

FiSMA r.y. Finnish Software Measurement Assoiciation M = Measurement, Management, Models

5 työryhmää ja foorumia: Standards and Models, Testing Models, IT Service Management, IT Effectiveness, SPIN

FiSMAn jäsenistössä on nelisenkymmentä suomalaista ohjelmistotyötä tekevää ja palveluja ostavaa yritystä, sekä yliopistoja ja muita julkisen hallinnon organisaatioita. Jäseniä kiinnostaa standardien hyödyntäminen ja keskinäinen kokemusten vaihto jäsenyritysten välillä.

Jäseniä mm. Trafi ja Endero.


Testauksen kaksi standardia

Kaksi standardia, jotka täydentävät toisiaan ISO 29119 Testausprosessi ISO 33063 Testausprosessin arviointimalli

ISO 29119 –standardi valmistunee tänä vuonna keskeisiltä osiltaan ISO 33063 –standardista on rakenne ja sisältö päätetty, työryhmä työstää

sitä äänestyskuntoon. ISO 29119 on jo nyt siinä kunnossa, että sitä voidaan ja kannattaa

hyödyntää Standardit täydentävät toisiaan, 29119 määrittelee prosessin ja 33063

perustuu siihen ja antaa perusteet arvioinnille.


Testauksen prosessit ja niiden kyvykkyyden arviointi


ISO 29119 Software Testing

ISO 29119 Software Testing –standardin tarkoitus on määritellä ohjelmistotestaukselle kansainvälinen standardi, jota voidaan käyttää kaikissa organisaatioissa kaikenlaiseen ohjelmistotestaukseen.

Standardi muodostuu neljästä osasta: 1. Concepts and Definitions Käsitteet ja määrittelyt 2. Test Process Testausprosessi 3. Test Documentation Dokumentointi 4. Test Techniques Testaustekniikat


Part 1 Concepts and Definitions

Systems and Software Engineering Software Testing Part 1: Concepts and Definitions


Part 1 Concepts and Definitions

Part 1 on informatiivinen osa, joka sisältää testauksen määrittelyt, käsitteet ja standardin sisällön.

Part1 on myös johdatus standardin muihin osiin. Siinä kuvataan testauksen rooli organisaation ja projektin kannalta. Siinä kuvataan miten testaus sovitetaan erilaisiin ohjelmiston

kehitysmallehin (agile, evolutionary, sequential ). Liitteissä kuvataan testauksen metriikkaa ja muita käytännön

esimerkkejä. Määrittelyesimerkkinä test case, miten se ilmenee standardin eri osissa.


Part 1 Multi-layered text context Diagram


Part 1 Concepts and Definitions Esimerkki

Test case a set of test inputs (and actions, where applicable), execution conditions,

and expected results developed to exercise specific test coverage item(s).


Part 2 Test Process

Systems and Software Engineering Software Testing Part 2: Test Process


Part 2 Test Process

Part 2 määrittelee yleisen testausprosessin, joka soveltuu kaikkien organisaatioiden käyttöön kaikessa ohjelmistojen testauksessa.

Se sisältää testausprosessin prosessikaaviot ja -kuvaukset. Testausprosessi on kolmitasoinen:

Organisaatiotason testausprosessi Testauksen hallintatason prosessi Dynaaminen testausprosessi

Koska testauksen tarkoitus on riskien vähentäminen, niin standardin lähestymistapa on riskiperusteinen


Part 2 Test Process


Part 2 Test Process


Part 2 Organizational Test process

Organizational test prosess -toiminto käsittää kaksi osaa: Testauspolitiikka Test Policy Testausstrategia Test Strategy

Test Policy on lyhyt, jossa kuvataan testauksen tarkoitus, tavoitteet ja laajuus organisaatiossa. Se määrittää testauksen käytännöt, antaa puitteet testauspolitiikan ja strategian luonnille, katselmoinnille ja jatkuvalle kehitykselle.

Test Strategy on yksityiskohtainen, tekninen dokumentti joka määrittelee miten organisaatio suorittaa testausta.

Dokumentit ovat yleisiä, eivät projektikohtaisia


Part 2 Organizational Test Process implementation


Part 2 Test management process

Muodostuu kolmesta aliprosessista Test Planning Process Tet Monitoring and Control Process Test Completion Process


Part 2 Test Management Process


Part 2 Test management process


Part 2 Dynamic Test Processes

Dynaaminen testausprosessi kuvaa kuinka testaus suoritetaan testaustyypeittäin tai testausvaiheittain.

Neljä prosessia Test design and implementation Test environment set-up and maintenance Test execution Test incident reporting


Part 2 Dynamic Test Processes


Part 2 Test design and implementation process


Part 2 Test Design & Implementation Process

The Test Design & Implementation Process describes how test cases and test procedures are derived; these are normally documented in a test specification, but may be immediately executed, for instance, if performing exploratory testing, in which case they are unlikely to be documented in advance. In figure 10 the activities are shown in a logical sequence, but in practice iteration will occur between many of the activities, often with activities TD3 to TD5 occurring in parallel for substantial periods.

7.2.1 Purpose 7.2.2 Outcomes 7.2.3 Activities and tasks

7.2.3.1 Identify Feature Sets (TD1)

7.2.3.2 Derive Test Conditions (TD2)

7.2.3.3 Derive Test Coverage Items (TD3)

7.2.3.4 Derive Test Cases (TD4)

7.2.3.5 Assemble Test Sets (TD5)

7.2.3.6 Derive Test Procedures (TD6)


Part 2 Esimerkki 7.2.3.4 Derive Test Cases (TD4) This activity consists of the following tasks: a) One or more test cases shall be derived by determining pre-conditions,

selecting input values and, where necessary, actions to exercise the selected test coverage items, and by determining the corresponding expected results. When deriving the test cases be aware that one test case may exercise more than one test coverage item and thus there is the opportunity to optimize derived test cases by combining coverage of multiple test coverage items in a single test case.

b) Where appropriate, the test cases should be recorded in the test case specification. If recorded, the traceability between the test basis, feature sets, test conditions, test coverage items and test cases shall be explicitly described.

d) Where appropriate, the contents of the test case specification should be approved by the stakeholders. This may require repeating tasks a) and b), and in some case first repeating the Derive Test Conditions (TD2) and/or Derive Test Coverage Items (TD3) activities.


Part 3 Test Documentation

Systems and Software Engineering Software Testing Part 3: Test Documentation



Standardi kattaa kaikenlaisen ohjelmistotestauksen dokumentoinnin mallit.

Siinä määritellään dokumenttien käyttö ja sisältö. Dokumentaatio noudattaa Part 2 mukaista tasomallia:

Organisaatiotason dokumentaatio Hallintatason dokumentaatio Testaustason dokumentaatio


Part 3 Test Documentation 7.3.4 Test cases

Defines the test cases derived from the test coverage items. A test case specifies how one or more test coverage item(s) are exercised to determine whether or not that part of the test item has been implemented correctly.

This section in the Test Case Specification could be formatted to list test cases under corresponding feature sets and/or test conditions. The test cases may be described in lists or in tables in a document or using a tool, e.g. A relational database or a dedicated test tool.

The information for a test case includes: 7.3.4.1 Unique identifier

Describes the unique identifier needed by each test case so that it can be distinguished from all other test cases. An automated tool may control the generation of the identifiers.

7.3.4.2 Objective Identifies and briefly describes the special focus or objective for the test case. This is typically

in the form of a title. 7.3.4.3 Priority

Defines the priority for the testing of this particular test case, if needed. 7.3.4.4 Traceability

Describes traceability to the test coverage item that the test case exercises or lists reference(s) to the associated requirement(s) and/or design description(s) in the test basis. This may be documented in a Test Traceability Matrix.


Part 3 Test Documentation 7.3.4.5 Preconditions

Describes the required state of the test environment and any special constraints pertaining to the execution of the test case, e.g. the state the test item shall be in before execution may start, including existence of specific test data and the currently active form or screen. This could be described explicitly or it could include references to other test cases, whose execution will set the preconditions.

The environment needed may be described collectively for one or more feature sets, or it may not be described in this specification if the description in the Test Plan is sufficient.

7.3.4.6 Inputs Specifies each action required to bring the test item into a state where the expected

result can be compared to the actual result. This may require provision of input data and events, e.g. button clicks, to the test item.

Some of the input data may be specified by value, while others, such as constant tables or transaction files, may be specified by name. All appropriate databases, files, terminal messages, memory resident areas, and values passed by the operating system must be considered.

All required relationships between input events, e.g. timing, must be described. The actions or steps may be numbered within the test case, if needed.



7.3.4.7 Expected results Specifies the expected outputs and behaviour (e.g. response time) that is required of

the test item in response to the inputs. Provides the expected value(s) (with tolerances where appropriate) for each required output.

The actions required to compare the expected result to the actual result may also be specified.

For instance, examining the output in a field on a form that is not active when the input is provided, waiting for a batch job to run and a report to be printed out and examined, or closing down the test item and restarting it to examine stored data.

7.3.4.8 Test outcome and test result The description of a test case may include placeholders to record test outcome and/or

test results during execution of the test case. Alternatively, this may be recorded in the Test Procedure Specification (see clause 7.10).


Part 4 Test techniques

Testaustekniikoiden kuvaus Testauksen mittaamisen tekniikoiden kuvaus


Part 4 Test techniques,luokittelu


Part 4 Test techniques, esimerkki

5.3 Structure-Based Testing Techniques 5.3.1 Statement Testing 5.3.1.1 Derive Test Conditions (TD2) A model of the source code of the test item (program) which identifies

statements as either executable or nonexecutable shall be derived. Each executable statement shall be a test condition. 5.3.1.2 Derive Test Coverage Items (TD3) The executable statements shall be identified as test coverage items (i.e.

they are the same as the test conditions).


Part 4 Test techniques, esimerkki

5.3.1.3 Derive Test Cases (TD4) The following steps shall be followed: 1. Identify control flow subpaths that reach one or more test coverage items

(executable statements) that have not yet been executed during testing. 2. Determine the test inputs that will cause the control flow subpath(s) to be

exercised. 3. Determine the expected outcome from exercising the control flow

subpath(s) by applying the corresponding test inputs to the test basis. 4. Repeat steps 1 to 3 until the required level of test coverage of the

executable statements is achieved.


Part 4 Test techniques, Testing of Quality Characteristics

A.1 Quality Characteristics A.1.1 Overview Software testing can be carried out to collect evidence that required

quality criteria have been satisfied by a test item. Required quality characteristics should be specified in the test

basis. Definitions of quality characteristics could be derived from ISO/IEC 25010 System and

Software Product Quality Requirements and Evaluation (SQuaRE) – System and Software Quality Models.


ISO/IEC 25010 Product Quality Model


Part 4 Test Techniques

A.2 Mapping Quality Characteristics to Test Design Techniques The test design techniques described in this standard can be used to test

a variety of the qualitycharacteristics listed above in Figure 3. The following table provides a mapping between them.

Table 1 – Mapping of ISO/IEC 25010 product quality characteristics to test design techniques

Statement Testing Functional Suitability

Functional completeness Functional correctness Functional appropriateness


Further information about SC7 standards

www.sfs.fi, IT standardisation at general level www.fisma.fi, Software, Systems, Services standards FiSMA: risto.nevalainen ( at ) fisma.fi

Documents

ISO/IEC 29119 Part 1, Part 2, Part 3, Part 4 Testauksen prosessit, dokumentointi ja menetelmät