Requirements Quality Assurance

Ch. 5 Lecture NotesIN4MTX 113January 2010

Requirements Quality AssuranceChapter 5 TopicsInspections and Reviews

Queries using a Requirements Database

Validation by Specification Animation

Verification through Formal Checks

22Congrats. You wrote your first RD.

but who checked it?3Why Have Inspections and Reviews?Validate that the RD:Reflects stakeholder needsExplains the system accuratelyGet feedbackProgressCustomer feedbackSubject matter experts that can catch your mistakes

End goal: have an accurate, complete, and consolidated RD

4Requirements Inspection Process5

Requirements Inspection Process6

Inspection Planning7Different terms: Walkthroughs, Colleague Reviews, Peer ReviewsSubtle differences involving scope and format of these reviewsWhich one to use?

Planning involves the basics: schedule, scope, have your document ready to review, decide who is getting invited, find a conference room, etcSome Guidance to Inspection Planning8Time it well

Limit the number of people attending: key experts and stakeholders only (max: 7, min: 3)

Dont invite any manager

Give your reviewers enough time

Get your comments ahead of time

Customers are tricky

Requirements Inspection Process9

Individual Reviewing10Free Form/Free StyleNo direction givenFind what you can

Checklist-basedSpecifics you want your reviewers to provide feedback: format, readability, clarity, consistency (defects), language and semantics

Process-basedAssign rolesSeek different perspectives from specific disciplines: safety, design, test, quality

Example: Defect-based Checklist11Omission: Was something greatly missed?Contradiction: Consistent with other requirements/concepts?Inadequacy:Did this meet the stakeholders needs?Ambiguity:Too many interpretations?Immeasurability: Are these requirements verifiable?Noise: Are these statements relevant?Over specificationDo these requirements add any value to verify?UnfeasibilityIs this possible?UnintelligibilityWhy is this statement/requirement here?Poor StructuringBad wording?Forward ReferenceIs the concept defined later in the document?RemorseHas the concept been used before definition?Propagated ChangesWould a change here propagate elsewhere?OpacityAre the dependencies visible?Some Guidance to Individual Reviewing 12Ask yourself: what are you seeking?Technical accuracy?Clarity in wording? then ask your reviewers for the same.

Providing direction will yield the best results: go with checklist-based or process-based reviews.

Requirements Inspection Process13

Defect Evaluation at Review Meetings14Have the inspection review meeting, collect comments.

Tips:Excel is very powerful / matrix comments, resolution, action itemsDocument the problem analyze later.15Defect Evaluation at Review Meetingsan example

Requirements Inspection Process16

RD Consolidation17Consolidate comments

Tip: decide your next actionResolve conflicting commentsDefer if the conflict gets out of handDisagreeing with a commentReconcile comments with your updated RD

Requirements Inspection Process18

Queries on a Database19 Consistency Checks

Metrics

Deltas

Data/Models InOutputsRequirements Database

AB20Full Inspection Review of Version A

Delta Inspection Review of Version B

Queries on a Database

ABDeltasQueries on a Database21Queries can be made to determine consistency in wording and assigned entitiesQueries for metrics:# of requirements: Volatile requirements comparison (Version A vs Version B) # of specific requirements: i.e. how many requirements related to interfaces? Safety requirements?Traceability: finding orphaned requirements, childless parentsDeltas from one baseline of an RD to the next

22

Validation by Specification Animation23The point: to check the adequacy of requirements and the assumptions youve made

Extracting an executable model from the specification: what perspective do we want to animate?

Validation ScenariosDoes the system behave as expected?

Mimic possible behaviors of the environmentDemonstrate that the software can respond to outside events

24

Pitfalls using Animation/Simulation25You still need a formal specification

Tricky things can happen in the environmentSimulated models will not catch these anomaliesExperts are needed

Not everything can be represented

Gaps will exist between animated model and original specificationAdequacy of model vs inadequacy of what was specified

Verification through Formal Checks26Language Checks

Dedicated Consistency and Completeness Checks

Model Checking

Theorem ProvingModel Checking27

28

Proof by Counterexampleinit: (doorsClosed, trainStopped)start: (doorsClosed, trainMoving)[speed = 0]: (doorsClosed, trainStopped)opening: (doorsOpen, trainStopped)start: (doorsOpen, trainMoving) Missing from DoorsState = closedFigure 5.1 Requirements inspection, review, and consolidationInspection

planning

Individual

reviewing

Defect evaluation

at review meetings

RD

consolidation

Figure 5.1 Requirements inspection, review, and consolidationInspection

planning

Individual

reviewing

Defect evaluation

at review meetings

RD

consolidation

Figure 5.1 Requirements inspection, review, and consolidationInspection

planning

Individual

reviewing

Defect evaluation

at review meetings

RD

consolidation

Figure 5.1 Requirements inspection, review, and consolidationInspection

planning

Individual

reviewing

Defect evaluation

at review meetings

RD

consolidation

Figure 5.1 Requirements inspection, review, and consolidationInspection

planning

Individual

reviewing

Defect evaluation

at review meetings

RD

consolidation

Figure 5.1 Requirements inspection, review, and consolidationInspection

planning

Individual

reviewing

Defect evaluation

at review meetings

RD

consolidation

Figure 5.4 Model checking P ( Q

Yes

Property

Figure 5.4 Model checking

Behavior model

Model checker

No,

+ counterexample

trainStopped

[speed = 0]

trainStart

[speed = 0]

Figure 5.5 A faulty SM model for the behavior of a controller of train doors and movements

closing

doorsOpen

trainMoving

opening

doorsClosed