January 16, 2017 Sam Siewert

SE420 Software Quality Assurance

Lecture 1 – Introduction Part-2

Course Learning Objectives Theory of Overall SQA Process Process Models (Waterfall, Spiral, XP) using Agile Strategy Terminology (IEEE SWEBOK – Chapter 4), Prepare for ISTQB Foundation Certification Principles and Methods for SQA Organization for SQA Practice – Design Methods, Software Construction and Testing (Exam-1)

1. Requirements and Use of Basic Design Models (SA/SD from SE300, OOA/D from SE310) 2. Code Walk-throughs 3. Debugging 4. Feature Addition and Performance Tuning 5. Unit Testing, Exit Criteria (CSUs) 6. Code Inspection 7. Validation and Verification (Code Implements Design, Design is Correct)

Practice, and Scaffold for Final – Requirements and Design (Exam-2) 1. Integration and Test (CSUs to Build and Test CSCI) 2. System Testing (CSC) 3. Acceptance Testing 4. Regression and Test Automation 5. Overall Metrics, Process Improvement (SEI CMM) 6. Delivery and Release Notes 7. SQA Inspection

Sam Siewert 2

January 12, 2016 Sam Siewert

Linux Skills

Introduction Session Part-2

Sam Siewert 4

C Code Unit Development and Test Assignment #1 Questions? Examples-Crypto.zip Hands on Sessions to:

1. Download and unzip code 2. Build code 3. Modify Makefiles (for debug with –g) 4. Re-build 5. Debug with DDD 6. Inspect variable values during debug 7. Set breakpoints 8. Step Over 9. Step Into 10. … 11. If code SEGFAULTS, dump core and

debug the core

Powerful Features of Visual Debug Code Walk-throughs – Static Analysis – Dynamic too

Display Data Structures – Evolution – Recursive – Link List – Trees – Graphs – Multiple

Indirections

Sam Siewert 5

CS317 B-tree Visualized

Discussion… SQA History (read SQA Text Chapter #1) – Come to Class Prepared to Discuss – Gurus (e.g. Edward Deming, Grady Booch, Barry Boehm,

Edward Yourdon, Tom DeMarco, …) – Organizations for SQA - http://www.sei.cmu.edu/ – Process and Validation of Process -

http://whatis.cmmiinstitute.com/

Assignment #1 Discussion – I will Post Every Other Tuesday, We’ll Discuss, Due Following

Week on Friday – Late Assignments – 10% Penalty for 3-day late Turn-in [Sunday],

After Monday, only with Instructor Permission

Sam Siewert 6

Wisdom from QA, SQA & SWE Gurus W. Edwards Deming – “"You can expect what you inspect."

Edsger Dijkstra - “program testing can be used to show the presence of bugs, but never to show their absence” – E.W. Dijkstra, “Notes on Structured Programming,” T.H.-Report

70-WSE-03, Technological University, Eindhoven, 1970; http://www.cs.utexas.edu/users/EWD/ewd02xx/EWD249.PDF

Barry Boehm – Are we building the right product? [validation]; Are we building the product right? [verification]

Sam Siewert 7

Overall SQA Process IEEE Computer SWEBOK version 3 – Organizations for SQA - http://www.sei.cmu.edu/ – Process and Validation of Process -

http://whatis.cmmiinstitute.com/

Process Must Be Defined, Repeatable, and Improved Over Time – Cost of Mistakes in Early Phases is Higher (E.g. Requirements

or Design mistakes compared to Implementation) – Organizational and Individual Engineer SQA – More than One Acceptable Process (E.g. Agile strategy hosting

Spiral process, using OO & Structured analysis and design)

Phases Do Not Have to Be Fully Completed? Waterfall?

Sam Siewert 8

Dimensions of SW Quality 1. Specification (What) – What will be built 2. Design (How) – How it will be Built (interface, function,

protocol) 3. Development (How) – Software Construction 4. Conformance – Was the right thing built (validation) and does

it work (verification)

Basic Interrogatives (Start of Any New Project) – Who? – Team – Where? – Organization (Distributed Team?) – Why? – Marketing Study, Product Research, Customer Request – What? – Capabilities and Requirements Specification – When? – Schedules

Enemy of Quality? Estimation with Defined Process? When is Testing Done? (Exit Criteria)

– How? – Design Details and Software Construction

Sam Siewert 9

Dimensions form Outline for SQA Process

Concept, Statement of Work, Marketing Study, Team Building 1. SW Requirements

– Analysis, Specification, Validation 2. SW Design

– High Level – Interfaces, Data flow, Concurrency, Objects/Modules – Detailed – State Machines, Features and Function, Algorithms – Design Validation (E.g. Simulation with MATLAB)

3. SW Construction 4. SW Testing

– Unit Testing – Integrated Testing – Regression Testing – Acceptance Testing

5. Maintenance – Field Support, Sustaining Engineering Sam Siewert 10

Waterfall Model Complete One Phase and Do Not Return Waterfall with Feedback Modification (Go Back One or More Phases) Defined and Disciplined, but Criticized as Impractical – Too Much Up Front Effort – Difficult to Collaborate – Rigid

Sam Siewert 11

Requirements

Design

Construction

Testing

Maintenance

Evolutionary or Spiral Model Phases, But Radial Distance and Area of Phase Represents Effort (Time / Cost) Plan to Re-visit Phases Goal is to Control Cost/Effort and Impact of Mistakes

Sam Siewert 12

Project Proposal

Reference Project

Specification (Analysis, requirements, Validation)

Design (High Level, Detailed, Validation)

Software Construction (Interfaces, Objects/Modules, Algorithms, Coding)

Compliance (Testing: Unit, Integrated, Regression, Acceptance)

Project HLD

Evolutionary Prototype

I & T Readiness

Detailed, Validated Specification

Detailed Design

All Modules Complete & Unit Tested

System Demonstration (Acceptance) Write/review

Define Services And interfaces

Concept Code (walk-through)

Unit Testing

Refine specification and re-validate Finalize Design

Finalize Code Modules and Unit Tests

(Code inspections)

System Integration Test

SQA – Many Activities to Coordinate SQA Management is Not Simple (Parallel with SWE Development) Activities Need to Be Scheduled at Appropriate Phase – Can’t have Code Inspection at a Walk-through – Can’t Do Integrated Testing Prior to Unit Tests

Defined, Repeatable, Measureable, Repeatable – Basic Requirements for Process – May Tailor for Market

Mission Critical Avionics Software In-Flight Infotainment System Mobile App for Android Storage Data Path Digital Media System

Sam Siewert 13

Stretch Goals and Objectives Focus on Applying Requirements and Design Methods for SQA Use Case Studies (SQA Pitfalls) What Can Go Wrong? Why it Did? Cost to Fix? Requirements V&V (System and Acceptance Test) Design V&V (Unit and I&T) Agile Strategy for Evolutionary Process in SQA

Sam Siewert 14

SQA Knowledge Survey Unit test drivers should attempt to modify the code being tested as little as possible: A_TRUE B_FALSE D_DON’T KNOW Edsger Dijkstra observed that “Program testing can be used to show the presence of bugs, but never to show their absence!”, so we most often define exit criteria for testing modules in terms of metrics such as path coverage rather than a zero defect claim: A_TRUE B_FALSE D_DON’T KNOW Path coverage criteria can be used only for White-box testing: A_TRUE B_FALSE D_DON’T KNOW Sam Siewert 15

SQA Knowledge Survey Black-box testing requires a mixed-mode source/assembly debugger, disassembly of source code into instructions and careful attention to compiler features such as short-circuit logic and conditional execution of instructions: A_TRUE B_FALSE D_DON’T KNOW Regression testing includes tests from all other phases of test development with the purpose to make sure that when a defect is fixed, that somehow new bugs are not introduced : A_TRUE B_FALSE D_DON’T KNOW Integration testing focuses on testing module-to-module interfaces and communication protocols to verify and validate module interfaces before they are integrated to compose an architecture design: A_TRUE B_FALSE D_DON’T KNOW Sam Siewert 16

SQA Knowledge Survey Module designs and code should be tested only for correct function and not stress, performance, or soak tested: A_TRUE B_FALSE D_DON’T KNOW Stress and soak time testing of a module with a unit test driver might reveal a memory leak in the module: A_TRUE B_FALSE D_DON’T KNOW Regression testing should only be run before a software product is shipped: A_TRUE B_FALSE D_DON’T KNOW If SQA Design is Concurrent with Software Design, Acceptance Testing would best be developed during development of: A_REQTS B_DESIGN C_CODING D_DON’T KNOW

Sam Siewert 17

SQA Knowledge Survey Ideally an acceptance test should be provided by the customer, but this rarely happens, so the software engineering team often must help define the acceptance test: A_TRUE B_FALSE D_DON’T KNOW System testing involves configuration of sub-system(s) from multiple software modules and a driver or external stimulus to exercise this sub-system: A_TRUE B_FALSE D_DON’T KNOW System test involves integration of sub-systems into a system tested in an end-to-end test environment, similar to how the system will be used: A_TRUE B_FALSE D_DON’T KNOW Sam Siewert 18