Embed Size (px)
Citation preview
Moving to Agile in an FDA Environment
An Experience Report
August 27, 2009
An Experience Report
Slide 2
Introduction
• Available for download at (insert link here)
• Agile Resources
– Agile Manifesto http://www.agilemanifesto.org/
– Agile Alliance http://www.agilealliance.org
• Agile Books
– http://www.agiletek.com/thoughtleadership/books
“Agile Software Development” by Alistair Cockburn
“Agile Project Management” by Jim Highsmith
An Experience Report
Slide 3
Outline
• The Abbott Experience
• Lessons Learned
1. The least burdensome approach?
2. A Risk Based Approach
3. Control what you don’t know, don’t let it
control you
4. Dispense with ceremony
• Results
An Experience Report
Slide 4
The Abbott Experience
• Comparing two FDA-regulated
medical device projects
– One not agile, one agile
– Class III devices (most stringent)
– Results of agile adoption• Lower cost
• Shorter duration
• Better, less prescriptive test cases
• Accommodated change
• Higher quality
• See paper “Adopting Agile in an
FDA Regulated Environment”
An Experience Report
Slide 6
Product Development Lifecycle
Product Development Lifecycle
ResearchDevelopment
Concept Feasibility
Concept Phase
Increment 1
Concept Phase
Increment n
Feasibility Phase
Increment 1
Feasibility Phase
Increment n
Development Phase
Increment 2Design Transfer
Development Phase
Increment nDevelopment Phase
Increment 1
Increment
Increment n, Iteration 1 Increment n, VerificationIncrement n, Iteration 1
Capability Inventory
(Comprehensive list of
features, scenarios,
architectural elements,
etc. – a living document)
An Experience Report
Slide 7
Software Development Lifecycle
An Experience Report
Slide 8
Iteration Model
Product Definition
Feature Summary
Feature Descriptions
Software Requirements
...
Planning
Software Development Plan
Software Process Definition
Software Engineering Development Practice
Software Test Plan
Software Configuration Management Plan
Software Quality Assurance Plan
Software Measurement Plan
Capability Allocation Matrix
Software Schedule
...
High Level Design
High Level Design Description
Software Architecture Description
User Interface Design Document
User Interface Framework
...
Software
Construction
Design
Feature Detailed Design
Presentation
Business Rule
Data Access
Persistence
Test Design
...
Implementation
Presentation
Business
Data Access
Database
Test Cases
...
Verification
Unit Test
Integration Test
Code Inspection
Design Review
...
Stakeholder Review
Application
Requirements
Feature Descriptions
Scenario Descriptions
...
Software Confirmation
System Test
Regression Analysis
Integration Test
Acceptance Test
Functional Testing
...
Software Release
Version Description
(Release Notes)
Traceability Matrix
Inputs -> Outputs
Validation / Verification
Data
...
Software Characterization
Iteration Model
An Experience Report
Slide 9
QSR Ref. ISO Ref. Design Control Element Documents
1. 820.30(b) 7.3.1 Design and development
planning
Design and Development Plan
2. 820.30(c) 7.3.2 Design input System Requirements Specification
Safety Risk Analysis
3. 820.30(d) 7.3.3 Design output System Design Description
Source code
Drawings/Schematics
4. 820.30(e) 7.3.4 Design review System review reports
Design review reports
5. 820.30(f) 7.3.5 Design verification Verification Test Procedures
6. 820.30(g) 7.3.6 Design validation Validation Test Procedures
Test Summary Report
Requirements traceability matrix
7. 820.30(h) NA Design transfer Design transfer procedure
8. 820.30(i) 7.3.7 Design changes Change control procedures
9. 820.30(j) 4.2.3-4 Design history file Document control procedures
Sample Design Control Documents
Courtesy of: Certified Compliance Solutions, Inc.
An Experience Report
Slide 10
Num. Requirement Document
1. Level of concern Safety Risk Analysis
2. Software Description Design and Development Plan
3. Device Hazard Analysis Safety Risk Analysis
4. Software Requirements
Specifications (SRS)
System Requirements Specification
5. Architecture Design Chart System Design Description
6. Software Design Specification System Design Description
7. Traceability Analysis System and safety requirements traceability matrix
8. Software Development
Environment Description
Design and Development Plan
9. Verification and Validation
Documentation
Verification Test Procedures
Validation Test Procedures
Test Summary Report
10. Revision Level History Initial market release
11. Unresolved anomalies As per procedure, anomalies that affect safety or
essential performance are not allowed in a release
Sample Submission Documents
Courtesy of: Certified Compliance Solutions, Inc.
Lessons Learned
An Experience Report
Slide 12
Least Burdensome
We [FDA] are defining the term “least burdensome” as a successful means of addressing a premarket issue that involves the most appropriate investment of time, effort, and resources on the part of industry and FDA.
--The Least Burdensome Provisions of the FDA Modernization Act of 1997: Concept and Principles; Final Guidance for FDA and Industry
An Experience Report
Slide 13
A Risk Based Approach
Courtesy of: Certified Compliance Solutions, Inc.
An Experience Report
Slide 14
• IEC 62304 section 5.1.1, page 31:
Note 1: The software model can identify different
activities for different software items according to
the safety classification
• FDA General Principles of Software
Validation, page 7, section 3.1.2:
The level of confidence, and therefore the level of
software validation, verification, and testing effort
needed, will vary depending on the safety risk
(hazard) posed by the automated functions of the
device.
Risk Based Approaches
Courtesy of: Certified Compliance Solutions, Inc.
An Experience Report
Slide 15
SOFTWARE
DOCUMENTATION
MINOR
CONCERN
MODERATE
CONCERN
MAJOR
CONCERN
Verification and
Validation
Documentation
Software
functional test
plan, pass /
fail criteria,
and results.
Description of
V&V
activities at
the unit,
integration,
and system
level. System
level test
protocol,
including
pass/fail
criteria, and
tests results.
Description of
V&V activities
at the unit,
integration, and
system level.
Unit,
integration and
system level
test protocols,
including
pass/fail
criteria, test
report,
summary, and
tests results.
FDA Reviewer Guidance 2005
Courtesy of: Certified Compliance Solutions, Inc.
An Experience Report
Slide 16
Control What You Don’t Know, Don’t Let It Control You
• What you do know:– A typical medical device is developed over a 3-5 year
time horizon
– It is a myth that you can predict in detail your end product requirements up-front
– Start with a core set of features that you must implement
– Implement the core features first
– Defer the most volatile features as long as possible
• Iterative approach allows the team to:– Manage scope and limit feature creep
– Negotiate scope and tradeoffs with key stakeholders
• “At time of commercial launch, a number of features, once thought to be essential, were not included. Some were deferred as long as three years. Nonetheless, the product was considered highly successful and trading off nice to have features for three years of sales is an easy choice.”
An Experience Report
Slide 17
Dispense With Ceremony
• If it is not adding value, and it is not required,
do not do it
• The design history files should contain the
minimum set of documentation that satisfies
the regulatory requirements
• There will be other activities that you will
want to document, no need to include in
design history file, make sure they add value
and do it in a least burdensome way
• Avoid doing things because “that’s the way
we’ve always done it”
• If it feels like you are wasting your time you
probably are
Results
An Experience Report
Slide 19
Results
• High visibility– Easier to manage and control
– Far fewer surprises
• Lower cost and shorter duration– Estimated schedule and team size reduction of 20%-
30%
– Estimated cost savings of 35%-50%
• Higher quality– Availability of working software facilitated continuous
testing instead of back loaded V&V
– Resulted in fewer overall defects, especially at the end of the project
• Better work life balance and team morale– Project death marches are rarer because the issues
are surfaced as you go and are managed accordingly, not all saved up for the end of the project
Q & A
An Experience Report
Slide 21
Tim Hughes J.R. Jenks
Managing Partner Managing Partner
[email protected] [email protected]
847-699-2260 847-699-2250
Thank You
John SkachSenior Technology Architect
847-699-2264
Rod RasmussenDirector, Informatics & Software Systems
847-938-3633
