Getting Your Medical Device FDA Approved

mentor.com/embedded

Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions.Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.

Qt is a registered trade mark of Digia Plc and/or its subsidiaries. All other trademarks mentioned in this document are trademarks of their respective owners.

Seminar June 2014

Getting Your Medical Device FDA Approved

2

Why and How Companies Fail Software documentation inadequate; includes: *

not provided at all, missing description, missing traceability matrix, missing list of anomalies, and/or missing validation

* “Analysis of Pre‐Market Review Times Under the 510(k) Program,” US Food and Drug Administration, CDRH, November 9, 2011

Affects 17% of submissions!

3

Why and How Companies Fail

Source: “Medical Device Recall Report: FY2003 to FY2012,” US Food and Drug Administration, CDRH, March 21, 2014

“Failure to implement software design controls, and where appropriate, testing procedures, as well as increasing complexity of the medical device use environment (with increased connectivity and interoperability) can lead to software anomalies often requiring a correction or removal.”

Software Change Control

Software Design

S/W Design (Mfg.)

Sum % of CDRH Recalls

2008 13 141 2 156 18.3%2009 9 111 1 121 15.4%2010 4 73 3 80 8.9%2011 11 182 10 203 15.8%2012 12 169 5 186 15.5%Total 49 676 21 746 15.1%

4

Why and How Companies Fail“Failure to follow or otherwise address current guidance document(s) or recognized standards ‐ FDA issues guidance documents or recognizes a national or international standard to help manufacturers determine what information to include in a 510(k) submission generally and for certain device types specifically. If a manufacturer fails to follow current guidance (i.e. that which is up‐to‐date) for a certain device type or a recognized standard, and offers no explanation for its failure to do so, FDA would consider that submission to be of poor quality and would issue an AI Letter that quotes current guidance to obtain the missing information.”

* “Analysis of Pre‐Market Review Times Under the 510(k) Program,” US Food and Drug Administration, CDRH, November 9, 2011

41% of submissions affected!

5

The Primary Roadmap Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices

http://www.fda.gov/MedicalDevices/ DeviceRegulationandGuidance/ GuidanceDocuments/ucm089543.htm

(Issued in 2005)

6

FDA Software Guidance Software “level of concern”

Estimate (in the absence of mitigations) of the severity of injury that a device failure or latent design flaw could permit or inflict, either directly or indirectly, on a patient or device operator: Major: could result in death or serious injury Moderate: could result in minor injury Minor: unlikely to cause any injury

Documentation FDA recommends submitting depends on the level of concern

7

Software‐related Documentation Overview

Describe the design of your device Document how your design was implemented Demonstrate how the device produced by your design implementation was tested

Show that you identified hazards appropriately and managed risks effectively

Provide traceability to link together design, implementation, testing, and risk management

8

Software‐related Documentation Device Hazard Analysis

Address all foreseeable hazards (including intentional or inadvertent misuse)

Identification of the hazardous event Severity of the hazard Cause(s) of the hazard Method of control (e.g., alarm, hardware design)

Corrective measures to eliminate, reduce, or warn

Verification of correct control implementation

9

Software‐related Documentation Software Requirements Specification

Hardware Requirements Programming Language Requirements SW Performance and Functional Requirements

Architecture Design Chart Software Design Specification Traceability Analysis SW Development Environment Description V&V Documentation Revision Level History Unresolved Anomalies (Bug List)

10

Software Development Standard IEC 62304:2006 Medical device software –Software life cycle processes SW development SW maintenance SW risk management SW configuration management

SW problem resolution

11

Overview – IEC 62304:2006 Describes software development and maintenance processes for medical devices but does not specify: An organizational structure or responsibilities The name, format or explicit content of documentation The specific life‐cycle model to be employed

FDA Consensus Standard (September 2008) IEC62304 conformance fulfills the “Software Development Environment Description” section of a 510(k) submission

Normative standard in Europe for CE marking

12

Software Verification and Validation (V&V) MINOR LOC:

System or device level testing, any integration testing Pass/fail criteria Summary of test results

MODERATE LOC: V&V activities at the unit, integration, and system level

Summary list of V&V activities Pass/fail criteria Test results

MAJOR LOC: V&V activities at the unit, integration, and system level

Unit, integration and system‐level test protocols Pass/fail criteria Test report, summary, test results

13

Software‐Related Documentation General Principles of Software Validation: Final Guidance for Industry and FDA Staff

http://www.fda.gov/ MedicalDevices/ DeviceRegulationand Guidance/ GuidanceDocuments/ ucm085281.htm

(Issued in 2002)

14

Software Design and Human Factors Weave human factors engineering into entire design and development process, including device design requirements, analyses, and tests

Consider device safety and usability issues when developing flowcharts, state diagrams, prototyping tools, and test plans

Perform task and function analyses, risk analyses, prototype tests and review, and full usability tests

Include participants from the user population(s)

15

Common User Interface (UI) Issues UI complexity causes user confusion, delay in use, or inability to use the device

UI makes it difficult for user to correct data entry errors or modify device settings in a timely fashion

UI falsely causes the user to believe a critical situation exists when it does not, or vice‐versa

UI does not draw attention to dangerous conditions of device operation or patient status

UI does not prevent known, likely data input errors

16

Regulatory Basis for Human Factors 21 CFR 820.30, Design Controls (The need for human factors is implied):

Design input – includes “needs of the user and patient”

Design verification – performance criteria met Design validation – “… devices conform to defined user needs and intended uses and shall include testing of production units under actual or simulated use conditions. Design validation shall include software validation and risk analysis….” [incl. use‐related risks]

17

Human Factors StandardsAAMI/ANSI HE75:2009Human factors engineering –Design of medical devices

18

ISO/IEC 62366:2007Medical devices – Application of usability engineering to medical devices

FDA Human Factors Guidance Applying HF&UE to Optimize Medical Device Design

http://www.fda.gov/ MedicalDevices/ DeviceRegulationandGuidance/ GuidanceDocuments/ ucm259748.htm NOTE: issued in 2011 –It is not yet in effect but it reflects FDA‐CDRH’s current thinking and approach to human factors

19

2011 Draft Human Factors GuidanceFDA’s best thinking regarding: Device Users, Use Environments and User Interfaces Use‐Related Hazard Analytical Methods Formative Evaluations Mitigation and Control of Use‐Related Hazards Design Verification Testing Human Factors Validation

20

“Use error caused by designs that are either overly complex or contrary to users' intuitive expectations for operation is one of the most persistent and critical problems encountered by FDA.”‐ General Principles of Software Validation, Final Guidance for Industry and FDA Staff (2002)

Submission Problems with HF/UE Not understanding that “usability goals” are not included in FDA’s recognition of the standard

Simply state “in compliance” with 62366 and submit no HF report

Not using US residents in validation studies (w/exceptions) Devices modified to mitigate use error issues, then the mitigation is not directly tested in validation testing

Formative evaluations either not done or pretending to be validation testing

Overemphasis on complex protocol, technique, and test conditions while the test focus, data, interpretations and report are impossible to interpret.

21

A Good HF/UE Validation Report Includes:

Intended device users, uses, use environments, and training

Device user interface (graphical & verbal description)

Summary of known use problems User task selection, characterization and prioritization

Summary of formative evaluations Validation testing

22

Validation Testing Rationale for test type selected (i.e., simulated use or

clinical evaluation) Number and type of test participants and rationale for

how they represent the intended user populations Test goals, critical tasks and use scenarios studied Technique for capturing unanticipated use errors Definition of performance failures Test results: Number of device uses, success and failure

occurrences Subjective assessment by test participants of any critical

task failures and difficulties Description and analysis of all task failures, implications

for additional risk mitigation23

FDA’s Cybersecurity Concern Network‐connected devices disabled by malware

Targeting of patient data using wireless technology

Uncontrolled passwords management Failure to address security vulnerabilities via update

Security vulnerabilities in OTS software

‐ Source: FDA Safety Communication (June 13, 2013)

24

FDA Cybersecurity Guidance Content of Premarket Submissions for Management of Cybersecurity in Medical Devices

http://www.fda.gov/ MedicalDevices/ DeviceRegulationand Guidance/ GuidanceDocuments/ ucm356186.htm NOTE: issued in 2013 –It is not yet in effect but it reflects FDA‐CDRH’s current thinking and approach to cybersecurity

25

2013 Draft Cybersecurity GuidanceGeneral Principles: Confidentiality – only authorized persons or entities have access

Integrity – Accurate and complete data, not improperly modified

Availability – Accessible and usable on a timely basis in the expected manner

26

2013 Draft Cybersecurity GuidanceDocumentation:Hazard analysis addressing intentional and unintentional cybersecurity risks

Traceability matrix linking controls to risks Systematic plan for updates and patchesDemonstrate how the device will be provided free of malware

IFU and specs for recommended anti‐virus s/w and/or firewall

27

Conclusions:Buy a copy of device‐relevant standardsGet free FDA guidance documents onlineRead themGap assess your processes and current records – then fix them

implement tools that drive consistent outputs

Hire a HF/UE Engineer (or make a consultant happy)

28

Thank You!

For questions or assistance, contact me at [email protected] or (972) 505‐1920

29

The Standard ‐ Illustrated

30

Medical Device Software under IEC 62304

George Romanski

©

IEC 62304

• Medical Device Software – Software Lifecycle Processes– Quality Management System*– RISK MANAGEMENT– Software Safety Classification– Development Process– Maintenance Process– Configuration Management*– Problem Reporting and Management*

IEC‐62304 Medical Software

* These processes are Universal between the standards

©

Software Safety Assessment

• For Avionics – ARP‐4761 (Safety Assessment) • For Medical – ISO 14971 (Safety/Risk) Normative reference • For Automotive

– Part of ISO 26262• For Industrial

– Part of IEC 61508• For Trains

– Part of EN 50128


Different terms: Design Assurance Level, Software Integrity Level, ClassSame Principle: Consequence/Exposure, Severity ‐> Level for software

Level chosen for System then applied to Software

‐‐‐How do you assess for RTOS?

©

What level is Device Software?

• Software Faults do not follow “Gauss Normal” Distribution (Bell Curve)– Given 10,000 lines of code– You test and find 10 software faults– What is the probability of finding another fault with another test?

• We Don’t know distribution of faults• Software must be assumed to be level C until shown by Risk Assessment that a lower level is applicable!


©

Software Risks

• Does it do what it should?• Does it do More than it should?• It does something wrong• Is an action late• Is an action too early• Are a sequence of actions in the wrong order?

How can we be sure? “enough”ALARP – As Low as Reasonably Practicable (RISK)


©

Software of Unknown Provenance (SOUP)

RTOSSoftware (SOUP)

TasksTasksTasksSemaphores

Semaphores

ISRs

Kalman Filter Software (SOUP)

“Wrapper”With ‘thin’ interface

Medical Device Application

Message QueuesMessage

Queues

‘others’

Behavior managed by RTOS


©

Software of Unknown Provenance (SOUP)

RTOSSoftware (SOUP)

TasksTasksTasksSemaphores

Semaphores

ISRs

Kalman Filter Software (SOUP)

“Wrapper”With ‘thin’ interface

Medical Device Application

Message QueuesMessage

Queues

‘others’

Behavior managed by RTOSDifferent SOUPs


©

Failure conditions potentially induced by RTOS

• Data is incorrectly modified. • Incorrect results are provided to the application.• Expected results are not provided or are provided past their

deadlines.• User code is not executed as expected (not run, incorrectly

run, and incorrectly sequenced).• Fault conditions are not detected.• Fault conditions are handled incorrectly.• False failure conditions are reported.• Incorrect or untimely response provided by the RTOS to

external or user generated events.


©

Failure conditions potentially induced by RTOS

• Data is incorrectly modified. • Incorrect results are provided to the application.• Expected results are not provided or are provided past their

deadlines.• User code is not executed as expected (not run, incorrectly

run, and incorrectly sequenced).• Fault conditions are not detected.• Fault conditions are handled incorrectly.• False failure conditions are reported.• Incorrect or untimely response provided by the RTOS to

external or user generated events.


Reduce risk byUsing Certified RTOS

©

Managing Risk ‐ ISO 14971

Risk Management Plan

Identify Risks Evaluate Control Verify

Risk Management Processes

Risk Management File


©

Medical Device – Based on Risk Assessment

Develop Hardware

Develop Software

IntegrateSystem

Risk Analysis

Risks

Functional RequirementsSystem


©

System Hazard – Software Class

• System Hazard– No Injury – A– Non‐Serious injury – B– Death or Serious injury – C

• If failure in Software leads to System Hazard• Software categorizes as

– Class – A– Class – B– Class – C


ISO 14971

IEC 62304

©

Risk Analysis

• Failure Mode Analysis• Identify Potential Risks• Determine Risk Exposure

– Continuous while operational– Frequent– Occasional

• Can Software contribute to Hazards• Can Hazards be mitigated• Finding potential hazards in Software can be TOUGH!


©

Requirements Hierarchies – DO‐178B

SystemRequirements

High‐LevelRequirements

Low‐LevelRequirements

ARP 4754A

Intended System Behavior

Intended Software Behavior

DO‐178C

Requirements basedon Software Architecture


©

Requirement/Design organization

RiskManagement

Requirements

Low‐LevelDesign

ISO 14971

Intended System Behavior

Intended System/ Software Behavior

IEC 62304

Software based onRequirements


©

Parameter Data Items

• Parameter Data Items can be developed and verified separately if certain conditions are met

• The high‐level requirements describe how the software uses the parameter data items

• The low‐level requirements define the structure, attributes and allowable values of the parameter data items

• Verification should show that every data element has the correct value – or correct value is in equivalence class and boundaries are verified

e.g. Configuration Vectors


©

Our Experience

• Develop verification evidence using a DO‐178C software Lifecycle

• All of the other Lifecycle processes will fall into place.

• Some additional documents required– Safety Plan– Safety Manual– Staff Competency Assessment

• Actual assessment required not just Resume


©

More Experience

• Interpretations and negotiations are prevalent in Automotive, Medical and Rail industries.

• TUV were strict on first Verocel certification– Plans, procedures and standards approved– Subsequent certifications were straightforward– “Manufacturing – reject tracking” needed to be addressed (for software)?


©

Finding and eliminating unintended behavior

• Requirements describe intended behavior• Software developed against requirements (TRACED)• Tests written against requirements (ONLY) and (TRACED)

• Software coverage by tests measured• Any software not covered demonstrates “unintended behavior”

• This is a risk that must be eliminated.


©

Code Coverage Analysis

• Requirements used to specify intended behavior• Write tests using Requirements ONLY

– Normal Range– Robustness– Equivalence Class– Boundary Value

• Run tests and measure how much code was executed• Assess is non‐executed code should not be there

Coverage Analysis not required explicitly by IEC 62304, but

Hard to mitigate “Unintended Functionality risk” without


©

Coding Standards

• Standards Required – used to show goodness of various properties

• Code Layout• Code consistency• Readability• Avoidance of risky constructs• Etc.


©

Code Review

• Verifies Conformance to Standards• Verifies conformance to review criteria• Verifies code against intended behavior

– Low‐level Design– Low‐level requirements


©

Code Analysis Tools (The silver bullet?)

• Perform consistency checks• Perform checks against defined coding rules (e.g. MISRA C) to find errors like:– Use of variable before initialization– Indexing out of bounds (simple cases)– Potential deadlock– Unreachable code (sometimes)– Arithmetic overflow (sometimes)

Good quality step, reduction of potential faults, BUT!


©

Code Analysis Tools for Certification

• Checks are incomplete• Hard to assess what checks must be completed manually

• No analysis against intended behavior– Low‐level design– Low‐level requirements

Only partial credit may be taken!


©

Configuration Management Processes

• Identification – Versions– Baseline

• Change Control – Documented process– Change Control Board

• Configuration Accounting– History tracking– Access Controls


©

Segregation of Software Components

RTOS

APP_OneClass C

APP_TwoClass A

Components can be segregated and treated as Class C on same computer.

Fault ContainmentMemory PartitioningTime Partitioning

Required

App‐Two cannot adversely affectClass C software


©

Audit Risk

• Scenario 1– Prepare Verification evidence on paper– Instruct Engineers to give YES/NO answers

– All information available, but difficult to locate.

– Auditor cannot make good assessment

– Applicant passes with substandard/incomplete audit.

Not the Verocel Way!


©

The Verocel Approach

• Plans, QA records, CM‐data, and Certification data: – Hyperlinked data – easy to find and trace

• All data open and put on DVD‐ROM– Auditors can trace their own copies

• All data extracted from Traceability database, and CM repository


©

• Develop Plans and Standards

• Develop Certification evidence using P&S

• QA Checks that P&S are followed

• Review Plans and Standards• Sample Cert evidence• Check QA Audits

Auditors approach

If a controlled process was used consistently And Sample is good Then we can trust the rest of the certification

evidence and the software

AuditorsDevelopers/Certifiers


mentor.com/embedded

Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions.Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.

Qt is a registered trade mark of Digia Plc and/or its subsidiaries. All other trademarks mentioned in this document are trademarks of their respective owners.

Andrew CaplesJune 2014

Mentor Embedded Solutions

2mentor.com/embedded

2

Trends for Mixed Criticality Needed to meet stringent non-functional requirements

— Cost— Space— Weight— Heat generation— Power consumption

Issues— Partitioning for safety assurance— Sharing for efficient resource usage— Hard Tasks must be guaranteed— Soft Tasks given best possible service— Must ensure the behavior of low criticality components does not

adversely impact the behavior of higher criticality components


3

Memory Protected

MemoryProtected

MemoryProtected

MemoryProtected

Nucleus Processes

Memory Protected Modules— Prevents sub-systems

from bringing down the system

— No Virtual Addressing

Multiple Types— Applications— Libraries— Hybrids

Integrated with SourceryCodeBench

— Build projects via wizards— Debug / load modules— Profile module execution

File Systems

Peripheral Bus DrivesGUINetworking

Power Aware Kernel

StorageLCDEthernet/Wireless Devices

MemoryProtectedApplication 1Task 1Task 2…Task n

Library 1Function 1Function 2…Function n

Hybrid 1Task 1Function 1…Task nFunction n

Application 2Task 1Task 2…Task n


4

Nucleus Process Model

Root Kernel Image

User Process

n

User Process

2

User Process

3

Hardware

User Process

1

User Process

2


5

Foreground / Background Mode

Static Application

Root Kernel Image

User Process

n

User Process

2

User Process

3

Hardware

User Process

1

User Process

2


66

Mentor Embedded Offerings


77

Mentor Embedded

United States

Canada

United KingdomIreland

NetherlandsGermany

Denmark

SwedenFinland

PolandArmenia

Russia China

JapanKorea

Taiwan

Australia

Singapore

IndiaPakistan

IsraelEgypt

Hungary

ItalyAustriaSwitzerland

Spain

France

Brazil

400+ engineers

50+ engineers in lead OSS community roles

10,000+ accepted OSS changes

Deployed in 3 Billion+ devices

20,000+ Sourcery CodeBench users

“Since 1996, Mentor Graphics has been the only EDA vendor with a broad product line of

embedded tools and software IP. Integrating our software and hardware expertise more

readily enables our customers to deal with the challenges of today’s multicore and power

management complexities.”

Walden Rhines, CEO

Technology

Getting Your Medical Device FDA Approved