Lessons Learned in Initiating Lessons Learned in Initiating and Conducting Risk and Conducting Risk

Assessments within a Risk Assessments within a Risk Analysis Framework: A Analysis Framework: A FDA/CFSAN ApproachFDA/CFSAN Approach

Robert BuchananDHHS Food and Drug AdministrationCenter for Food Safety and Applied

Nutrition

PerspectivePerspective

Risk Assessor vs. Risk Manager

Risk Manager vs. Risk Assessor

The ParadoxThe Paradox

Codex Alimentarius “Principles and Guidelines for the Conduct of Microbial Risk Assessments” Principle #2: There should be a Principle #2: There should be a

functional separation between risk functional separation between risk assessment and risk managementassessment and risk management

Important principle that doesn’t work if you achieve it

Setting the StageSetting the Stage

Experience:Experience: conducted two large quantitative microbial risk assessments Vibrio parahaemolyticus Listeria monocytogenes

Learning:Learning: address the “bumps” encountered to discuss lessons learned to develop a risk analysis framework

for future major risk assessments

The BumpsThe Bumps Resource intensive Setting time frames and

establishing deadlines Establishing why risk assessment is

being done Establishing expectations

What a risk assessment can do? Establishing boundaries

Who is in charge? Who gets to decide? Who settles conflicts?

The BumpsThe Bumps Communicating with risk managers

(Why can’t risk assessors speak English?) Modeling Explaining uncertainty vs. variability

Lack of understanding of risk assessment techniques (Why are risk managers so dense?)

Getting risk managers to pay attention early

The BumpsThe Bumps Presenting the results

Interpretation Transparency Plain language (Who rights the report?) Dealing with unexpected results

Review of risk assessments Impact on risk management programs Protecting the risk assessors against

outside pressure

Risk Analysis Working Risk Analysis Working GroupGroup Senior managers Risk managers Risk assessors

Scientists “Modelers” Project leaders

Risk Communicators Facilitator

Risk Analysis Working Risk Analysis Working GroupGroupOverall Goal:Overall Goal:

To improve the quality and consistency of “major” risk assessments conducted by the Center for Food Safety and Applied Nutrition

What Are ‘Major’ Risk What Are ‘Major’ Risk Assessments?Assessments?

Non-routine, complex, and/or quantitative

Involve multiple program offices and/or cross-cutting in nature

Require a commitment of significant resources to complete

Risk Analysis Working Risk Analysis Working GroupGroup

Specific Tasks:Specific Tasks:1) Identify boundaries and

responsibilities of key participants in the risk analysis process

2) Develop a process for identifying and selecting risk assessments conducted by CFSAN

3) Establish procedures for the conduct of risk assessment within a risk analysis framework

Risk Analysis – The TheoryRisk Analysis – The Theory

Traditional definition: risk analysis is composed of : Risk Risk

ManagementManagement Risk Risk

AssessmentAssessment Risk Risk

CommunicatioCommunicationn

Risk Management

Risk Communication

Risk Assessment

Risk Analysis – The Risk Analysis – The RealityReality In CFSAN risk

analysis is a “stew” of : Center

Management Risk

Management Risk Assessment Risk

Communication Research Etc

Risk

MgmtRisk

Communi-cation

Risk Assess- ment

Research

CFSAN

Mgmt

FDA

Mgnt

FDA

GC

Procedures to Initiate and Procedures to Initiate and Conduct Risk AssessmentsConduct Risk Assessments

INITIATE:INITIATE: A decision-based approach to identify and select all ‘major’ risk assessments Four phase process: concept

generation, problem identification, data feasibility evaluation, disposition

CONDUCT:CONDUCT: A systematic and iterative approach to conducting risk assessments Four step process: Plan, Perform,

Review, Publish

The Identification and The Identification and Selection ProcessSelection Process

Selection of Major Risk Selection of Major Risk AssessmentsAssessments

Select major risk assessments on basis of

Center’s risk management Center’s risk management needsneeds

Center’s resourcesCenter’s resources Data feasibilityData feasibility

Personal ObservationPersonal Observation

If science is complex, there is no consensus among stakeholders, and there are multiples ways of managing the risk – consider a quantitative risk assessment

If science is straight forward, there is general consensus among stakeholders, and there is only one risk management option – do not conduct a quantitative risk assessment

Where do risk Where do risk assessment ideas come assessment ideas come from?from?

Stakeholders (public)

Follow up to completed risk assessment

Regulatory staff Management Researchers Other agencies

A Four Phase A Four Phase Selection ProcessSelection Process

Phase 1:Phase 1: Concept Concept

GenerationGeneration

Phase 2:Phase 2: Problem Problem

FormulationFormulation

Phase 3:Phase 3: Feasibility Feasibility

DeterminationDetermination

Phase 4:Phase 4: Disposition Disposition

Selection ProcessSelection Process

Concept GenerationConcept Generation

Problem Formulation

Feasibility DeterminationDisposition (selection)

Identify and maintain list of potential RM questions for which a RA would assist with policy decisionsIdentify hazards/commodities State the RM question(s)State the RA question(s)

Selection ProcessSelection Process

Concept Generation

Problem FormulationProblem Formulation

Feasibility DeterminationDisposition (selection)

Management reviews list and approves specific RAs for further evaluation (feasibility determination)

Selection ProcessSelection Process

Concept Generation

Problem Formulation

Feasibility Feasibility DeterminationDeterminationDisposition (selection)

Review literature to determine availability of data needed to conduct RA to answer RM questions

Recommend action:More research neededModify RM questionQualitative RA vs. Quantitative RA

Selection ProcessSelection Process

Concept Generation

Problem Formulation

Feasibility DeterminationDisposition (selection)Disposition (selection)

Using the results of the feasibility determination as an aid, management selects RAs to be conducted based on technical merit, resource availability, and other factors and get commitment of resources

Conducting the Risk Conducting the Risk AssessmentAssessment

Conducting the Risk Conducting the Risk AssessmentAssessment

A four step process: Step 1: Plan

Step 2: Perform

Step 3: Review

Step 4: Publish

Conducting the Risk Conducting the Risk AssessmentAssessment

Step 1: PlanningStep 1: Planning Define scope Identify resource needs Assign teams Develop timelines

Conducting the Risk Conducting the Risk AssessmentAssessment

Step 2: PerformStep 2: Perform Answer risk management

questions

Receive Charge from Risk Managers

Receive Assumptions from RMRefine Assumptions for modeling

Assemble data/model inputsVerify data/model inputs

Develop modelAudit model

Run model/iterations

Review results

New data

Draft Report

Conduct sensitivity analysis

Conducting the Risk Conducting the Risk AssessmentAssessment

Step 3:Step 3:Review – ongoing process

Advisory CommitteesPeer Review SGE PanelsPublic Comment

ApproveClear

Technical and Scientific Reviews of Technical and Scientific Reviews of the Draft Risk Assessment the Draft Risk Assessment

DocumentsDocuments

Final D ocum ent

D raft D ocum ents for Public C om m entFederal R egister N otice

D ocum ents Available on FD A/C FSAN W ebsitePublic M eeting

Internal and External R eviewD ata and Assum ptions

D raft D ocum entM odel

Request for Data and In form ationFederal R egister N otice

Public M eetingsAdvisory C om m ittee (N AC M C F)

Conducting the Risk Conducting the Risk AssessmentAssessment

Step 4: PublishStep 4: PublishDevelop “roll out” strategyPublic release of documentsHandling commentsPublic comment

Organization of Risk Organization of Risk AnalysisAnalysis Risk analysis should be

conducted by teams: Risk Management TeamRisk Management Team Risk Assessment TeamRisk Assessment Team Risk Communication TeamRisk Communication Team

Meet regularly to discussion process and surface issues

Organization of Risk Organization of Risk AnalysisAnalysis Established three unique Established three unique

positions to help with positions to help with consistency, coordination, and consistency, coordination, and making decisions making decisions Science Advisor for Risk Analysis Risk Analysis Coordinator Risk Assessment Project Manager

Summary of Recommendations Summary of Recommendations ImplementedImplemented

Decision-based approach to identify and select all ‘major’ risk assessments

Procedure for the conduct of risk assessment within risk analysis framework

Established new organizational structure

Summary of Recommendations Summary of Recommendations ImplementedImplemented

Criteria to evaluate data quality Procedures for the approval and

release of risk assessments Formal peer review process Enhance capabilities to conduct

risk assessments and interaction between chemical and microbiological risk assessments

Report AvailableReport Available

“Initiation and Conduct of All ‘Major’ Risk Assessments within a Risk Analysis Framework”

Available on FDA website: http//:www.cfsan.fda.gov/~dms/http//:www.cfsan.fda.gov/~dms/

rafw-toc.htmlrafw-toc.html

New TwistsNew Twists

OMB Guidelines on Data Quality

Court decisions of status risk assessment when used as part of a regulatory activity

Security issues when risk assessment used for food security activities