Assessment Methodology Unit Scientific Cooperation …securite-alimentaire.public.lu/actualites/evenements/2009/06/2e... · Assessment Methodology Unit Scientific Cooperation and

1

When go quantitative in risk assessment

Didier VerlooAssessment Methodology Unit

Scientific Cooperation and Assistance Directorate

La sécurité dans mon assiette, OSQCAJune 3, 2009

OSQCA, 3 June 20092

EU regulation

In order to achieve the general objective of a high level of protection of human health and life, food law shall be based on risk analysis except where this is not appropriate to the circumstances or the nature of the measure.

OSQCA, 3 June 20093

o Three components of Risk Analysis

o Risk management (the decision making)

o Risk assessment

o Risk communication

Risk Analysis

OSQCA, 3 June 20094

o What specific questions do we want the risk assessment to answer?o What assessment assumptions are we willing to accept?o What can be done to reduce the impact of unwanted events?o What can be done to reduce the likelihood of unwanted events?o What are the trade-offs involved among risk management options?

o Which is the best option?

Risk management is the work required to adequately answer these questions:

The decision maker

Risk management

OSQCA, 3 June 20095

o With whom do you communicate?o What are their agendas?o What do people know about the risks and how? What do they want to

know?o How do you get both the information you need and the information others

have?o How and when do you bring the information you want to communicate?

Risk communication is the work required to adequately answer these questions:

Risk communication

OSQCA, 3 June 20096

o What can go wrong (hazard)?o How can it happen (drivers)?o How bad can it be (impact)?o How likely is it (probability)?o Would the incorrectness of any assumption change the management decision,

and if so can we test it?o What effect would the risk management options have?o Would the practical collection of further data alter a decision?o How much would the action(s) or inaction cost and to whom (or is this Risk

Management?)?

Risk assessment is the work required to adequately answer these questions:

Risk assessment

Ref: Vose 2000

OSQCA, 3 June 20097

Risk assessment: codex (tox)

• Hazard identification– Identification of biological, chemical and physical agents capable of causing

adverse health effects which may be present in particular food or groups of food

• Hazard characterisation– The evaluation of the nature of the adverse health effect (dose-response)

• Exposure assessment– Evaluation of likely intake or exposure from other sources

• Risk characterisation– Estimation of the probability of occurrence and severity of the adverse

health effects given all above

OSQCA, 3 June 20098

o Provides method to the decision maker to make decision under uncertainty that is

o Logical

o Science based

o Transparent

o Reproducible

o Provided within the asked (usually short) timeframe

Risk assessment rules of thumb

OSQCA, 3 June 20099

EFSA MISSION

Provide independent scientific advice for

European Community’s (EC) legislation and policies in fields with impact on

Food Safety, Nutrition, Feed Safety, Plant health and Plant protection, Animal Health

and Animal Welfare

OSQCA, 3 June 200910

From “question” to “answer”

European Commission

European Parliament

Member States

EFSA (“self mandate”)

Question?

Risk Assessment

Opinion

RiskManagement

Risk Communication

Industry

Media

Consumers

Professionals


EFSA RemitFounding Regulation Reg. 178/2002

• Scientific opinions (artt. 29 and 30)• Uniform risk assessment methodologies (art.

23)• Scientific and technical assistance to Commission

(art. 31)• Scientific studies (art. 32)• Data collection (art. 33)• Emerging risks (art. 34 & 35) and Rapid alert

systems (art. 35)• Networking of organisations operating in the

field of EFSA’s mission (art. 36)


Insert new organigram


• Not Risk Assessments - Remit of Panels (exceptpesticide safety review)

• SCA provides support to Risk Assessment Units/Panels (exposure, data collection and analysis, methodology)

• Scientific cooperation with MS• Shared best practices with Scientific Panels

– Working groups: Selection of Experts– Transparency: Declaration(s) of Interest – Openness: Reports on the web

SCA: Scientific Cooperation and Assistance: Modus operandi


Data collection and analyses: activities

• Data Collection and analyses– strategy drafted– Operating Procedures developed– Harmonisation of terminology onging– Harmonisation of data collection and transmission

standards ongoing

• Areas for data collection and analyses

– Zoonoses

– food consumption

– chemical occurence

– pesticide residues


Insert new organigram


What AMU does

1. Collates and summarizes data from scientific literature and existing databases

2. Evaluates and revises statistical and modelling methods used in risk assessments

3. Carries out and supports epidemiological and statistical data analyses

4. Develops quantitative risk assessment and quantitative decision support tools for risk managers

5. Contributes to the development and application of new or refined risk assessment approaches


Methodological support to Commission/panels/units

• Epidemiology– Bluetongue (Art 31 support to COM)– Colony Collapse Disorder in bees

• Statistical/modelling support– AFC food colours (South Hampton)– PPR Q10– Zoonoses baseline studies– GMO MON863– AHAW Tuberculosis in deer– Cadmium– Pinewood nematodes

• Data management– PPR acute tox data birds and mammals– AFC/GMO/PRAPeR internal workload tracking databases– Data management support to all statistical/modelling activities

• Scientific support– SCAF literature review cloning and nanotechnology– Systematic review guidelines– Aspartame– Isoflavones


The AMU team

• 13 persons today• Different backgrounds (veterinary epidemiology,

mathematics, statistics, toxicology, nutrition, chemistry, data management, librarian)

• In 2008 AMU had projects with ALL panels and units in EFSA

• Multidisciplinary• ‘Methodological’ is common denominator

19

RISK


Ability to define what may happen in the futureand to choose among alternatives lies at the heartof contemporary societies

allocating wealth to safeguard public healthbuy a housewear a seatbeltpaying insuranceplay the lottery

A brief history of risk


Serious study of Risk began during the Renaissance

although it is rooted in (and wouldn't have been possible without) the Hindu-Arabic numbering system which reached the West 800 years ago

people broke loose from constraints of the past and subjected to beliefs of open challenge

before that future was a whim of the Gods

Blaise Pascal and Pierre de Fermat theorie of probability, the mathheart of the concept of Risk (based on a gambling question from leChevalier de Méré, 1654)

A brief history of risk


"Risk" early italian "risicare" = "to dare“

In this sense risk is a choice rather than a fate


o Provides method to the decision maker to make decision under uncertainty that is (should be)

o Logical

o Science based

o Transparent

o Reproducible

o Provided within the asked (usually short) timeframe

Risk assessment


Different categories of RA

Qualitative Quantitative?

models


o Is in fact a good review on everything known about the risko Based on literature, brainstromings, interviewso Discuss pathogenecity, infection routes, survivability and growth of the

pathogeno Discuss infection doses, susceptible populationso Visualize possible pathwayso Discuss economic?, environmental, public health, ..., effects of the risko Discuss the level of scientific knowledge

o Focused on the question of the decision makero Should be the start of any RA processo Can be done within a relative short timeframe

Qualitative risk assessment


o Assess the probability of a risko Let the problem drive the analysis o Probability theory, usually modelled through simulationo Requires good data (prior information)

o PH: Requires good epidemiologic data and/or make use of results of good epidemiologic studies

o Math complex, might be stat heavyo Based on assumptions !!!

o Box 1976: "All statistical models are wrong but some are useful" is also applicable for Quantitative RA models

o Provides most objective assessment given sufficient good data

Quantitative risk assessment


o Model is a system of connected probability distributionso Probability theoryo Conditional probabilities!o Knowledge of stochastic processes

o binom, hypergeo, poisson, multinomial, normal, weibull,...o Uncertainty about distribution parameters is superimposed

o Good applied statisticso Statistics: Frequentist, Bayesian, Bootstrapo Expert opinion is not a replacement for data

o Distinction between uncertainty and variabililty



Uncertainty and Variability

"Variability is a phenomenon in the physical world to be measured, analysed and where appropriate explained. By contrast, uncertainty is an aspect of knowledge"

Sir David Cox

o Variability is the effect of chance and is a function of the system1o Uncertainty is the assessor's lack of knowledge about the parameters

that characterise the physical system to be modelled1

1Risk Analysis, a quantitative guide. David Vose, Wiley 2nd edition



Monte Carlo simulation: the motor

• Iterative process• Random samples drawn from distributions• Each iteration the calculation is made and output is

stored• n=n+1• If n< a large number then go back to start• Plot distribution of output


Output can be shown as a relative distribution …

1998 nominal mean number of people with enteric, non-bloody diarrhea fluoroquinolone-resistance Campylobacter infection from

chicken that receive fluoroquinolone as treatment

0 2000 4000 6000 8000 10000 12000

Rel

ativ

e co

nfid

ence


… or as a cumulative distribution

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2,000 4,000 6,000 8,000 10,000 12,000



Cum

ulat

ive

conf

iden

ce


MC simulation versus versus calculation with point estimates or ranges

• Realistic as values which are more likely to occur will be sampled more

• No focus on WCS



0 2000 4000 6000 8000 10000 12000

Rel

ativ

e co

nfid

ence


Data

Parameters

epi/stat

Risk Assessment

What we see

What we might see

(simulate orcalculate)

What we estimate

What has beenestimated

Communicate to RM Decision

What is known

What is thought

Variability Uncertainty(+ sometimes Variability)

Uncertainty+Variability

RA and epidemiology/statistics


QMRA an example of MC FTF modelling

data data data data datastat stat stat stat stat

p q r s t

QMRA model

Outputnr of cases

Scenario analysisInformed, transparent, science basedconclusions


Monte Carlo FTF modelling

• The backbone is the risk pathway, different compartments– Farm,…, slaughterhouse,…, retail,…, kitchen,…, exposure,…, cases

• Parameters different compartments– statistical analysis of epidemiological/experimental data– expert opinion

• FTF model represents the probabilistic relationship within and between compartments

• Monte Carlo simulation– probability model calculated repeatedly– values sampled from the input parameter distributions

• Outcome (number of cases)

• Assessing the influence of input parameters on the output, scenario analysis

Predict

Understand and predict


Combining simulation and statistics: MCMC

Markov Chain Monte Carlo simulation (MCMC)

• Bayesian approach• Incorporate the statistics with the probabilistic model• In fact it becomes on big (rather complicated) statistical

model• Parameter could be updated through different data

sources occurring before or after in the chain (bi directional and interactive inference)


Bayesian network

data data data data data

p q r s t

nr of cases

data

Bayesian network


Advantages of BN

• Bi directional inference possible• MC simulation over a BN is straightforward (scenario

simulation)• Move from risk assessment to decision analysis in food

safety– Inclusion in the model of an appropriate utility function– Model risk management actions that could be taken– Possible to determine the conditional distribution of relevant

parameters and outcomes given a certain decision.– Increase the transparency of the decision making process and

optimize the model as a decision making tool for risk managers


But…

• Computationally intensive algorithms• With the software available today

– not realistic that a complex farm to fork model could be modelled as one gigantic Bayesian network

– approach would be (is) the use of different modules, each or some modules being a Bayesian network which are connected to each other by ‘simple’ unidirectional Monte Carlo simulation

– Data available within a specific Bayesian network compartment would influence the relevant parameters within the compartment but only influence the outcome of the other compartments to the right


Qual to quant: how far to go?

• Roughly based on:– EFSA Scientific Committee opinion

Uncertainties in Dietary Exposure AssessmentRequest No EFSA-Q-2004-019Adopted on 14 December 2006

NOT AN EFSA OPINION BUT A PERSONAL VIEW


How far to go: ‘tiered’ approach

• Qualitative

• Quantitative

Increasing complexity

Decision

Decision

Decision

Decision


Key decision questions to increase complexity of the RA

• Should we do it?

• Can we do it?


Questions to be answered before increasing complexity

Should we do it? interaction with RM– Is it needed to come to a RM conclusion?– Could it alter the RA and RM conclusion?– Will it be used?– How will it be used?– Can RM accept the assumptions?

Can we do it?– Data– Deadlines– Resources– Guidelines or start from scratch?– When to stop? (value of information could it alter the RA and RM

conclusions)– Can we communicate it? (how will it be used?)


Thank you for your attention

Documents

Assessment Methodology Unit Scientific Cooperation …securite-alimentaire.public.lu/actualites/evenements/2009/06/2e... · Assessment Methodology Unit Scientific Cooperation and