Food SafetyResearch Consortium A MULTI-DISCIPLINARY COLLABORATION TO IMPROVE PUBLIC HEALTH Ranking Pathogens in Foods for Broad Priority Setting The Foodborne

Food SafetyResearch ConsortiumA MULTI-DISCIPLINARY COLLABORATION TO IMPROVE PUBLIC HEALTH

Ranking Pathogens in Foods for Broad Priority

SettingThe Foodborne Illness Risk Ranking

Model

Michael BatzResearch Associate, Resources for the Future

[email protected], (202) 328-5020

RAC Workshop on Food and Waterborne Pathogen Risk Ranking Models: From

Policy to PracticeCollege Park, Maryland

18 August 2005

Food SafetyResearch Consortium 2

Food Safety Research Consortium Multi-disciplinary collaboration to improve

public health, focused on creating tools and analysis to foster a science- and risk-based food safety system in the United States

Member institutions / steering committee U. California at Davis (Jerry Gillespie) U. Georgia (Mike Doyle) Iowa State (Cathie Woteki) U. Maryland (Glenn Morris) U. Massachusetts (Julie Caswell) Michigan State (Ewen Todd) Resources for the Future (Mike Taylor)


Additional FIRRM Researchers

Glenn Morris (U of Maryland School of Medicine) Mike Taylor (Resources for the Future) Alan Krupnick (Resources for the Future) Sandy Hoffmann (Resources for the Future) Holly Gaff (U of Maryland School of Medicine) David Hartley (U of Maryland School of Medicine) Marisa Caipo (U of Maryland School of Medicine) Jody Tick (Resources for the Future) Diane Sherman (Resources for the Future)


What does FIRRM do?

Ranks food-pathogen combinations by public health impact 28 pathogens 13 food categories, 48 subcategories 5 measures of public health impact

Illnesses Hospitalizations Deaths Dollars QALY loss

Choice of assumptions and data sources


Some Characteristics Not a predictive model –not a risk

assessment Created in Analytica

Graphical user interface: point-and-click, drop-down menus, follow the arrows

Changeable assumptions and choices of data

Uncertainty (Monte Carlo) Relatively user friendly: takes some time

to learn, but no command-line prompts Open and free to download/use/change


Some more characteristics

Transparency Built-in documentation No secrets – the math is right there,

though it might take some work to follow the dots

Decisions make explicit uncertainties that are usually hidden or glossed over

Adaptable to new data Vetted via workshops, policy input


Why was it created? First step in priority setting Complex food system: many pathogens,

many foods, many points of contamination

Use data driven approach Compare food-pathogen vectors, not just

pathogens Determine the economic impacts of

illnesses

The Question: Which pathogen-food vectors have the most significant impacts on public health?


Phase II Development Phase I: 2002- 2003

Thanks to Robert Wood Johnson Foundation Resulted in FIRRM as presented today Significant data gaps Not ready for prime time

Phase II: 2004 - 2006 Thanks to CSREES Fill many data gaps Incorporate more uncertainty information Create web-interface Ready to inform policy


Where does FIRRM fit in? FIRRM is part of a larger conceptual framework

of priority setting tools and models CSREES Project: Prioritizing Opportunities to

Reduce Foodborne Disease 3 Regional Workshops 1 National Conference:

National Conference for Stakeholders and ExpertsSeptember 14, 2005RFF Conference Center, Washington, DC

http://www.card.iastate.edu/food_safety/national_conference/


Conceptual FrameworkRisk Ranking

Priority Setting Decision

- Purpose I: Resource allocation, research, data, etc

Conceptual Framework for Prioritizing Food Safety Interventions


Conceptual Framework

Intervention Assessment

-Cost of Interventions-Effectiveness (in terms of contamination indicators)-Cost-Effectiveness (indicator)

Risk Ranking


- Purpose I: Resource allocation, research, data, etc- Purpose II: Risk management, private intervention, etc





-Cost of Interventions-Effectiveness (indicators)-Cost-Effectiveness (indicator)

Risk Ranking

Health BenefitAssessment

-Health Outcomes-Health Valuation

CombinedAssessment

-Cost-Benefit-Cost-Effectiveness


- Purpose I: Resource allocation, research, data, etc- Purpose II: Reg. action, private intervention, etc





-Cost of Interventions-Effectiveness (indicators)-Cost-Effectiveness (indicator)

Risk Ranking

Health BenefitAssessment

-Health Outcomes-Health Valuation

CombinedAssessment

-Cost-Benefit-Cost-Effectiveness


- Purpose I: Resource allocation, research, data, etc- Purpose II: Reg. action, private intervention, etc

Post HocEvaluation

Data Collection



How is the model structured? Incidence Estimates

National Maryland

Health Valuation Economic QALY

Food Attribution Based on outbreak data Based on expert judgment Based on risk assessments and other data

Rankings


Module 1: IncidenceNational estimates based on Mead et al. (1999)

Reported illnesses multiplied by underreporting factors

Similar underreporting factors for hosps, deaths Foodborne is percent of Total illness (for each path)

FIRRM adaptations: Uncertainty as probability distributions Alternate multipliers, hospitalization & fatality rates Estimates for Maryland based on FoodNet laboratory

data (two years only of stripped, summarized data)

Mead, P. S., L. Slutsker, V. Dietz, et al., Food-Related Illness and Death in the United States, Emerging Infectious Diseases (1999), 5, 607-625.


Incidence in Phase II

Year-by-year data Add more years through 2003 New underreporting factors

Now: based on Mead (1999), single factor per pathogen

Soon: based on Voetsch et al. (2004), a three-tiered approach (at least for FoodNet paths)

Voetsch, A. C., T. J. V. Gilder, F. J. Angulo, et al., FoodNet Estimate of the Burden of Illness Caused by Nontyphoidal Salmonella Infections in the United States, Clinical Infectious Diseases (2004), 38, S127-134.


Module 2: Valuation Aggregate measure Economic impact of disease Useful for later cost-benefit Create outcome trees for each

pathogen to capture symptoms, severities, treatments

Compute dollars & QALYs for each health state


Health Outcome Tree (example)

65% are mild cases and recover fully1,300 cases

35% are severe cases

700 cases

50% do not visit a physician and recover fully5,000 cases

30% visit a physicianand recover fully3,000 cases

20% are hospitalized2,000 cases

55% recover fully385 cases

25% chronic sequelae175 cases

20% die in first year140 cases

Total cases of Pathogen A

10,000 cases


Module 2: Valuation (cont’d)

Economic valuation Cost of Illness (COI) – morbidity Willingness to Pay (WTP) – mortality (VSL) COI values drawn primarily from ERS studies WTP values drawn from literature

Quality Adjusted Life Years (QALYs) Quantify based on scale of 0 to 1 Values drawn from surveys Subtract from baseline & multiply by duration Numerous health indices available (QWB, HUI, EQ5D) FIRRM currently uses Quality of Well Being (QWB)

index


Chronic sequelae in Phase I FIRRM Campylobacter

Guillain-Barre Syndrome (GBS) Hospitalized with and without ventilation Eventual recovery (return to work) Permanent disability (never return to work)

E. coli O157:H7 Hemolytic Uremic Syndrome (HUS)

Dialysis and transplants Kidney transplants Premature death

Listeria monocytogenes Stillbirths and newborn deaths Mild, moderate, and severe retardation

Nontyphoidal Salmonella No chronic sequelae


Valuation in Phase II 8 additional pathogens:

Cyclospora Cryptosporidium Shigella Vibrio vulnificus Vibrio parahaemolyticus & other marine Vibrios Yersinia enterolotica Norovirus Toxoplasma gondii

Additional chronic sequelae Reactive arthritis Irritable Bowel Syndrome

New QALY index Probably will use EQ-5D (EuroQoL)


Impact of VSL on Valuation

Mean Estimates

Total Costs (Millions 2001$)Campylobacter 1,203 1,754Escherichia coli O157:H7 162 396Listeria monocytogenes 573 3,131Salmonella nontyphoidal 906 3,045

Costs per Case (2001 $)Campylobacter 613 894Escherichia coli O157:H7 2,601 6,334Listeria monocytogenes 229,800 1,256,000Salmonella nontyphoidal 675 2,269

VSL = $5M (Viscusi Midpoint)

VSL = $1.66M (Landefeld & Seskin)


Module 3: Food AttributionFor each pathogen, apply percent of total

due to each food categoryNo ideal data sourcePrimary data options:

Outbreak data Expert elicitation FDA/USDA Listeria risk assessments “Shorthand” risk assessment approach

(consumption/contamination)Two-tier food categorization (eg. seafood/

finfish)


Module 3: Food Attribution

Outbreak data (1990-2000) CSPI compilation: mostly CDC data (88%) Approx 2000 outbreaks & 80,000 cases Percents based on cases summed across

years Expert Elicitation

Mail survey, peer-reviewed set of respondents

101 contacted, 45 completed 11 pathogens Best estimates, also low/high estimates Self-assessed expertise, confidence in

answers


Food Attribution for Campy.

Campylobacter spp.Seafood 9% 1%Eggs 0% 3%Produce 39% 5%Beverages 0% 0%Dairy 21% 7%Breads and Bakery 0% 0%Game 0% 2%Beef 5% 4%Poultry 16% 65%Pork 1% 9%Luncheon/Other Meats 2% 1%Unattributable/Other 7% 2%Total 100% 100%

Outbreak Data

Expert Elicitation


Food Attribution in Phase II Update food categories Update outbreak data

Add years: 2001 - 2003 Focus on CDC line listings Allow user to choose which years to use

Expand incorporation of expert elicitation

Incorporate FoodNet case-control studies Further develop “shorthand” risk

assessment approach based on food consumption and contamination data


Interface 1

Insert screen grab: main model screen

Say: open the model and interact by double-clicking… double-click on ‘model interface’ to run some scenarios


Interface 2

Insert screen grab: main interface screen


Ranking by Dollars

Pathogen-Food Combination1 Listeria monocytogenes / Luncheon - Other Meats 1,074 990 215 691.0 3,7892 Listeria monocytogenes / Dairy - Milk 680 627 136 437.5 2,3993 Salmonella nontyphoidal / Eggs - Egg Dishes 362,707 4,219 149 434.5 3,8924 Campylobacter / Produce - Vegetables 488,604 2,623 26 346.6 2,1655 Campylobacter / Dairy - Milk 380,995 2,045 20 270.2 1,6886 Listeria monocytogenes / Luncheon - Luncheon Meats 355 327 71 228.2 1,2527 Campylobacter / Poultry - Chicken 283,565 1,522 15 201.1 1,2578 Campylobacter / Produce - Produce Dishes 213,764 1,148 11 151.6 9479 Salmonella nontyphoidal / Produce - Vegetables 93,288 1,085 38 111.7 1,00110 Listeria monocytogenes / Breads - Bakery 158 145 32 101.4 55611 Escherichia coli O157:H7 / Beef - Ground Beef 23,838 703 20 88.5 76512 Salmonella nontyphoidal / Poultry - Chicken 72,871 848 30 87.3 78213 Campylobacter / Seafood - Seafood Dishes 119,243 640 6 84.6 782

QALYCases Hosps Deaths2001 $ (Mill)

These rankings are provided as an example. They are based on midpoint values and were computed in 2003 using default model settings, including a VSL of $2.2M and attribution based on outbreak data, among other assumptions. Only four pathogens are currently valued in dollar or QALY terms.


Ranking by Deaths

Pathogen-Food Combination1 Toxoplasma gondii / Unattributable Food 112,500 2,500 375 -- --2 Listeria monocytogenes / Luncheon - Other Meats 1,074 990 215 691.0 3,7893 Salmonella nontyphoidal / Eggs - Egg Dishes 362,707 4,219 149 434.5 3,8924 Listeria monocytogenes / Dairy - Milk 680 627 136 437.5 2,3995 Listeria monocytogenes / Luncheon - Luncheon Meats 355 327 71 228.2 1,2526 Salmonella nontyphoidal / Produce - Vegetables 93,288 1,085 38 111.7 1,0017 Listeria monocytogenes / Breads - Bakery 158 145 32 101.4 5568 Salmonella nontyphoidal / Poultry - Chicken 72,871 848 30 87.3 7829 Salmonella nontyphoidal / Poultry - Turkey 69,342 807 28 83.1 74410 Salmonella nontyphoidal / Poultry - Chicken Dishes 68,590 798 28 82.2 73611 Salmonella nontyphoidal / Produce - Fruits 65,485 762 27 78.4 70312 Escherichia coli nonO157 STEC / Unattributable Food 31,229 921 26 -- --13 Campylobacter / Produce - Vegetables 488,604 2623 26 346.6 2,165

QALYCases Hosps Deaths2001 $ (Mill)

These rankings are provided as an example. They are based on midpoint values and were computed in 2003 using default model settings. Note that Toxoplasma and E coli STEC do not have enough outbreaks in the attribution dataset to estimate food-pathogen combinations.


Phase II Tasks Already mentioned

Update & improve incidence estimates More pathogens valued Different QALY index Update & improve food attribution

Treatment of uncertainty Incorporate more variance information Uncertainty and sensitivity analysis Importance assessment

Web-based model Simplified interface Ability to save changes, compare different runs Contract with Enrich Consulting


Conclusions Lots of uncertainty

Underreporting multipliers in incidence estimates Mortality valuation Food attribution estimates

Largest data gaps in food attribution Valuation changes ranking Norovirus and Toxoplasma are important Preliminary results are useful for priority setting

For more information about the Foodborne Illness Risk Ranking Model, or to download a draft version:http://www.rff.org/fsrc/firrm.htm


Appendices


Analytica

Modeling environment developed primarily for risk and decision analysis

Visual modeling framework Hierarchical influence diagrams Point and click interaction

Embedded uncertainty Inputs as probability distributions Monte Carlo simulation to propagate

uncertainties


Why Analytica Transparency

Data and math is explicitly visible; documentation of sources and assumptions

Flexibility and adaptability Visual programming means fast development;

modular; collaborative tool; easy to expand and/or change;

Accessibility modest software costs; distribution; web

interface; Ease of use

Drop down menus allow to easily change assumptions; don’t have to be an expert or programmer to use it.


Key Activities

Map current information landscape for food safety

Convene public and private stakeholders to establish guiding principles

Identify key obstacles to sharing existing data and strategies for resolution

Develop cost effective strategies and priorities for collecting new data

Establish mechanisms for data housing and dissemination

Maintain links to decision processes at all levels


Food CategoriesMajor category Sub-category Major category Sub-category

Finfish BreadsMolluscan Shellfish BakeryOther Seafood Breads and Bakery ComboSeafood Dishes Game GameSeafood Combo Ground BeefEggs Other BeefEgg Dishes Beef DishesEggs Combo ChickenFruits TurkeyVegetables Other PoultryProduce Dishes Chicken DishesProduce Combo Turkey DishesJuices HamOther Beverages Other PorkBeverage Combo Pork DishesMilk Luncheon MeatsCheese Other MeatsIce Cream Other Meat DishesOther Dairy USDADairy Combo FDASalads Both USDA/FDARice/Beans/Stuffing/Hot Pasta Dishes Unattributable Unattributable and OtherSandwichesSauces/Dressings/OilsOther FoodsMulti-Ingredient Combo

SeafoodBreads and Bakery

BeefEggs

PoultryProduce

Beverages Pork

Dairy

Luncheon/ Other Meats

Multi-Source

Multi-Ingredient


Food Attribution: E. coli O157:H7

Escherichia coli O157:H7Seafood 1% 0% 0%Eggs 0% 0% 0%Produce 15% 18% 63%Beverages 4% 3% 0%Dairy 2% 4% 8%Breads and Bakery 0% 0% 0%Game 0% 2% 0%Beef 66% 64% 27%Poultry 0% 1% 1%Pork 0% 1% 0%Luncheon/Other Meats 3% 2% 0%Unattributable/Other 9% 6% 0%

CSPI Outbreak Data Expert Elicitation

Contamination & Consumption

Data


Food Attribution: Salmonella

Salmonella nontyphoidalSeafood 2% 3% 0%Eggs 31% 20% 0%Produce 16% 8% 31%Beverages 2% 2% 0%Dairy 7% 7% 59%Breads and Bakery 4% 0% 0%Game 0% 1% 0%Beef 6% 10% 1%Poultry 16% 37% 7%Pork 6% 5% 1%Luncheon/Other Meats 4% 2% 0%Unattributable/Other 7% 3% 0%

CSPI Outbreak Data

Expert Elicitation


Data


Food Attribution: Toxoplasma

Toxoplasma gondiiSeafood 0% 1% 0%Eggs 0% 0% 0%Produce 0% 5% 0%Beverages 0% 0% 0%Dairy 0% 1% 0%Breads and Bakery 0% 0% 0%Game 100% 15% 0%Beef 0% 18% 0%Poultry 0% 3% 0%Pork 0% 37% 0%Luncheon/Other Meats 0% 2% 0%Unattributable/Other 0% 19% 0%

CSPI Outbreak Data

Expert Elicitation


Data


Method: Expert Elicitation Survey

Peer-reviewed list of potential respondentsMail survey with phone follow-upSurvey Content

Developed in collaboration with national expert on elicitation (Paul Fischbeck, CMU)

Respondent background information Respondent self-evaluation of level of expertise Quantitative (%) attribution with best judgment

and upper and lower bounds Reporting on information used in response


Example of Expert Elicitation Survey

Part 1

Campylobacter spp.

Likely to be a source?

Best Estimate

Low Estimate

High Estimate

100%

Food Category

Percent of U.S. Foodborne Cases in a Typical Year

Seafood

Eggs

Produce

Beverages (not water)

Dairy

Breads and Bakery

Game

Beef

Poultry

Pork

Luncheon/Other MeatsOther


Example of Expert Elicitation Survey

Part 2

Questions:

1. What information did you principally rely on to fill out this table?(list all that apply)

general knowledge

your own research or clinical experience

specific journal articles, data sets, or other specific professionalpublications. Please list:

3. Is there a factor other than the type of food that determinesWhether a food is associated with illnesses caused by this pathogen? For example, for listeria, it may be that illnesses are associated withrefrigerated foods regardless of whether the food is dairy or poultry.

2. Please give your best estimate of the total number of foodbornecases of illness caused by this pathogen in a typical year.

Provide a brief description of your reasoning.


Rankings Sorted by Cases

Pathogen-Food Combination Cases Hosps Deaths 2001 $ QALYNorwalk-like viruses / Seafood - Molluscan Shellfish 1 2 19 NA NANorwalk-like viruses / Multi-Ingredient - Salads 2 3 24 NA NANorwalk-like viruses / Produce - Produce Dishes 3 4 25 NA NANorwalk-like viruses / Produce - Fruits 4 8 33 NA NANorwalk-like viruses / Produce - Vegetables 5 10 38 NA NACampylobacter / Produce - Vegetables 6 5 13 4 4Norwalk-like viruses / Breads and Bakery - Bakery 7 14 44 NA NANorwalk-like viruses / Multi-Ingredient - Sandwiches 8 16 45 NA NACampylobacter / Dairy - Milk 9 7 21 5 5Salmonella nontyphoidal / Eggs - Egg Dishes 10 1 3 3 1Norwalk-like viruses / Beverages - Other Beverages 11 28 56 NA NACampylobacter / Poultry - Chicken 12 9 30 7 6


Ranked by Dollars VSL Comparison

Landefeld and Seskin

(Adjusted Max $1.66M)

Mrozek and Taylor

(Adjusted $2.39M)

Viscusi Midpoint

(Unadjusted $5M)

EPA (Adjusted $6.49M)

Campylobacter / Produce 1 2 4 4

Listeria monocytogenes / Luncheon/Other Meats 2 1 1 1

Salmonella nontyphoidal / Eggs 3 4 3 3

Campylobacter / Dairy 4 5 7 7

Campylobacter / Poultry 5 8 8 9

Listeria monocytogenes / Dairy 6 3 2 2

Salmonella nontyphoidal / Poultry 7 6 5 5

Salmonella nontyphoidal / Produce 8 7 6 6

Campylobacter / Seafood 9 10 15 15

Escherichia coli O157:H7 / Beef 10 9 9 8


Data Challenges

Utility of models requires multiple types of data: Attribution of illnesses to foods Effectiveness and cost of interventions Link between interventions and health outcome

Data is already collected but spread out Federal and state agencies Food industry Academic researchers

Focused effort needed to access and use existing data and fill critical gaps


The Data Opportunity

Key agencies embrace the systems approach to food safety and the need to set priorities

Data needs have been highlighted by the NAS, GAO, and FSRC

Technical tools now exist to collect and manage the needed data

Fragmentation of the data “system” is a recognized problem


A Food Safety Information Infrastructure is Needed To... Build buy-in on system goals, data

needs, and priorities Make certain that the right questions

are being asked Assure technical compatibility of data

systems Assure data access and sharing Provide information at all levels to

decision makers

Documents

Food SafetyResearch Consortium A MULTI-DISCIPLINARY COLLABORATION TO IMPROVE PUBLIC HEALTH Ranking Pathogens in Foods for Broad Priority Setting The Foodborne