2003 National Conference on 2003 National Conference on Tobacco or HealthTobacco or Health

Thursday, 11 December 2003: 1:30 pm – 3:00 pm

Boston Marriott Copley Place Regis Suite

EVAL-143: Local Evaluation and Surveillance Programs

Local Evaluation and Surveillance Local Evaluation and Surveillance ProgramsPrograms

Thursday, 11 December 2003Thursday, 11 December 2003

1:30 pm – 3:00 pm1:30 pm – 3:00 pm

Marriott Copley Place Regis SuiteMarriott Copley Place Regis Suite

Session OverviewSession Overview

Seth Emont, Ph.D.* & Jeff Soukup**Seth Emont, Ph.D.* & Jeff Soukup***White Mountain Research Associates, L.L.C.*White Mountain Research Associates, L.L.C.

**Tobacco Free Nebraska, Nebraska Health and Human **Tobacco Free Nebraska, Nebraska Health and Human ServicesServices

Evaluating a Tobacco Control Program with Uncertain Evaluating a Tobacco Control Program with Uncertain Funding LevelsFunding Levels

Session SpeakersSession Speakers

Argelia Figueroa, MSc & Scott Pollack, MSArgelia Figueroa, MSc & Scott Pollack, MS

Tobacco Use Prevention ProgramTobacco Use Prevention Program

Georgia Division of Public HealthGeorgia Division of Public Health

Georgia Local Tobacco Ordinance StudyGeorgia Local Tobacco Ordinance Study

Session SpeakersSession Speakers

Thomas Melnik, DrPHThomas Melnik, DrPH

Division of Chronic Disease Prevention & Adult HeathDivision of Chronic Disease Prevention & Adult Heath

New York State Department of HealthNew York State Department of Health

Local Level Tobacco Surveillance in New York StateLocal Level Tobacco Surveillance in New York State

Session SpeakersSession Speakers

Jim Chen, MD, PhDJim Chen, MD, PhDDepartment of PediatricsDepartment of Pediatrics

Wayne State University School of MedicineWayne State University School of Medicine

State-Specific Smoking Prevalence Affecting State-Specific Smoking Prevalence Affecting National Trends in the U.S., 1990-2001National Trends in the U.S., 1990-2001

Session SpeakersSession Speakers

Evaluating a Tobacco Control Evaluating a Tobacco Control Program with Uncertain Funding Program with Uncertain Funding

LevelsLevels

Jim ChenJim Chen

Peter FinnPeter Finn

Sherry MillsSherry Mills

Tandi NjobeTandi Njobe

Abt Associates Inc.Abt Associates Inc.

Cheryl WieseCheryl Wiese

Jeff WillettJeff Willett

Ian NewmanIan Newman

Seth Emont (consultant)Seth Emont (consultant)

University of Nebraska University of Nebraska - Lincoln- Lincoln

Jeff SoukupJeff Soukup

Judy MartinJudy Martin

Tobacco Free NebraskaTobacco Free Nebraska

NE Health & Human NE Health & Human Services SystemServices System

Evaluating a Tobacco Control Program with Evaluating a Tobacco Control Program with Uncertain Funding LevelsUncertain Funding Levels

Learner ObjectivesLearner Objectives

At the end of the presentation, attendees will be able to At the end of the presentation, attendees will be able to plan and implement a scaled-down evaluation, which plan and implement a scaled-down evaluation, which utilizes:utilizes:

data-driven briefsdata-driven briefsevaluation technical assistance, andevaluation technical assistance, and““built-in” monitoring strategies to track the impact of local built-in” monitoring strategies to track the impact of local tobacco control programming.tobacco control programming.

Tobacco Free NebraskaTobacco Free NebraskaEvaluation Advisory PanelEvaluation Advisory Panel

Monica EischenMonica Eischen, Office on Smoking and Health, Centers , Office on Smoking and Health, Centers for Disease Control and Preventionfor Disease Control and PreventionPebbles Fagan, Ph.D.Pebbles Fagan, Ph.D., National Cancer Institute, , National Cancer Institute, National Institutes of HealthNational Institutes of HealthJean Forster, Ph.D.Jean Forster, Ph.D., Director, Prevention and Policy , Director, Prevention and Policy Program, University of MinnesotaProgram, University of MinnesotaKaren Gerlach, Ph.D.Karen Gerlach, Ph.D., Senior Program Officer, The , Senior Program Officer, The Robert Wood Johnson FoundationRobert Wood Johnson FoundationEric WiesenEric Wiesen, Office on Smoking and Health, Centers for , Office on Smoking and Health, Centers for Disease Control and PreventionDisease Control and Prevention

Tobacco Free Nebraska:Tobacco Free Nebraska:Program GoalsProgram Goals

Prevent the initiation of tobacco use among Prevent the initiation of tobacco use among young peopleyoung peopleEliminate exposure to second hand smokeEliminate exposure to second hand smokeIdentify and eliminate disparities related to Identify and eliminate disparities related to tobacco usetobacco useTo promote tobacco use cessation among To promote tobacco use cessation among adults and youthsadults and youths

Tobacco Free Nebraska:Tobacco Free Nebraska:Program Summary (continued)Program Summary (continued)

Source of Chart: A Broken Promise to Our Children, The 1998 State Tobacco Source of Chart: A Broken Promise to Our Children, The 1998 State Tobacco Settlement Five Years Later. American Heart Association, Campaign for Settlement Five Years Later. American Heart Association, Campaign for Tobacco Free Kids, American Cancer Society, American Lung Association, Tobacco Free Kids, American Cancer Society, American Lung Association, November 12, 2003 (page 67).November 12, 2003 (page 67).

$405,000 (3.0%)

Tobacco Free Nebraska:Tobacco Free Nebraska:Program Summary (continued)Program Summary (continued)

Source of Chart: A Broken Promise to Our Children, The 1998 State Tobacco Source of Chart: A Broken Promise to Our Children, The 1998 State Tobacco Settlement Five Years Later. American Heart Association, Campaign for Tobacco Free Settlement Five Years Later. American Heart Association, Campaign for Tobacco Free Kids, American Cancer Society, American Lung Association, November 12, 2003 (page Kids, American Cancer Society, American Lung Association, November 12, 2003 (page 67).67).

0.40

Program Components Program Components •Community/School-Community/School-based based

interventions interventions •Policy initiatives Policy initiatives •Media-based Media-based interventions interventions •Statewide Quitline Statewide Quitline •Outreach to Outreach to racial/ethnic racial/ethnic minorities minorities

•Youth empowerment Youth empowerment (“No Limits”)(“No Limits”)

Data Sources (Existing) Data Sources (Existing) •Youth Risk Behavior Survey Youth Risk Behavior Survey •Adult Behavior Risk Factor Survey Adult Behavior Risk Factor Survey •Youth Tobacco Survey Youth Tobacco Survey •Adult Tobacco Survey Adult Tobacco Survey •School Administrator Survey School Administrator Survey •Social Climate Survey Social Climate Survey •Pregnancy Risk Assessment Monitoring Pregnancy Risk Assessment Monitoring System System

•NCIAA/Youth Access Compliance DataNCIAA/Youth Access Compliance Data•TRAIN TRAIN •Media print monitoring - UNLMedia print monitoring - UNL

Data Sources (in Development)Data Sources (in Development) •Quitline database Quitline database

Measures Measures •Tobacco use Tobacco use •Quit attempts Quit attempts •ETS exposure ETS exposure •Policies/ordinances Policies/ordinances (state, county, local, (state, county, local, workplace) workplace) •Attitude/risk perceptionAttitude/risk perception

Implementation/Implementation/Mediating FactorsMediating Factors

•Level of exposure to Level of exposure to TFN programs TFN programs

•Compliance with CDC Compliance with CDC Best Practices Best Practices

•County-specific County-specific variations of model variations of model

•Level of policy Level of policy tracking and tracking and

enforcementenforcement

External InfluencesExternal Influences •Budget allocations to TFN and Budget allocations to TFN and staffingstaffing

•Tobacco control advocacy Tobacco control advocacy •Social climate Social climate •Effect of other agencies’ Effect of other agencies’ programs and Federal policies programs and Federal policies (e.g., RWJF, HCFA, ALA, CDC) (e.g., RWJF, HCFA, ALA, CDC)

Documenting Achievement of TFN Goals Documenting Achievement of TFN Goals Short-Term GoalsShort-Term Goals

•TFN Program growth (e.g., implementation of TFN Program growth (e.g., implementation of program components) program components) •Youth empowerment Youth empowerment •Public/stakeholder awareness of TFN programs Public/stakeholder awareness of TFN programs •Establishing local infrastructure for tobacco control Establishing local infrastructure for tobacco control

Long-Term GoalsLong-Term Goals •Policy change (e.g., tobacco-free schools, youth Policy change (e.g., tobacco-free schools, youth access, smoke-free workplaces) access, smoke-free workplaces) •Changes in public attitudes Changes in public attitudes •Changes in tobacco use behaviors Changes in tobacco use behaviors •Comparison of program success with “model” statesComparison of program success with “model” states

OutcomesOutcomes

Program modifications Program modifications based on interim findingsbased on interim findings

Expect the Unexpected?Expect the Unexpected?

Bird’s-Eye View of Bird’s-Eye View of EvaluationEvaluation

User Friendly Evaluation Logic Model: User Friendly Evaluation Logic Model: Documenting Strategies for Achieving ChangeDocumenting Strategies for Achieving Change

Barriers toBarriers toChangeChange

Facilitators ofFacilitators ofChangeChange

Bird’s-Eye View of Evaluation ActivitiesBird’s-Eye View of Evaluation ActivitiesMonitor trends in adult and youth tobacco use through existing Monitor trends in adult and youth tobacco use through existing statewide surveysstatewide surveysCompare trends in adult and youth tobacco use to surrounding Compare trends in adult and youth tobacco use to surrounding states and the nationstates and the nationMonitor trends in adult tobacco use cessation through State Monitor trends in adult tobacco use cessation through State QuitlineQuitlineMedia monitoringMedia monitoringEvaluation technical assistance (to state and to local grantees)Evaluation technical assistance (to state and to local grantees)TRAIN: Tobacco Reporting and Information NetworkTRAIN: Tobacco Reporting and Information NetworkCoalition leaders and participants surveyCoalition leaders and participants surveyQualitative interviews of key stakeholdersQualitative interviews of key stakeholdersFocus group of TFN staffFocus group of TFN staffVariety of Evaluation-Based Publications:Variety of Evaluation-Based Publications:

State Snapshots: 2001-2003State Snapshots: 2001-2003Local Briefs: “No Limits Snapshot”, “Building Public Health Infrastructure”Local Briefs: “No Limits Snapshot”, “Building Public Health Infrastructure”Process and outcome measures blueprintProcess and outcome measures blueprint

Evaluation Activities...RevisitedEvaluation Activities...RevisitedMonitor trends in adult and youth tobacco use through existing Monitor trends in adult and youth tobacco use through existing statewide surveysstatewide surveysCompare trends in adult and youth tobacco use to surrounding Compare trends in adult and youth tobacco use to surrounding states and the nationstates and the nationMonitor trends in adult tobacco use cessation through State Monitor trends in adult tobacco use cessation through State QuitlineQuitlineMedia monitoringMedia monitoringEvaluation technical assistance (to state and to local grantees)Evaluation technical assistance (to state and to local grantees)TRAIN: Tobacco Reporting and Information NetworkTRAIN: Tobacco Reporting and Information NetworkCoalition leaders and participants surveyCoalition leaders and participants surveyQualitative interviews of key stakeholdersQualitative interviews of key stakeholdersFocus group of TFN staffFocus group of TFN staffVariety of Evaluation-Based Publications:Variety of Evaluation-Based Publications:

State Snapshots: 2001-2003State Snapshots: 2001-2003Local Briefs: “No Limits Snapshot”, “Building Public Health Infrastructure”Local Briefs: “No Limits Snapshot”, “Building Public Health Infrastructure”Process and outcome measures blueprintProcess and outcome measures blueprint

Evaluation Activities...RevisitedEvaluation Activities...Revisited

Monitor trends in Monitor trends in adult and youth adult and youth tobacco use through tobacco use through existing statewide existing statewide surveyssurveys

T.R.A.I.N.: Tobacco T.R.A.I.N.: Tobacco Reporting and Information Reporting and Information NetworkNetwork

Local evaluation Local evaluation technical assistancetechnical assistance

Tracking Outcome Measures

Tracking Process Measures/Program Implementation

“Built In” Evaluation Capacityand Sustainability

““Data-Driven” BriefsData-Driven” Briefs

Very targeted topicsVery targeted topicsBrief and to the point! (3 pages)Brief and to the point! (3 pages)Highlights complementary activities at state and local Highlights complementary activities at state and local levelslevelsDeveloped by an independent evaluation teamDeveloped by an independent evaluation teamBalances qualitative and quantitative dataBalances qualitative and quantitative dataWritten for policy makers as key audienceWritten for policy makers as key audienceEasily communicates main messagesEasily communicates main messagesKey stakeholders’/partners’ perspectives are incorporatedKey stakeholders’/partners’ perspectives are incorporated

Examples of Data-Driven BriefsExamples of Data-Driven Briefs

State Funding Cuts to Tobacco Free Nebraska (TFN) Are Likely to Reverse Gains

Summary: Based on the experience of other states, funding cuts to Nebraska’s tobacco prevention and cessation program are expected to slow down or reverse the positive trends made in reducing adult smoking prevalence, youth access to tobacco products, and exposure to secondhand smoke in the state.

Examples of Data-Driven BriefsExamples of Data-Driven Briefs

Nebraskans Support Funding State Tobacco Prevention and Cessation Initiatives

The 2003 Nebraska state legislature decided to no longer use money from the Tobacco Settlement Trust Fund, or use new money from a tax increase on cigarettes, to fund tobacco prevention and cessation initiatives. This decision will likely derail many of the state's tobacco control initiatives that were previously supported by settlement funds. However, a large majority of Nebraskans (including smokers) support using settlement or tobacco tax monies to fund these initiatives.

How Have the Data-Driven Briefs Been How Have the Data-Driven Briefs Been Used by Tobacco Free Nebraska?Used by Tobacco Free Nebraska?

Posted on websitePosted on websiteDisseminated to:Disseminated to:

Tobacco control coordinatorsTobacco control coordinatorsPartnersPartnersAdvocatesAdvocatesLegislatorsLegislatorsMediaMedia

Key tool for lobby teamKey tool for lobby team

For More Information...For More Information...

Contact Jeff Willett, Cheryl Wiese, Ian Newman, Seth Contact Jeff Willett, Cheryl Wiese, Ian Newman, Seth EmontEmont–– University of Nebraska - LincolnUniversity of Nebraska - Lincoln

Contact Peter FinnContact Peter Finn –– Abt Associates Inc.Abt Associates Inc.

Contact Jeff Soukup, Judy MartinContact Jeff Soukup, Judy Martin – Tobacco Free Nebraska – Tobacco Free Nebraska

For an electronic copy of these and other reports, please visit For an electronic copy of these and other reports, please visit the Tobacco Free Nebraska website:the Tobacco Free Nebraska website: http://www.hhs.state.ne.us/tfn/tfnpubrep.htmhttp://www.hhs.state.ne.us/tfn/tfnpubrep.htm