Aligning Outcomes with Population Needs

Kathy Atwood and Deirdre Danahar SAMHSA/CSAP’s Southeast CAPT

July 10, 2007

2

Key Principles of SPF

• Based on a public health approach • Focused on outcomes based prevention• Widens the scope to population-based

prevention• Provides a systematic strategic planning process

using epidemiologic data to drive decision making

• Provides a framework for building systems capacity

Integration of Substance Abuse and HIV Prevention Efforts

• Complex social problem• Best addressed by targeting the local circumstances,

conditions and individual-level behaviors.• Requires comprehensive solutions at both the individual

and community level – Variety of intervention approaches directed to multiple aspects of

the problem

• Determining optimal mix of strategies is complicated by limited information on evidence of effectiveness of hybrid (Substance and HIV prevention) interventions

SAMHSA/CSAP Expert Workgroup on Evidence-Based Programming

Convened nationally recognized prevention experts to develop recommendations and guidelines for State and community preventionplanners

– Prevention research indicates that few problems are amenable to change through direct influence

– Most problems must be influenced indirectly by addressing underlying factors that contribute to the problem

– Need broad array of evidence-based interventions– Flexibility to choose options that fit community

circumstances

Underlying Factors

• Conditions that lead to the development of problems and consequences– May include specific local policies and practices,

community realities and population shifts

• Risk and protective factors – Present across the course of human development– Make individuals and groups either more or less

prone to substance abuse and HIV risk behaviors in certain social contexts

Your Program Logic model is a…

• Picture of your program or intervention• Graphic representation of the “theory of

action” – what is invested, what is done, and resulting changes in participants and/or community conditions

• Core of planning and evaluation• Provides a common framework for your

work

INPUTS OUTPUTS OUTCOMES

Program investments

Activities Participation Short Medium

What we invest

What we do

Who we reach

Resulting changes

Long-term

Program Logic Model Design

My Program’s Logic Model

• What outcome(s) is my program aiming to achieve?

• What activities are we implementing to accomplish this outcome(s)?

• What changes in causal factors (e.g. risk and protective factors) does the outcome of your program contribute to?

• What program/policy/practice is being implemented?

• Enhances team work and provides focus• Guides prioritization and allocation of

resources• Motivates staff• Helps to identify important variables to

measure; use evaluation resources wisely• Increases resources, opportunities,

recognition• Supports replication• Often is required!

Benefits of Program Logic Models

Benefits of Program Logic Models

• Provides a common language

• Helps differentiate between program activities and results --- outcomes

• Increases understanding of program

• Guides and helps focus work

• Increases intentionality and purpose

• Provides coherence across complex tasks, diverse environments

• Helps you explain your program to others—especially funders

• Provides a picture of your program on one page

• Helps you set boundaries: if it doesn’t fit within your logic model, you can say no.

• Increases your confidence in your program• Helps you think as a team

What’s In It For You…

Logic Models are…

…Used at different conceptual and practical levels

• Program level

• Community level

Substance-Related

Consequences and Use

Causal (e.g. Risk/Protective)

Factors

Programs/Policies/Practices

Implementing the Strategic Prevention Framework

Planning, Monitoring, Evaluation and Replanning

Outcome-Based PreventionCommunity Logic Model

The Community Logic Model: Key Tool for Community Planning

• Community logic model serves as a map of the problem (e.g., substance use and HIV risk behaviors) and the factors leading to that problem

• Community logic model represents systematic plan for attacking local problem within a specific context

• Community logic model starts by defining problem not choosing the solutions (program, practices, or policies)

Community Logic Model

• Essential tool for comprehensive, effective community prevention effort

• Links the risk behavior to underlying factors based on research and local data

• Provides explicit rationale for selecting programs, policies and practices to address the risk behavior

Community Logic Model ExampleCommunity Logic Model Example

Community Logic ModelReducing alcohol

-related youth traffic fatalities

High rate of alcohol-

related crash mortality

Among 15 to 24 year olds

Low or discount PRICING of alcohol

Easy RETAIL ACCESS to Alcohol for youth

Easy SOCIAL ACCESS to Alcohol

Media Advocacy to Increase Community

Concern about Underage Drinking

Restrictions on alcohol advertising in

youth markets

SOCIAL NORMS accepting and/or encouraging

youth drinking

PROMOTION of alcohol use (advertising, movies,

music, etc)

Low ENFORCEMENT of alcohol laws

Underage

DRINKING AND DRIVING

Social Event Monitoring and Enforcement

Bans on alcohol price promotions and

happy hours

Young Adult

BINGE DRINKING

Enforce underage retail sales laws

Causal Factors

Strategies(Examples)

Substance-Related

Consequences

SubstanceUse

Low PERCEIVED RISK of alcohol use

Young Adult

DRINKING AND DRIVING

Underage

BINGE DRINKING

Each causal factor provides an opportunity or potential point of entry for types of interventions or strategies (policies, practices, programs) that may change the targeted problem

Community Logic Models Fit Unique Community Needs

• Communities must tailor the logic model to fit their particular context, needs, capacities, and readiness

18

Analytic Process to Select Best Fit Prevention Interventions

Conceptual Fit of Strategies

• Identify strategies that are relevant to the variables in the community’s logic model– Strategies that don’t fit the community logic model are

unlikely to effectively address the identified problem

• Select strategies to effect the community’s most significant risk/protective factors and conditions

• Target multiple points of entry to impact the target population across social environments and sectors

Comprehensive Community Plans

Time

School

PeersFamily

Work

Neighborhood

ReligiousGroups

Law

s and

Po

liciesS

oci

al S

erv

ice

s

Work

Media

Practical Fit and Utility of Strategies

Is the strategy a practical fit with the resources, capacity, and readiness of the community organizations responsible for implementing interventions

Developed/adaptable for the target population– Age, gender, culture, language

Implemented successfully with a similar population Delivered in a similar setting Implementation materials, Training/TA, evaluation tools

available to ensure implementation quality

Assess Feasibility of Strategies

• Cultural fit with community values• Political fit with local power structure • Organizational fit with mission, vision,

culture• Administrative fit with policies and

procedures• Technical fit with staff capability and time• Financial fit with existing resources

Benefits of logic modeling to community partnerships

• Provides framework to discuss and articulate joint work

• Helps facilitate conversation with focus on agreed upon goals

• Shows contribution of each partner and how result depends upon all

• Keeps end outcome upfront and center• Provides way to communicate about the

partnership to others

Prevention System Pre-Alignment Stakeholders i.e. Providers, Police,

Preventions professionals,

community leaders

Community ATOD Coalition

Programs/ Policies/

Practices

community wide reductions in identified ATOD use and

related consequences

Stakeholders efforts & resources target different outcomes

Efforts are diluted and not coordinated to effect change

Prevention System Alignment Stakeholders i.e. Providers, Police,

Preventions professionals,

community leaders

Community ATOD Coalition

Programs/ Policies/

Practices

community wide reductions in identified ATOD use and

related consequences

Stakeholders efforts & resources target same outcomes

Efforts are targeted and coordinated to effect change

My Community Logic Model

• What is the outcome my community is aiming to achieve?

• What behaviors (e.g.: specific substance use) are present in my community?

• What causal factors (e.g.: risk and protective factors) are supporting the behaviors?

• What local contributing factors are present?• What is the overlap between my community

logic model and my program level logic model?

Benefits of Community Logic Models

• Provides common language and understanding among partners

• Guides and focuses work• Provides coherence across work done by

partners• Guides prioritization and allocation of

resources• Enhances collaboration and provides

focus

Logic Model and Evaluation Questions

Needs assessment:

What are the characteristics, needs, priorities of target population?

What are potential barriers/facilitators?

Process evaluation:

How is program implemented?

Are activities delivered as intended?

Are participants being reached as intended?

What are participant reactions?

Outcome evaluation:

To what extent are desired changes occurring? For whom?

Is the program making a difference?

What seems to work? Not work?

What are unintended outcomes?