Indicatorsfor ACSM

Objectives

• Explain the role of indicators in monitoring and evaluation for ACSM.

• Describe the characteristics of well-defined indicators.

• Demonstrate how to develop indicator descriptions.

Creating a Framework

E dW' ŽĂů͗� ZĞĚƵĐĞŵŽƌďŝĚŝƚLJĂŶĚŵŽƌƚĂů͗ŝƚLJƌĞů͗ĂƚĞĚƚŽd ͘

E dWK ďũĞĐƚŝǀ Ğ� /ŶĐƌĞĂƐĞĐĂƐĞĚĞƚĞĐƚŝŽŶƌĂƚĞĨƌŽŵϰϮй ƚŽϲϬй ďLJϮϬϭϱ ͘͘

^D K ďũĞĐƚŝǀ Ğ� ZĂŝƐĞŬŶŽǁ ů͗ĞĚŐĞŽĨdƐLJŵƉƚŽŵƐ ĂŶĚdƐĞƌǀ ŝĐĞƐƚŽŝŶĐƌĞĂƐĞƚŚĞŶƵŵďĞƌŽĨƉĞŽƉů͗ĞŝŶŝƚLJyƐĞĞŬŝŶŐĐĂƌĞĨŽƌdƐLJŵƉƚŽŵƐĂƚK d^ĐĞŶƚĞƌƐďLJϯϬй ďLJĞĐĞŵďĞƌϮϬϭ ϯ͘͘

/E Wh d^ • ^Ƶƌǀ ĞLJ • ^Ƶƌǀ ĞLJƐƚĂĨĨ • &ƵŶĚŝŶŐ • >ŝƐƚŽĨƐƵďǁ ĂLJ

ƌŽƵƚĞƐ • >ŝƐƚŽĨƌĂĚŝŽ

ƐƚĂƚŝŽŶƐ • D ĞĚŝĂďƵLJĞƌ

d/s /d/̂

• ŽŶĚƵĐƚ<WƐƵƌǀ ĞLJ

• Ğǀ Ğů͗ŽƉĂŶĚĂŝƌƌĂĚŝŽƐƉŽƚƐ

• Ğǀ Ğů͗ŽƉĂŶĚƉƌŽĚƵĐĞƐƵďǁ ĂLJĂĚƐ

K h dWh d^ • ηŽĨƉĞŽƉůĞ

ƐƵƌǀ ĞLJĞĚ

• ηŽĨƌĂĚŝŽƐƉŽƚƐĚĞǀ ĞůŽƉĞĚ

• # of times radio spots aired

• # of subway ads developed

• ηŽĨƐƵďǁ ĂLJƚƌĂŝŶƐǁ ŝƚŚĂĚƐ

K h dK D ^ • /ŶĐƌĞĂƐĞĚ

ŬŶŽǁ ůĞĚŐĞŽĨd ĂŶĚK d^ĐĞŶƚĞƌƐ

• /ŶĐƌĞĂƐĞĚηŽĨƉĞŽƉůĞƐĞĞŬŝŶŐĐĂƌĞĨŽƌdƐLJŵƉƚŽŵƐĂƚK d^ĐĞŶƚĞƌƐ

/D Wd • /ŶĐƌĞĂƐĞĚ

ŶƵŵďĞƌŽĨdĐĂƐĞƐ ĚĞƚĞĐƚĞĚ

RESULTS

What Is an Indicator?

• Clues, signs, or markers.

• Used to:

1. Track inputs, activities, outputs, and outcomes.

2. Measure progress toward the goal and objectives.

Crow and the Pitcher!

Indicators are Part of the M&E Plan

Examples

Framework Component Indicator

INPUT: IPCC training (curriculum) 100 copies of training manuals, slide sets, and handouts on IPCC

ACTIVITY: Lobbying Meeting with the finance minister and NTP director

OUTPUT: Patient coalition meeting Number of patients attending the meeting

OUTCOME: Increased funding Percentage of NTP budget covered by the Ministry of Health

OUTCOME: Increased support for community-based DOTS

* Policy change to allow volunteer health workers to serve as treatment supporters

OUTCOME: Increased political support for TB control

* Parliament declares TB a national emergency

Goal What am I trying to achieve?Objective What are the major steps I need

to take to reach my goal?Activities What am I going to do to reach

my objective?Indicators MONITOR progress: Have I done what I said I would do?Inputs What resources do I need to

complete each activity?Outputs What will the immediate product

of my activities be?Outcomes What do I hope will happen as a

result of my activities?Indicators EVALUATE results: Did my work have the desired effect and contribute to my goal?

Impact What do I think these activities will contribute to my goal?

Indicators: Important M&E Evidence

Outputs versus Output Indicators

• Output is the immediate result of an activity:

# of people trained # of people with TB symptoms going to health facility for evaluation REPORT WHAT YOU ACTUALLY ACCOMPLISHED

• Indicators for outputs measure your result against a target value:

# of people actually trained versus how many you wanted to train # of people with TB symptoms going for evaluation versus how

many you thought would go ASSESS WHETHER YOU REACHED YOUR TARGET VALUE

Activity

Develop and produce subway

ads

Outputs

# ads produced

# subway trains with

ads

Outcome

Increased knowledge of TB and

DOTS centers

Impact

Increased # of TB cases detected

NTP Objective: Increase case detection rate from 42% to 60% by 2015.

NTP Goal: Reduce morbidity and mortality due to

TB in Country X.

ASCM Objective: Raise knowledge of TB symptoms and TB services to increase the number of people in City X seeking care for TB symptoms at DOTS centers by 30% by December 2013 .

Inputs

Funding

List of subway routes

Indicator: Number of TB deaths per 100,000 per year

Indicator:Difference in case detection rate between intervention and control cities

Indicator: # of people requesting screening at the City X DOTS center compared to baseline

Indicator: Ads produced according to schedule (Yes/No)

Indicator: Number of ads and subway trains versus target

Indicators: •% of people who know two TB symptoms and have heard of the DOTS center•# of people requesting screening at City X DOTS center

Indicator: Lists

developed(Yes/No)

Indicator: Case detection rate

Main Steps of Creating Indicators

1. Select an indicator

2. Test against criteria

3. Write a description

Characteristics of Good Indicators

Is This Indicator Valid?

• Does it tell us what we really want to know?

• Could it actually measure something else?

Example: % of Ministry of Health budget dedicated to TB as an indicator

of government commitment to TB control.

Improved indicator: % of NTP budget covered by government (compared to % covered by donors).

Is This Indicator Reliable?

• Will everyone interpret or calculate it the same way?

Example: Number of partners actively participating in advocacy coalition.

Improved indicator: Number of partners who attend at least 75% of coalition meetings.

Example: Smear conversion rate.

Is This Indicator Activity Specific?

• Does it tell us about our activity only? Could any other factor (ACSM or not) influence this indicator?

Example: % of TB screening clients receiving a smear test as an indicator that providers from our training are referring TB suspects properly.

Improved indicator: % of TB screening clients with documented referral for smear microscopy.

Is This Indicator Feasible?

• Do we have a realistic data source? Do we have enough money and staff?

Example: % of population with correct knowledge about TB symptoms, annually (indicator of a successful communication activity).

Alternative indicator: Number of people with possible TB symptoms presenting for diagnosis at specified DOTS clinics.

Comparable

• Do the results mean the same thing in different geographic areas at different times?

Example: Number of nongovernmental organizations in each region mobilized to participate in World TB Day.

Alternative indicator: % of nongovernmental organizations in each region mobilized to participate in World TB Day.

Indicator Descriptions

• What is the complete definition?

• What is the data source for this indicator?

• How do we calculate the value of the indicator?

Numerator, denominator

Qualitative criteria

Indicator Description

Indicator: Percentage of district DOTS nurses receiving IPCC training

Definition Data Source Calculation

Percentage of DOTS nurses who attend the full training and receive a completion certificate.

IPCC training attendance sheet and certificate list submitted to NTP.

Number of nurses completing the training.

Total number of DOTS nurses in the district.

Questions?