Indicatorsfor ACSM
Objectives
• Explain the role of indicators in monitoring and evaluation for ACSM.
• Describe the characteristics of well-defined indicators.
• Demonstrate how to develop indicator descriptions.
Creating a Framework
E dW' ŽĂů͗� ZĞĚƵĐĞŵŽƌďŝĚŝƚLJĂŶĚŵŽƌƚĂů͗ŝƚLJƌĞů͗ĂƚĞĚƚŽd ͘
E dWK ďũĞĐƚŝǀ Ğ� /ŶĐƌĞĂƐĞĐĂƐĞĚĞƚĞĐƚŝŽŶƌĂƚĞĨƌŽŵϰϮй ƚŽϲϬй ďLJϮϬϭϱ ͘͘
^D K ďũĞĐƚŝǀ Ğ� ZĂŝƐĞŬŶŽǁ ů͗ĞĚŐĞŽĨdƐLJŵƉƚŽŵƐ ĂŶĚdƐĞƌǀ ŝĐĞƐƚŽŝŶĐƌĞĂƐĞƚŚĞŶƵŵďĞƌŽĨƉĞŽƉů͗ĞŝŶŝƚLJyƐĞĞŬŝŶŐĐĂƌĞĨŽƌdƐLJŵƉƚŽŵƐĂƚK d^ĐĞŶƚĞƌƐďLJϯϬй ďLJĞĐĞŵďĞƌϮϬϭ ϯ͘͘
/E Wh d^ • ^Ƶƌǀ ĞLJ • ^Ƶƌǀ ĞLJƐƚĂĨĨ • &ƵŶĚŝŶŐ • >ŝƐƚŽĨƐƵďǁ ĂLJ
ƌŽƵƚĞƐ • >ŝƐƚŽĨƌĂĚŝŽ
ƐƚĂƚŝŽŶƐ • D ĞĚŝĂďƵLJĞƌ
d/s /d/̂
• ŽŶĚƵĐƚ<WƐƵƌǀ ĞLJ
• Ğǀ Ğů͗ŽƉĂŶĚĂŝƌƌĂĚŝŽƐƉŽƚƐ
• Ğǀ Ğů͗ŽƉĂŶĚƉƌŽĚƵĐĞƐƵďǁ ĂLJĂĚƐ
K h dWh d^ • ηŽĨƉĞŽƉůĞ
ƐƵƌǀ ĞLJĞĚ
• ηŽĨƌĂĚŝŽƐƉŽƚƐĚĞǀ ĞůŽƉĞĚ
• # of times radio spots aired
• # of subway ads developed
• ηŽĨƐƵďǁ ĂLJƚƌĂŝŶƐǁ ŝƚŚĂĚƐ
K h dK D ^ • /ŶĐƌĞĂƐĞĚ
ŬŶŽǁ ůĞĚŐĞŽĨd ĂŶĚK d^ĐĞŶƚĞƌƐ
• /ŶĐƌĞĂƐĞĚηŽĨƉĞŽƉůĞƐĞĞŬŝŶŐĐĂƌĞĨŽƌdƐLJŵƉƚŽŵƐĂƚK d^ĐĞŶƚĞƌƐ
/D Wd • /ŶĐƌĞĂƐĞĚ
ŶƵŵďĞƌŽĨdĐĂƐĞƐ ĚĞƚĞĐƚĞĚ
RESULTS
What Is an Indicator?
• Clues, signs, or markers.
• Used to:
1. Track inputs, activities, outputs, and outcomes.
2. Measure progress toward the goal and objectives.
Crow and the Pitcher!
Indicators are Part of the M&E Plan
Examples
Framework Component Indicator
INPUT: IPCC training (curriculum) 100 copies of training manuals, slide sets, and handouts on IPCC
ACTIVITY: Lobbying Meeting with the finance minister and NTP director
OUTPUT: Patient coalition meeting Number of patients attending the meeting
OUTCOME: Increased funding Percentage of NTP budget covered by the Ministry of Health
OUTCOME: Increased support for community-based DOTS
* Policy change to allow volunteer health workers to serve as treatment supporters
OUTCOME: Increased political support for TB control
* Parliament declares TB a national emergency
Goal What am I trying to achieve?Objective What are the major steps I need
to take to reach my goal?Activities What am I going to do to reach
my objective?Indicators MONITOR progress: Have I done what I said I would do?Inputs What resources do I need to
complete each activity?Outputs What will the immediate product
of my activities be?Outcomes What do I hope will happen as a
result of my activities?Indicators EVALUATE results: Did my work have the desired effect and contribute to my goal?
Impact What do I think these activities will contribute to my goal?
Indicators: Important M&E Evidence
Outputs versus Output Indicators
• Output is the immediate result of an activity:
# of people trained # of people with TB symptoms going to health facility for evaluation REPORT WHAT YOU ACTUALLY ACCOMPLISHED
• Indicators for outputs measure your result against a target value:
# of people actually trained versus how many you wanted to train # of people with TB symptoms going for evaluation versus how
many you thought would go ASSESS WHETHER YOU REACHED YOUR TARGET VALUE
Activity
Develop and produce subway
ads
Outputs
# ads produced
# subway trains with
ads
Outcome
Increased knowledge of TB and
DOTS centers
Impact
Increased # of TB cases detected
NTP Objective: Increase case detection rate from 42% to 60% by 2015.
NTP Goal: Reduce morbidity and mortality due to
TB in Country X.
ASCM Objective: Raise knowledge of TB symptoms and TB services to increase the number of people in City X seeking care for TB symptoms at DOTS centers by 30% by December 2013 .
Inputs
Funding
List of subway routes
Indicator: Number of TB deaths per 100,000 per year
Indicator:Difference in case detection rate between intervention and control cities
Indicator: # of people requesting screening at the City X DOTS center compared to baseline
Indicator: Ads produced according to schedule (Yes/No)
Indicator: Number of ads and subway trains versus target
Indicators: •% of people who know two TB symptoms and have heard of the DOTS center•# of people requesting screening at City X DOTS center
Indicator: Lists
developed(Yes/No)
Indicator: Case detection rate
Main Steps of Creating Indicators
1. Select an indicator
2. Test against criteria
3. Write a description
Characteristics of Good Indicators
Is This Indicator Valid?
• Does it tell us what we really want to know?
• Could it actually measure something else?
Example: % of Ministry of Health budget dedicated to TB as an indicator
of government commitment to TB control.
Improved indicator: % of NTP budget covered by government (compared to % covered by donors).
Is This Indicator Reliable?
• Will everyone interpret or calculate it the same way?
Example: Number of partners actively participating in advocacy coalition.
Improved indicator: Number of partners who attend at least 75% of coalition meetings.
Example: Smear conversion rate.
Is This Indicator Activity Specific?
• Does it tell us about our activity only? Could any other factor (ACSM or not) influence this indicator?
Example: % of TB screening clients receiving a smear test as an indicator that providers from our training are referring TB suspects properly.
Improved indicator: % of TB screening clients with documented referral for smear microscopy.
Is This Indicator Feasible?
• Do we have a realistic data source? Do we have enough money and staff?
Example: % of population with correct knowledge about TB symptoms, annually (indicator of a successful communication activity).
Alternative indicator: Number of people with possible TB symptoms presenting for diagnosis at specified DOTS clinics.
Comparable
• Do the results mean the same thing in different geographic areas at different times?
Example: Number of nongovernmental organizations in each region mobilized to participate in World TB Day.
Alternative indicator: % of nongovernmental organizations in each region mobilized to participate in World TB Day.
Indicator Descriptions
• What is the complete definition?
• What is the data source for this indicator?
• How do we calculate the value of the indicator?
Numerator, denominator
Qualitative criteria
Indicator Description
Indicator: Percentage of district DOTS nurses receiving IPCC training
Definition Data Source Calculation
Percentage of DOTS nurses who attend the full training and receive a completion certificate.
IPCC training attendance sheet and certificate list submitted to NTP.
Number of nurses completing the training.
Total number of DOTS nurses in the district.
Questions?