Data Driven Management - Visioning Slides CARE CML Indrajit

DATA DRIVEN MANAGEMENT

Overview and Application

A discussion on How to Use Data in Management and Decision-Making in Public

Health Scenario in Bihar?

Suggestive slides for Visioning Workshop

Prepared by:Indrajit Chaudhuri, CARE India23rd July 2012

Indrajit Chaudhuri, 23rd July 2012

What is Data?• Data is the value of different variables. • Quantitative Data are generally represented by number or

percentage• In the context of MCH, data on coverage, practices / behavior,

services provided etc. can provide understanding of performance of the program. E.g.,

• No. of institution delivery (in a block / district)• % of children received immunization• No. of children breast-fed within one hour of delivery• % of mothers who were visited at home by FLW thrice in first week after delivery• No. of mothers received information on maternal complications by FLWs during last trimester

of pregnancyEtc. etc.

• It is important to MEASURE / ASSESS to generate DATA

23%

372

58

8.1

12.9%

7%

28

6


Why do we need data?• For upward reporting:

– Calculation of cumulative national and sub-national estimates– Planning – Budget allocation– Supports policy makers to develop policies and guidelines

• For decision making on the ground:– Development of local-level strategies for implementation– Targeting of issues on which performance is low– Targeting of geographic areas or specific population groups

where indicators are poor


DATA IS OFTEN NOT USED FOR DECISION MAKING


REASON FOR DATA OFTEN NOT BEING USED FOR DECISION MAKING

- specifically at the implementation level

Information need for managers at all level are

not assessedData collection decisions are not made considering the decision making need

at the implementation level

Contextual data of interest of Program Managers are not

collectedData is often not

available for the Program Manager in a useable

form

Flow of data does not ensure that it reaches

Program Managers at all levels in a timely manner

And, also… Lack of capacity to use the data

and realization of its usefulness

WHY ARE DATA NOT USED?


What do we mean by Data Driven Management?

• Data Driven Management (DDM) is the way of program management where major decisions are taken on the basis of data.

• A Program Manager needs to take lots of decisions – day-to-day implementation decisions, strategic decisions etc. – depending on the nature of program and level of management.

• Any decision is taken on the basis of certain information. If the program manager does not have those information – it may lead to wrong or imperfect decisions.

• The Program Manager knows issues on which she needs to take decision. So, she can identify well ahead what information she requires for taking those decisions.

• In DDM, data is collected in order to provide those information to Program Manager for facilitating the decision making process. Managers takes any decision based on those data ensuring an objective decision-making process.


How Data Driven Management works?

• Data Driven Management follows following few steps:– Identification of information requirement:

– What all information do we need to manage the program and take important decisions (at various levels) ?

– Preparing strategy for capturing those information: – What data should be collected for making those information

available in a timely manner? How should those data be collected? How should data flow?

– Analyzing Data in order to make relevant information available: – How should the data be analyzed? How can the analysis of data be

presented in a usable form, which provide timely, optimal and required information for decision-making?

– Using analyzed data for taking important program decisions – mainly in terms of evaluating progress and setting future target


THE DATA DRIVEN MANAGEMENT FRAMEWORK

MEASURE

IDENTIFY GAPS

STRATEGIZE

TAKE ACTION

Measure output / outcome / impact level indicator TO GENERATE DATA

ANALYSE DATA to identify low

performing areas (SC / blocks etc.) &

reasons for low performance

Prepare / modify strategies and

prepare plans for particular

geographic area

Measure again !

Take appropriate

action as per strategy

SET A TARGET – after measuring each time as

reference for the next assessment


Home visit by FLW at right

time

Delivery of appropriate

messages and effective

counseling

Change in behavior

Measure

Identify Gaps

Strategize

Take Action

Measure

Identify Gaps

Strategize

Take Action

Measure

Identify Gaps

Strategize

Take Action

Application of DDM Framework An example of application of Data Driven Management Framework in improving behavioral outcomes by applying it at for intermediary

outputs responsible for the final outcome


MEASUREVARIOUS INDICATORS COULD BE MEASURED TO GENERATE DATA

Measuring the final impact is important. But, the changes in impact-level indicators depend on changes in many smaller actions, which could be measured through various output / outcome level indicators. Therefore, for program managers, it is important to measure intermediary outcomes and outputs – in order to identify the gaps clearly.

An example is used for describing this in the next slides…. In the example first week PNC visits are taken as example.


FINALLY… MEASURE IMPACT

MEASURE OUTCOMES

MEASURE FREQUENCY

OF HOME VISIT –

WHETHER FLW VISITED?

WHETHER MESSAGES WERE DELIVERED?

Home Visit by FLWs during

the first week of delivery Delivered

message on “nothing to be applied on the

cord”

Clean cord care practiced

Delivered message on “skin to skin

care”

Thermal care practiced

Delivered message on “only breast

milk” EBF practiced

Delivered message on maternal danger signs

Recognized & treated of maternal

complication

Reduced Maternal Mortality

Delivered message on

neonatal danger signs

Recognized & treated of neonatal

complication

Reduced Neonatal Mortality

EXAMPLE


IDENTIFY GAPSDATA COULD BE ANALYZED TO IDENTIFY GAPS

After measurement of indicators at various output and outcome level –generated data are analyzed in order to identify gaps.

It is important to identify gaps in terms of:• Where in the chain of output and intermediary outcomes is there a drop

in performance?• Finding out specifically low performing regions: Blocks, Sub-centers or

catchment areas etc. which are not performing below the acceptable standard

• Finding out socio-economic groups – where performance of some indicators are low


EXAMPLE

ImpactOutcome 2Outcome 1

Output XAction A

Action B

Output YAction C

Where in the chain is there a drop in performance or achievement?

Which geographical areas are low performing?

(with respect to a particular indicator)

?


STRATEGIZESTRATEGIES AND PLANS ARE PREPARED FOR MITIGATING THOSE GAPS

Data driven management helps in sharp identification of gaps – which helps in building the strategy.

The preparation of strategy should consider the TARGET that is set after measurement, which is specific to the indicator in the particular area.


TAKE ACTIONAPPROPIATE ACTION SHOULD BE TAKEN AS PER STRATEGY

After the action is taken, measurement should be repeated. There should be an agreed newly-set target and agreed time-frame within which the changed strategy should reflect in the measurement.

If the measurement reveals large gap from the target – the process of gap finding and re-strategizing should be repeated.


In the first three quarters home visit

increased, but advices did not

improve

Q1 Q2 Q3 Q4 Q5 Q60

20

40

60

80

% of women received 3 PNC visits by FLW

% of women received any advice from FLW

% of women received all advices from FLW

EXAMPLE PNC visit data of a particular area for consecutive quarters

Data is continuously measured and analyzed in terms of visit and advice by FLWs

MEASURE & ANALYSE




improve

Gap identification

exercises show poor content

delivery because of lack of

capacity of FLWs


Analysis of data of visit and advice by FLWs helps in identifying gaps

Q1 Q2 Q3 Q4 Q5 Q60

20

40

60

80




IDENTIFY GAPS


Q1 Q2 Q3 Q4 Q5 Q60

20

40

60

80





Identification of gaps help in preparing better strategies and plan for improvement of outcome



improve

Gap identification



capacity of FLWs

STRATEGIZE

Increased emphasis on

content delivery


Q1 Q2 Q3 Q4 Q5 Q60

20

40

60

80





Action taken as per revised strategy helps in improved outcome



improve

TAKE ACTION & MEASURE

AGAIN

Gap identification



capacity of FLWs

Increased emphasis on

content delivery

Q1 Q2 Q3 Q4 Q5 Q60

20

40

60

80





APPLICATION OFDATA DRIVEN MANAGEMENT

IN OUR CONTEXT – IN THE CONTEXT OF PUBLIC HEALTH IN BIHAR


CURRENT STATUS WITH RESPECT TO DATA IN HEALTH SECTOR IN BIHAR

• The value of data has immensely increased in recent years. Lots of data are being collected. They are being used for reporting above, planning and setting overall target and budget. But use of data on taking management decisions at the implementation level is very limited.

• Available Data Sources: HMIS, different large population surveys (NFHS, DLHS etc.)

• Issues with available datao HMIS: Self-reported – possibility of errors; Meant for upward

reportingo Large population Surveys: Data for smaller geographic areas are

not available; Data are available after long time – loss relevance.o Mainly coverage / final outcome data – data on intermediary steps

are mostly not available• Therefore, all these available data are generally not used for day-to-day

decision making


HOW CAN WE RESOLVE THIS ISSUE? WHAT DO WE NEED FOR EFFECTIVE DDM?

• Population level surveys are best ways to measure output and outcome level indicators.

• Random sample ensures unbiased estimate.• Data flow should be fast – in order to have minimum time lag between

data capture, analysis and use – so as to generate almost real-time estimates.

• So, a population level random sample survey covering all the important output and outcome level indicators with capability of generating real-time data can help in initiating the Data-driven Management on the ground.

A probable solution, which can work well for the block and district level, could be employing LQAS methodology – with real-time data transfer and analysis mechanism.


WHAT IS LQAS?• Lot quality assurance sampling (LQAS) is a random sampling

methodology, that helps us generate an understanding of performance / achievement in a supervisory area (e.g., block) with very small number of random sample.

• In our context there can be two way use of LQAS:– It can be used at the block-level to identify ‘priority blocks’ or ‘priority

indicators in a block’ – which are not achieving the target or an established benchmark

– It can also provide a measure of coverage or estimate of various indicators at the district-level

• The beauty of employing LQAS is that it can work effectively with a very small sample size at the block-level – which is as small as 19 – when randomness of the sample is ensured.


• A small number of random sample is selected from each block. – 19 is the most common and most efficient sample size for LQAS – These samples are checked for any specific indicator which can be

binomially expressed (like – “yes/no”, “achieved/not achieved”, “received/not received” etc.).

• A target (expressed by the term ‘decision rule’) is pre-set to indicate the accepted result in that indicator at the block level. The ‘decision rule’ indicates number of respondents from sample that should be found to meet the criteria for that indicator (Decision rule is determined from the table shown in a slide later). – If the pre-set target for an indicator is 80%, then the LQAS table shows

that the decision rule should be 13 out of 19 samples. This means: if less than 13 samples of a block meet the criteria for an indicator, then the target for that block is not achieved.

HOW IS LQAS USED?


How to interpret LQAS data?• At the district level: District estimates are available with fair

precision.• At the block level:

– We do not get any coverage estimate, but, we get to know • which of the blocks do not meet the “target” in a particular

indicator • which particular indicators did not meet the “target” in a

particular block– The best use of LQAS (at the block-level) is to find under-

performing blocks and underperforming indicators in a block.– A TARGET SHOULD BE PRE-SET FOR GENERATING THESE

ESTIMATES AT THE BLOCK-LEVEL.


FROM OUR LQAS - ROUND 1IFHI has already undertaken LQAS between the months of December and February. The method is operationally tested now.

IFHI block coordinators collected data from 19 mothers of each of the 4 age groups of children (0-2 months, 3-5 months, 6-8 months and 9-11 months).

The data was captured also through hand-held devices and real-time estimates and analyses were available for use at the block and district level.

Results of the round-1 are expressed in next few slides. As targets were not set beforehand in the block with block-level managers – we are using dummy targets for analysis.


PLEASE INSERT RELEVANT SLIDES FROM YOUR “LQAS DISTRICT

RESULT” PRESENTATIONS(which were shared earlier)


TARGET SETTING– LET US SET A TARGET FOR THE NEXT ROUND ON A

FEW SIMPLE INDICATORS

for the district (overall) &

for a block (if necessary)


HOW TO SET TARGETS?• We can select some indicators which we feel will be improved in next three

month. Say, indicators on home visit and delivery of some contents through FLW interaction (say, on BP).

• We can get the district estimate and a rough idea about block situation (from color – red/green and indicative number) from Round 1. We can also get an estimate of various indicators from the Ananya Baseline data.

• We can discuss about these few indicators in district-level visioning workshops to set a overall district target for the next three months.

• We can also discuss about these indicators in visioning workshops at the block-level to agree with them on the same target – If some of the blocks feel that the target set at the district level is too

high and not contextual for their district revision of the target for the particular block could be done.

• We can discuss these targets with FLWs in ANM Tuesday meetings, ASHA divas meetings and AWW monthly meetings to get their ownership.

• Then, after the next round we can see whether blocks met those targets or not (from the LQAS decision rule table).


HOW KEEP OUR FOCUS & MONITOR THESE TARGETS DURING THE QUARTER?

• LQAS data will be available after the quarter. But it is important to keep the attention of FLWs on these targets. There are few possible ways to do that:

• These few indicators should be discussed in all possible forums with FLWs to keep the attention of FLWs maintained. The discussion with FLWs can happen in Sub-center Platform Meetings, ANM Tuesday meetings, ASHA divas meetings and AWW monthly meetings.

• Some of these measures may be available from HMIS or some other data source of IFHI. These data should be analyzed and presented to FLWs on a monthly basis to keep a track on the progress.

• Information available from Home Visit Registers should be discussed in reference to these indicators in all the monthly sub-center meetings.

Etc.

ANNEXURES

SLIDES TO BE USED FOR EXPLAINING CERTAIN CONCEPTS –

IF NECESSARY

LQAS Decision Table

Taken from A Participant’s Manual, Joseph J Valadez et al.


Why use a Sample Size of 19?

• Little is added to the precision of the measure by using a sample larger than 19.

• Sample sizes less than 19, however, see a rapid deterioration in the precision of the measure.

Sample size


What do we need to remember from the decision table?

– For 19 samples:

– Therefore, for a sample size of 19 per block, if the target is 50% for an indicator (say, initiation of breast-feeding within one hour of delivery), then all the blocks, which had less than 7 samples as ‘yes’ (i.e., if less than 7 women out of 19 found initiated breast-feeding within one hour) will be identified as ‘not met the target’ and will be marked in “Red”.

– Target can be set by the implementation team (generally, district team) and the decision rule will change accordingly. E.g., if target is 60%, decision rule will be 9, 11 for 70% and 13 for 80%.

Target 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95%

Decision Rule 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16


USE OF DATA DRIVEN MANAGEMENT IN CONTEXT OF ULTIMATE VISION OF MCH

• Ultimate vision of MCH is to reduce IMR, Malnourishment, TFR and MMR• But, it is difficult to measure these. We can measure proximal outcome

indicators – which can indicate whether we are in right direction. • In order to see whether our program is in right direction to reduce IMR – we

should find out:• Whether identification of newborn complications are increasing• Whether more newborns are getting treatment for complications• Whether FLWs are providing right message and right counseling regarding newborn

complications at right time through home visit Etc.

• In order to see whether our program is in right direction to reduce Malnourishment – we should find out:

• Whether exclusive breast feeding rates (till six-month) are increasing• Whether age-appropriate frequency and quantity of complementary feeding is

increasing with continuation of breast-feeding from six-month age of the child• Whether initiation of complementary feeding at the age of six month is increasing.

Etc.CONTINUED…


USE OF DATA DRIVEN MANAGEMENT IN CONTEXT OF ULTIMATE VISION OF MCH (Continued) • In order to see whether our program is in right direction to reduce TFR –

we should find out:• Whether unmet need for contraception is decreasing• Whether • Whether more newborns are getting treatment for complications• Whether FLWs are providing right message and right counseling regarding newborn

complications at right time through home visit

• In order to see whether our program is in right direction to reduce MMR – we should find out:

• Whether identification of maternal complications are increasing• Whether more mothers are getting treatment for complications• Whether FLWs are providing right message and right counseling regarding maternal

complications at right time through home visit Etc.

• Thus Data Driven Management provides us much easier and simpler ways to understand the progress towards achievement of these high-level indicators like MMR, IMR, TFR and malnourishment.

Data & Analytics

Data Driven Management - Visioning Slides CARE CML Indrajit