Click here to load reader

Developing Public Health Performance Measures: Benefits and Challenges Mare Schumacher Maricopa County Department of Public Health APHA Meetings October

  • View

  • Download

Embed Size (px)

Text of Developing Public Health Performance Measures: Benefits and Challenges Mare Schumacher Maricopa...

  • Developing Public Health Performance Measures: Benefits and ChallengesMare SchumacherMaricopa County Department of Public HealthAPHA Meetings October 2001

  • About UsMaricopa County Department of Public Health in Phoenix, AZ440 employees47 distinct programs (from childhood immunizations to HIV education)85% grant funded

  • Developing Measures

  • HistoryPre-1998: Collected some metrics (mostly grants)1998-99: Dept. leaders request collection of measures for all programs2000: Countywide initiative2000-01: Complete family of measures for all 47 programs

  • Developing MeasuresOne employee dedicated to process coordination (about time)Trained managers/supervisors on:Types of pms (e.g. output, outcome)Proper collection methodsProper documentation

  • Developing MeasuresSupervisors sat down with staff to determine goalsWhen needed, coordinator worked individually with programsBiggest challenge: the psychology

  • Stages of PM Process*Denial: This is flavor of the month. If I ignore it, it will go away shortly.Anger: I havent got time for this!Panic/Freeze: PMA (Performance Measure Anxiety)Depression: I cant wait to retire.Acceptance: We have to do this.* Apologies to Elisabeth Kubler-Ross

  • Two Sales OptionsCheerleader Model:Telling employees how super-fantastic these performance measures will be!!!Empathy (I Feel Your Pain) Model:Acknowledging hard work that will be required by employees. Helping them through difficulties.

  • Collection and MaintenanceMaintain PMs in central document/ electronic location w/gatekeeperImportant to keep narrative explaining significant shifts in numbersUpdated semi-annually, now quarterly

  • Example: ImmunizationsResult: Coverage level - % of clients receiving entire seriesOutput: # of immunizations, # of clients, # of clients 0-2 years oldEfficiency: Cost per immunization

  • Example: Nutrition Education

    Output: # of studentsEfficiency: Cost per studentResult: Increase in knowledge, attitudes, behavioral intent from pre- to post-test

  • Using and Abusing Measures

  • PhilosophySet GoalsRun ProgramFunding Decisions (Accountability)Measure and Analyze Program Results

  • BenefitsLong term resource planningStaffBudgetA goal to shoot for, rally aroundSticking with priorities are we using our energies for our top priority?

  • Do we really measure performance?

  • Performance Measurement ExperimentCounty XMaricopa CountyGoal for Fiscal Year 01:Actual Performance:Goal for Fiscal Year 01:2525Actual Performance:

  • Performance Measurement ExperimentCounty XMaricopa CountyGoal for Fiscal Year 01:Actual Performance:Goal for Fiscal Year 01:2525Actual Performance:At this point in the presentation, two people representing each of the counties must fill a cup with marbles. Maricopa County uses a spoon and County X has to use chopsticks. The result is that County X scores lower on its performance measure. Without knowing that County X had chopsticks, we might conclude that it exhibits poor performance.

  • The Chopstick EffectFundingStaff shortagesChange in grant requirementsMarlboro has big ad/promotionAnd so on

  • IssuesMay not measure program effort or potential effectivenessWidget Factory model often not applicable in PHCost of collecting data some- times not worth itGoal setting as art

  • AlternativesProcess evaluationUp front research (usually from Academia)Statistical modelingStandard research practices: Test/Control, etc.National Public Health Performance Standards (self-assessment)

  • ConclusionsYou can develop measures for your organizationBut dont forget the psychological component.Use performance measures wisely better for planning than evaluating.Lets find other ways to accurately determine success (for true accountability).

  • InformationMare Schumacher Maricopa County Department of PH 1845 E. Roosevelt, Phoenix, AZ [email protected]: Taking Stock of Performance Measurement: Information Resources for Public Managers, Government Finance Review, April 2001Resource: Performance Measures Basics (presentation for program staff). Send e-mail to Mare Schumacher for copy.

  • The following is a presentation I used to educate program staff on performance measures at our Department. (It was not presented at the APHA.) Feel free to use it or alter it for your own purposes.I ask nothing in return except that you think of me for a moment every time you use it

    Mare SchumacherDepartment of Public HealthMaricopa County, Arizona (Phoenix)

  • Performance MeasuresPresentation to Public HealthStrategic Planning TeamMarch 1999

  • The PastThe night before due, make up #s to put in as performance measuresNo one looks at themNo one checks to see if you measuredNo one thinks about it for 11 months and 27 daysNext year, repeat process

  • The FutureCounty Administration will be looking closely at performance measuresDepts./Programs will be judged on performance measuresBudget may be tied to performance measures

  • Making the Best of ItThe night before due, make up #sLive with consequencesCome up with performance measures that are meaningful to County and youWeve got two choices:

  • Suggested ProcessMay be other options...

  • Meet with StaffIn a meeting with all or many staffHelps set realistic goalsEmployees know what is expected of them

    What gets measured gets done.-- Rose Howe, December 1998

  • Decide Program MissionWhy do we exist?What contribution to we make to the people of MC?NOT What do we do?

  • Example: HIV PlanningWhat we do:Have meetingsContact coalition membersPrepare reportsGet trained in how to build teamsWhy we existTo create a comprehensive HIV prevention plan that will reduce the spread of HIV

  • Just talk thisWere not measuring what we doWe are measuring our contribution to the public healthDoesnt mean what we do is not important, just not the thing were measuring

  • Create Output MeasurementOutput measurement:ActivitiesThings we doExamples:Number of applications processedNumber of clientsNumber of immunizationsNumber of coalition meetings

  • Example: Immunizations*Conducted 7500 immunizations in 97/98, etc.Expect to administer 9500 in 99/00Reasons: Population growing by 5%, will have increase in staff in 99/00 * All numbers made up -- demonstration purposes only

  • Create Efficiency MeasurementEfficiency measurement:Cost per outputResources used to produce the activityExamples:Cost per clientTime spent per clientCost per client compared to certain standard(s)

  • Example: ImmunizationsClients per hour = 4 in 97/98Expect 5 by 2000/01 due to new methods Cost per client = $1.22Expect cost = $1.45 by 00/01 due to increase cost of vaccine.

  • Create Impact (Outcome) MeasurementImpact (outcome) measurement:Indicate the impact of service on the communityShould flow directly from mission

    Examples:Percentage of clients rehabilitatedReduction in incidence of disease

  • Example: Immunizations90% coverage of school-aged children by 00/0157% coverage of clients at public health immunization sitesPossibly: 400 cases of measles avoided

  • All MeasuresWhat data will we use to measure it?If none available now, how will we get it?

  • Goals (Future Years)Make them specificMake them realistic - how much control do we have over that measure? What are the reasons for expecting the goal to be higher, lower, etc.Document why you picked that goal? How did you calculate it?

  • TimingDevelop measures with staffDraft meeting with John/MareFinal draft meeting

    Final measures dueEnd of March/April

    End of April (sign-up sheet)May 11 SP Meeting

    June 1

Search related