National Outcome MeasuresNational Outcome Measures

Department of Health and Human ServicesDepartment of Health and Human ServicesHealth Resources and Services AdministrationHealth Resources and Services Administration

Bureau of Health ProfessionsBureau of Health Professions

All Grantee ConferenceAll Grantee ConferenceJune 1, 2005June 1, 2005

National Outcomes Breakout SessionsNational Outcomes Breakout Sessions

PresentersPresentersDan Blumenthal, M.D.Dan Blumenthal, M.D.Morehouse School of MedicineMorehouse School of Medicine

Gary Hart, Ph.D.Gary Hart, Ph.D.University of WashingtonUniversity of Washington

Jan Towers, Ph.D.Jan Towers, Ph.D.American Academy of Nurse American Academy of Nurse

PractitionersPractitioners

Kirsten Thomsen, PA-CKirsten Thomsen, PA-CSociety of Primary Care Policy Society of Primary Care Policy

FellowsFellows

Maria Clay, Ph.D.Maria Clay, Ph.D.East Carolina UniversityEast Carolina University

Marina Matthew, RHIAMarina Matthew, RHIAPennsylvania Department of HealthPennsylvania Department of Health

Elyse Perweiler, MPP, RNElyse Perweiler, MPP, RNNational Association of Geriatric National Association of Geriatric

Education CentersEducation Centers

Julia Rose, Ph.D.Julia Rose, Ph.D.Case Western Reserve UniversityCase Western Reserve University

Tanya PagTanya Pagán án Raggio, M.D.Raggio, M.D.Division of Medicine & Dentistry, Division of Medicine & Dentistry,

HRSA/BHPRHRSA/BHPR

David Goodman, M.D.David Goodman, M.D.Dartmouth Medical SchoolDartmouth Medical School

Structure of the SessionStructure of the Session

Overview of the performance measurement Overview of the performance measurement development processdevelopment process

Brief presentation on Brief presentation on National OutcomeNational Outcome MeasuresMeasures

Panelists’ commentsPanelists’ commentsGeneral comments and questionsGeneral comments and questions

The Development ProcessThe Development Process

Strategic Planning WorkgroupStrategic Planning Workgroup Performance Measurement WorkgroupPerformance Measurement Workgroup Health Outcomes WorkgroupHealth Outcomes Workgroup Program StaffProgram Staff A 3-level Performance Measurement A 3-level Performance Measurement

System:System:1.1. Program Specific MeasuresProgram Specific Measures2.2. CoreCore Measures Measures 3.3. National OutcomesNational Outcomes

“Measure what is measurable, and make measurable what is not so.”

--Galileo Galilei (1564-1642)

Work in ProgressWork in Progress

Are We Asking the Right Are We Asking the Right QuestionsQuestions??

Do the proposed measures help tell the story of BHPr Do the proposed measures help tell the story of BHPr programs and their accomplishments?programs and their accomplishments?

– Are they policy relevant?Are they policy relevant?

– Are they easy to understand?Are they easy to understand?

– Are they clearly related to a variety of BHPr program activities?Are they clearly related to a variety of BHPr program activities?

– Are they broad enough to encompass a variety of health Are they broad enough to encompass a variety of health professionals, conditions, and target populations?professionals, conditions, and target populations?

– Do they complement the Program Specific and Core Do they complement the Program Specific and Core Performance Measures?Performance Measures?

Expectations of GranteesExpectations of Grantees

The Bureau is responsible for reporting on the The Bureau is responsible for reporting on the National Outcome Measures, and will use data National Outcome Measures, and will use data from National Data Sets.from National Data Sets.

Grantees are Grantees are notnot responsible for reporting responsible for reporting National Outcome Measures.National Outcome Measures.

Grantees are encouraged to link their goals and Grantees are encouraged to link their goals and activities to National Outcome Measures.activities to National Outcome Measures.

Grantees are responsible for meeting Grantees are responsible for meeting grant grant specific targetsspecific targets on Program Specific Measures, on Program Specific Measures, and selected Core Measures determined by and selected Core Measures determined by BHPr.BHPr.

Conceptual FrameworkConceptual Framework

40 Programs

40 Logic Models

~400 activities

~400 outcomes

Diversity

PC Career Choice

Distribution

Quality of Education &

Training

Infrastructure

Improved Access to

Primary Care

Reduced Morbidity

Reduced Mortality

Reduced Health

Disparities

Improved Access to

Public Health Services

5 Common Purposes

Three Levels of MeasurementThree Levels of Measurement

1.1. Core Performance MeasuresCore Performance Measures Capture accomplishments across 5 common purposes, at the Bureau Capture accomplishments across 5 common purposes, at the Bureau

LevelLevel

Use aggregated Program dataUse aggregated Program data

2.2. Program Specific Performance Measures:Program Specific Performance Measures: Capture the unique accomplishments of each ProgramCapture the unique accomplishments of each Program

3.3. National Outcome MeasuresNational Outcome Measures Measures of access to primary careMeasures of access to primary care

Measures of health statusMeasures of health status

Logically related to Program and Bureau activitiesLogically related to Program and Bureau activities

Based on National data sources.Based on National data sources.

Detailed ExamplesDetailed Examples The slides that follow provide two detailed examples of The slides that follow provide two detailed examples of

National Outcome Measures.National Outcome Measures.

These are not These are not PerformancePerformance Measures. Measures.

The Bureau is responsible for reporting these National The Bureau is responsible for reporting these National Outcome Measures. Grantees are encouraged to link their Outcome Measures. Grantees are encouraged to link their activities to National Outcome Measures.activities to National Outcome Measures.

Data will be obtained from national data sets, not grantees.Data will be obtained from national data sets, not grantees.

The outcomes are logically linked to BHPr program efforts.The outcomes are logically linked to BHPr program efforts.

Changes in the outcomes can be Changes in the outcomes can be partiallypartially attributed to BHPr attributed to BHPr efforts.efforts.

NOM1:NOM1: Infant mortality rateInfant mortality rate

GOAL To reduce the number of infant deaths

DEFINITIONS Numerator: All infant deaths within 12 months of birth (Will also assess by race and ethnicity)

Denominator: All live births

Units: 1000 Text: Rate per thousand

HEALTHY PEOPLE 2010 OBJECTIVE

Related to objective 16-1 Reduce fetal and infant deaths

DATA SOURCES National Vital Statistics System (NVSS), CDC, NCHS

NOM2:NOM2: Percent of children between the ages 19 Percent of children between the ages 19 and 35 months who have completed immunizations and 35 months who have completed immunizations

recommended for the first 18 months of liferecommended for the first 18 months of life

GOAL To reduce direct and indirect costs of infectious diseases for which there are effective, universally-recommended vaccines.

DEFINITIONS Numerator: Fully vaccinated children aged 19 to 35 months.

Denominator: All children aged 19 to 35 months.

Units: 100 Text: Percent

HEALTHY PEOPLE 2010 OBJECTIVE

Related to Objective 14-22 (Universally recommended vaccination of children aged 19 to 35 months)

DATA SOURCES National Immunization Survey; National Health Interview Survey; HHS National Vaccine Program Office & National Vaccine Advisory Committee; CDC National Immunization Program

Brief Overview of National Brief Overview of National Outcome MeasuresOutcome Measures

NOM3:NOM3: Pediatric asthma hospitalization ratePediatric asthma hospitalization rate

NOM4:NOM4: Ratio of the untreated dental caries rate for Ratio of the untreated dental caries rate for underrepresented minority children to the underrepresented minority children to the untreated dental caries rate for White untreated dental caries rate for White

childrenchildren

NOM5:NOM5: Percent of adolescents 12-17 years of age Percent of adolescents 12-17 years of age who who have smoked in the past yearhave smoked in the past year

NOM6:NOM6: Proportion of pregnant women who received Proportion of pregnant women who received early and continuous prenatal care early and continuous prenatal care

Brief Overview of National Brief Overview of National Outcome MeasuresOutcome Measures

NOM7:NOM7: Percent of women aged 40 and over who Percent of women aged 40 and over who have received a mammogram within the have received a mammogram within the preceding 2 years preceding 2 years

NOM8:NOM8: Rate of preventable/avoidable Rate of preventable/avoidable hospitalizations hospitalizations

NOM9:NOM9: Percent of adults who smoke tobacco Percent of adults who smoke tobacco productsproducts

NOM10:NOM10: Proportion of adults with high blood Proportion of adults with high blood cholesterol cholesterol

Brief Overview of National Brief Overview of National Outcome MeasuresOutcome Measures

NOM11:NOM11: Percent of adults with diabetes whose Percent of adults with diabetes whose glycosylated hemoglobin measurement in the glycosylated hemoglobin measurement in the last year is 7.0 percent or lesslast year is 7.0 percent or less

NOM12: Odds Ratio (Self-reported examinations for oral NOM12: Odds Ratio (Self-reported examinations for oral and pharyngeal cancer reported in adult and pharyngeal cancer reported in adult African- American males compared to White African- American males compared to White males over 40 years of age)males over 40 years of age)

NOM13: Coronary Heart Disease Death RateNOM13: Coronary Heart Disease Death Rate

NOM14: Age-adjusted suicide rateNOM14: Age-adjusted suicide rate

What’s Next?What’s Next?

Determine final set of measuresDetermine final set of measuresDevelop modified data collection forms Develop modified data collection forms

(for Program Specific and Core)(for Program Specific and Core)Develop analysis and reporting planDevelop analysis and reporting planApply for Paperwork Reduction Act Apply for Paperwork Reduction Act

Clearance (OMB)Clearance (OMB)Modify Grant GuidanceModify Grant GuidanceImplement the SystemImplement the System

For Further InformationFor Further Information……

Please contact your Please contact your Program OfficerProgram Officer, , oror

Janet T. Schiller, Ed.D.Janet T. Schiller, Ed.D.

Senior Program AnalystSenior Program Analyst

HRSA Bureau of Health ProfessionsHRSA Bureau of Health Professions

5600 Fishers Lane, Room 8-1035600 Fishers Lane, Room 8-103

Rockville MD 20857Rockville MD 20857

301-443-0889301-443-0889

jschiller@hrsa.govjschiller@hrsa.gov

Are We Asking the Right Are We Asking the Right QuestionsQuestions??

Do the proposed measures help tell the story of BHPr Do the proposed measures help tell the story of BHPr programs and their accomplishments?programs and their accomplishments?

– Are they policy relevant?Are they policy relevant?

– Are they easy to understand?Are they easy to understand?

– Are they clearly related to a variety of BHPr program activities?Are they clearly related to a variety of BHPr program activities?

– Are they broad enough to encompass a variety of health Are they broad enough to encompass a variety of health professionals, conditions, and target populations?professionals, conditions, and target populations?

– Do they complement the Program Specific and Core Do they complement the Program Specific and Core Performance Measures?Performance Measures?