Measuring need using population health indicators ... · Rural Health Research Center Measuring Need Using Population Health ... Example of a Summary Table of Data Characteristics

South Carolina Rural Health Research Center

Measuring Need Using Population Health Indicators: Some Composite Methods Shortchange Rural Counties

Charity B. Breneman, PhD Post-Doctoral Fellow,

SC Rural Health Research Center

International Conference on

Health Policy Statistics, January 2018

Rural Health Research Center South Carolina

Composite Health Indices

• Constructed from a set of population health indicators from multiple sources that have been transformed (e.g., scaled, normalized, standardized) and aggregated together in a single measure (Rothenberg et al., 2015)

Summary measure of health

• Geographical units (e.g., countries, states, counties) • Socioeconomic groups

Used to monitor and compare the health between populations

Rothenberg, et al. (2015). Urban health indicators and indices – current status. BMC Public Health, 15:494.


Multiple Uses of Composite Health Indices

• Public communication • Track changes overtime • Problem identification • Policy design and

adoption • Stimulate efforts to

improve population health

The United Health Foundation – America’s Health Rankings. Available at https://www.americashealthrankings.org/ The University of Wisconsin Population Health Institute – County Health Rankings. Available at http://www.countyhealthrankings.org/

America’s Health Rankings

County Health Rankings

Oliver, TR. (2010). Population health rankings as policy indicators and performance measures. Prev Chronic Dis, 7:A101.

Examples of Composite Health Indices in the United States

Presenter

Presentation Notes


County Health Rankings

The University of Wisconsin Population Health Institute – County Health Rankings. Available at http://www.countyhealthrankings.org/

Methods 1. Gather data 2. Impute missing data using state mean 3. Normalize county values within each state

for each measure using the average of counties in that state (z-scores)

4. Eliminate outliers 5. Multiply by scientifically-informed weights 6. Sum weighted scores 7. Rank counties by the sum of all measure

scores


Distribution of Rural and Urban Counties Across the United States

Category Number of Counties

Metropolitan 1,166

Micropolitan 641

Small Adjacent 674

Remote Rural 655

Counties were characterized based on level of rurality using Urban Influence Codes: Urban (UICs 1, 2), Micropolitan (UICs 3, 5, 8), Small Adjacent (UICs 4, 6, 7,), and Remote Rural (UICs 9, 10, 11, 12).

Presenter

Presentation Notes

In the United States, there are 1,970 counties that are considered rural which as you can see from the map, rurality covers the majority of the United States. So the question is, how do you go about identifying which rural counties have the poorest health outcomes. The literature on composite indices that measure population health in rural counties is quite sparse. And then when you look at some of the current composite health indices being used, the methodologies do not adequately accommodates the data challenges associated with small area populations.


County County Health Rankings (CHR)

YPLL Age-adjusted All-cause Mortality

Absolute Difference (CHR and YPLL)

Absolute Difference (CHR and All-cause)

Coahoma, MS 1907 1895 1872 12 35

Wilcox, AL 1906 1898 1825 8 781

Holmes, MS 1905 1884 1805 21 100

McDowell, WV 1904 1901 1903 3 1

Phillips, AR 1903 1887 1882 16 21

Mingo, WV 1902 1891 1899 11 3

Sharkey, MS 1901 1885 1828 16 73

Pemiscot, MO 1900 1893 1869 7 31

Jefferson Davis, MS

1899 1866 1603 33 296

Jefferson, MS 1898 1848 1518 50 380

Comparison of ranks for the 10 rural counties with poorest

health outcomes

Presenter

Presentation Notes

Rural counties were ranked using the same methods as CHR with some slight modifications. I standardize the individual components of Health Outcomes composite index using the national mean instead of the state mean. Differences are noted in the three different measures of population health. Single measures of population health, like Years of Potential Life Lost (YPLL) and all-cause mortality, may not sufficiently capture the complex interactions between the many social, economic, physical, clinical, and other factors that influence population health. This is one reason why composite health indices are desirable because they summarize several health factors in an easy to understand form.


Construction of Composite Health Indices Step 1.

Theoretical framework

Step 2. Data Selection

Step 3. Imputation of Missing Data

Step 4. Multivariate

Analysis

Step 5. Normalization

Step 6. Weighting and Aggregation

Step 7. Robustness and

Sensitivity

Step 8. Back to the Real

Data

Step 9. Links to Other

Variables

Step 10. Presentation and

Dissemination

• Multiple steps in the process • Caution required during all

steps

Organisation for Economic Co-Operation and Development. (2008). Handbook on Constructing Composite Indicators – Methodology and User Guide.

May shortchange rural counties

Presenter

Presentation Notes

The construction of a composite health index is not a straight forward process and often the methods vary from one index to the next. See the Handbook on Constructing Composite Indicators – Methodology and User Guide for more details. This is a great resource. The rest of the presentation will explain how steps 2 and 3 may shortchange rural counties.


Things to Consider during Data Selection

1. Availability of data at the desired geographic level of analysis

• Use the average of multiple years of data

• Proxy measures may be substituted

2. Timeliness 3. Accessibility 4. Accuracy

Example of a Summary Table of Data Characteristics from County Health Rankings

The University of Wisconsin Population Health Institute – County Health Rankings. Available at http://www.countyhealthrankings.org/ Organisation for Economic Co-Operation and Development. (2008). Handbook on Constructing Composite Indicators – Methodology and User Guide.

Focus Area Measure Weight Source Year(s) Missing Length of life (50%)

Premature death

50% Mortality files

2012-2014 9% (169)

Quality of life (50%)

Poor or fair health

10% BRFSS 2015 0

Poor physical health days

10% BRFSS 2015 0

Poor mental health days

10% BRFSS 2015 0

LBW 20% Natality files 2008-2014 5% (93)

Presenter

Presentation Notes

Missing and unreliable data are a common phenomenon for geographic areas with small populations (see last column for number of rural counties missing data for that particular variable).


Imputation of Missing Data Types

1. Case deletion Ignores differences between cases with complete versus incomplete data

2. Single imputation Mean/median/mode substitution

3. Multiple Imputation

Overuse of imputation techniques can impact the overall

quality of the composite index.


Presenter

Presentation Notes

There are three general methods for dealing with missing data: Case deletion, single imputation, and multiple imputation


Population Health Indicator

Worse Health Outcomes

State Average (only) % (n)

National Average (only) % (n)

Both % (n)

Clin

ical

Car

e

Uninsured 8% (5) 17% (11) 52% (33)

Uninsured children 0% 6% (4) 92% (58)

Poverty 25% (16) 0% 8% (5)

Child poverty 11% (7) 3% (2) 32% (20)

No primary care providers - 78% (49) -

No dentists - 86% (54) -

No mental health providers - 48% (30) -

Comparison of Unranked Rural Counties (n=63) in County Health Rankings to State and National Averages

Exa

mpl

e of

Cas

e D

elet

ion

Presenter

Presentation Notes

To account for unreliability nationwide, County Health rankings excludes the least unpopulated counties from the Rankings. This included 63 rural counties that were unranked due to problematic (missing or unreliable) values for premature death or low birth weight. Comparison of the unranked counties to the state average or national average demonstrates that those unranked rural counties had higher un-insurance rates compared to ranked counties and had a fewer number of health care providers. (Table above shows the number of unranked counties with worse clinical care outcomes compared to the state average, nation average, or both).


Bris

tol B

ay, A

K Ya

kuta

t, AK

Al

pine

, CA

Chey

enne

, CO

H

insd

ale,

CO

Sa

n Ju

an, C

O

Cam

as, I

D Cl

ark,

ID

Stan

ton,

KS

Wal

lace

, KS

Cart

er, M

T G

arfie

ld, M

T M

cCon

e, M

T Pe

trol

eum

, MT

Prai

rie, M

T Tr

easu

re, M

T W

ibau

x, M

T Ar

thur

, NE

Bann

er, N

E Bl

aine

, NE

Deue

l, N

E G

arde

n, N

E G

rant

, NE

Hay

es, N

E Ke

ya P

aha,

NE

Loga

n, N

E Lo

up, N

E M

cPhe

rson

, NE

Siou

x, N

E Th

omas

, NE

Whe

eler

, NE

Har

ding

, NM

Bi

lling

s, N

D Lo

gan,

ND

Sher

idan

, ND

Slop

e, N

D Ca

mpb

ell,

SD

Har

ding

, SD

Hyd

e, S

D Jo

nes,

SD

Sully

, SD

Bord

en, T

X G

lass

cock

, TX

Kene

dy, T

X Ke

nt, T

X Ki

ng, T

X Lo

ving

, TX

McM

ulle

n, T

X M

otle

y, T

X Ro

bert

s, T

X St

erlin

g, T

X Te

rrel

l, TX

Da

gget

t, U

T Pi

ute,

UT

Uni

nsur

ed

Uni

nsur

ed

Chi

ldre

n

Prim

ary

Care

Ph

ysic

ians

Dent

ists

Men

tal

Heal

th

Prov

ider

s

NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA

Worse health outcomes compared to state average

Worse health outcomes compared to national average

Worse health outcomes compared to both state and national averages Clinical Care

Presenter

Presentation Notes

The following slide presents a table that is designed to demonstrate that some of these unranked counties may be considered vulnerable populations given by how several of the clinical care outcomes are worse in comparison to their corresponding state-level average and the national average. Additionally, some of the unranked rural counties have a clustering of several poor health outcomes and factors that further demonstrate the vulnerability of the county.


Example of Single Imputation

State

Number of Rural Counties Missing data

Average Years of Potential Life Lost

(YPLL) Rural

Counties All

Counties

Alaska 4 9246.05 8813.51

Colorado 4 7017.81 6663.9

Kansas 18 8077.68 7749.7

Montana 8 8868.80 8648.0

Nebraska 12 6872.43 6674.7

North Dakota 16 8189.12 8482.89

South Dakota 16 9420.28 8838.4

Texas 11 8680.02 8237.0

County Health Rankings uses single imputation methods to replace missing data Use corresponding state-level

means Method chosen for ease of

communicating methods and final rankings with the public

Modifications are needed to accommodate the unique characteristics of data from rural counties

Presenter

Presentation Notes

The use of the state-level mean may lead to biased rankings especially since rural counties tend to have poorer health outcomes in comparison to urban counties as well as to the overall state average. Compare columns for the average of rural counties compared to average of all counties.


Pros Cons Can summarize several elements into a single measure.

Poorly constructed composite index may be misinterpreted or send misleading policy messages.

Easier to interpret. May invite simplistic policy conclusions. Can assess progress over time. May be misused if poorly constructed or lacks

transparent methodology. Reduce the visible size of a set of indicators without dropping the underlying information base.

May disguise limitations of data.


Use of Indices


Future Directions

• Determine which method works best for rural counties with missing data.

• Identify proxy measures and determine how they may impact the ranks


Thanks! Our web site:

rhr.sph.sc.edu Core funding from:

Federal Office of Rural Health Policy, Health Resources & Services Administration, USDHHS

Contact: [email protected]


The Rural Health Research Gateway provides access to all

publications and projects from eight different research centers.

Visit our website for more information.

ruralhealthresearch.org

Sign up for our email alerts! ruralhealthresearch.org/alerts Center for Rural Health

University of North Dakota 501 N. Columbia Road Stop 9037

Grand Forks, ND 58202

http://www.ruralhealthresearch.org/

Documents

Measuring need using population health indicators ... · Rural Health Research Center Measuring Need Using Population Health ... Example of a Summary Table of Data Characteristics