Kansas Department of Health and EnvironmentCenter for Health Disparities

2008 Health Disparities ConferenceTopeka, Kansas, Apr. 1, 2008

Measuring Health Disparities

James P. ScanlanAttorney at LawWashington, DC

jps@jpscanlan.com

Objectives1. Explain the problematic nature of standard measures of differences between rates (relative differences, absolute differences, odds ratios)

2. Explain a plausible alternative approach that avoids the problems with standard measures

Part 1

Problematic Nature of Binary Measures of Differences between Rates

References Health Disparities Measurement tab on jpscanlan.com

Can We Actually Measure Health Disparities? Chance (Spring 2006) (A12)

Race and Mortality, Society (Jan-Feb 2000) (A10)

The Misinterpretation of Health Inequalities in the United Kingdom, British Society for Population Studies Conference 2006 (B7)

Measurement Problems in the National Healthcare Disparities Report, American Public Health Association Conference 2007 (B 12)

Items D23, D41, D43, D45, D46, D48, D52, D53

Four Binary Indicators of Differences Between Rates

Rates of experiencing some beneficial outcome:

Advantaged group (AG) = 50% Disadvantaged group (DG) = 40%

1 Relative difference between rates of experiencing an outcome (in terms of ratio of AG’s rate to DG’s rate (Ratio 1)): 1.25 (50/40)

2 Relative difference between rates of failing to experience the outcome (Ratio 2): 1.20 (60/50)

3 Odds ratio (in terms of DG’s to AG’s odds of failing to experience the outcome) : 1.50 ((60/40)/(50/50)0

4 Absolute differences between rates: 10 percentage points (50% -40%)

Table 1: Examples of Changing Rates and Changing Differences Between Rates

Period Yr 0 dir Yr 5 dir Yr 10 dir Yr 15

AG Rate 40% I 58% I 76% I 94%

DG Rate 23% I 39% I 58% I 85%

Ratio 1 1.77 D 1.50 D 1.31 D 1.10

Ratio 2 1.29 I 1.46 I 1.75 I 2.42

Odds Ratio 2.29 D 2.19 I 2.28 I 2.67

Absol Diff .17 I .19 D .18 D .09

Question

In the prior slide, which measure provides the most accurate information as to the change in disparity?

Answer

None. There was no change in disparity. The patterns are based on hypothetical test score data simulating the situation where two groups have somewhat different distributions of factors associated with some outcome. Each measure changed in the manner that would occur if, with no change in differences between averages, a cutoff was lowered to allow everyone scoring just below a cutoff now to pass the test (or if test performance were improved such as to allow everyone between two points to achieve the higher score)

Crucial Point Not that various measures tend to support different

interpretations of the direction of a change in disparity (though that is a matter of some consequence)

Rather, that no standard measure can alone provide information as to whether there occurred a meaningful change in disparity over time, because each measure tends to change as the overall level of an outcome changes

Caveat

Standard Patterns of Changes in Binary Measures as the Overall Prevalence of an Outcome Changes

As an outcome increases from being very rare to being almost universal:

1. Relative differences in experiencing it (Ratio 1) tend to decrease

2. Relative differences in failing to experience it (Ratio 2) tend to decline

3. Odds ratios tend to decrease until the approximate intersection of Ratios 1 and 2 and thereafter increase

4. Absolute differences tend to move in the opposite direction of odds ratios

Fig 1. Ratio of (1) AG Success Rate to DG Success Rate (Ratio 1) at Various Cutoffs Defined by AG Success Rate

0

1

2

3

4

5

1 3 5 10 20 30 40 50 60 70 80 90 95 97 99

Cutoffs Defined by AG Success Rate

Ra

tio

s (1) AG Succ Rate/DG Succ Rate

Fig 2. Ratios of (1) AG Success Rate to DG Success Rate (Ratio 1) and (2) DG Fail Rate to AG Fail Rate (Ratio 2)

0

1

2

3

4

5

1 3 5 10 20 30 40 50 60 70 80 90 95 97 99

Cutoffs Defined by AG Success Rate

Ra

tio

s (1) AG Succ Rate/DG Succ Rate

(2) DG Fail Rate/AG Fail Rate

Zone A Zone B

Pt X

Fig 3. Ratios of (1) AG Success Rate to DG Success Rate (Ratio 1), (2) DG Fail Rate to AG Fail Rate (Ratio 2), and (3) DG Fail Odds to AG Fails Odds

0

1

2

3

4

5

1 3 5 10 20 30 40 50 60 70 80 90 95 97 99

Cutoffs Defined by AG Success Rate

Ra

tio

s (1) AG Succ Rate/DG Succ Rate

(2) Ratio DG Fail Rate/AG Fail Rate

(3) DG Fail Odds/AG Fail OddsPt X

Zone A Zone B

Fig 4. Ratios of (1) AG Success Rate to DG Success Rate, (2) DG Fail Rate to AG Fail Rate, and (3) DG Fail Odds to AG Fails Odds; and Absolute Diff Between Rates

0

1

2

3

4

5

1 3 5 10 20 30 40 50 60 70 80 90 95 97 99

Ra

tio

s

(1) AG Succ Rate/DG Succ Rate

(2) Ratio DG Fail Rate/AG Fail Rate

(3) DG Fail Odds/AG Fail Odds

0

10

20

1 3 5 10 20 30 40 50 60 70 80 90 95 97 99

Cutoffs Defined by AG Success Rate

Per

cen

tag

e P

oin

ts

Absolute Diff Betw Rates

Zone A Zone B

Zone A Zone B

Pt X

Fig. 5. Ratios of (1) Wh to Bl Rate of Falling above Percentages of the Poverty Line, (2) Bl to Wh Rate of Falling below the Percentage, (3) Bl to Wh Odds of Falling Below the Percentage; and (4)Absolute Difference Between Rates

●0

1

2

3

4

600 500 400 300 250 200 175 150 125 100 75 50

Rat

ios

(1) Wh Rt Ab/Bl Rt Ab

(2) BL rt Bel/Wh Rt Bel

(3) Bl Odds Bel/Wh Odds Bel

0

10

20

30

600 500 400 300 250 200 175 150 125 100 75 50

Percentage of the Poverty Line

Perc

en

tag

e P

oin

ts

(4) Absolute Diff betw Rates

Zone A Zone B

Pt X

Pt X

Zone A Zone B

Fig. 6. Ratio of (1) Wh to Wh Rate of Falling Above Various SBP Levels, (2) Wh to Bl Rate of Falling below the Level, (3) Bl to Wh Odds of Falling Above the Level; and (4) Absolute Difference Between Rates (NHANES 1999-2000, 2001-2002, Men 45-64)

●1

2

3

4

5

110 120 130 140 150 160 170 180 190

Ra

tio

s

(1) Wh Rt Bel/Bl Rt Bel

(2) Bl Rate Ab/Wh Rate Ab

(3) Bl Odds Bel/Wh Odds Bel

0

10

20

110 120 130 140 150 160 170 180 190

Systolic Blood Pressure

Pe

rce

nta

ge

Po

ints

(4) Absolute Diff betw Rates

Zone A Zone B

Zone A Zone B

Pt X

Pt X

Interpretive Implications of Described Patterns of Change

Mortality and acute morbidity declines in adverse outcomes tend to increase relative

differences in adverse outcomes but decrease relative differences in favorable outcomes (mortality and survival)

since activity tends to be well into Zone B, reductions in adverse outcomes tend to reduce absolute differences (increase odds ratios)

Healthcare outcomes improvements in care (e.g., increases in rates of receiving

procedures) tend to reduce relative differences in receipt of procedures but increase relative differences in failure to receive procedures

since (depending on the procedure) activity can be in Zone A or B, improvements in care may tend to increase or decrease absolute differences and odds ratios

issues with AHRQ and NCHS (A12, B12, D23a, D42, D52, D53)

Illustrations from Recent Journal Articles

Patterns of Black and White Rates of Adequate Hemodialysis

Sehgal AR. Impact of quality improvement efforts on race and sex disparities in hemodialysis. JAMA 2003;289:996-1000

Rates of adequate hemodialysis:Year White Black1993 46% 36%2000 87% 84%

Summary of changes in rate differences:Absolute diff: decreased from 10 to 3 percentage points

Ratio 1 (adequate dialysis): decreased from 1.27 to 1.10Ratio 2 (inadequate dialysis): increased from 1.19 to 1.23

See B12, D23, D23a, D42

Difference between means of hypothetical underlying distributions: 1993: .26 standard deviations2000: .14 standard deviation

See Part 2 and D43

Two Contrasting Studies

Jha et al. Racial trends in the use of major procedures among the elderly. N Engl J Med 2005;353:683-691: found (mainly) increasing absolute differences during periods of increasing prevalence of procedures

Trivedi et al. Trends in the quality of care and racial disparities in Medicare managed care. N Engl J Med 2005;353:692-700: found (mainly) declining absolute differences during periods of increasing prevalence of appropriate care

Reconciliation: Jha et al. principally in Zone A; Trivedi et al. principally in Zone B; see D23, D23a, D40, D40a, D41, D41a, B11

Further examples Pickett et al. Widening social inequalities in risk for sudden

infant death syndrome. Am J Public Health 2005;95:97-81. (very successful “back-to-sleep” program seemed to increase SES disparities in SIDS) See D3.

Morita et al. Effect of school-entry vaccination requirements on racial and ethnic disparities in Hepatitis B immunization coverage among public high school students. Pediatrics 2008;121:e547-e552. (very successful vaccination requirement seems to reduce racial and ethnic disparities in vaccination rates). See D52.

Baicker et al. Who you are and where you live: how race and geography affect the treatment of Medicare beneficiaries. Health Affairs 2004:Var-33-Var-44 (varied comparisons re relative and absolute differences in procedures). See D53.

Pay for Performance and Healthcare Disparities

Werner et al. Racial profiling: The unintended consequences of coronary artery bypass graft report cards. Circulation 2005;111:1257–63. Increasingly cited as evidence the pay-for-performance will tend

to increase healthcare disparities

Casalino et al. Will pay-for-performance and quality reporting affect health care disparities? Health Affairs 2007;26(3):405-414. Recommends that pay-for-performance be tied to effects on

disparities as now being implement in Massachusetts

See D46, D48 (explaining Werner findins in light of tendencies described above), D49, D51 (explaining patterns one typically would observe in Massachusetts)

Implications of Focus Upon Subpopulation

Subpopulations that are truncated parts of overall populations tend not to have normal distributions of factors associated with an outcome when the distributions in the overall population are perfectly normal

Nevertheless, since the truncated distributions tend to have regular shapes, standard patterns of changes in binary measures (save for odds ratios) tend to apply

Even so, there are interpretive implications of the fact that some studies examine subpopulations defined by need for special attention (e.g., hypertensive) rather than overall populations

Absolute differences in process outcome versus control outcomes More serious implications with regard to “Approach 2”

Fig 7. Ratios (1) AG Success Rate to DG Success Rate, (2) DG Fail Rate to AG Fail Rate, (3) DG Fail Odds to AG Fail Odds: and Absolute Differences within Subpopulation Falling Below Point Defined by 30 Percent Fail Rate for AG

●1

2

3

4

1 10 20 30 40 50 60 70 80 90 99

Ra

tio

s(1) AG Rate Above/DG RateAbove

(2) DG Rate Bel/AG Rate Bel

(3) DG Odds Bel/AG OddsBelow

0

10

20

1 10 20 30 40 50 60 70 80 90 99Cutoffs Defined by AG Success Rate

w/in Truncated Subpopulation

Per

cen

tag

e P

oin

ts

(4) Absolute Diff betw Rates

Zone A Zone B

Zone A Zone B

Fig.8. Absolute Difference Between Rates within the Total Population, and with Population Below the 30 Percent Fail Rate for the AG, according to AG Fail Rate Within Each Population.

●0

10

20

1 10 20 30 40 50 60 70 80 90 99

Cutoffs Defined by AG Fail Rate - All

Rat

ios Absolute Diff betw Rates - All

0

10

20

1 10 20 30 40 50 60 70 80 90 99Cutoffs Defined by AG Fail Rate within

Universe Below AG Fail Ratio of 30

Ra

tio

s Absolute Diff betw Rate - Subpop

Zone A Zone B

Zone A Zone B

Fig. 9. Ratio of (1) Wh to Bl Rate of Falling below Various SBP Levels (favorable outcome), (2) Bl to Wh Rate of Falling above the Level (adverse outcome), (3) Bl to Wh Odds of Falling above the Level; and (4) Absolute Difference between Rates (NHANES 1999-2000, 2001-2002, Men 45-64), Limited to Population with SBP Above 139

●1

2

3

4

5

144

148

152

156

160

164

168

172

176

180

184

188

Ra

tio

s

(1) Wh Rt Bel/Bl Rate Bel

(2) Bl Rt Ab/Wh Rt Ab

(3) Bl Odds Bel/Wh Odds Bel

0

10

20

Systolic Blood Pressure

Pe

rce

nta

ge

Po

ints

TADnum

Zone A Zone B

Zone A Zone B

Fig. 10. Absolute Differences Between Rates of Falling Above Certain SBP Levels for Overall Population and Population with SBP above 139

●0

10

20

110 120 130 140 150 160 170 180 190

Systolic Blood Pressure

Rati

os

Absolute Diff betw Rates

0

10

20

Systolic Blood Pressure

Ra

tio

s

Absolute Diff betw Rates

Zone A Zone B

Zone A Zone B

Part 2

Alternative Approaches to Measurement

Measurement Possibilities on a Seemingly Continuous Scales Longevity – no (see B7, B11) SF 36 scores – no (see B11) Metabolic syndrome measures – no (see B11) Cardio risk indexes – no (see B11) Allostatic load – possibly (see B11) Components of allostatic load – possibly (see B9, B11) Cortisol level – possibly (see B11) Self rated health on a continuous scale - possibly (see

B7, B11) Gini coefficient, concentration index etc (see A12, D43)

Measurement Possibilities Using Outcome Rates Approach 1 – departures from standard

patterns (A12, B7, D41, D43)

Approach 2 – identifying the difference between means of hypothetical underlying distributions based on group rates in settings being compared (D43, D45, D46, D48)

Table 2. Hypothetical Illustration of Approach 2

Period AG Rate DG Rate EES

Yr 0 76% 58% .50

Yr 5 94% 88% .38

*Estimated effect size – difference between hypothesized means in terms of percentage of a standard deviation

Table 3: Illustration of Approach 2 Based on Data in Article to which D48 Responds

Coronary angiogramYearWh Rate* Bl Rate EES 1988 86 43 .251997 228 161 .14

Coronary angioplastyYear Wh Rate Bl Rate EES 1986 10 3 .321997 26 16 .15

Coronary artery bypass surgeryYearWh Rate Bl Rate EES1986 31 8 .411997 59 26 .27

*All rates are per 10,000

Conclusions Regarding Approach 2

Further examples on D43, D45, D46, D48, D52, D53

Procedure speculative because it rests on hypotheses as to normality of underlying distributions (see D43)

Procedure unsuitable for truncated distributions, which we know not to be normal (see D43, D46a)

Despite weaknesses, procedure is superior to standard measures of differences between rates for evaluating size of disparity in different settings

Where to go from here?

Other References

Keppel K., Pamuk E., Lynch J., et al. 2005. Methodological issues in measuring health disparities. Vital Health Stat 2 (141) (http://www.cdc.gov/nchs/data/series/sr_02/sr02_141.pdf) (see A12, B12, D6)

Carr-Hill R, Chalmers-Dixon P. The Public Health Observatory Handbook of Health Inequalities Measurement. Oxford: SEPHO; 2005 (http://www.sepho.org.uk/extras/rch_handbook.aspx) (see A7, D8)

Houweling TAJ, Kunst AE, Huisman M, Mackenbach JP. Using relative and absolute measures for monitoring health inequalities: experiences from cross-national analyses on maternal and child health. International Journal for Equity in Health 2007;6:15 (http://www.equityhealthj.com/content/6/1/15) (see D43, D50)