1

Transparency in Health Care Quality

What you need to know about public reporting

Elizabeth Mort, MD, MPH

Vice President Quality & Safety, MGHAssociate Chief Medical Officer, MGH

Team Leader for Uniform High Quality,Partners HealthCare Inc.

2

Transparency in health care

• Transparency involves being open about what you do, how you do it, and the results that you get.

• In health care, transparency encompasses

– Clinical quality and safety

– Service and access

– Pricing and cost

• Purpose:

– Increase public accountability

– Inform consumers’ decision-making

– Rationalize resource use (costs) in health care

– Inspire providers to improve

3

Outline• How did we get here?

• What information is out there?

• A short primer on quality measurement, ranking, tiering

• Landmark litigation

• Current initiatives in MA

• Discussion

4

How did we get here?

• Rising cost of health care– Longstanding problem, now in crisis

• Gaps in quality– Striking variation in quality and service delivery

• Consumerism– Consumer empowerment driving transparency

and accountability– Consumer directed health plans as a new tactic

to reduce costs

International Comparison of Spending on Health, 1980-2005

$-

$1,000

$2,000

$3,000

$4,000

$5,000

$6,000

$7,000

1980

1982

1984

1986

1988

1990

1992

1994

1996

1998

2000

2002

2004

United StatesGermanyCanadaFranceAustraliaUnited Kingdom

0

2

4

6

8

10

12

14

16

1980

1982

1984

1986

1988

1990

1992

1994

1996

1998

2000

2002

2004

United StatesGermanyCanadaFranceAustraliaUnited Kingdom

* PPP=Purchasing Power Parity.Data: OECD Health Data 2007, Version 10/2007.

Average spending on healthper capita ($US PPP*)

Total expenditures on healthas percent of GDP

5

RISING COSTS

Source: Commonwealth Fund National Scorecard on U.S. Health System Performance, 2008 5

7681

88 84 89 8999 97

8897

109 106116 115 113

130134

128

115

6571 71 74 74 77 80 82 82 84 84

90 93 96101 103 103 104

110

0

50

100

150

Fra

nc

e

Ja

pa

n

Au

str

alia

Sp

ain

Ita

ly

Ca

na

da

No

rwa

y

Ne

the

rla

nd

s

Sw

ed

en

Gre

ec

e

Au

str

ia

Ge

rma

ny

Fin

lan

d

Ne

w Z

ea

lan

d

De

nm

ark

Un

ite

d K

ing

do

m

Ire

lan

d

Po

rtu

ga

l

Un

ite

d S

tate

s

1997/98 2002/03

Deaths per 100,000 population*

* Countries’ age-standardized death rates before age 75; including ischemic heart disease, diabetes, stroke, and bacterial infections.See report Appendix B for list of all conditions considered amenable to health care in the analysis.Data: E. Nolte and C. M. McKee, London School of Hygiene and Tropical Medicine analysis of World Health Organization mortality files (Nolte and McKee 2008).

Mortality Amenable to Health Care

GAPS IN QUALITY

Source: Commonwealth Fund National Scorecard on U.S. Health System Performance, 2008 6

32

46

53

47

49

50

58

39

0 20 40 60 80 100

Uninsured all year

Uninsured part year

Insured all year

<200% of poverty

200%–399% of poverty

400%+ of poverty

2005

2002

GAPS IN QUALITY

Recommended Screening & Preventive Care for Adults

Percent of adults (ages 18+) who received all recommended screening andpreventive care within a specific time frame given their age and sex*

* Recommended care includes seven key screening and preventive services: blood pressure, cholesterol, Pap, mammogram,fecal occult blood test or sigmoidoscopy/colonoscopy, and flu shot. See report Appendix B for complete description.Data: B. Mahato, Columbia University analysis of Medical Expenditure Panel Survey.

U.S. Variation 2005

U.S. Average

Source: Commonwealth Fund National Scorecard on U.S. Health System Performance, 2008 7

8

Emerging models of payment reform: new combinations of old ideas

• Incremental reforms such as nonpayment for never events

• Primary care payment reform, medical home, tiered case-management fees, capitation

• Episode-based payments, global case rates• Shared savings models, providers share in

savings, quality monitored • Consumer directed plans

Rosenthal MB, NEJM 359;12 Sept 18, 2008

9

Consumer-directed health plans are emerging

• Rationale: patients with more out of pocket expenses will drive more rationale use of resources (hopefully data-driven)

• Several varieties– Higher co-payments and deductibles– Health savings accounts– Tax credits– Tiering of physicians

• Tiering has been the tactic of choice in MA used by the Group Insurance Commission

10

“For Your Benefit,” Group Insurance Commission Newsletter, Fall 2008

11

So, what’s at issue?

• “The appropriate way to measure physicians’ (quality and) efficiency is a matter of disagreement between those that pay for (use) health care and those who provide it.”

Arnold Milstein, MD Thomas Lee, MD

NEJM 357:26 December 27, 2007

12

Providers worry about…

• Poorly designed performance reporting can lead to risk aversion

• The risk of misclassifying a physician threatens their reputation and livelihood

• There are more effective ways to address cost of care

• There are more accurate ways of measuring quality

13

Consumers and purchasers

• Consumers want more information about the quality of care their doctor’s provide

• Consumers want more information about the value they are purchasing

14

Our challenge

• Not measuring MD competency in some way is simply not an option

• Not controlling costs in some way is simply not an option

• Goal this afternoon:– Review the current measurement initiatives– Discuss what we can get behind in terms of assessing

the quality of care of MDs

15

What’s out there?

16

Sources of MD-specific information

• Word of mouth• BORIM physician profiles• Health grade profiles• MHQP profiles• Health plan products tiers• Angie’s list• Vitals.com• Consumers checkbook• Rate MD • Google

17

BORIM

18

DPH specialty profiles: CABG

19

DPH specialty profiles: CABG

20

http://www.mhqp.org

21

Healthgrades

22

23

24

Vitals.com

25

26

27

Benefits manager

• Husband and wife have just moved to Boston and are employed by the state and covered through the GIC

• They’re signing up for a health plan and need access to: Cardiology

28

29

GIC members pick a plan

30

31

32

A short primer on quality measurement

• Measures of quality and efficiency• Physician profiling• Tiering methodologies

33

Defining quality is a challenge

• Donabedian: structure, process, outcome

• IOM six aims: safe, effective, patient-centered, timely, efficient, equitable

• FACCT domains: staying healthy, getting better, living with illness or disability, coping with end of life

• Internal vs. External audience

34

What we need for a good system

• Standardized performance measures representing all relevant domains

• Access to pt level data

• Data verification and auditing

• Comparative analyses and reporting

Performance Measurement Accelerating Improvement

IOM 2007

35

Health care settings are not equally covered

• Hospitals - most mature

• Groups - somewhat developed

• Provider-level - very spotty

• Systems – nascent

• Health plans – NCQA led the way

• States - spotty

• Community - undervalued

Service line coverage is spotty

Confidential and Proprietary © March 2008 Sg2

Steps toward transparency: where are we on this steep climb?

Confidential and Proprietary © March 2008 Sg2

Meanwhile….on-line tools are proliferating…

Source: The Advisory Board Company. Drivers of Consumer Choice Implications from the 2007 Consumer

39

Loose talk about accuracy

• Accuracy of measurement– Reliability– Validity

• Misclassification of physicians– Reliability and validity– Cut-off points

40

Reliability

• Reliability speaks to the consistency of a measure– Internal consistency, (Cronbach’s Alpha)

usually measured between 0-1.0) – Test-retest– Inter-rater

• Reliability is a prerequisite for validity!!!

41

Validity

• Face validity (sounds good)• Content (are all dimensions of the construct

measured, assumes this is possible)• Construct (considered with that which is being

measured cannot be operationally defined)• Predictive (cholesterol and CAD risk)• Concurrent (high scores on safety culture and

low rates of SREs)

42

Risk of mis-classification

= area of uncertainty

Score Significantly below Significantly above

Risk of misclassification is low <2.5 % with sample size of 45 and measurement reliability of 0.7

Dana Safran. et al; J Gen Intern Med 20-06; 21:13-21

50th percentile

0.7

0.8

0.9

43

Efficiency measures

• Currently, the majority of efficiency measures rely on the MD as the unit of analysis

• Data sources: encounter and claims data

• Risk adjustment relies on same source

44

Efficiency measures

• Episode of treatment groupers (ETGs)– Pooled claims data are used to derive the

total cost for a particular episode– Care is then attributed to a physician– Physicians average cost is determined for

each ETG – Cost per ETG is averaged across all ETGs

that relate to that doc– Proprietary

45

Commentary on ETG validity

• We have reason to be concerned

Elizabeth McGlynn, PhD Associate Director, RAND Health; Distinguished Chair in Health Quality

46

47

• Measure cost efficiency via “ETG” methodology

• Measure quality via HEDIS, etc.

• Squeeze quality and cost scores from claims data

• Incent patient and physician behavior via differentials in co-payments

• Implemented in 2006

Tiering

48

GIC’s rules for 2008-2011

• Must individually rate MD’s in six specialtiesCardiology EndocrinologyOrthopedics GastroenterologyRheumatology OB-GYN

• Three tiers for all plans predeterminedTier 1: 20% Tier 2: 65%Tier 3: 15%

• Must use GIC’s data• Standardized reports to make the rankings

interpretable for the physicians (developed collaboratively with MMS input)

49

Tufts Navigator (GIC): Tiering Explanation

50

Tufts Navigator (GIC): Tiering Explanation continued

51

Landmark litigation

52

53

54

NY Principles for MD Tiering

• Core principles of the settlement– Accuracy– Transparency of information– Oversight of process

• Ratings examiner: a 501 c 3 organization– National standard setting organization– Regular reporting to NY AG

55

56

MMS sues GIC

57

58

What Physicians Are Saying

• “I am apparently treating patients for epilepsy, according to the GIC. Somewhat unusual for an ophthalmologist.”

• “Several patients listed couldn't possibly be mine, as I don't perform the designated type of surgery.”

• “There are procedures on my list that I have never performed. I called GIC and there was only voice mail.”

• “We received our data March 11 and were told that all feedback was due by March 14. I called the health plan to give feedback March 12. I left a message and my call was not returned.”

• “This program is unfair, and I'm hopping mad. Please let me know what I can do.”

Courtesy MMS

59

The Litigation

• Asks courts to “correct the wrongs” of the CPI• Defendants: GIC, Tufts, Unicare• Allegations

– Physicians falsely ranked and defamed– Patients misled and financially penalized

• Petition: Stop tiering, or require that it been done right, e.g.:– Transparency and 60 days prior notice– Feedback and correction process– Meaningful physician input– Accuracy, validity and reliability

Courtesy MMS

60

1. Aim to strengthen patient-physician relationships2. Involve physician in the design and implementation of all programs3. Use clinically important and sound performance measures4. Ensure sample sizes are adequate to support meaningful data analysis5. Rely on meaningful data and analytic techniques6. Share and review data with physician or practices prior to public release7. Ensure transparency of all quality and cost-effectiveness measure and

methods8. Identify and consider practice characteristics that may require special

attention in quality and cost-effectiveness monitoring9. Use uniform reporting formats10. Minimize unintended harmful consequences of quality and cost-

effectiveness monitoring and public reporting11. Be pre-tested before implementation.

MMS Principles for MD Tiering

61

62

63

Current initiatives & discussion

• BORIM & MMS developing credentialing guidelines• TJC requires us to conduct periodic assessments of provider • CMS has developed PQRI to advance MD quality• MHQP advancing provider measurement• PCHI has been evolving its approach to MD measures