TAI: Julie Bershadsky presentation

DATA AND ALL THAT FUN STUFF

Julie Bershadsky, PhDHuman Services Research Institute

2014 ITACC TA Institute

Data - surveys, questionnaires, phone interviews, administrative data, etc…

Data – surveys, questionnaires, phone interviews, administrative data, etc…

BUT…• Data are important!• Measure the right things

– Why, what, when– How

• Why measure?– Anecdotal evidence?– Required

• Policy makers want to see it– Evidence of change

• Baseline• Improvement

– Accountability

What to measure?• Process vs. outcomes• Outcomes:

– Reflect goals– Address issues that can be influenced– Directional– Accurate, valid and reliable– Affordable to collect– Culturally competent– Interpretable– Measurable!

• When to measure– How often?

How to measure?

• Mode– In-person– Phone– Mail– Web

• Response options• Sample?

– Who?

Using the data

• Do nothing• Can compare to an internal standard or goal• Can compare to external standard or goal• Can compare to others

• Last two: complicated…..

The Complication

?

If things were simple:

is better than

But things are not simple

?

Hmmmm…..

?

Solution: Risk adjustment

Aha!

Still complicated…• What should be adjusted?• What to adjust for?

– Differ between GA and MA• E.g. number of nice people• E.g. age

– Affect the outcome– BUT: legitimately affect the outcome (e.g.

biological factors)

National Core Indicators (NCI)

• Multi-state collaboration of state DD agencies interested in measuring how well public systems for people with developmental disabilities perform along several areas, including: employment, community inclusion, choice, rights, and health and safety

• HSRI-NASDDDS collaboration• Started in 1997• Now in over 40 states

• Why: – Allow state-to-state comparisons of performance of state DD systems

– Influence national and state policy

– Inform strategic planning and priority setting at state and national levels

National Core Indicators (NCI)• What:

– Indicator-based, focus on outcomes and quality of life

– Individual characteristics of people receiving services

– The locations where people live

– The activities they engage in during the day including whether they are working

– The nature of their experiences with the supports that they receive (e.g., with case managers, ability to make choices, self-direction)

– The context of their lives – friends, community involvement, safety

– Health and well-being, access to healthcare

National Core Indicators (NCI)• When:

– Yearly cycle• How (NCI Adult Consumer Survey):

– Standardized face-to-face interview– Response options: simple– Sample in each state

• No a priori exclusion criteria, all 18+ adults with IDD receiving at least one service in addition to case management

National Core Indicators (NCI)Using the data

• Reporting reqs• Track improvement• Comparing across groups• Comparing with national “average”• Comparing with other individual states• Formulating policy• Identify areas and priorities for QI• Raising awareness

Specialized Institutional facility

Group Home

Apartment Program

Independent Home/ Apt

Parent or Relative's home

Foster care/ Host home

Nursing Facility

Other

Don't know

0% 5% 10% 15% 20% 25% 30% 35%

6%

27%

4%

17%

34%

6%

1%

4%

0%

Residence

At Least One Psychotropic Medication by Living Arrangement

institu

tion

community

-based re

sidence

independent home/a

pt

parents/

relatives h

ome0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

53%

68%

53%

36%

20

Psych meds

21

Basic Exams & Screenings

• Higher percentages in provider-based settings

• Lowest for people living in parent/relative home

• Similar trend across indicators

institution community-based residence

independent home/apt

parents/relatives home

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100% 94%97%

91%

85%

95%

88%

73%69%68%

69%

58%

43%

90%

68%

49%46%

Physical exam

Dental visit

Vision screening

Hearing test (5 years)

Choice

0%

10%

20%

30%

40%

50%

60%

70%

80%

50%45%

64%

80%

60% 60% 58% 59%

22

Wages (‘09-’10)

Table 12.4 Proportion of people who reported they have enough privacy at home

State Overall In State

N Institution Community-Based

Individual Home

Parent’s Home

Significantly Above Average

ME 97% 220 n/a 98% 100% n/a

AZ 97% 311 n/a 95% 95% 98%

KY 95% 319 n/a 94% 100% 98%

GA 95% 381 n/a 96% 100% 93%

PA 94% 835 n/a 95% 96% 95%

Within Average Range

CT 93% 247 n/a 92% 95% 94%

MEORC 93% 279 n/a 92% 94% 93%

LA 93% 258 n/a 87% 95% 95%

OH 91% 345 n/a 89% 88% 94%

NY 91% 1,637 n/a 89% 93% 93%

MA 91% 379 n/a 90% 93% 91%

NC 91% 499 n/a 90% 94% 91%

AR 90% 226 90% 80% 93% 93%

MI 89% 244 n/a 81% 88% 94%

AL 88% 226 n/a 90% n/a 86%

MO 87% 341 n/a 79% 92% n/a

IL 87% 231 n/a 85% n/a 88%

Significantly Below Average

HI 82% 201 n/a n/a n/a 81%

NJ 81% 338 n/a 80% n/a 83%

SC 79% 300 n/a 76% 86% 77%

NCI Average 90% 7,817 90% 88% 94% 91%

Unadjusted:

Table 4.1 Number of times people reported they went out shopping in the past month (Adjusted Variable)

State Adjusted Average

N (observed)

HI 5.8 430

ME 5.8 317

NC 5.3 883

AZ 4.5 351

MI 4.3 408

NY 4.3 2,504

MO 4.2 497

LA 4.2 417

MA 4.2 494

PA 4.1 1,234

CT 4.0 346

GA 3.8 485

AR 3.6 387

AL 3.2 345

IL 2.9 333

MEORC 2.8 307

KY 2.6 471

OH 2.5 273

SC 2.5 402

NJ 2.5 409

NCI Average 3.9 11,293

Adjusted:

Example: Georgia

• Use of psychotropic medications above average• Further study:

– After transition from institutions• Prescribed for the first time• Number of prescribed meds wend up

• Initiative to reduce use • Reflected in data

Example: Kentucky• Behind other states in employment, health

and exercises, psych meds, loneliness and friendship

• Employment: revised waiver, increased payment for supported employment, decreased payment for day activity services

• Psych meds: revised waiver• New service: community access• Targeted health and wellness training

Example: Arizona• Identified deficits in

women’s healthcare outcomes

• Took action to improve women’s ob/gyn access resulting in rate of annual visits increasing from 30% to 70%– Educating parents

– Educating doctors

Prioritizes actions and quality efforts on:• Aging caregivers• Well-woman checkups• Loneliness & self advocacy• Choice of case managers• Exercise• Service Costs• Physician access

Example: Alabama

• Choice and decision-making identified as area for improvement; • Five year plan to increase # of PCP facilitators and offering SRV training

to families

• Using consumer and family surveys to measure satisfaction pre- and post- community placement from closing of state facility

Why Are Data Important?Without Data

• Could not distinguish policy goals and actual outcomes of those policies

• Could not compare the effectiveness and outcomes of specific services

• Could not track important system changes – negative or positive

• Could not track the impact of system reform

With Data

• Can measure and developstrategic goals

• Can enhance transparency• Can involve individuals and

families in the interpretation of results

• Can communicate system values – e.g., choice, health, relationships

• NCI CA Movers Presentation

THE ENDWhat did she say?

CONTACT INFO

JULIE BERSHADSKY, PhDJBERSHADSKY@HSRI.ORG

617-876-0426 ext 2331