Measuring Participation andMeasuring Participation and ... · Participation and Environmental Factors 1 Measuring Participation andMeasuring Participation and Environmental Factors:

[email protected] 28 October 2010

Participation and Environmental Factors 1

Measuring Participation andMeasuring Participation and Environmental Factors:

Key Aspects of the ICF Framework

Allen W. Heinemann, PhD, ABPPProfessor, Physical Medicine and Rehabilitation

Feinberg School of Medicine, Northwestern UniversityDirector, Center for Rehabilitation Outcomes Research

Rehabilitation Institute of Chicago1

Presentation Objectives

• Illustrate the state-of-the-art in measuring it ti i ticommunity participation

• Describe challenges in measuring environmental factors that affect participation

2



3

International Classification of Functioning Disability and Health

Body Functions&

Structures

Activities &

Participation

Environmental Factors

BarriersFunctions Capacity

4

Barriers

Facilitators

Functions

Structures

Capacity

Performance



Participation Measurement Project Task Overview

1. Conduct literature review2. Conduct focus groups with

8. Collect population data from persons with and without g p

consumers, caregivers, providers, payers, policy makers

3. Develop items and rating scales to operationalize participation

4. Conduct cognitive interviews with consumers and general

disabilities9. Refine instrument10. Collect and analyze

population data from persons with and without disabilities (Behavioral Risk Factor Surveillance System)

public5. Revise items, rating scales6. Pilot test participation

instrument7. Refine instrument

13. Collect 3rd round of data, augmented with panel sample (~1100)

14. Evaluate CPI as part of routine post-discharge follow-up assessment - underway

5

Construct Development: Stakeholder Focus Groups

• Consumers (5)– Chicago, IL– Englewood, CO

• Caregivers (6)– Chicago, IL– Englewood, CO

• Payers (1)– Insurance Research Group

• Policy Makers (2)– Federal: Washington, DC– State: Chicago, Illinois

• Total Participants• Rehabilitation

professionals (4)– Chicago, IL– Englewood, CO

– 18 groups– 138 participants

6



Focus Group Questions

1. What does the word “participation” mean to you? a) What does it mean “to participate”a) What does it mean to participateb) Why is participation important to you?

2. What areas of everyday life are most important to you? a) If taken away, what areas would you miss most/would be the

biggest loss to you?

3. What issues or barriers most affect your participation? 4 What are your biggest supports to participation?4. What are your biggest supports to participation? 5. What recommendations do you have for improving

participation opportunities for people with disabilities?

7

Focus Group Quotes

Consumers “It means to make some sort of contribution in life”“Working and living”“It's important to socialize”It s important to socialize“Just being able to do the things that you enjoy”

Care-givers “Just doing what you want to do”“Being able to go to the store, to school, being able to do all of

the things that normal people do”Providers “It goes beyond just daily living activities”

“What you want when you want with who you want”

8

y y y“You are seen as having something to give”

Payers “Allowed to fail, take on challenge”“Lack of information can be as isolating as any physical barrier”

Policy Makers

“Just the stuff we do and take for granted”



Qualitative Analytic Procedures

• Analyses and interpretation• Open coding of data by 2

•Codes represent • Content themesOpen coding of data by 2

researchers, including team members with disabilities

• Topical coding to identify key emerging themes

• Detailed coding of • individual data within groups

Content themes• Narrative stories & examples

• Areas of agreement and differences on specific issues

E d d t• individual data within groups• comparative analysis across

individuals• across stakeholder groups

•End product• Participation item pool that reflects the themes and concerns & priorities of multiple stakeholders 9

What We Learned: Participation Enfranchisement

Meaningful Engagement/

ParticipationValues

Choice & Control

Access & Opportunity

Personal & Societal

Responsibilities

Having an Impact & Supporting

Others

Engagement/Being a Part of

Enfranchisement

10

• Hammel J, Magasi S, Heinemann AW, Whiteneck G, Bogner J, Rodriguez E. What does participation mean? An insider perspective from people with disabilities. Disability and Rehabilitation, 30:19,1445‐1460.

• Magasi S, Hammel J, Heinemann AW, Whiteneck G, Bogner J. Participation: A comparative analysis of multiple rehabilitation stakeholders’ perspectives. Journal of Rehabilitation Medicine, 41, 936‐944, 2009.

Social Connection, Inclusion &

Membership



Community Participation Indicators

• Source Material– Focus group thematic codes – Comprehensive literature review

• Content Domains– Productive roles (student, worker, and homemaker)– Social roles (spouse or partner, family member,

caregiver, caregiver to children or aging parents, relationships with adults and friends

– Community roles (volunteer, civic, political, organization, or community group member)

11

Community Participation IndicatorsVersion 2• 20 engagement items

F f ti it• Frequency of activity• Importance of activity• Evaluation of activity frequency

• 48 enfranchisement items• Higher “ceiling” soughtHigher ceiling sought• Revised rating scale – from 4 to 5 agreement

categories

12



13

14



15

16



Round 3 Data: Sample Characteristics• N = 1163• Source

– Panel survey: 51%

• Impairments– Mobility 63%– Mental 23%Panel survey: 51%

– Community sites: 31%– Former inpatients: 18%

• Median age = 53 yr (+ 17)• Women = 49%• Race

– Caucasian 72%– African-American 12%

– Hearing 14%– Vision 13%– Learning 11%– Communication 9%

• Disability Severity

17

– African-American 12%• Hispanic 7%• Self-report 72%

Results: Activity Patterns

• Activity Categories• Social activities• Productive activities• Low frequency activities

• Activity ImportanceY• Yes vs. no.

18



Social Activities

80%

100%

40%

60%

80%

7 days5-6 days3-4 days1-2 daysNone

0%

20%

Spend time wfamily

Keep in touchw family

Get out &about

Keep in touchw friends

Spend time wfriends

Socialactivities

‘Productive’ Activities

80%

100%

40%

60%35+ h20-34 h10-19 h5-9 h1-4 hNone

0%

20%

Work Child care Homemaintenance

Householdfinances

Volunteer Learn



100%

Low Frequency Activities

20%

40%

60%

80%

5+ d4 d3 d2 d1 d0 d

0%

20%

Hobbies Active recreation

Religious activities

Support Groups

Entertainment Community organizations

Civic activities

Percent of Important Activities Performed Often Enough

Mean = 55%SD = 28N = 1149

22



Scoring Decision:Activities and Activity Importance

• Avoid creating a “busy-ness” index• Personal preferences, opportunities,

environmental factors influence activity patterns

• Report descriptive information about activity patternspatterns

• Evaluate utility of “percent of important activities performed often enough”

23

Confirmatory and Exploratory Factor Analyses

1. Examined response distributions and sum scores across item categories separately forscores across item categories separately for recoded and unrecoded items

2. Completed CFA on recoded & unrecoded items with Sample 1

3. Completed EFA on recoded & unrecoded items with Sample 2with Sample 2– 8 eigenvalues > than 1; 4 eigenvalues > 1.5

24



Rating Scale Analysis Summary:Enfranchisement Subscales

• Involvement in Life Situations– Items well targeted to sample– High person reliability

• Control over participation– Items well targeted to sample

One slightly misfitting item (I speak up for– One slightly misfitting item (I speak up for myself)

– High person reliability

25

Item MeasureModel

SEInfit

MnSq

Involvement in Life SituationsItem Statistics

I spend time doing things that improve my community .89 .04 1.03I have a say on decisions in my community .61 .03 1.15I am actively involved in my community .60 .03 .71I assume leadership roles in organizations .55 .03 1.34I have influence in my community .39 .03 .84I participate in a variety of activities .01 .03 .95I contribute to the well-being of my community -.11 .03 .71

l l k h ll 11 03 1 09I regularly seek out new challenges -.11 .03 1.09I contribute to society -.18 .03 .81People see my potential -.43 .03 .97I spend time helping others -.43 .03 1.10I feel safe participating in community activities -.77 .04 1.23People count on me -1.02 .04 1.19



Control over ParticipationItem Statistics

Item MeasureModel

SEInfit

MnSqI am able to go out and have fun .75 .04 .97I actively pursue my dreams and desires .74 .04 .89I have opportunities to make new friends .72 .04 1.19I do important things with my life .45 .04 .96I live my life fully .42 .04 .81I live my life the way that I want .36 .04 .97I participate in activities when I want .10 .04 .85I participate in activities that I choose 08 04 1 11

27

I participate in activities that I choose .08 .04 1.11I do things that are important to me -.14 .04 .68I am in control of my own life -.25 .04 1.00I speak up for myself -.29 .04 1.41I have choices about the activities I do -.52 .04 .92I have control over how I spend my time -.57 .04 1.08I have the freedom to make my own decisions -.74 .04 1.04I take responsibility for my own life -1.09 .05 1.23

Disability Severity Affects Involvement in Life Situations

Known Groups Validity Evidence



Disability Severity Affects Control over Participation

Known Groups Validity Evidence

Conclusions

1. Participation as measured by activity frequency, evaluation and enfranchisement items is not a unidimensional construct• Don’t model activity frequency, importance and evaluation in a

composite measure• Individuals’ preferences and opportunities vary greatly• Personal preferences determine individual’s participation profile

2. Involvement in and control over participation are distinct constructs that can be measured reliablyconstructs that can be measured reliably

• Expanded set of community enfranchisement items contains 2 distinct item sets that are suitable for item banking and computer adaptive administration

3. Construct validity of involvement in and control over participation is promising

30



D l i E i lDeveloping Environmental Factor Measures for Persons with Stroke, Traumatic Brain Injury and Spinal Cord Injury

31

Injury, and Spinal Cord Injury

Person in Environment Theorists

• Kurt Lewin B = f (P, E)• M. Powell Lawton B = f (P, E, P x E)

– Geriatrician, described personal, suprapersonal, social, and physical, non-personal aspects of the environment

• Rudolph Moos– Social Ecology Model describes institutionalized psychiatric and

geriatric patients• Mary Law

– Person, Environment, Occupation (PEO) model– Canadian Occupational Performance MeasureC O p

• W. Dunn (KU)– Ecology of Human Performance Framework

• Patrick Fougeyrollas– Disability Creation Process emphasizes environmental factors

32



The ICF Provides a Taxonomy of Environmental Factors

Craig Hospital Inventory of Environmental Factors (CHIEF)

• Identifies barriers in physical/architectural, attitudes support, services/assistance, work/schoolattitudes support, services/assistance, work/school policies

• 25 items (12 item short form)• Scored in terms of

– Frequency of environmental barriers encountered – Impact on participation– Frequency-magnitude product score

• Whiteneck, Gerhart, Cusick. Identifying environmental factors that influence the outcomes of people with traumatic brain injury, JHTR, 2004



Facilitators and Barriers Survey (FABS/M) • Describes perceived influence of the environment for

people with mobility impairments• 133 items cover 5 ICF environmental dimensions• Scoring focuses on whether, how much and how often

environmental features influence participation at home and in the community– Frequency scale: never, to daily– Facilitator scale: None, helps some, helps a lot– Barrier scale: None, limit some, limits a lota e sca e o e, so e, s a o

• Gray, Hollingsworth, Stark, Morgan. A subjective measure of environmental facilitators and barriers to participation for people with mobility limitations. Disabil Rehabil, 2008

Measure of the Quality of the Environment (MQE)

• Assesses perceived influence of environmental factors on social participation of individuals infactors on social participation of individuals in relation to their abilities and limitations

• 109 items• Scored on 7-point obstacle/facilitator scale

– “major, moderate, and minor obstacle” through “no influence” to “minor, moderate, or major facilitator”

B h N F ll M f th• Boschen, Noreau, Fougeyrollas. Measure of the Quality of the Environment: Reliability Study. Canadian Journal of Rehabilitation, 1997



Study Goals

• Develop, test and evaluate measures of EF and their influence on participation for persons withtheir influence on participation for persons with stroke, TBI and SCI

• Determine if an item response theory approach to measuring participation facilitators and barriers yields item banks that complement recently developed EF instruments

37

recently developed EF instruments• Determine if a validated set of EF instruments

will allow us to determine to what extent they predict social health and participation for people with long term disabilities

Progress

• Analyzed Focus Group Data– Identified themes of EF that influence participation

• Review of current EF literature – Gathered project relevant information from 100 unique sources

• Developed Conceptual Framework– Informed by analysis of focus group data and literature review

• Item pooling, binning, and winnowing– Pooled 3500 items relevant to framework

Binned 2300 items to specific domains

38

– Binned 2300 items to specific domains– Winnowed 350 items within domains

• Item Development– Drafted and revised items– Computed reading level– Evaluated Spanish translatability of items



Conceptual Framework

Assistive Technology Built

EnvironmentEconomicFinancial

Environment

Natural Environment

Transportation

Environmental Factors

39

Systems Services

and PoliciesAccess to

Information and

Technology

Social Supports

and Attitudes

Example of Domain Definition:Social Environment

• Pertains to factors in the social i t i l di i d itienvironment, including perceived positive

and negative disability-related attitudes and behaviors (e.g., acceptance, stigma, marginalization) exhibited by society and members of respondents’ communities and social networks.

40



Binned and Winnowed Items

Domain Binned WinnowedE i / Fi i lEconomic / Financial 42 35

Built Environment 605

Natural Environment 79 37

Assistive Technology 178 104

Social Supports & Attitudes 710 59

S S i & P li i

41

Systems, Services & Policies 411 35

Transportation 136 28

Access to Information & Technology 112 37

Total 2273 335

Next Steps

• Cognitive Testing– Administer representative selection of items to disability samplesp y p– Analyze cognitive interview data and revise items if necessary

• Pilot Testing– Recruit 100 subjects for each targeted disability group (n=300)– Administer EF item banks– Analyze data to identify item specific characteristics

• Field Testing

42

Field Testing– Recruit 200 subjects for each targeted disability group (n=600)– Administer EF items banks– Compare with legacy environmental measures



Archives of Ph i lPhysical Medicine and Rehabilitation, September 2010 (supplement)

Discussion

For More Informationh i @ th t [email protected]

www.ric.org/cror•Rehabilitation Research and Training Center on Measuring Rehabilitation Outcomes and Effectiveness NIDRR Award H133B040032 and

•Rehabilitation Research and Training Center on

44

Improving Measurement of Medical Rehabilitation Outcomes (H133B090024)

Documents

Measuring Participation andMeasuring Participation and ... · Participation and Environmental Factors 1 Measuring Participation andMeasuring Participation and Environmental Factors: