30. Manly - In-Home Evaluation - rsvpBOOK

10/22/2016

1

In‐home cognitive assessment: research and clinical applications for an increasingly diverse world

Jennifer Manly, PhD

[email protected]

Financial Disclosure

I have no financial relationships to disclose:

Employee of: Columbia University

Consultant for: none

Stockholder in: none

Research support from: National Institutes on Aging

Honoraria from: none

10/22/2016

2

Collaborators

Maria Glymour

Laura Zahodne

Adam Brickman

Karen Siedlecki

Supported by

• NIA R01 AG16206 & AG028786 (PI: Manly)

• NIA R01 AG037212 (PI: Mayeux)

Danurys Sanchez

Sze Liu

Yaakov Stern

Richard Mayeux

Nicole Schupf

Learning Goals

1. Identify the advantages of home‐based cognitive testing for research and clinical work, as well as challenges to validity that are specific to in‐home assessment.

2. Describe currently available tools available for in‐home neuropsychological assessment, and their validity and reliability for assessment of cognitive impairment across diverse individuals.

3. Describe how emerging technologies for in‐home testing can address ecological validity and deepen our understanding of cognitive function beyond the select group who receive neuropsychological testing in the clinic or office.

10/22/2016

3

Where is this?

What is this?

Photo credit: Adam M. Brickman, PhD

Focus of this workshop(caveats)

• Adults & older adults

• Neuropsychological assessment• (not rehabilitation)

• Reduction of disparities in research participation and equitable access to neuropsychological assessment

10/22/2016

4

INCREASING DIVERSITY

0

10

20

30

40

50

60

70 1990

20002025

2050

White, not Hispanic

BlackAmerican Indian, Eskimo and Aleut

Asian and Pacific Islander

Hispanic origin(of any race)

Percent of the Population, by Race and Hispanic Origin: 1990 ‐ 2050

Source: U.S. Bureau of the Census, decentennial census and population projections

An unavoidable future for neuropsychology• We must be responsive to the rapidly changing and diverse nature of the population by taking into account the impact of cultural, linguistic, and educational background of the people we assess in all aspects of our practice, including not only the measures and normative standards that we use, but also the languages in which we are competent to assess, the setting in which neuropsychological assessment takes place, the educational materials we provide, and the recommendations we make.

10/22/2016

5

Learning Goals




Neuropsychological research is WEIRDHenrich, Heine, Norenzayan, 2010

• W estern• E ducated• I ndustrialized• R ich• D emocratic

• 80 percent of study participants• 12 percent of the world’s population

• Even basic cognitive processes are culturally bound• Fundamental aspects of our scientific literature are based on a small, unrepresentative subset of the population, yet we make assumptions about human cognition

10/22/2016

6

How has this affected neuropsychological research?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

10/14 ‐ 10/15 10/15 ‐ 10/16

Archives of Clinical Neuropsychology manuscripts mentioning race/ethnicity/language use over 2 years

Mentions race/ethnicity Does not mention race/ethnicty

Selection Bias

• Differences in recruitment across racial/ethnic groups may lead to non‐generalizable results

• Ethnic minority participants may not be broadly representative of the community

• Consider how barriers to participation may influence sample characteristics and bias results

• Racial and ethnic minorities are less likely to present to Memory Disorders Clinics, are less likely to receive a formal diagnosis of AD than non‐Hispanic Whites

• Minorities who present to clinics are more likely to have neuropsychiatric symptoms than Whites

10/22/2016

7

How does in‐home assessment change the landscape of neuropsychological research?

• Appears to be critical for increasing representativeness of participants

• Leads to new knowledge:• Mechanisms of disparities in cognitive function across the lifecourse

• Sources of resilience for people at risk for cognitive impairment

Examples of studies using in‐home assessment• Washington Heights Hamilton Heights InwoodColumbia Aging Project (WHICAP)

• Health and Retirement Survey (HRS) ADAMS

• REasons for Geographic and Racial Differences in Stroke (REGARDS) Study

• University of Alabama at Birmingham Study of Aging

• Developmental Epidemiology Network cohort ‐Neuropsychological Outcomes Study

10/22/2016

8

WHICAP• Washington Heights‐Inwood Columbia Aging Project

• N > 6,500 older adults• Inclusion criteria: Medicare‐eligible residents, age 65+,

Spanish or English speaking

• Seen in home at 18‐24 month intervals

PI: Richard Mayeux

0

5

10

15

20

25

30

35

40

Hispani African American

Non‐Hispanic White

Proportion of sample

10/22/2016

9

WHICAP Participants

10/22/2016

10

WHICAP

Annual age‐specific inciden

ce

Tang et al., 2001; Neurology 56: 49‐56

Evidence of DisparitiesIncidence of AD by Age and Race/Ethnicity WHICAP 2001

10/22/2016

11

Race, education, literacy, & incident ADRelative Risk

** p < .01; *** p < .001

***

***

***

***

**

Zahodne et al., (2014)

Survival curves by (a) age, (b) education, (c) gender, and (d) degree of bilingualism

10/22/2016

12

Aging And Demographics Memory Study (ADAMS)• Supplement to the Health and Retirement Survey (HRS)

• Conducted a population‐based study of dementia

• First of its kind to conduct in‐home assessments of dementia in a national sample that is representative of the U. S. elderly population

Evidence of DisparitiesPrevalence of Cognitive Impairment by Age and Race/Ethnicity HRS 2006

Alzheimer’s Association, 2010

10/22/2016

13

HRS used ADAMS to estimate national prevalence and risks of AD

Racial disparities by US region of primary school education in HRS

Liu, Glymour, Zahodne, Weiss, & Manly, JINS (2015)

10/22/2016

14

REGARDS telephone based cognitive assessment• Six‐item screener (SIS)

• CERAD Words list Learning

• CERAD Word list Recall

• Animal fluency

• Letter Fluency

10/22/2016

15

Incident cognitive impairment in the US

Wadley et al., 2011

REGARDS – Stroke Belt is associated with incident cognitive impairment

• A national cohort of adults ≥ age 45• 38% African Americans and 62% European Americans• 56% Stroke Belt residents and 44% from the remaining contiguous United States and the District of Columbia

• Residents of the Stroke Belt had greater adjusted odds of incident cognitive impairment than non‐Belt residents (OR = 1.18; 95% CI 1.07 – 1.30)

• Regional disparities in cognitive decline mirror regional disparities in stroke mortality, suggesting shared risk factors for these adverse outcomes

• Efforts to promote cerebrovascular and cognitive health should be directed to the Stroke Belt.

Wadley et al., 2011

10/22/2016

16

University of Alabama at Birmingham Study of AgingRichard Allman

• Longitudinal observational study

• Community‐dwelling Medicare beneficiaries in five counties of central Alabama

• 65 or older at baseline

• 433 completed in‐home assessments of cognitive function at baseline and 4 ‐year follow‐up

Alabama counties of residence during participants’ childhood schooling

Michael Crowe et al. J Gerontol A Biol Sci Med Sci2012

• Student–teacher ratio and school year length, not expenditures, were associated with baseline cognitive function

• Independent of education level, age, race, gender, income, reading ability, vascular risk factors, and health behaviors

• Associations were stronger in those with lower levels of education (≤12 years)

• Not related to 4‐year change in cognitive function

10/22/2016

17

Developmental Epidemiology Network Cohort ‐Neuropsychological Outcomes StudyMaureen Durkin, Louise Kuhn, Veronica Hinton, et al.

• 1,665 infants

• Weighing 1,500 g or less at birth

• Born in one of five hospitals in Massachusetts, New York, or New Jersey in 1991–1993

• 1,442 infants survived to hospital discharge

• Maternal postpartum interviews

• Data were abstracted from obstetric and neonatal charts

• 439 children aged 6–8 years enrolled in follow‐up studies

• All neuropsychological assessments were done in child’s home

American Journal of EpidemiologyCopyright © 2003 by the Johns Hopkins Bloomberg School of Public Health All rights reserved

Vol. 158, No. 11Printed in U.S.A.

DOI: 10.1093/aje/kwg257

Influence of Breastfeeding on Cognitive Outcomes at Age 6–8 Years: Follow-up of Very Low Birth Weight Infants

Melanie M. Smith1,2, Maureen Durkin1,2,3, Veronica J. Hinton1,4, David Bellinger5, and LouiseKuhn1,2

1 Gertrude H. Sergievsky Center, Columbia University, New York, NY.2 Department of Epidemiology, Joseph L. Mailman School of Public Health, Columbia University, New York, NY.3 Department of Population Health Studies, University of Wisconsin Medical School, Madison, WI.4 Department of Neurology, College of Physicians and Surgeons, Columbia University, New York, NY.5 Neuroepidemiology Unit, Children’s Hospital and Harvard Medical School, Harvard University, Boston, MA.

Received for publication February 25, 2003; accepted for publication June 3, 2003.

10/22/2016

18

How does in‐home assessment change the landscape of neuropsychological research?• Address barriers to diverse research participation:

• Mistrust• Financial Resources• Disability

• May enrich breadth and quality of certain types of data• ADL and IADL• Social interactions• Physical and social supports

• Must pair with:• Equal community partnership• Representative recruitment method• Reliable and valid instruments• Appropriate norms• Clinician competency

Causes of mistrust of research on cognition and neuropsychological testing

• Lack of cultural and linguistic competency• Scheduling• Practitioner• Billing

• Impact of Tuskegee and other medical abuses• Legacy of IQ testing among minorities• Little evidence of benefits of medical research for minorities

• Intractable disparities

• Recent experiences of discrimination in the medical setting

• Lack of access to informationAdapted from Scharff et al., 2010

10/22/2016

19

Disability is a barrier to travel to clinic

• Physical limitations make travel to the clinic more difficult

• Frailty

• Wheelchairs

• Visual impairment

• Amplified for people with low SES

• Reliance on caregivers increases overall burden of a clinic visit

Financial resources are needed to come to the clinic/hospital• Travel time

• Transportation costs

• Parking

• Food

10/22/2016

20

ADL and IADL assessment at home

• Up‐close examples of how people manage daily activities

• Personal

• Home care

• View medication set‐up

• Not reliant on an informant

Social interactions

• Scheduling and planning

• Greeting

• Novel social situation

10/22/2016

21

Assess physical and social supports• Ramp for wheelchair?

• Layout accommodate visual impairment?

• Cluttered medicine cabinet?

• Family interactions

• Easier to include family members as informant

Equal community partnerships

• Community‐based participatory research• Create a trusting relationship• Share concerns and determine community priorities• Define questions• Identify important constructs• Choose or create measures• Decide on all procedures, weighing costs and benefits• Monitor participant burden• Communication of results• Plan next steps, maintaining relationships

• Maintain a community advisory board

• Consider word‐of‐mouth

10/22/2016

22

Representative recruitment method• Convenience samples, volunteers, and clinic‐based samples rarely lead to representative samples

• Random Digit Dialing

• Representative school‐based

• Medicare or regional health plan

• Voter registration lists

• Door‐to‐door

Reliable and valid instruments

10/22/2016

23

Goals of Neuropsychological Assessment

• Determine standing of test score relative to others• Descriptive

• Detecting/classifying impairment• Diagnostic

• Measuring change

• Predicting everyday functioning

Appropriate norms

• If diagnostic, match demographics and setting of testing

• Age, sex, years of school• Race/ethnicity, language, school quality• Geographic region

• If descriptive, match age and setting of testing

• These variables could affect both level of score and confidence interval

10/22/2016

24

Clinician Competency

• Training and experience are needed to interpret home‐based assessment just as they are needed to interpret assessment in other settings

• Lack of cultural competency could defeat the purpose of offering home‐based assessment


10/22/2016

25

Neuropsychological testing in the Dominican Republic


Safety Concerns

• Are home visits less safe than in‐clinic visits?

• Safety concerns have been addressed in prior research studies, including:

• Inclusion of familiarity of neighborhood and cultural competency in hiring testers

• Sending more than one tester or another staff member

• Scheduling in the morning

• Asking participants if they have safety concerns during scheduling

• Considering neighborhood norms

10/22/2016

26

Learning Goals




What traditional neuropsychological measures are appropriate for home assessment?

10/22/2016

27

What traditional neuropsychological measures are appropriate for home assessment?

• Given that the conditions under which the tests were normed can be closely replicated:

• The majority of them!

• Exception = Orientation

Making neuropsychological assessment accessible• Practicing outside the clinic is part of our field’s tradition

• Bedside assessment

• Nursing homes

• Community‐based “satellite” clinics

• Retirement communities and NORCs

• Schools

10/22/2016

28

How can the reliability and validity of home‐administered assessments been established?

Reliability and validity Bolarinwa, 2015

10/22/2016

29

How can the reliability of in‐home testing be established?• The uniformity of a test when repeated measures are conducted

• Consistency

• Reproducibility when scores remain consistent over time

• Test‐retest

• Split half or internal consistency

• Alternative form

Making neuropsychological assessment accessible• Practicing outside the clinic is part of our field’s tradition

• Bedside assessment

• Nursing homes

• Community‐based “satellite” clinics

• Retirement communities and NORCs

• Schools

How many of the measures and batteries have documented reliability testing in non‐clinic‐based settings?

10/22/2016

30

Construct ValidityMemory

Language

Speed

SRT-Total

SRT- Delayed recall

SRT- Delayed recog

Naming total

Letter Fluency

Category Fluency

Similarities

Repetition

BVRT recognition

Comprehension

BVRT Matching

Rosen

Identities/Oddities

Color Trails 1

Color Trails 2

e1

e2

e3

e7

e8

e9

e10

e11

e12

e13

e14

e15

e6

e5

e4

Visual-spatial

2 = 1022.27RMSEA = .04CFI = .93

Configural invariance • The pattern of relationships, or the factor structure, is identical across groups

• Each factor is associated with the same set of items across the groups

• For example, the memory factor was represented by the same three observed variables (i.e., the Selective Reminding Test total recall, delayed recall, and delayed recognition variables) in both English and Spanish speakers.

K. Siedlecki et al., 2008

10/22/2016

31

Metric invariance

• The magnitudes of the unstandardized factor loadings are the same across the language groups

• Provides evidence that the corresponding latent factors have the same meaning across groups, because the latent factors reflect what is common among the observed variables

• For example, the relationship between the observed variables and the latent variables was the same across English and Spanish speakers


Structural invariance

• Refers to the invariance of the relationships (correlations) between or among the latent factors

• The correlations between the memory construct and the other constructs were higher for English speakers than Spanish speakers


10/22/2016

32

Construct ValidityMemory

Language

Speed

SRT-Total

SRT- Delayed recall

SRT- Delayed recog

Naming total

Letter Fluency

Category Fluency

Similarities

Repetition

BVRT recognition

Comprehension

BVRT Matching

Rosen

Identities/Oddities

Color Trails 1

Color Trails 2

e1

e2

e3

e7

e8

e9

e10

e11

e12

e13

e14

e15

e6

e5

e4

Visual-spatial

2 = 1022.27RMSEA = .04CFI = .93

MIMIC model

Aiken Morgan et al., Exp Aging Res. 2010

10/22/2016

33

NIH Toolbox

• NIH Toolbox for the Assessment of Neurological and Behavioral Function

• Modules• Motor function• Sensation• Emotion• Cognition ‐ Seven measures, takes 30 minutes

• Executive Function• Episodic Memory• Language• Processing Speed• Working Memory• Attention

• Validation• English ‐ 476 participants• test‐retest reliability, age effects on performance, and • convergent and discriminant construct validity

• Access to Cognition measures require approval, which is granted to researchers and clinicians with knowledge of how to use neuropsychological tests

Portability

10/22/2016

34

NIH Toolbox Picture Sequence Memory Test

NIH ToolboxDimensional Change Card Sorting

10/22/2016

35

NIH Toolbox Oral Reading Recognition Test

Learning Goals




10/22/2016

36

Emerging technologies for non‐clinic‐based cognitive assessment

• Video Teleconference (Cullum et al.)**

• Internet Assessment

• Mailed tablet

• Cell Phone burst measures (Sliwinski)

• Unobtrusive Assessment (Kaye)• Wearables

• Home‐Based

Digital PenRhoda Au et al.

• Number of strokes• Total ink length• Time it took to draw• Pen speed • Pause, hesitation, and time spent simply

holding the pen • Length of the major and minor axis and

eccentricity of the fitted ellipse• Largest angular gaps in the clockface• Digits that are missing or repeated• Height and width of digit bounding boxes• Omissions or repetitions of hands• Hour hand to minute hand size ratio• Presence and direction of arrowheads• Play it back later

10/22/2016

37

testmybrain.com & manybrains.netLaura Germine

testmybrain.com & manybrains.netLaura Germine

• “began as a way of providing cognitive testing tools directly to members of the public who didn't have access to local resources for neuropsychological assessment”

• “citizen science research platform” & “open, participant‐centered science”

• 50 cognitive tests

• 20 publications

• 1.5 million participants

10/22/2016

38

Testmybrain.comFamous Faces Test

Testmybrain.comFamous Faces Test

10/22/2016

39

Testmybrain.comLaura G

Ecological Momentary AssessmentStress and cognitive agingMartin Sliwinski

• Assessment “bursts”

• 14 days, twice a year for 4 years

• Specially‐programmed smartphones

• Complete “brain games” assessing cognitive performance

• Morning, five times throughout the day, and at bedtime

10/22/2016

40

Project Talent, 1960

• Schools were randomly sampled from cells based on region and type of school across the US

• All students in grades 9‐12 were selected to participate in two full days of data collection

• Study collected extensive personality, knowledge, and cognitive ability data, and completion of an extensive demographic questionnaire

• Follow‐up surveys were administered at 1, 5, and 11 years after each grade cohort graduated high school

• Nationally representative longitudinal study

• In 1960 377,000 9th‐12th

graders from 1237 schools were assessed

10/22/2016

41

Project Talent Aging StudyTwins and Sibs R01 AG056163

• Carol Prescott, University of Southern California• Assess cognitive functioning and AD in a twin/sib sample and in a parallel sample of siblings drawn from the same high schools as the twins

• Including schoolmate controls provides leverage for separating the etiological relevance of family environmental, school and socio‐environmental factors on AD risk and cognitive outcomes in later life and evaluating the mediating and moderating roles of educational experiences, occupational activities, and later‐life cognitive engagement

Project Talent Aging StudySchools RF1 AG056164

• Jennifer Manly, Columbia University• Evaluate the role of educational experiences in potentially promoting cognitive resilience to early life adversity

• Evaluate racial/ethnic differences in the benefits of attending higher quality schools or the mechanisms though which education influences dementia risk

10/22/2016

42

PTAS Internet based measures

(1) (2) (3) (4) (5)

Abstract ReasoningIn this exercise, each item consists of a set of figures arranged in a pattern that has been formed according to certain rules. Look at the

pattern in Sample Item 1.

10/22/2016

43

PTAS Cognitive Measures

• N 15,000

• Exploring Interactive Voice Recognition (IVR) for verbal memory, fluency, and reading recognition

• Send tablet in the mail for those without internet access

• Jeff Kaye, MD et al.,• Ambient Independence Measures (AIMs)• Unobtrusively monitor important health changes due to chronic disease and

aging• Sensors distributed throughout an individual's natural living environment• BIG DATA

from Patel et al, 2012

10/22/2016

44

Emerging technologies for in home assessment: from research to practice

• Are these measures applicable to clinical practice?• Diagnostic accuracy

• Predictive accuracy

• Utility in crafting treatment• Efficacy over and above existing methods

• Cost

• Utility among diverse individuals

• Confidentiality of data, encryption, other technological risks must be further addressed

Getting started tips for in‐home research• Engage community advisors first

• Present in‐home as an option

• Consider satellite location as an option as well• Plan to reimburse participants generously for time and travel

• Incorporate time on phone to plan visit and walk through experience

• Aim for positive word of mouth – informed and happy participants are the key

• Visit in the morning

10/22/2016

45

Conclusions

• Research studies that have used reliable and valid in‐home neuropsychological measures

• Increased the representativeness of participants

• Were able to reveal mechanisms of disparities in cognitive function across the lifecourse

• Were able to reveal sources of resilience for people at risk for cognitive impairment

Conclusions

• Advantages of home‐based neuropsychological assessment

• Reduces the burden of coming in to the clinic

• Addresses barriers to diverse research participation• Mistrust

• Financial Resources

• Disability

• May enrich breadth and quality of certain types of data• ADL and IADL

• Social interactions

• Physical and social supports

10/22/2016

46

Conclusions

• Disadvantage of in‐home cognitive testing• Formal reliability and validity has been established on several measures used in research but not traditional instruments when used in the home

• Conditions under which tests were validated must be as closely matched as possible

• Uncontrollable distractions may increase

• Travel burden is shifted to neuropsychological testing team (and associated cost)

• Some settings might pair with safety concerns

Conclusions

• Most currently available neuropsychological measures are appropriate for in‐home assessment IF:

• Reliability and validity have been established for the conditions of testing

• Appropriate norms match• demographics and conditions of testing (diagnostic)

• Age, sex, years of school

• Race/ethnicity, language, school quality

• Geographic region

• age and conditions of testing (descriptive)

10/22/2016

47

Conclusions

• Non‐traditional tools for cognitive assessment with established reliability and validity include

• Telephone‐administered measures

• Computerized adaptive testing

• Internet‐based measures

• These measures have been more widely used in studies of longitudinal change and risk for cognitive impairment, less for studies of diagnostic accuracy

Conclusions

• Emerging technologies for in‐home assessment can enhance traditional neuropsychological tests and include

• Digital capture• Mailed tablets• IVR• Cell Phone burst measures• Unobtrusive assessment including wearables or ambient sensors

• The utility of these instruments over and above traditional measures administered at home is largely untested

Documents

30. Manly - In-Home Evaluation - rsvpBOOK