Upload
vuongmien
View
215
Download
0
Embed Size (px)
Citation preview
Using observational methods to measure treatment integrity in
psychosocial intervention research
Kimberly S Van Haitsma PhD Director Polisher Research Institute
Madlyn amp Leonard Abramson Center for Jewish Life (formerly Philadelphia Geriatric Center)
1425 Horsham Road North Wales PA 19454-1320
215 371 1895 kvanhaitsmaabramsoncenterorg
Presented at the Gerontological Society of America 64th Annual Scientific Meeting
Pre-Conference Workshop Current and Future Challenges in Designing Behavioral Interventions From Randomized Trials to Community Implementation
November 18th 2011 1230 to 430 1
K2A Knowledge to Action Framework for Public Health
EVALUATION
Translation Supporting Structures Research
Supporting Structures
Institution-
alization
Supporting
Structures
DISCOVERY
STUDIES
EFFICACY
STUDIES
EFFECTIVENESS
AND IMPLEMENTATION
STUDIES
TRANSLATION PHASE
RESEARCH PHASE
INSTITUTION-
ALIZATION
PHASE
KNOWLEDGE
INTO
PRODUCTS
DISSEMINATION
PRACTICE
INSTITUTION-
ALIZATION
DECISION
TO
TRANSLATE
ENGAGEMENT
DECISION
TO
ADOPT
DIFFUSION
Practice-based
Discovery
Practice-based
Evidence
Wilson K Brady T ampLesesne C (2011) An organizing framework for Translation in public health The knowledge to action framework Preventing chronic disease Public health research practice and policy 8(2)1-7 wwwcdcgovpcdissues2011mar10_0012htm
Overview of presentation
bull Illustration of how real-time observations can be used to enhance fidelity in psychosocial intervention studies
ndash What behavior observation measures are available
ndash What aspects of fidelity measurement can be optimized by real time observational techniques
3
An illustrative example The Individualized Positive Psychosocial Intervention Study
Funded by
bull To examine the impact of an individualized recreational intervention on quality of life outcomes for persons with dementia residing in nursing homes ndash Focus on CNA as the interventionist of choice (with the assistance of
RT)
ndash Intervention focus was on enhancing CNA communication skills and individualizing content of intervention
ndash Outcomes focused on enhancing resident positive emotion and behavior and diminishing negative emotion and behavior
ndash Methodology focused on real-time observation of dyadic interactions
Intervention Study Design
bull Randomized Controlled Trial Residents randomly assigned to one of two
intervention conditions (IPPI or Attention Control) or Usual Care
CNAs were assigned to provide intervention based on their existing permanent assignment to a given resident and were blind to intervention condition
bull Residents (n=180)
bull CNAs (n=84)
bull Number of real-time observations (n=2638 occasions)
Resident Group Assignments
Individualized Positive Psychosocial Intervention (IPPI) (n=44 residents)
bull Content of activity selected based on resident preferences
Attention Control Intervention (N= 43 residents) bull Standardized 1-1 interaction (eg reading a magazine
aloud) Both Interventions ndash CNA intervention delivery facilitated by coaching from Recreational
Therapist ndash 10 minute sessions 3xweek for 3 weeks ndash Half of CNAs received additional Communication training half did
not
Usual Care (n=93 residents) bull No prescribed interactions Captured normative behaviors
Intervention delivery focused on Communication Skills of CNAs
7
Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in
dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Prescribed Positive Communication Behaviors
Greeting Saying hello goodbye handshake
Explanation Explains what activity is or will be occurring
Courtesy Saying excuse me please thank you yoursquore welcome
Offers choice Offers choice by explicitly identifying 2 alternatives
Asks preference Solicits resident preferences for past present or future activities
Praisecompliment Expresses praise compliment or approval
Reassurance Provides reassurance or validates feelings
Positive touch Gently touching resident stroking arm or back holding hands hugging kissing
Positiveneutral prompt Verbal prompt to initiate an activity without negative tone
Nonverbal prompt Points shows something to orient toward object
Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects
Proscribed Negative Communication Behaviors
Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior
Talks to other Talks to third party while interacting with resident
Rebukedisapprove Reprimand admonish express disapproval
Negative touch Handles resident roughly grabs resident moves resident without warning
Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes
related to Quality of Life
9
Definitions of Observed Resident Behavior Variables
Positive Behavior
Task Engagement engages in conversation manually manipulates or gestures toward an object
Positive Verbal coherent conversation responding to question
Very Positive Verbal complimenting joking singing
Positive Touch physically receptive to another person gently touching stroking hugging kissing
Negative behavior
Aggression hitting kicking throwing things grabbing spitting
Uncooperative pulling away saying ldquonordquo turning head or body away
Very Negative Verbal swearing screaming mocking making strange noises
Negative Verbal incoherent repetitious statements muttering
General Restlessness pacing fidgeting disrobing repetitive movement
Eyes Closed Sits with eyes closed
Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Resident Affective Responses
11
Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel
K (1999) Observed affect and quality of life in dementia
Further affirmations and problems Journal of Mental Health
and Aging 5 69-81
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
K2A Knowledge to Action Framework for Public Health
EVALUATION
Translation Supporting Structures Research
Supporting Structures
Institution-
alization
Supporting
Structures
DISCOVERY
STUDIES
EFFICACY
STUDIES
EFFECTIVENESS
AND IMPLEMENTATION
STUDIES
TRANSLATION PHASE
RESEARCH PHASE
INSTITUTION-
ALIZATION
PHASE
KNOWLEDGE
INTO
PRODUCTS
DISSEMINATION
PRACTICE
INSTITUTION-
ALIZATION
DECISION
TO
TRANSLATE
ENGAGEMENT
DECISION
TO
ADOPT
DIFFUSION
Practice-based
Discovery
Practice-based
Evidence
Wilson K Brady T ampLesesne C (2011) An organizing framework for Translation in public health The knowledge to action framework Preventing chronic disease Public health research practice and policy 8(2)1-7 wwwcdcgovpcdissues2011mar10_0012htm
Overview of presentation
bull Illustration of how real-time observations can be used to enhance fidelity in psychosocial intervention studies
ndash What behavior observation measures are available
ndash What aspects of fidelity measurement can be optimized by real time observational techniques
3
An illustrative example The Individualized Positive Psychosocial Intervention Study
Funded by
bull To examine the impact of an individualized recreational intervention on quality of life outcomes for persons with dementia residing in nursing homes ndash Focus on CNA as the interventionist of choice (with the assistance of
RT)
ndash Intervention focus was on enhancing CNA communication skills and individualizing content of intervention
ndash Outcomes focused on enhancing resident positive emotion and behavior and diminishing negative emotion and behavior
ndash Methodology focused on real-time observation of dyadic interactions
Intervention Study Design
bull Randomized Controlled Trial Residents randomly assigned to one of two
intervention conditions (IPPI or Attention Control) or Usual Care
CNAs were assigned to provide intervention based on their existing permanent assignment to a given resident and were blind to intervention condition
bull Residents (n=180)
bull CNAs (n=84)
bull Number of real-time observations (n=2638 occasions)
Resident Group Assignments
Individualized Positive Psychosocial Intervention (IPPI) (n=44 residents)
bull Content of activity selected based on resident preferences
Attention Control Intervention (N= 43 residents) bull Standardized 1-1 interaction (eg reading a magazine
aloud) Both Interventions ndash CNA intervention delivery facilitated by coaching from Recreational
Therapist ndash 10 minute sessions 3xweek for 3 weeks ndash Half of CNAs received additional Communication training half did
not
Usual Care (n=93 residents) bull No prescribed interactions Captured normative behaviors
Intervention delivery focused on Communication Skills of CNAs
7
Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in
dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Prescribed Positive Communication Behaviors
Greeting Saying hello goodbye handshake
Explanation Explains what activity is or will be occurring
Courtesy Saying excuse me please thank you yoursquore welcome
Offers choice Offers choice by explicitly identifying 2 alternatives
Asks preference Solicits resident preferences for past present or future activities
Praisecompliment Expresses praise compliment or approval
Reassurance Provides reassurance or validates feelings
Positive touch Gently touching resident stroking arm or back holding hands hugging kissing
Positiveneutral prompt Verbal prompt to initiate an activity without negative tone
Nonverbal prompt Points shows something to orient toward object
Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects
Proscribed Negative Communication Behaviors
Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior
Talks to other Talks to third party while interacting with resident
Rebukedisapprove Reprimand admonish express disapproval
Negative touch Handles resident roughly grabs resident moves resident without warning
Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes
related to Quality of Life
9
Definitions of Observed Resident Behavior Variables
Positive Behavior
Task Engagement engages in conversation manually manipulates or gestures toward an object
Positive Verbal coherent conversation responding to question
Very Positive Verbal complimenting joking singing
Positive Touch physically receptive to another person gently touching stroking hugging kissing
Negative behavior
Aggression hitting kicking throwing things grabbing spitting
Uncooperative pulling away saying ldquonordquo turning head or body away
Very Negative Verbal swearing screaming mocking making strange noises
Negative Verbal incoherent repetitious statements muttering
General Restlessness pacing fidgeting disrobing repetitive movement
Eyes Closed Sits with eyes closed
Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Resident Affective Responses
11
Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel
K (1999) Observed affect and quality of life in dementia
Further affirmations and problems Journal of Mental Health
and Aging 5 69-81
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Overview of presentation
bull Illustration of how real-time observations can be used to enhance fidelity in psychosocial intervention studies
ndash What behavior observation measures are available
ndash What aspects of fidelity measurement can be optimized by real time observational techniques
3
An illustrative example The Individualized Positive Psychosocial Intervention Study
Funded by
bull To examine the impact of an individualized recreational intervention on quality of life outcomes for persons with dementia residing in nursing homes ndash Focus on CNA as the interventionist of choice (with the assistance of
RT)
ndash Intervention focus was on enhancing CNA communication skills and individualizing content of intervention
ndash Outcomes focused on enhancing resident positive emotion and behavior and diminishing negative emotion and behavior
ndash Methodology focused on real-time observation of dyadic interactions
Intervention Study Design
bull Randomized Controlled Trial Residents randomly assigned to one of two
intervention conditions (IPPI or Attention Control) or Usual Care
CNAs were assigned to provide intervention based on their existing permanent assignment to a given resident and were blind to intervention condition
bull Residents (n=180)
bull CNAs (n=84)
bull Number of real-time observations (n=2638 occasions)
Resident Group Assignments
Individualized Positive Psychosocial Intervention (IPPI) (n=44 residents)
bull Content of activity selected based on resident preferences
Attention Control Intervention (N= 43 residents) bull Standardized 1-1 interaction (eg reading a magazine
aloud) Both Interventions ndash CNA intervention delivery facilitated by coaching from Recreational
Therapist ndash 10 minute sessions 3xweek for 3 weeks ndash Half of CNAs received additional Communication training half did
not
Usual Care (n=93 residents) bull No prescribed interactions Captured normative behaviors
Intervention delivery focused on Communication Skills of CNAs
7
Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in
dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Prescribed Positive Communication Behaviors
Greeting Saying hello goodbye handshake
Explanation Explains what activity is or will be occurring
Courtesy Saying excuse me please thank you yoursquore welcome
Offers choice Offers choice by explicitly identifying 2 alternatives
Asks preference Solicits resident preferences for past present or future activities
Praisecompliment Expresses praise compliment or approval
Reassurance Provides reassurance or validates feelings
Positive touch Gently touching resident stroking arm or back holding hands hugging kissing
Positiveneutral prompt Verbal prompt to initiate an activity without negative tone
Nonverbal prompt Points shows something to orient toward object
Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects
Proscribed Negative Communication Behaviors
Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior
Talks to other Talks to third party while interacting with resident
Rebukedisapprove Reprimand admonish express disapproval
Negative touch Handles resident roughly grabs resident moves resident without warning
Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes
related to Quality of Life
9
Definitions of Observed Resident Behavior Variables
Positive Behavior
Task Engagement engages in conversation manually manipulates or gestures toward an object
Positive Verbal coherent conversation responding to question
Very Positive Verbal complimenting joking singing
Positive Touch physically receptive to another person gently touching stroking hugging kissing
Negative behavior
Aggression hitting kicking throwing things grabbing spitting
Uncooperative pulling away saying ldquonordquo turning head or body away
Very Negative Verbal swearing screaming mocking making strange noises
Negative Verbal incoherent repetitious statements muttering
General Restlessness pacing fidgeting disrobing repetitive movement
Eyes Closed Sits with eyes closed
Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Resident Affective Responses
11
Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel
K (1999) Observed affect and quality of life in dementia
Further affirmations and problems Journal of Mental Health
and Aging 5 69-81
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
An illustrative example The Individualized Positive Psychosocial Intervention Study
Funded by
bull To examine the impact of an individualized recreational intervention on quality of life outcomes for persons with dementia residing in nursing homes ndash Focus on CNA as the interventionist of choice (with the assistance of
RT)
ndash Intervention focus was on enhancing CNA communication skills and individualizing content of intervention
ndash Outcomes focused on enhancing resident positive emotion and behavior and diminishing negative emotion and behavior
ndash Methodology focused on real-time observation of dyadic interactions
Intervention Study Design
bull Randomized Controlled Trial Residents randomly assigned to one of two
intervention conditions (IPPI or Attention Control) or Usual Care
CNAs were assigned to provide intervention based on their existing permanent assignment to a given resident and were blind to intervention condition
bull Residents (n=180)
bull CNAs (n=84)
bull Number of real-time observations (n=2638 occasions)
Resident Group Assignments
Individualized Positive Psychosocial Intervention (IPPI) (n=44 residents)
bull Content of activity selected based on resident preferences
Attention Control Intervention (N= 43 residents) bull Standardized 1-1 interaction (eg reading a magazine
aloud) Both Interventions ndash CNA intervention delivery facilitated by coaching from Recreational
Therapist ndash 10 minute sessions 3xweek for 3 weeks ndash Half of CNAs received additional Communication training half did
not
Usual Care (n=93 residents) bull No prescribed interactions Captured normative behaviors
Intervention delivery focused on Communication Skills of CNAs
7
Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in
dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Prescribed Positive Communication Behaviors
Greeting Saying hello goodbye handshake
Explanation Explains what activity is or will be occurring
Courtesy Saying excuse me please thank you yoursquore welcome
Offers choice Offers choice by explicitly identifying 2 alternatives
Asks preference Solicits resident preferences for past present or future activities
Praisecompliment Expresses praise compliment or approval
Reassurance Provides reassurance or validates feelings
Positive touch Gently touching resident stroking arm or back holding hands hugging kissing
Positiveneutral prompt Verbal prompt to initiate an activity without negative tone
Nonverbal prompt Points shows something to orient toward object
Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects
Proscribed Negative Communication Behaviors
Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior
Talks to other Talks to third party while interacting with resident
Rebukedisapprove Reprimand admonish express disapproval
Negative touch Handles resident roughly grabs resident moves resident without warning
Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes
related to Quality of Life
9
Definitions of Observed Resident Behavior Variables
Positive Behavior
Task Engagement engages in conversation manually manipulates or gestures toward an object
Positive Verbal coherent conversation responding to question
Very Positive Verbal complimenting joking singing
Positive Touch physically receptive to another person gently touching stroking hugging kissing
Negative behavior
Aggression hitting kicking throwing things grabbing spitting
Uncooperative pulling away saying ldquonordquo turning head or body away
Very Negative Verbal swearing screaming mocking making strange noises
Negative Verbal incoherent repetitious statements muttering
General Restlessness pacing fidgeting disrobing repetitive movement
Eyes Closed Sits with eyes closed
Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Resident Affective Responses
11
Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel
K (1999) Observed affect and quality of life in dementia
Further affirmations and problems Journal of Mental Health
and Aging 5 69-81
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Intervention Study Design
bull Randomized Controlled Trial Residents randomly assigned to one of two
intervention conditions (IPPI or Attention Control) or Usual Care
CNAs were assigned to provide intervention based on their existing permanent assignment to a given resident and were blind to intervention condition
bull Residents (n=180)
bull CNAs (n=84)
bull Number of real-time observations (n=2638 occasions)
Resident Group Assignments
Individualized Positive Psychosocial Intervention (IPPI) (n=44 residents)
bull Content of activity selected based on resident preferences
Attention Control Intervention (N= 43 residents) bull Standardized 1-1 interaction (eg reading a magazine
aloud) Both Interventions ndash CNA intervention delivery facilitated by coaching from Recreational
Therapist ndash 10 minute sessions 3xweek for 3 weeks ndash Half of CNAs received additional Communication training half did
not
Usual Care (n=93 residents) bull No prescribed interactions Captured normative behaviors
Intervention delivery focused on Communication Skills of CNAs
7
Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in
dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Prescribed Positive Communication Behaviors
Greeting Saying hello goodbye handshake
Explanation Explains what activity is or will be occurring
Courtesy Saying excuse me please thank you yoursquore welcome
Offers choice Offers choice by explicitly identifying 2 alternatives
Asks preference Solicits resident preferences for past present or future activities
Praisecompliment Expresses praise compliment or approval
Reassurance Provides reassurance or validates feelings
Positive touch Gently touching resident stroking arm or back holding hands hugging kissing
Positiveneutral prompt Verbal prompt to initiate an activity without negative tone
Nonverbal prompt Points shows something to orient toward object
Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects
Proscribed Negative Communication Behaviors
Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior
Talks to other Talks to third party while interacting with resident
Rebukedisapprove Reprimand admonish express disapproval
Negative touch Handles resident roughly grabs resident moves resident without warning
Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes
related to Quality of Life
9
Definitions of Observed Resident Behavior Variables
Positive Behavior
Task Engagement engages in conversation manually manipulates or gestures toward an object
Positive Verbal coherent conversation responding to question
Very Positive Verbal complimenting joking singing
Positive Touch physically receptive to another person gently touching stroking hugging kissing
Negative behavior
Aggression hitting kicking throwing things grabbing spitting
Uncooperative pulling away saying ldquonordquo turning head or body away
Very Negative Verbal swearing screaming mocking making strange noises
Negative Verbal incoherent repetitious statements muttering
General Restlessness pacing fidgeting disrobing repetitive movement
Eyes Closed Sits with eyes closed
Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Resident Affective Responses
11
Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel
K (1999) Observed affect and quality of life in dementia
Further affirmations and problems Journal of Mental Health
and Aging 5 69-81
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Resident Group Assignments
Individualized Positive Psychosocial Intervention (IPPI) (n=44 residents)
bull Content of activity selected based on resident preferences
Attention Control Intervention (N= 43 residents) bull Standardized 1-1 interaction (eg reading a magazine
aloud) Both Interventions ndash CNA intervention delivery facilitated by coaching from Recreational
Therapist ndash 10 minute sessions 3xweek for 3 weeks ndash Half of CNAs received additional Communication training half did
not
Usual Care (n=93 residents) bull No prescribed interactions Captured normative behaviors
Intervention delivery focused on Communication Skills of CNAs
7
Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in
dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Prescribed Positive Communication Behaviors
Greeting Saying hello goodbye handshake
Explanation Explains what activity is or will be occurring
Courtesy Saying excuse me please thank you yoursquore welcome
Offers choice Offers choice by explicitly identifying 2 alternatives
Asks preference Solicits resident preferences for past present or future activities
Praisecompliment Expresses praise compliment or approval
Reassurance Provides reassurance or validates feelings
Positive touch Gently touching resident stroking arm or back holding hands hugging kissing
Positiveneutral prompt Verbal prompt to initiate an activity without negative tone
Nonverbal prompt Points shows something to orient toward object
Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects
Proscribed Negative Communication Behaviors
Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior
Talks to other Talks to third party while interacting with resident
Rebukedisapprove Reprimand admonish express disapproval
Negative touch Handles resident roughly grabs resident moves resident without warning
Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes
related to Quality of Life
9
Definitions of Observed Resident Behavior Variables
Positive Behavior
Task Engagement engages in conversation manually manipulates or gestures toward an object
Positive Verbal coherent conversation responding to question
Very Positive Verbal complimenting joking singing
Positive Touch physically receptive to another person gently touching stroking hugging kissing
Negative behavior
Aggression hitting kicking throwing things grabbing spitting
Uncooperative pulling away saying ldquonordquo turning head or body away
Very Negative Verbal swearing screaming mocking making strange noises
Negative Verbal incoherent repetitious statements muttering
General Restlessness pacing fidgeting disrobing repetitive movement
Eyes Closed Sits with eyes closed
Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Resident Affective Responses
11
Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel
K (1999) Observed affect and quality of life in dementia
Further affirmations and problems Journal of Mental Health
and Aging 5 69-81
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Intervention delivery focused on Communication Skills of CNAs
7
Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in
dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Prescribed Positive Communication Behaviors
Greeting Saying hello goodbye handshake
Explanation Explains what activity is or will be occurring
Courtesy Saying excuse me please thank you yoursquore welcome
Offers choice Offers choice by explicitly identifying 2 alternatives
Asks preference Solicits resident preferences for past present or future activities
Praisecompliment Expresses praise compliment or approval
Reassurance Provides reassurance or validates feelings
Positive touch Gently touching resident stroking arm or back holding hands hugging kissing
Positiveneutral prompt Verbal prompt to initiate an activity without negative tone
Nonverbal prompt Points shows something to orient toward object
Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects
Proscribed Negative Communication Behaviors
Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior
Talks to other Talks to third party while interacting with resident
Rebukedisapprove Reprimand admonish express disapproval
Negative touch Handles resident roughly grabs resident moves resident without warning
Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes
related to Quality of Life
9
Definitions of Observed Resident Behavior Variables
Positive Behavior
Task Engagement engages in conversation manually manipulates or gestures toward an object
Positive Verbal coherent conversation responding to question
Very Positive Verbal complimenting joking singing
Positive Touch physically receptive to another person gently touching stroking hugging kissing
Negative behavior
Aggression hitting kicking throwing things grabbing spitting
Uncooperative pulling away saying ldquonordquo turning head or body away
Very Negative Verbal swearing screaming mocking making strange noises
Negative Verbal incoherent repetitious statements muttering
General Restlessness pacing fidgeting disrobing repetitive movement
Eyes Closed Sits with eyes closed
Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Resident Affective Responses
11
Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel
K (1999) Observed affect and quality of life in dementia
Further affirmations and problems Journal of Mental Health
and Aging 5 69-81
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Definitions of Observed CNA Behavior Variables Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in
dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Prescribed Positive Communication Behaviors
Greeting Saying hello goodbye handshake
Explanation Explains what activity is or will be occurring
Courtesy Saying excuse me please thank you yoursquore welcome
Offers choice Offers choice by explicitly identifying 2 alternatives
Asks preference Solicits resident preferences for past present or future activities
Praisecompliment Expresses praise compliment or approval
Reassurance Provides reassurance or validates feelings
Positive touch Gently touching resident stroking arm or back holding hands hugging kissing
Positiveneutral prompt Verbal prompt to initiate an activity without negative tone
Nonverbal prompt Points shows something to orient toward object
Task engagement Verbal interaction related to the task at hand Physical manipulation of objects related to task at hand assisting resident in manipulation of objects
Proscribed Negative Communication Behaviors
Ignores resident Does not look at resident or respond to resident verbal or nonverbal behavior
Talks to other Talks to third party while interacting with resident
Rebukedisapprove Reprimand admonish express disapproval
Negative touch Handles resident roughly grabs resident moves resident without warning
Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes
related to Quality of Life
9
Definitions of Observed Resident Behavior Variables
Positive Behavior
Task Engagement engages in conversation manually manipulates or gestures toward an object
Positive Verbal coherent conversation responding to question
Very Positive Verbal complimenting joking singing
Positive Touch physically receptive to another person gently touching stroking hugging kissing
Negative behavior
Aggression hitting kicking throwing things grabbing spitting
Uncooperative pulling away saying ldquonordquo turning head or body away
Very Negative Verbal swearing screaming mocking making strange noises
Negative Verbal incoherent repetitious statements muttering
General Restlessness pacing fidgeting disrobing repetitive movement
Eyes Closed Sits with eyes closed
Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Resident Affective Responses
11
Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel
K (1999) Observed affect and quality of life in dementia
Further affirmations and problems Journal of Mental Health
and Aging 5 69-81
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Intervention Outcomes Focused on Enhancing Resident Affective and Behavioral Outcomes
related to Quality of Life
9
Definitions of Observed Resident Behavior Variables
Positive Behavior
Task Engagement engages in conversation manually manipulates or gestures toward an object
Positive Verbal coherent conversation responding to question
Very Positive Verbal complimenting joking singing
Positive Touch physically receptive to another person gently touching stroking hugging kissing
Negative behavior
Aggression hitting kicking throwing things grabbing spitting
Uncooperative pulling away saying ldquonordquo turning head or body away
Very Negative Verbal swearing screaming mocking making strange noises
Negative Verbal incoherent repetitious statements muttering
General Restlessness pacing fidgeting disrobing repetitive movement
Eyes Closed Sits with eyes closed
Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Resident Affective Responses
11
Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel
K (1999) Observed affect and quality of life in dementia
Further affirmations and problems Journal of Mental Health
and Aging 5 69-81
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Definitions of Observed Resident Behavior Variables
Positive Behavior
Task Engagement engages in conversation manually manipulates or gestures toward an object
Positive Verbal coherent conversation responding to question
Very Positive Verbal complimenting joking singing
Positive Touch physically receptive to another person gently touching stroking hugging kissing
Negative behavior
Aggression hitting kicking throwing things grabbing spitting
Uncooperative pulling away saying ldquonordquo turning head or body away
Very Negative Verbal swearing screaming mocking making strange noises
Negative Verbal incoherent repetitious statements muttering
General Restlessness pacing fidgeting disrobing repetitive movement
Eyes Closed Sits with eyes closed
Van Haitsma K Lawton MP Kleban M Klapper JA amp Corn JA (1997) Methodological aspects of the study of streams of behavior in dementing illness Alzheimer Disease and Associated Disorder 11(4) 228-238 PMID 9437440
Resident Affective Responses
11
Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel
K (1999) Observed affect and quality of life in dementia
Further affirmations and problems Journal of Mental Health
and Aging 5 69-81
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Resident Affective Responses
11
Lawton M P Van Haitsma K Perkinson M amp Ruckdeschel
K (1999) Observed affect and quality of life in dementia
Further affirmations and problems Journal of Mental Health
and Aging 5 69-81
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Detailed information about measurement properties of observational measures of outcomes
12
bullCuryto K Van Haitsma K Vriesman D (2008) Direct Observation of Individual Behavior A
Review of Current Methods and Measures for Use with Older Adults with Dementia Research in
Gerontological Nursing 1(1) 52-76 bullUpdates pertinant to this review since 2008
Burgio L D Park N S Hardin J M amp Sun F (2007) A longitudinal examination of agitation and resident characteristics in the nursing home The Gerontologist 47(5) 642-649
Cohen-Mansfield J Thein K Dakheel-Ali M amp Marx M S (2010) Engaging nursing home residents with dementia in activities The effects of modeling presentation order time of day and setting characteristics Aging amp Mental Health 14(4) 471-480 doi10108013607860903586102
Ersek M Polissar N amp Neradilek M B (2011) Development of a composite pain measure for persons with advanced dementia Exploratory analyses in self-reporting nursing home residents Journal of Pain and Symptom Management 41(3) 566-579 doi101016jjpainsymman201006009
Horgas A L Elliott A F amp Marsiske M (2009) Pain assessment in persons with dementia Relationship between self-report and behavioral observation Journal of the American Geriatrics Society 57(1) 126-132 doi101111j1532-5415200802071x
Husebo B S Strand L I Moe-Nilssen R Husebo S B amp Ljunggren A E (2009) Pain behaviour and pain intensity in older persons with severe dementia Reliability of the MOBID pain scale by video uptake Scandinavian Journal of Caring Sciences 23(1) 180-189 doi101111j1471-6712200800606
Pulsford D Duxbury J A amp Hadi M (2011) A survey of staff attitudes and responses to people with dementia who are aggressive in residential care settings J Psychiatr Ment Health Nurs 18(2) 97-104 doi101111j1365-2850201001646x
Razani J Bayan S Funes C Mahmoud N Torrence N Wong J Josephson K (2011) Patterns of deficits in daily functioning and cognitive performance of patients with Alzheimer disease Journal of Geriatric Psychiatry and Neurology 24(1) 23-32 doi1011770891988710390812
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
wwwnolduscom
Observational Method used in the IPPI study ldquoThe Observerrdquo
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Measuring Behavior with The Observer
Customized coding scheme
Precisely detail
behaviors amp affect
Time-stamped event log
Annotate behavior via simple key press Coding can be done live or via videorecording
Record who does what where when and howto whom Behaviors can be coded with modifiers eg intensity
Maintain time-stamped event log Synchronize with multimodal data
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Pocket Observer be free to move
Code on a handheld device
Mobile observations outdoor use
Add behaviorsmodifiers on the handheld
Supports Smart phones rugged handhelds
PDAs (Windows Mobile 6x)
Gather amp score behavior data live using
Pocket Observer
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Data collection Continuous Coding via Video
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
The ObserverXT Analyzing the data
Descriptive statistics
Frequency How many times did the resident strike out
Duration How long did the resident express pleasure
Latency rate per minute percent observation inter-rater
reliability etc
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Bellg A Borrelli B Resnick B et al (2005) Enhancing treatment fidelity in health behavior change studies Best practices and recommendations form the NIH Behavior Change Consortium Health Psychology 23 443-451
What is fidelity in psychosocial intervention research
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Receipt Recipient
Assures that the treatment has been received
and understood by the individual
Enactment Recipient
Monitors that the individual performs
treatment-related behavioral skills and
cognitive strategies in relevant real life
settings as intended
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
What aspects of fidelity are optimally measured by real- time observational
methods
19
Focus Purpose
Design Study
Ensure that a study can adequately test its
hypotheses in relation to the underlying
theory and clinical processes
Training Interventionist
Ensure that interventionists have been
satisfactorily trained to deliver the
intervention to participants
Delivery Interventionist
Ensure that the intervention is delivered as
intended
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Fidelity of Treatment Delivery
20
Do CNArsquos adhere to communication skills (prescribed and proscribed) in intervention protocols while delivering a one to one intervention to persons with dementia
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Protocol fidelity for adherence to proscribed communication skills for CNAs (N=84) trained to deliver psychosocial
interventions to nursing home residents with dementia
Percent who used communication behaviors
CNA Communication Behaviors Total N=84 ()
Po
sit
ive
Greeting 86
Explanation 89
Courtesy 79
Offers choice 80
Asks preference 98
Praisecompliment 79
Reassure 60
Positive announcement 89
Positive touch 96
Positive verbal prompt 92
Non-verbal prompt 89
Task engagement 71
Neg
ati
ve
Negative announcement 1
Ignores resident 6
Talks to another 87
Rebukedisapprove 13
Negative touch 0
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Benefits of real- time observation in measuring treatment delivery integrity
22
bullInterventionist self reports of adherence are heavily influenced by demand characteristics or need for social approval1 bullDifferences between observed and interventionist reported adherence can be substantial (eg 100 for self report 44 for observed)2
bullPrecision in measuring adherence can prevent premature abandonment of potentially effective interventions that are simply not delivered effectively
1 Perepletchikova F amp Kazdin A (2005) Treatment integrity and therapeutic change Issues and research recommendations ClinicPal Psychology Science and Practice 12(4) 365-383 2 Hardeman W Michie S Fanshawe T et al (2008) Fidelity of delivery of a physical activity intervention Predictors and consequences Psychology and Health 23 (1) 11-24
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Fidelity and Study Design How Observational Methods Can Help
23
bullCan enhance study power by reducing random or unintended variability resulting in a more modest ldquonrdquo in future studies2
bullFacilitate identification of crucial active ingredients and their relationship to immediate (during intervention sessions) and longer term outcomes1
Observational methods can enhance the ability to empirically validate which intervention ingredients are most or least crucial to outcomes Observed intervention components should be theory-based and tied to outcomes of choice
1 National Advisory Mental Health Council Workgroup Report (2010) From discovery to cure Accelerating the development of new and personalized interventions for mental illnesses httpwwwnimhnihgovaboutadvisory-boards-and-groupsnamhcreportsfromdiscoverytocurepdf 2 Horner S Rew L amp Torres R (2006) Enhancing intervention fidelity A means of strengthening study impact JSPN 11(2) 80-89
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Lawton M P amp Nahemow L (1973) Ecology and the aging process The psychology of adult development and aging 619-674
Theoretical basis for focusing on verbal and nonverbal communication behaviors of CNAs in the context of intervention delivery to persons with dementia
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement 11
Ignores resident
Talks to another 08 12 09 26 10 28 08
Rebukedisapprove 06 12 10
Negative touch
Greeting 15 11 -09 08 35 16 38 20
Explanation 15 15 -12 20 12 46 20 51 24
Courtesy -14 47 27 46 32
Offers choices 11 12 -12 34 21 42 13
Asks preference 17 14 -14 10 11 41 19 52 16
Praisecompliment 07 -13 38 35 54 20
Reassurance 20 12 16 22 10
Positive verbal prompt 13 17 -12 13 10 29 08 48 11
Positive announcement 08 11 -10 12 34 12 37 14
Positive touch 09 12 -08 22 14 40 23
Non-verbal prompt 09 -07 -15 07 09 43 26 42 16
General conversation 11 09 -08 -18 09 09 66 09 19
Verbally engaged in task 10 -12 39 43 90 21
Physically engaged in task 15 07 12 22 20
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Pearson Correlations (N=2638 real-time observation occasions) Resident Negative Behavior Resident Positive Behavior
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Ne
gati
ve C
NA
co
mm
un
icat
ion
b
eh
avio
rs
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Po
siti
ve C
NA
co
mm
un
icat
ion
be
hav
iors
Greeting
Explanation
Courtesy
Offers choices
Asks preference
Praisecompliment
Reassurance
Positive verbal prompt
Positive announcement
Positive touch
Non-verbal prompt
General conversation
Verbally engaged in task
Physically engaged in task
Correlation is significant at the 0001 level (2-tailed)
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt 12
Negative announcement
Ignores resident
Talks to another 08 12 09
Rebukedisapprove 06 12
Negative touch
Greeting 35 16 38 20
Explanation 46 20 51 24
Courtesy 47 27 46 32
Offers choices 34 21 42 13
Asks preference 41 19 52 16
Praisecompliment 38 35 54 20
Reassurance 16 22 10
Positive verbal prompt 29 08 48 11
Positive announcement 34 12 37 14
Positive touch 22 14 40 23
Non-verbal prompt 43 26 42 16
General conversation 66 09 19
Verbally engaged in task 39 43 90 21
Physically engaged in task 07 12 22 20
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement 11
Ignores resident
Talks to another 26 10 28 08
Rebukedisapprove 10
Negative touch
Greeting 15 11 -09 08
Explanation 15 15 -12 20 12
Courtesy -14
Offers choices 11 12 -12
Asks preference 17 14 -14 10 11
Praisecompliment 07 -13
Reassurance 20 12
Positive verbal prompt 13 17 -12 13 10
Positive announcement 08 11 -10 12
Positive touch 09 12 -08
Non-verbal prompt 09 -07 -15 07 09
General conversation 11 09 -08 -18 09 09
Verbally engaged in task 10 -12
Physically engaged in task 15
Very neg
verbal Neg
verbal Gen
restless Eyes closed
Uncoop- erative
Aggres- sion
Pos verbal
Very pos verbal
Task en-gagement
Pos touch
Negative prompt
Negative announcement
Ignores resident
Talks to another
Rebukedisapprove
Negative touch
Greeting 35 38
Explanation 46 51
Courtesy 47 46
Offers choices 42
Asks preference 41 52
Praisecompliment 38 35 54
Reassurance
Positive verbal prompt 48
Positive announcement 37
Positive touch 40
Non-verbal prompt 43 42
General conversation 66
Engaged in task 39 43 90
Physically engaged in task
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Why focus on positive emotion as an outcome of choice
1 Fredrickson B (2004) The broaden-and-build theory of positive emotions Phil Trans R Soc Lond B Biol Sci September 29 359(1449) 1367ndash
1378 doi 101098rstb20041512
2 Fredrickson BL amp Losada MF (2005) Positive affect and complex dynamics of human flourishing American Psychologist 60 678-686
doi1010370003-066X607678
3 Meeks S VanHaitsma K Kostiwa I amp Murrell S (in press) Positivity and Well-Being among Community-Residing Elders and Nursing Home
Residents What is the Optimal Affect Balance Journals of Gerontology Psychological Sciences
Broaden-and-Build Theory of Positive Emotion (Fredrickson 2004)1
bullPositive emotions are a primary means to improve psychological and physical well-being over time bullPositive emotions broaden peoplersquos momentary thought-action repertoires (enhance attention flexibility openness to new experiences) and build their enduring personal resources (social attachments resiliency creativity enhanced cardiovascular recovery) bullPositive emotions serve to ldquoundordquo or ldquocorrectrdquo the after effects of negative emotional experiences bullIn order for an individual to flourish the ratio of positive to negative emotional experiences needs to be at least 3 to 1 23
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11 15
Explanation 30 12 17 20 08
Courtesy 38 14
Offers choice 25 11 12
Asks preference 34 14 10 16
Praise
compliment 42 12
Reassure 12 11 08 13
Positive
announcement 26 09 09 09
Positive touch 25 12 13
Positive verbal
prompt 27 12 14
Non-verbal
prompt 37 20
Verbal task
engagement 51 10 07
Physical task
engagement 14 07 17
Negative
announcement
Ignores resident
Talks to another 17 09 09 07
Rebuke
disapprove 07
Negative touch
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Pearson Correlations (N=2638 real-time observation occasions)
Resident Positive Affect Resident Negative Affect
Pleasure Interest Anger Anxiety Sadness
Po
sit
ive C
NA
co
mm
un
icati
on
be
ha
vio
rs
Greeting
Explanation
Courtesy
Offers choice
Asks preference
Praise
compliment
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt
Verbal task
engagement
Physical task
engagement
Neg
ati
ve C
NA
co
mm
un
icati
on
beh
avio
rs
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Correlation is significant at the 0001 level (2-tailed)
Pleasure Interest Anger Anxiety Sadness
Greeting 24 11
Explanation 30 12
Courtesy 38 14
Offers choice 25 11
Asks preference 34 14
Praise
compliment 42 12
Reassure 12
Positive
announcement 26 09
Positive touch 25 12
Positive verbal
prompt 27 12
Non-verbal
prompt 37 20
Verbal task
engagement 51 10
Physical task
engagement 14 07
Negative
announcement
Ignores resident
Talks to another 09 07
Rebuke
disapprove 07
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting 15
Explanation 17 20 08
Courtesy
Offers choice 12
Asks preference 10 16
Praise
compliment
Reassure 11 08 13
Positive
announcement 09 09
Positive touch 13
Positive verbal
prompt 14
Non-verbal
prompt
Verbal task
engagement 07
Physical task
engagement 17
Negative
announcement
Ignores resident
Talks to another 17 09
Rebuke
disapprove
Negative touch
Pleasure Interest Anger Anxiety Sadness
Greeting
Explanation
Courtesy 38
Offers choice
Asks preference
Praise
compliment 42
Reassure
Positive
announcement
Positive touch
Positive verbal
prompt
Non-verbal
prompt 37
Verbal task
engagement 51
Physical task
engagement
Negative
announcement
Ignores resident
Talks to another
Rebuke
disapprove
Negative touch
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31
Summary Pros and Cons afforded by real time observation of treatment fidelity
compared to other methods
30
Cons bullExpensive and time intensive to train researchers bullIs not efficient for use in ldquoreal worldrdquo of clinical practice
Pros bullProvide a more accurate and objective account of treatment delivery bullEnhanced understanding what treatment ingredients give you the biggest ldquobang for your buckrdquo in outcomes bullKnowledge of potent treatment ingredients can lead to 1) refinements of treatment protocols 2) more targeted treatment training materials 3) refined platform for building fidelity measures used for quality improvement purposes in clinical settings
Thank you
31