Upload
ginger
View
45
Download
0
Embed Size (px)
DESCRIPTION
Understanding user engagement with digital interventions. Dr Jason Rentfrow , Dr Leanne Morrison, & Dr Sharon Lin On behalf of the UBhave , Emotion sense, LifeGuide , and POWeR teams. Understanding user engagement with power: digital intervention for weight management. Leanne Morrison - PowerPoint PPT Presentation
Citation preview
Understanding user engagement with digital interventions
Dr Jason Rentfrow, Dr Leanne Morrison, & Dr Sharon Lin
On behalf of the UBhave, Emotion sense, LifeGuide, and POWeR teams
UNDERSTANDING USER ENGAGEMENT WITH POWER: DIGITAL INTERVENTION
FOR WEIGHT MANAGEMENT
Leanne MorrisonUniversity of Southampton
How can we encourage better engagement with online interventions?
STUDY 1: Role of supplemental human support (“coaching”)
– RCT of online POWeR programme + supplemental telephone coaching across communities in North East England (n = 786)
STUDY 2: Role of supplemental mobile support (“apps”)– Series of in depth mixed methods ‘N-of-1’ case studies to
explore engagement with and impact of online POWeR programme + POWeR Tracker app (N = 13)
How do users feel about these forms of supplemental support?
What impact do they have on engagement and health-related outcomes?
• Theory and evidence-based online programme to support users to adopt a sustainable and positive approach to weight management
• Developed using LifeGuide
• POWeR Tracker: Android smartphone application to accompany the online programme
– Maintain awareness of
personal POWeR goals – Monitor progress
Study 1: Community based RCT
Consent and RegistrationN=1131
CoachN=247 (31.4%)
Web onlyN=264 (33.6%)
ControlN=275 (35%)
RandomisationN=786 (69.5%)
Responded to f/upN=162 (58.9%)
Responded to f/upN=40 (15.2%)
Responded to f/upN=53 (21.5%)
Qualitative interviews (n=19, purposively
sampled)
Coaching Protocol:
2 short phone calls from a ‘POWeR coach’ in week 1 and week 4
Mohr et al. (‘Supportive Accountability’)
In collaboration with public health teams (Scott Lloyd, NHS Tees, NHS Durham and Darlington)
Is usage enhanced by the addition of brief human support in the form of telephone coaching?
Did telephone coaching encourage usage?
Participants in the coach arm significantly more likely to complete the core POWeR sessions
Web only Web+ Coach Significance testN (% )completing 3 core sessions
47 (17.8%) 64 (25.9%) ᵪ2(1,n=511)=4.93, p=.026
1 2 3 4 5 6 7 80
10
20
30
40
50
60
70
80
% of sample still using POWeR at each session
webcoach
Perceptions of coaching
• Low uptake– only 23.5% had one phone call, 18.6% had both
• How did coaching help? – Praise and positivity – Accountability
DEMOGRAPHICS: Older, lower health literacy, higher BMI, hypertension, previous referral to weight loss scheme
USAGE AND ENGAGEMENT: More sessions completed, more log ins, more time spent online, Satisfaction with POWeR, fewer doubts about how to use POWeR, autonomous motivation
PROGRESS: Greater weight loss
• Variety of measures:– Daily questionnaire measures – Step counts (via pedometer)– Weekly telephone interviews – Objective data on web and app usage
Q1. Does an app improve goal perceptions/progress?
Study 2: POWeR Tracker• 13 participants followed over 4 weeks in series
of n-of-1 case studies (ABAB vs. BABA) • Compare web-based POWeR with and without
POWeR Tracker app
Q2. When, why and how do people engage with a web + mobile intervention?
A: Week 1 B: Week 2 A: Week 3B: Week 4
Did the POWeR Tracker app improve goal perceptions?
• No clear effect of app availability on daily goal perceptions or step count
• Measurement effect Goal Effort Goal
awarenessGoal
motivationGoal self-efficacy
Goal achievement
Step count
1 2 3 4 5 6 Diet PA Diet PA Diet PA Diet PA
Alex 0.50 -0.16 -0.30 -0.01 -0.22 -0.13 -0.74 -0.16 -0.55 -0.61 -0.14 -0.61 0.28 -0.76 0.26
Susan 0.14 0.68 0.34 -0.22 -0.67 -0.26 0.21 0.05 0.60 -0.44 0.54 -0.32 -0.07 0.14 0.26
Hannah 0.05 -0.33 -0.20 -0.24 -0.28 -1.19 -0.15 -0.12 0.23 0.12 -0.98 -0.73 -0.78 -0.67 -0.17
Dan 0.11 0.25 0.12 -0.04 -0.56 0.00 0.19 0.73 0.10 -0.15 -0.30 0.44 -0.09 0.55 0.01
Natalie 0.77 -0.08 0.40 0.05 -0.33 0.57 -0.51 -0.21 0.37 0.02 -0.76 -0.22 -0.10 -0.87 0.12
Lucy 0.28 0.37 -0.20 0.24 0.34 0.26 0.66 0.76 0.57 0.64 0.40 0.62 0.81 0.60 0.74
Rachel -0.04 0.22 -0.25 0.42 0.95 0.85 0.25 0.55 0.61 0.64 0.71 0.68 0.34 0.47 -0.11
Lisha 0.34 -0.01 0.09 0.00 -0.16 -0.53 0.05 -0.58 0.16 -0.78 0.10 -0.51 -0.26 -0.19 0.43
Marcus 0.18 0.18 -0.32 -0.43 -0.22 -0.41 0.03 -0.39 -0.29 -0.45 0.18 -0.66 -0.17 -0.52 -0.08
Ian -0.68 -0.72 -0.34 -0.64 0.65 -0.92 0.96 0.69 1.09 1.09 0.91 0.20 0.38 0.42 0.00
Laura - - - - - - - - - - - - - - -
Chris - - - - - - - - - - - - - - -
Andrew - - - - - - - - - - - - - - -
“I guess if you weren’t like…with the questionnaires every day, um…if you weren’t doing the questionnaires every day then I think you would miss the app more…..Because…I guess the questionnaires every day were making you think about how well have I done today, or kept you motivated and yeah...it was just a period of evaluation. Whereas if you didn’t have those then you…yeah I think you would miss the app or miss the website more.” (Susan)
• Short bursts of on-the-go access or time-relevant use
• Notifications prompted app use (when used)• Use app primarily for a reminder of key
information (e.g. food lists, goals) • Variation in approach to using hybrid web +
app intervention
Engagement with POWeR Tracker: When, where, why?
Example: Dan
• Use at key times (e.g. lunchtime – food choices, spare time in between lectures)
• Short bursts of use ~ up to 10 minutes at a time
• Response to app notifications
1 2 3 4 5 6 7 8 9 10 11 12 130
5
10
15
20
25
30
35
40
Diary completionsGoal updateInformation viewsGoal checks
Participant
Engagement with POWeR: Summary
• Telephone coaching appears to improve engagement with online interventions and offer benefits to particular groups of user
• Offering mobile tools or apps appears to improve the convenience and accessibility of health behaviour change interventions– Individual variation in tool preferences and
patterns of use
ANALYSING COMPLEX DATA SETS
Sharon LinUniversity of Southampton
Data analyses
• Visualization• N-of-1 studies
Time spent on groups of pages
Re-ordered time spent on groups of pages – clustered time
Analyses of N-of-1 studies
• To draw in the regression model:,
)
A sample of Ubhave dataDay
Dan_Total step Inhibitor
1 NA 02 11471 03 9760 04 3558 05 4739 06 3662 07 NA 08 5729 19 2794 1
10 7636 111 3996 112 7467 113 10587 114 3863 115 1649 016 5659 017 8390 018 2221 019 8980 020 3566 021 3457 022 NA 123 5038 124 4798 125 5414 126 5380 127 3678 128 6335 1
Effect size = 0.006α = 5592, β = 17.27, ρ(rho) = 0.06, σ = 2698
1 3 5 7 9 11 13 15 17 19 21 23 25 270
2000
4000
6000
8000
10000
12000
14000
Dan’s total Steps
p242M15
Statistical challenges of N-of-1 studies
• Challenges– Small sample size – Autocorrelated errors in repeated measures
arising from the individual under study– Non-normality of the responses
• Remedy under investigations– Parametric bootstrap tests
N-of-1 Power Function Off (7D)-On(7D)-Off (7D)-On(7D) or ABAB
0 0.5 1 1.5 2 2.5 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Bootstrapped Tests, Nint=1000, B=1000
ρ=0, Tsta,OLSρ=0, Tsta,GLSρ=0.2, Tstaρ=0.5, Tstaρ=0.7, Tsta
Measure
Individual
Effect size
Ρ (rho) Power
M15 Dan 0.01 0.1 0.05
M12 Lucy 0.61 0.2 0.25
M14 Dan 0.55 0.5 0.13
M2 Chris 0.59 0.7 0.15
Tools will be available in the future
• Functions for visualisation• Analysis tool for N-of-1 studies
– correcting small sample and correlated data problems
• Power functions for N-of-1 studies– giving guidance for N-of-1 study design
Acknowledgements Funded by the EPSRC under the UBhave projectFor more information please visit: http://ubhave.org
22
Behavioural/Social Scientists:
Professor Lucy Yardley
Professor Susan Michie
Professor Peter Smith
Dr Jason Rentfrow
Dr Leanne Morrison
Dr Laura Dennison
Dr Sharon Lin
Computer Scientists:
Dr Cecilia Mascolo
Dr Mark Weal
Dr Mirco Musolesi
Dr Danius Michaelides
Dr Charlie Hargood
Dr Neal Lathia
Dr Veljko Pejovic