Supporting Bodily Communication in Video Consultations of PhysiotherapyDeepti Aggarwal

Friday, 10 March 17

Video Consultationsvan Dyk 2014; Yellowlees 2005

Friday, 10 March 17

Bodily CommunicationArgyle 2013; Heath 1986, 2002

Friday, 10 March 17

Bodily Communication in Physiotherapy

Friday, 10 March 17

Research Gaps

Friday, 10 March 17

Main Research Question

How can interactive technologies support physiotherapists in

understanding bodily information during video consultations?

Friday, 10 March 17

Research Phases

#1 Observations

#2 Designing & Lab study

#3 Field Deployments

Friday, 10 March 17

Research Phases

#1 Observations

#2 Designing & Lab study

#3 Field Deployments

Friday, 10 March 17

Research Question

How do physiotherapists interpret bodily information in

current practices of video consultations?

Friday, 10 March 17

#1 Observations

Face-to-Face (3) Video (7)

2 physiotherapists & 5 patients with chronic pain Prescribed exercises: tip-toes, squats

Friday, 10 March 17

ACM DIS 2016 (full paper)

Friday, 10 March 17

Face-to-Face Video

1. Opening

2. History taking

3. Examination

4. Diagnosis

5. Treatment

6. Closing

Byrne & Long 1976

Friday, 10 March 17

Face-to-Face Video

Appearance(Full body)

Posture

Movement(Walking)

Orientation

Appearance(Upper Torso)

Posture

Movement

Orientation

1. Opening

Friday, 10 March 17

Face-to-Face Video

Facial Expressions(Tears, red cheeks, tensed

eyes)

Quality of Movements

(Depth, range, smoothness)

Eye Contacts(for encouragement,

willingness to engage)

2. History Taking

Eye Contacts(encouragement,

willingness to engage)

Quality of Movements

(depth, range, smoothness)

Facial Expressions(Tears, red cheeks, tensed

eyes)

Friday, 10 March 17

Face-to-Face Video

Tactile feedback(Tightness, inflammation,

temperature)

Response to touch(Fear, protective spasm,

facial expression)

Touch on patient’s body

3 & 4. Examination & Diagnosis

Response to touch(Fear, protective spasm,

facial expression)

Tactile feedback(Tightness, inflammation,

temperature)

Touch on own body

Friday, 10 March 17

Face-to-Face Video

Touch on patient’s body

5. Treatment

Touch on own body

Body posture

Tone of speech(pitch)

Efforts(fatigue, tremor)

Body posture

Tone of speech(pitch)

Efforts(fatigue, tremor)

Friday, 10 March 17

Face-to-Face Video6. Closing

Body language

Facial expressions

Body language

Facial expressions

Friday, 10 March 17

Key Takeaways

Lower limb movements are difficult to observe

Less specific treatment

Video limits a wide range of bodily cues

Friday, 10 March 17

Research Phases

#1 Observations

#2 Designing & Lab study

#3 Field Deployments

Friday, 10 March 17

Key aspects of lower body movements

Weight Distribution

Range of Movements

Foot Orientation

Friday, 10 March 17

Research Question

How can we collect and visualize lower body movements to support

physiotherapists during video consultations?

Friday, 10 March 17

Wii-Fit boardMcGough et al. 2012

Friday, 10 March 17

Sensoria socksSarnow et al. 1994

Friday, 10 March 17

SoPhy

SOcks for PHYsiotherapy

Socks with sensors Web interface

Friday, 10 March 17

SoPhy socks

IMU

Pressure Sensors

Bluetooth Shield

Arduino Pro Mini

Friday, 10 March 17

SoPhy web-interface

Friday, 10 March 17

Friday, 10 March 17

Research Question

How does SoPhy help physiotherapists in assessing

lower body movements during video consultations?

Friday, 10 March 17

#2 Lab study

Clinician’s room

10 physiotherapists & 1 actor (patient)

Patient’s room

Friday, 10 March 17

Study Design (2x2)

Extreme pain Low pain

Friday, 10 March 17

Tasks for Participants

Assess the patient’s exercises

Organize 4 video sessions

Report confidence in a questionnaire

Friday, 10 March 17

Tasks for Actor

Dorsiflexion & Plantarflexion

1-leg tip-toes & 2-leg tip-toes

Squats

Friday, 10 March 17

Increase in Confidence

Without SoPhy, I may not be able to pick up things just from video. Like I thought, ‘Ooh!

that foot looks tilted outside’, but then whether it has any relation with their weight distribution or not, I can’t tell just from the

video. (P2)

Exercises with SoPhy without SoPhy

SquatsF(1,36) = 10.97, p<.01

M = 5.75,

SD = 1.06M = 4.17,SD = 1.13

Friday, 10 March 17

Fewer Repetitions of Exercises

Exercises with SoPhy without SoPhy

DorsiflexionF(1,36) = 6.99, p<.05

M = 8.10,

SD = 3.21M = 11.25,SD = 4.54

PlantarflexionF(1,36) = 6.14, p<.05

M = 7.45,

SD = 3.88M = 10.60,SD = 4.24

SquatsF(1,36) = 8.36, p<.01

M = 6.05,

SD = 2.48

M = 8.05,

SD = 3.18

Friday, 10 March 17

Fewer Repetitions of Exercises

Over video, I can see the person only from one direction. [...] The sock system provides me this detailed information irrespective of

how the person is standing or sitting. I did not ask the patient to turn backwards or sideways

when I had the sock data - the system was already doing it for me. (P3)

Friday, 10 March 17

Deeper Insights on Weight Distribution

It’s always challenging to understand weight bearing because the pressure points are not visible. It’s easy in cases when the person is putting more weight on one foot than other.

But it is difficult to understand how the pressure is distributed across the foot, is it on

the heels, or on the balls. (P8)

Friday, 10 March 17

Challenges

Interference to visual observation

Alignment with clinical practice

Friday, 10 March 17

ACM CHI 2017 (full paper)

Friday, 10 March 17

Limitations

Controlled lab study

Focus on assessment

Friday, 10 March 17

Research Phases

#1 Observations

#2 Designing & Lab study

#3 Field Deployments

Friday, 10 March 17

Research Question

How does SoPhy support physiotherapists in assessing

and treating patients, with lower limb issues, during naturally

occurring video consultations?

Friday, 10 March 17

Revised SoPhy socks

Use of bright colored socks to support visual observations

Friday, 10 March 17

Revised SoPhy web-interface

To align with clinical practice

Friday, 10 March 17

Study Design

2 physiotherapists & 5-7 patients with lower limb issuesSimulated video sessions at Royal Children’s Hospital

Hospital Room #1 Hospital Room #2

Friday, 10 March 17

Study Design

Face-to-Face (60 mins)

Video Consultation

(45 - 50 mins)

Face-to-Face follow up (15 - 10 mins)

First session Second session

Participant Observations & Interviews

Friday, 10 March 17

Study Setup

Friday, 10 March 17

Session in Action

Friday, 10 March 17

Video without SoPhy (study 1)

1. Opening

2. History taking

3. Examination

4. Diagnosis

5. Treatment

6. Closing

Video with SoPhy (study 3)

Friday, 10 March 17

Study 1Video with

SoPhy

Appearance(Upper Torso)

Posture(Upper Torso)

Movement(sitting)

Orientation

Appearance(Upper Torso)

Posture(upper torso + feet)

Movement(walking, sitting)

Orientation

1. Opening

Friday, 10 March 17

Thesis Contributions

Conceptual understanding of bodily communication in video consultations of physiotherapy

Designed and studied a technology to support physiotherapists during video consultations

Friday, 10 March 17

Dissertation Status

Commenced study 3

Finished study 1, 2

Thesis Chapters 4 & 6 - first draft finished

Friday, 10 March 17

Completion Plan

Time Tasks

March - April 2017

Continue study 3Finish Chapters 2, 3 & 5Revise Chapters 2 to 6

May - June 2017

Attend CHI 2017Finish Study 3Finish Chapters 7, 8, 9

July - August 2017

Finish Chapter 1Revise complete thesisThesis Submission

Friday, 10 March 17

Thank you

Friday, 10 March 17