Opportunities and Challenges of Using Smartphones for Health Monitoring and Intervention@neal_lathiaComputer LaboratoryUniversity of Cambridge

http://en.wikipedia.org/wiki/Macintosh_SE My first computer.

My latest computer.

Macintosh SE19878 MHz1 MB20 MB7.7 kg$2,900

Model:Released:Processor:RAM:Hard Drive:Weight:Price:

Samsung Galaxy S420131.6 GHz x 42 GB16/32 GB130 g~$500

1/5: the “basics” why health via smartphones?why smartphones for health?

why health via smartphones? health “in the moment” vs. “reconstructed” ubiquity of technology vs. limited face-to-face

Smartphones are incredibly personal devices: they are not often shared.

Research shows that owners regularly keep their smartphone within arms length of them.

A. Dey et. al. Getting Closer: An Empirical Investigation of the Proximity of Users to their Smartphones. In ACM Ubicomp 2011.

why smartphones for health? 2/5: interactivity 3/5: sensors4/5: machine learning

2/5: interactivity * notifications check-insgamessituation-awareness

momentary, context-aware, engaging* I don't research interaction/design

notifications

check-ins

games

situation-aware

Emotion Sense Q Sensemood tracking smoking cessationDept. of Psychology Behavioural Sciences

check-ins'volunteering'

check-ins'volunteering'

notifications'prompting'

“games”'guiding'

situation-aware'informing'

2/5: interactive apps a) active behavioural monitoringb) momentary assessmentc) information delivery

3/5: smartphone sensors passive behavioural monitoring

Sensors were originally added to smartphones for purely functional purposes.

E.g., an accelerometer lets the device know when to display the screen in landscape mode; the GPS allows the device to support maps/driving apps.

Only later did researchers uncover that all of these sensors could be a valuable source of behavioural data.

what is a “sensor?”AccelerometerGPS / Wi-FiGyroscopeBluetoothMicrophoneEnvironmentPhone / Text LogsDevice LogsSocial Media APIsApp Usage

Accelerometer → activityGPS / Wi-Fi → location, mobilityGyroscope → device orientationBluetooth → co-locationMicrophone → audio processingEnvironment → light / temperature / pressurePhone / Text Logs → socialisingDevice Logs → network / batterySocial Media APIs → socialising, connectivityApp Usage → context, searching

Sensors do not “directly” encode behaviour. For example, sampling from the accelerometer provides time series data of changes in acceleration.

All sensor data needs to be processed in order to extract/infer behaviours. How, for example, does the accelerometer indicate physical activity?

raw sensor data

features

does the accelerometer feature correlate with reports of current levels of physical activity?r = 0.369

does the accelerometer feature correlate with reports of levels of physical activity on that day?r = 0.172

...batteries are still headaches:

Sensors were originally added to smartphones for purely functional purposes.

These sensors were not built to efficiently collect continuous streams of data*.

* This is changing...?

K. Rachuri. Smartphones Based Social Sensing: Adaptive Sampling, Sensing and Computation Offloading. PhD Thesis, Computer Laboratory. 2012.

4/5: machine learning using data to infer behaviour

Machine Learning (vs. Behaviour Theory?)

Behaviours are often too complex and/or abstract to directly encode them into software.

Machine learning a statistical approach that centres around using data to learn to identify and predict behaviours. Often without knowing much (or anything) about what those behaviours actually look like.

Two broad categories of learning algorithms, which are often referred to as unsupervised and supervised learning.

Unsupervised learning, or clustering, assumes you have:

(a) a large dataset of many representations of a behaviour, and

(b) a way of measuring the extent that two representations of behaviours are similar.

… without knowing precisely how to code for that behaviour

What are the behaviours that emerge when a city uses the stations in a bicycle sharing scheme?

Define a way of representing the behaviour:

8AM7AM... 11PM

... 50% 25% 32%Station A

What are the behaviours that emerge when a city uses the stations in a bicycle sharing scheme?

Define a way of comparing behaviour:

...

... 50% 25% 32%Station A

... 23% 34% 52%Station B

What are the behaviours that emerge when a city uses the stations in a bicycle sharing scheme?

What diurnal patterns of physical activity emerge from smartphone accelerometers?

What diurnal patterns of physical activity emerge from smartphone accelerometers?

8AM7AM... 11PM

... 1.23 2.33 3.12User A

...

... 1.23 2.33 3.12User A

... 2.33 3.43 2.33User B

What diurnal patterns of physical activity emerge from smartphone accelerometers?

What diurnal patterns of physical activity emerge from smartphone accelerometers?

How does this relate to happiness?

4/5: machine learning using data to infer & predict behaviour

* I didn't include supervised learning, which is awesome

5/5: challenges & opportunities

1. Software Engineering / Expectations2. Marketing3. Control over target population4. Understanding sensor data5. Writing code6. Finding research value

1. Blurred lines between research and practice2. High potential for multi-disciplinary impact

3. Cheap to roll-out to huge audiences4. Accessible to 'everyone'

5. Rising demand for quality healthcare technology6. Wearables are coming!

Opportunities and Challenges of Using Smartphones for Health Monitoring and Intervention@neal_lathiaComputer LaboratoryUniversity of Cambridge

“Can I run an ESM study like Emotion Sense?”