Upload
eric-larson
View
46
Download
0
Tags:
Embed Size (px)
Citation preview
eric c. larson | eclarson.com
mobile health for the masses
Assistant Professor Computer Science and Engineering
health monitoring using mobile phones
> slide about me
Shwetak Patel James Fogarty Jon Froehlich
the health landscape
75% of all US healthcare spending is on chronic disease
source: center for disease control, 2014
state of health in the US
US infant mortality rate ranks ~23rd in the world, but 1st in delivery cost
source: national vital statistics reports, 2014
9 out of 10 of US doctors feel medications are overprescribed
source: american medical association, 2012
solution: more actionable data, better managed care,
preventative care, but with lower cost
one strategy: mhealth
what is mhealth?
mHealth (em-‘helth )an abbreviation for mobile health, a term used for the practice of medicine and public health supported by mobile devices
the promise of mHealth:eliminate doctor visits remote / automatic diagnosis equalize developing countries
stress check
fitness trainer
heart rate
current mhealth
telemedicine
remote training
stress check
telemedicinefitness trainer
heart rate
current mhealthremote training
~30,000 apps for health ~95% are for calorie counting & exercise ~5% are remote monitoring, wellness,
smoking cessation, references, etc.
yet to be a disruptive mHealth technology
mHealth sensing outside the clinic
compliance?cost?
privacy?data reliability?
phone as a sensorbaseline process
clinical quantity
sensor or
human
embedded sensors processing
estimated
accelerometer gyroscope barometric pressure temperature magnetometer /compass dual camera / flash 1+ microphones proximity sensor capacitive sensor gps motorized actuator wireless antenna (s)
compliance++; cost--;
data reliability?
what can the mobile phone sense with clinical accuracy?
pupillary response
lung functionjaundice
pupillary response
lung functionjaundice
Total Serum Bilirubin
Medical GoldStandard
Transcutaneous
TcB
Bilirubinometer
TcBNon-invasive
Correlates Well
$7000
Quick results
Screening tool for TSB
20
15
10
5
0
0.5 1 2 3 4 5 6 Age (days)In Hospital
Biliru
bin
(mg/
dL)
Newborn Bilirubin Levels
75th percentile
25th percentile
Visual Assessment • Parents • Many physicians • Traveling practitioners
In Hospital At Home
Screening Challenges
Tend to underestimate
bilirubin level in blood
jaundice levelblood draw
yellowness camera processing
estimated
bilicam
Shwetak PatelJim Taylor Lilian DeGreef Mayank Goel Jim Stout
Study Evaluation
100 newborn participants • <1 day old when
enrolled
Data collected by medical professionals using iPhone 4S
BiliCam
TSB (ground truth)
TcB (control)
3 - 5 days old
Noisy Data
Automatic Quality Control
✔ ✖ ✖
✖ ✖ ✖
Ideal Glare Overexposed
Occlusion Shadow Underexposed
Algorithm Overview
Bilirubin Estimate
Color Balance
Extract Features
Apply Regressions
400 500 600
Wavelength (nm)
Rela
tive
Bili
rubi
n
Abs
orpt
ion
Prob
abili
ty
RGB
CrCbY
YCbCr
a*b*
L*
L*ab
with & without flash
skin Gradient (of RGB channels)
Extract Features
Color Balance
Regression Ensemble
No
90th percentileYes
regressions agree
mean
Sigmoidal
LARS-Lasso Elastic Net
Support Vector Regressions
Encapsulated Neighbor
Random Forest Regression
Bilirubin Estimate
0
5
10
15
20
25
0 5 10 15 20 25
Results
TSB Ground Truth (mg/dl)
Estim
ated
Bilir
ubin
(mg/
dl)
0
5
10
15
20
25
0 5 10 15 20 25
BiliCamrank order 0.85 correlation
Results
TSB Ground Truth (mg/dl)
Estim
ated
Bilir
ubin
(mg/
dl)
0
5
10
15
20
25
0 5 10 15 20 25
TcBs correlate 0.75 - 0.93
BiliCamrank order 0.85 correlation
TcBrank order 0.92 correlation
Results
TSB Ground Truth (mg/dl)
Estim
ated
Bilir
ubin
(mg/
dl)
Interpretation20
15
10
5
0
high risk
high intermediate risk
low intermediate risk
low risk
Biliru
bin
(mg/
dL)
0.5 1 2 3 4 5 6 Age (days)
20
15
10
5
0
high risk
high intermediate risk
low intermediate risk
low risk
Biliru
bin
(mg/
dL)
0.5 1 2 3 4 5 6 Age (days)
Bhutani Nomogram
20
15
10
5
0
high riskhigh intermediate risk
low intermediate risk
low risk
Biliru
bin
(mg/
dL)
0.5 1 2 3 4 5 6
Age (days)
9 high risk cases based on TSB
Interpretation
20
15
10
5
0
high riskhigh intermediate risk
low intermediate risk
low risk
Biliru
bin
(mg/
dL)
0.5 1 2 3 4 5 6
Age (days)
BiliCam 2/9 missed high risk (22%)85% blood draws avoided
Interpretation
20
15
10
5
0
high riskhigh intermediate risk
low intermediate risk
low risk
Biliru
bin
(mg/
dL)
0.5 1 2 3 4 5 6
Age (days)
BiliCam 2/9 missed high risk (22%) 85% blood draws avoided
TcB 2/9 missed high risk (22%) 88% blood draws avoided
BiliCam is sufficient for newborn Jaundice screening, but it is unknown how user error affects reliability
Interpretation
Next steps: developing world
Kernicterus:+21+($8+mill)+
Hazardous+jaundice:++1158+
($50,000)+
Extreme+jaundice:++2,317+($20,000)+
Severe+jaundice:+35,000+($8,500)+
Phototherapy:+290,000++($1,000)+
Visible+jaundice:+3.5+million+
Births/year:+4.1+million+
In the US Middle- & low-income countries:
• 75,000 cases kernicterus/year
• 114,000 newborn deaths/year
• 65% newborn deaths from kernicterus
kernicterus 21
($8 million)
Bhutani et al. 2013, Pediatric Research 2010
pupillary response
lung functionjaundice
pupillary response drug impairment
concussion stroke
pain fatigue
cognitive disabilities arousal
cognitive load
cognitive load refers to the total amount of mental effort being used in the working memory
multiply these two numbers
3 6
measurement pupillometer
IR gaze trackers
$4000$1000
pupillary change
cognitive loadcontact
IR camera
visible iris camera regression
estimated
PupilWare
Suku Nair Sohail Rafiqi Mark Chatchai Ephrem Fernandez
pupilware cognitive load study
uniquely brighter than the iris [43]. This contrast makes it rela-tively easy to measure the pupil’s center. It is important to note that eye tracking in general does not require pupil size measure-ment; it only requires measuring the center of the pupil or center of the iris. Therefore much of the techniques used in eye tracking are not directly applicable to pupillary response measurement.
Starburst [42] is a pupil segmentation algorithm that uses a hybrid of feature based and model-based approaches. Starburst estimates eye center and iteratively grows the pupil region to find the pupil edge using ellipse fitting techniques and RANSAC to eliminate outliers [44]. Starburst then finds the inliers from the ellipse fit-ting iterations. These inliers are sent to a final ellipse-fitting algo-rithm and, finally, pupil diameter is found by averaging the el-lipse width and height. We use a modified version of the Starburst algorithm in PupilWare, modified to more appropriately find the pupil edge without the high contrast provided by infrared light.
In recent work, Wood and Bulling were able to track eye gaze from tablet cameras [45]. Their prototype, EyeTab, uses a model-based approach to estimate the gaze without an infrared light source. EyeTab uses means of gradient algorithm (where most of the image gradients meet) [46] to determine the center of the eye and then identify points on the limbus edge (i.e., the edge of the iris) to fit an ellipse model. While EyeTab is closely related to PupilWare, the iris center is used to infer a participant’s gaze and therefore EyeTab does not measure pupillary dilation, which clearly separates the contributions of our analyses. However, this work showed that head pose and lighting could be controlled and compensated using a mobile device’s camera, clearly influencing the design of PupilWare.
3 DATA COLLECTION To validate the design of PupilWare algorithm we conducted an IRB approved human subject study at Southern Methodist Uni-versity as shown in Figure 1. As part of the study we replicated one of the classics Cognitive psychology experiments namely Digit Span task. The test is used to artificially induce the cogni-tive load on the participant. This is one of the classic cognitive pupillometry tests [19][18][32] that was later repeated by Klingner [30] using eye-trackers. A user is presented with a spe-cialized, short duration task meant to artificially induce mental workload. At the beginning of the task, the baseline pupil size is captured. At the end of the task subjective rating and task comple-tion time (TCT) are used to validate the workload.
Each trial is started with stabilization of pupil for 5 seconds. Se-quences of digits are spoken aloud to the user at the rate of 1 number per second. After a short period participants orally report back the sequence. This test assesses how much the pupil diame-ter increases as participants memorize the digit and decreases as these digits are reported back. We also capture any errors that the users make. We control the difficulty level by the number of dig-its in the sequence, i.e., 5, 7, or 9 (four iterations of each diffi-cult). In addition to collecting the pupil size we also keep track of number of times participants make a mistake.
For ground truth, we use two accepted devices from the psychol-ogy community—the Neuroptics VIP-200 Pupillometer [27] and Gazept Remote Eye Tracker [47]. The Neuroptics VIP-200 is a portable, battery operated, hand-held device that accurately measures pupil size with a resolution of 0.1 mm. This is same as used by ophthalmologist during an eye exam. The device can measure pupil diameter either with no background illumination for 2 seconds or variable light levels in one sequence for a total of
10 seconds. It reports average pupil size and standard deviation. The VIP-200 is placed over the participant’s eye (Figure 2). The Gazept Remote Eye Tracker emits near infrared (NIR) light over the participant and captures participant’s eyes using two optically zoomed NIR cameras. It constantly captures the pupils of the participant reflecting any changes in their size in real-time. For the device under test, a web camera, we use a Microsoft Lifecam [48] of resolution 1280x720 pixels. The web camera stores the video of the participant during each task at 15 frames per second.
Figure 1 -- Experiment Setup
Figure 2 -- Calipers and the VIP-200 in use
Distance between participant’s eyes is measured using a pair of calipers (Figure 2). This measurement is used to convert the pupil size in pixels to size in mm. Prior to performing any task, each participant’s baseline pupillary diameter is determined using the VIP-200 for both eyes separately as well as using a remote eye tracker. The VIP-200 is used to determine the baseline only, veri-fying that the Gazept pupil dilation measurement is accurate to within 0.25 mm. After verification, the VIP-200 is removed from over the participant’s eye.
3.1 Participants In this study we recruited 12 subjects all but 2 were associated with SMU either as a student or staff. At the onset, each partici-pant was asked to fill out a questionnaire on an iPad application about his or her medical history, caffeine intakes, etc. This infor-mation was collected primarily to determine if a participant should be excluded from the experiment. Following Table 1 shows the demographics of the participants.
pupilware 20 participants
over 500 iterations of digit span tasks using both laptop embedded camera and smartphone
for light brown and colored eyes: identical to pupillometer captures sub millimeter pupil response first automatic classification of cognitive load
pupilware 20 participants
over 500 iterations of digit span tasks using both laptop embedded camera and smartphone
for light brown and colored eyes: identical to pupillometer captures sub millimeter pupil response first automatic classification of cognitive load
markers of painPupilWare
head injury
cravings context aware computing
attention
future work in
sympathetic nerve damage fatigue and sleep deprivation
pupillary response
lung functionjaundice
spirometer
device that measures amount of air inhaled and
exhaled.
using a spirometer
flow
volume
volum
e
time
using a spirometer
flow
volume
volum
e
time
flow
volume
volum
e
time
FEV1
FVC
FEV1: Forced Expiratory Volume in 1 second FVC: Forced Vital Capacity
FEV1% = FEV1/FVC
> 80% healthy60 - 79% mild40 - 59% moderate
< 40% severe
clinical spirometry
flow rate volume
lung functionairflow sensor
sound pressure microphone processing
estimated
SpiroSmart Shwetak PatelMayank Goel
Gaetano Boriello
Jim StoutMargaret Rosenfeld
Elliot Saba
2012 study: appropriate for
trending and screening
if you have access to a smartphone
2012: Using SpiroSmart
2015: Using SpiroCall
SpiroCall
any phone in the world can be used as a spirometer
2015 study: 50 participants
4 styles of phone including feature phone
head to head with spirometer
SpiroCall vs spirometer ~5% spirometer vs spirometer ~5%
in 10 years, COPD will surpass AIDS/HIV as the leading cause of death in low income nations
global initiative for chronic obstructive lung disease (GOLD 2014) world health organization, global burden of disease (2013)
ongoing research third party validation FDA device approval gamification and compliance, reliability scalable training of public health workers
pupillary response
lung functionjaundice
> slide to unlock
Thank You!
eric c. larson | eclarson.com
mobile health for the masses
Assistant Professor Computer Science and Engineering
health monitoring using mobile phonescollaborators: Suku Nair Eric Bing Sohail Rafiqi Mark Wang Ephrem Fernandez, MD, PhD Gaetano Boriello Shwetak Patel Jim Stout, MD Jim Taylor, MD Margaret Rosenfeld, MD Mayank Goel Lilian DeGreef Joseph Camp
eclarson.com [email protected] @ec_larson
eric c. larson | eclarson.comAssistant Professor Computer Science and Engineering
BiliCamhome screening for newborn jaundice
SpiroCallmobile and smartphone spirometry
MobiScreenmobile training for cervical cancer screening
PupilWarepupillary response using
everyday cameras