ii

ACKNOWLEDGEMENTS

This thesis represents not only my work, but a milestone achieved through the guidance and

mentoring of all people who shared their support in this. I am indebted to many people for making

this endeavor an unforgettable experience.

I gratefully acknowledge my advisor Prof. Michael Gennert for his encouragement,

inspiration and his great efforts to help me surface clear ideas through meaningful discussions. I

also thank my committee members, Prof Joseph Beck for his timely inputs and guidance on the

technical aspects of this research and Prof Serge Roy for helping me to keep in mind the actual

impact of this work and bringing in the perspective of the community that will be benefitted by

this work.

I reserve special thanks for Dr. Joshua Kline whose guidance and encouragement has been

a steady influence throughout my thesis. He has inspired me to become an independent researcher

and helped me realize the power of critical reasoning. His mentoring has been instrumental in

brainstorming the key insights of the thesis as well as in rational thinking and impactful writing.

I am grateful to my colleagues at Delsys Inc, for their assistance and support. It has been a

pleasure to work with such a driven team and to learn from their experiences.

With final words, I deeply thank my family for their continuous support and encouragement.

It’s their unwavering belief in my dreams that kept me motivated throughout my masters.

iii

AUTONOMOUS REAL-TIME TRACKING OF BRADYKINESIA

IN PARKINSON’S DISEASE DURING UNSCRIPTED ACTIVITIES

BHAWNA SHIWANI

Worcester Polytechnic Institute, 2017

ABSTRACT

Bradykinesia is one of the most prevalent yet poorly monitored motor symptoms of

Parkinson’s Disease (PD). Generally defined by slowness of movements, the specific symptoms

and severity of bradykinesia fluctuate throughout the day in PD patients and are therefore difficult

to assess and treat during relatively short duration visits to a clinician. Body-worn sensors and AI

algorithms could provide valuable tool for the clinical assessment of bradykinesia. However, to

date no algorithm has been developed that can solve this problem. Therefore, this thesis aimed to

design a software platform consisting of machine learning algorithms and clinically-informed

metrics to provide real-time tracking of the motor symptoms associated with bradykinesia.

Activity-specific neural-network detection algorithms were designed, trained and tested to classify

sensor data from activities associated with minimal movement, such as non-walking, separately

from those associated with more continuous movement, such as walking. Clinically-informed

metrics were identified through rigorous signal analysis of EMG and IMU sensor data of muscle

activity and limb movement to quantify the motor symptoms used for clinical assessment of

bradykinesia. Real-time processing of more than 2000 minutes of movement data from PD patient

provided the following results: 1) walking from non-walking activities were separated using a

neural-network activity classifier with an accuracy of 99.5%; 2) a second neural network algorithm

trained for walking data provided minute-by-minute detection of bradykinesia with an accuracy of

iv

93.8%; 3) a third neural network algorithm trained for non-walking data provided minute-by-

minute detection of bradykinesia with 97.4% accuracy. In addition, the clinically-informed metrics

successfully quantified changes in motor symptoms of bradykinesia – such as poverty of

movement, reduced limb velocity and reduced range of movement – that occurred before and after

administration of PD medication. Together the detection algorithm and sensor-derived metrics

provide a novel, proof-of-concept framework that establishes the clinical viability of a real-time

tracking system for therapeutic interventions and patient-specific treatment.

v

TABLE OF CONTENTS

Title Page ...................................................................................................................... i

Acknowledgements ....................................................................................................... ii

Abstract ........................................................................................................................ iii

Table of Contents ......................................................................................................... v

List of Tables ............................................................................................................... vi

List of Figures ............................................................................................................... vii

List of Abbreviations .................................................................................................... viii

Introduction ................................................................................................................. 1

Specific Aims ............................................................................................................... 6

Methods ....................................................................................................................... 8

Results ……………...................................................................................................... 29

Discussions ...………….............................................................................................. 37

Future Work …………….............................................................................................. 41

Appendix A .................................................................................................................... 43

Appendix B .................................................................................................................... 44

Bibliography................................................................................................................... 45

vi

LIST OF TABLES

Table 1 Subject Population Characteristics ................................................................. 8

Table 2 Complete List of identified features during non-walking activities …........... 19

Table 3 Complete List of identified features during walking activities ….................. 20

Table 4 Clinically Informed Metrics …………………............................................... 25

Table 5 The classification accuracies of Walking Detector ........................................ 29

Table 6 Bradykinesia classification accuracy using Neural Network Architecture …. 30

Table 7 Bradykinesia classification accuracy using Logistics Regressions …………. 31

vii

LIST OF FIGURES

Figure 1 Manifestations of different symptoms in Parkinson’s Disease ……………… 2

Figure 2 Sensor data captured concurrently from EDS and TA muscles ……………… 9

Figure 3 Sensor data during walking and non-walking activities …………………… 12

Figure 4 Normal Healthy Walking …………………………………………………… 13

Figure 5 Temporal segmentation of signal data ........................................................... 14

Figure 6 Algorithm architecture to capture bradykinesia symptoms during

unscripted activities ....................................................................................... 15

Figure 7 Flow diagram of logistic regression algorithm .............................................. 22

Figure 8 Flow diagram of neural network algorithm .................................................. 23

Figure 9 Voluntary activities highlighted within 2-minute window of gyroscope data 26

Figure 10 Highlighted swing phase during step cycle .................................................. 27

Figure 11 Step detector algorithm results with an example of 10-minute segment of

ADL as captured by gyroscope placed at lower extremity …........................ 30

Figure 12 Plot I of clinically informed metrics during non-walking activities ……...... 32

Figure 13 Plot II of clinically informed metrics during non-walking activities ……...... 33

Figure 14 Plot I of clinically informed metrics during walking activities …………...... 34

Figure 15 Plot II of clinically informed metrics during walking activities …………...... 35

viii

LIST OF ABBREVIATIONS

ACC Accelerometer

ADL Activities of Daily Living

APDA American Parkinson’s Disease Association

EDS Extensor Digitorum Superficialis

EMG Electromyography

GYR Gyroscope

IMU Inertial Measurement Unit

PD Parkinson’s Disease

TA Tibialis Anterior

UPDRS Unified Parkinson’s Disease Rating Scale

VL Vastus Lateralis

WIRB Western Institutional Review Board

1

INTRODUCTION

Parkinson’s Disease (PD) is a neurodegenerative, chronically progressive disease that has affected

mobility and cognitive function of more than 10 million people worldwide [1]. Despite such a

daunting prevalence, there are few options to automatically and accurately monitor its progression

or evaluate treatment efficacy based upon the patient’s motor symptoms [2]. Often the limitations

to track motor symptoms such as tremor, slowness of movements and gait instability, result in an

incomplete picture of the disease [3,4]. However, a more comprehensive view of the disease

symptoms is crucial in determining the appropriate medication regimen. The goal of this research

was to design a framework to autonomously monitor motor symptoms of PD utilizing wearable

sensor technology during unscripted daily activities.

In PD, the cells that are responsible for producing a vital neurotransmitter called dopamine

are progressively depleted. Deficiency of dopamine results in a variety of involuntary movement

disorders as well as a reduced ability to produce normal voluntary movements [5]. Although PD

symptoms may vary throughout the day and manifest differently across patients, the primary motor

symptoms of PD typically include: cyclic shaking movements at rest (Tremor); slowness, paucity,

and hesitancy of movements (Bradykinesia); stiffness in muscles (Rigidity); shortened, shuffling

steps (Parkinsonian Gait); and medication-induced rapid, jerky involuntary movements

(Dyskinesia) (refer to Figure 1). Manifestation and severity of these symptoms can fluctuate

throughout the day depending on individual differences in stage of the disease and the

timing/dosage of dopamine-replacement therapy [6].

Among the symptoms of PD, bradykinesia is one of the most common and early stage

manifestations. Although primarily marked by an overall slowness of movements, body

2

bradykinesia manifests in a variety of symptoms such as hesitation to initiate or stop the voluntary

action, abnormally slow and shortened movements, reduced arm-swing, and poverty of movement.

Traditionally, clinicians assess bradykinesia and other PD symptoms using a protocol known as

the Unified Parkinson’s Disease Rating Scale (UPDRS) [7]. The UPDRS is a clinician-guided

rating tool that is often used to monitor the longitudinal course of PD by conducting periodic

evaluations during clinic visits. It consists of three parts on 1) Mentation, Behavior, and Mood, 2)

Activities of Daily Living (ADL) and 3) Motor Examination. The motor examination scale

assesses PD manifestations based on scripted activities and segments them into 5 different severity

levels based on the clinician’s impression of the magnitude of the impairment(s) during the PD

test. For example, the clinician may ask the patient to tap their fingers together as fast as possible

to rate the amplitude and frequency of the taps on the 0-4 rating scale (Item 23 of Part III Motor

Figure 1. Manifestations of different symptoms in Parkinson’s Disease – Different

symptoms of Parkinson’s Disease are presented along with their characteristic

manifestations. (Image Credit: http://www.afcares.com/blogs/parkinsons-disease-

seeking-in-home-care/)

3

Exam). This approach rests on the assumption that the severity of bradykinesia symptoms during

these coached activities correlates to the same severity levels for unscripted activities in daily

living for all PD patients. However, PD symptoms manifest differently for every patient depending

on their work and daily living habits [8]. UPDRS has also developed a broader guiding scale for

the clinicians to assess “body bradykinesia” (Item 31 of Part III Motor Exam) by considering the

reduced arm swing, slowness, paucity, and reduced range of movements during daily living

activities [7]. In both cases, assessment of body bradykinesia under the observation of a clinician

provides only a momentary picture of disease symptoms and fails to capture the fluctuations of

bradykinesia symptoms between the medication replacements. A more detailed tracking of

fluctuations of bradykinesia symptoms is critical to capture medication response for better disease

management.

Traditional way to track bradykinesia symptoms is to advise PD patients to maintain diaries

to log the disease symptoms daily [9]. However, manual logging methods demonstrate only a

limited efficacy as the objective assessment of highly versatile PD symptoms is difficult for

patients [10,11] and often results in poor compliance and incomplete logs due in part to the fact

that they may suffer from cognitive impairment as a result of PD. In addition to burdening the

patient, the absence of a continuous and more objective measure of PD symptoms greatly

minimizes its effectiveness in monitoring the oftentimes rapid fluctuations in PD symptom severity

throughout the various stages of PD. Proper medication titration for symptom management

therefore requires an autonomous and continuous tracking of PD symptoms in the context of ADL.

Wearable sensors can be leveraged to fill this gap by objectively quantifying the clinically

relevant motor symptoms with precise temporal resolution [12,13,14]. Even though it is not

possible to measure the cognitive symptoms of PD using body-worn sensors, the captured motor

4

symptoms acceptably correlate with the overall disease progression. Initial approaches to track

bradykinesia and other PD symptoms using wearable sensors relied on either accelerometers or

EMG activity [15]. However, the more recent development of inertial sensors, that integrate

gyroscopes to measure angular velocity and accelerometers to measure linear acceleration has

resulted in new studies exploiting the improved measuring capacity of this technology to monitor

motor symptoms [16]. Automated tracking of these symptoms using wearable sensors would be

significantly helpful to relieve patients from a constant pressure of maintaining logs or recalling

symptoms during the clinical visits. Although there have been prior attempts described in the

literature to design autonomous tracking systems capable to operate in patients home environment,

most of these systems have been limited to automating the PD detection during scripted activities

as mentioned earlier. This applies especially to bradykinesia detection [17] which has only been

successfully reported for specific activities such as hand opening and closing, finger tapping

[18,19] or walking [20,21,22]. There is an unfulfilled need to autonomously track and quantify the

symptoms of bradykinesia in such a manner that these clinically informed features can be directly

used by clinicians to inform the presence and severity of bradykinesia during ADL.

This thesis describes the design of a robust algorithm to track bradykinesia during

unscripted activities using wearable sensors that provides information regarding limb movement

and muscle activity. The algorithm is a key requirement to developing a wearable device that will

enable the clinician to obtain objective, minute-by-minute measures of the bradykinesia

manifestations during ADL. The outputs of the algorithm include the identification of occurrences

of bradykinesia and objective metrics to quantify the magnitude of impairments (symptoms)

caused by bradykinesia. The experimental protocol for this research was designed to record several

continuous hours of sensor data to serve as a proof of concept to the intended full day monitoring

5

of the PD patient. The sensor data were captured from combined inertial measurement unit (IMU)

and electromyography (EMG) wireless sensors to capture the limb movement and muscle activity

respectively. Experiments were video recorded and annotated by movement disorder experts to

establish a truth file. The problem of autonomous tracking of bradykinesia incidences was

approached by successfully separating the sensor dataset into walking and non-walking activity

subclasses using neural network. This approach led to capturing the activity-specific symptoms of

bradykinesia more robustly.

After sucessfully separating the unscripted activity data into walking and non-walking

activity sub-classes, activity-related features were developed to capture specific symptoms of

bradykinesia during walking and non-walking. Using these features as inputs, separate

classification algorithms were designed for tracking the symptoms of bradykinesia during walking

and non-walking activities. The algorithms demonstrated a high accuracy in detecting body

bradykinesia on previously unseen test data. In order for clinicians to have a continuous record of

the fluctations in severity of body bradykinesia, quantitative measures such as speed, range and

total time of movements were provided and their clinical usefulness was demonstrated for

individual cases from the data set. Together the detection algorithm and sensor-derived metrics

provide a complete software platform to quantify the quality of voluntary and involuntary

movements in PD. This work demonstrates the first ever algorithm capable of tracking symptoms

of movement impairments in bradykinesia during ADL that equip the clinicians with consistent

real-time tracking to better inform the progression of the disease and provide outcome measures

for therapeutic interventions, ultimately improving their quality of life.

6

SPECIFIC AIMS

To develop an autonomous algorithm for detecting the presence and absence of body bradykinesia

and monitoring the different symptoms of bradykinesia a rigorous signal processing analysis was

performed and the key signal attributes of bradykinesia symptoms were coupled with machine

learning algorithms. The following aims identify key stages of the project development:

Aim 1: Acquire EMG and IMU sensor data during unscripted activities. The target was to

include experimental data from 16 patients to create an adequate sample set capturing a broad

range of regular daily activities from PD subjects with different severity levels and different

presentations of bradykinesia as well as other concomitant PD motor symptoms. An experimental

protocol was designed to collect sensor data while PD subjects engage in ADL in a simulated home

environment.

Aim 2: Autonomously classify unscripted activities into walking and non-walking subclasses:

To identify all characteristic attributes of bradykinesia, the movement data was separated into

walking and non-walking activity subclasses as the symptoms of bradykinesia manifest differently

during these activities. A walking detection algorithm was designed to identify stepping instances

by utilizing the key signal features of leg movement during walking. The identified steps were

used to obtain walking segments, which in turn guided the partitioning of movement data into

walking and non-walking subclasses. As a pre-stage to classifying bradykinesia during ADL, this

approach targeted more specific feature selection of bradykinetic symptoms in isolated walking

and non-walking activity subclasses.

Aim 3: Real-time detection of bradykinesia and tracking of motor symptoms: The walking

and non-walking activity sub-classes were separately targeted to design classification algorithms

to identify bradykinesia. Objective measurements designed through rigorous signal processing of

7

IMU and EMG sensors were employed as inputs to train and test these machine learning

algorithms. The decisions from these activity specific classifications were compiled in a real-time

bradykinesia tracking report throughout the range of unscripted and unscripted activities.

Additionally, a complete profile of clinically informed features including range of movements,

speed of movements, paucity of movements and reduced arm swing was recorded using gyroscope

data. The final deliverable includes the real-time profile of clinically informed features of

bradykinesia symptoms as well as the periods of bradykinesia occurrences identified during

unscripted daily activities.

8

METHODS

Subjects: I worked with a team of clinicians and engineers to contact patients from local

neurological clinics, American Parkinson’s Disease Association (APDA) support groups, and the

Michael J Fox Trial Finder to recruit a total of 16 subjects that volunteered for the study. Prior to

their participation, subjects signed a consent form approved by Western Institutional Review

Board (WIRB) indicating their understanding of the procedures and risks of the study, and

willingness to participate. All the subjects were independently ambulatory and were diagnosed

with mild to moderately severe PD based on the Hoehn and Yahr staging scale (Hoehn–Yahr stages

II–III while “on” and Hoehn–Yahr stages III–IV while “off”) [7]. All patients were on a regimen

of levodopa-replacement parkinsonian medications and reported wearing off with motor

fluctuations between doses. All the patients presented with body bradykinesia, based on the rating

criteria illustrated by the UPDRS (Appendix A), as well as other movement disorders associated

with Parkinson’s disease. Based on their bradykinesia symptoms, the patient data were divided

equally (n=8) into training and testing sets, each consisting of approximately 1000 minutes of

recorded data. A summary of the subject population characteristics is provided in Table 1,

PD Subjects Data Corpus Training Data Corpus Testing

Number n = 8 n = 8

Age (y) 57.5 ± 12.5 63.2 ± 12.1

Men/Women 6/2 6/2

Disease Duration (y) 8.6 ± 5.4 5.4 ± 1.7

Total Data (min) 1000 1000

Bradykinesia Prevalence (%) 58.7% 76.4%

Table 1. Subject Population Characteristics – The bradykinesia prevalence indicates the percentage of

recording period when the PD subjects demonstrated symptoms of bradykinesia.

9

organized according to their use for Training or Testing of the Classification Algorithms. The

subsets were selected such that all symptoms of bradykinesia were covered in both sets.

Study Location: The data were recorded in a simulated home settings in an approximately 900 sf

research laboratory within Delsys Inc. (Natick, MA). The experimental room included household

items such as a refrigerator, a set of table and chairs, a couch and a bed to provide different options

to sit, move around or lie down. Food, drinks, and snacks were also provided during the

experiments. Patients were encouraged to freely engage in activities that they would be doing at

home as a part of their daily routine, and were not coached on how or when those activities should

be performed. Figure 2 demonstrates an example of a subject engaged in daily activities during the

experiment.

Figure 2. Sensor data captured concurrently from EDS and TA muscles – The highlighted locations

indicate the finger extensor muscles of the forearm (EDS) and ankle flexor (TA) muscles of the leg on the

more symptomatic side. Wireless sensors placed on each of these muscle locations captured movement

and muscle activity in 9 channel IMU data comprising of accelerometer, gyroscope and magnetometer

and 1 channel EMG data.

10

Data Acquisition: Trigno™ IM wireless sensors (Delsys, Inc, Natick, MA) were used to record

10 channel IMU (tri-axis accelerometer, tri-axis gyroscope, tri-axis magnetometer) and EMG

(electromyography) data from the subjects. Eight sensors were placed on the skin above the mid-

belly of the following muscles: 1) left and right finger extensor muscles of the forearm (Extensor

Digitorum Superficialis(EDS)); 2) left and right knee extensor muscles of the thigh (Vastus

Lateralis(VL); 3) left and right ankle flexor muscles of the leg (Gastrocnemius; and 4) left and

right ankle extensor muscles of the leg (Tibialis Anterior(TA)). A ninth sensor was placed on the

hip. The number and locations of the sensors used maintained consistency with prior studies [15]

and allowed for analysis of optimal sub-sets of sensor locations to improve usability.

The accelerometer and gyroscope data were sampled at 148.148 Hz, the magnetometer data at

74.074 Hz, and EMG data at 1111.11 Hz. Based on recommendations from prior work [15] the

sensor signals from the EDS and TA muscles from the more symptomatic side reported by PD

patients during their pre-screening interview were selected for the analysis. The selection of these

muscle locations for developing algorithms for monitoring bradykinesia was reinforced by the goal

of utilizing a reduced sensor set for practical ease-of-use by a patient while maintaining sufficient

representations of movement of different limbs. The signal data from the sensors, were acquired

wirelessly by EMGworks Acquisition software (Delsys Inc) in the form of High Performance

Fortran (HPF) files for offline data analysis. Time-synchronized video recordings of the patient

were also captured during the experiment which was later annotated by movement disorder

experts. Data were automatically saved into data files of 10-minute length to minimize asynchrony

between sensor and video data.

Protocol: The sensor data were collected for a total duration of 3 hours while the subjects carried

out unscripted activities in the laboratory to capture the change in bradykinesia symptoms between

11

dopamine replacement dosages as a proof of concept for future applications of all day recordings.

The experiments were preferably timed in the middle of their On-period1 to capture any motor

fluctuations produced by wearing-off of their medication or dyskinesia side-effects that may

follow medication intake. During the recording period, the subjects were free to move about at will

except at the beginning, middle and the end of the overall recording period, when the Part III

standardized UPDRS Motor tasks (Items 23-30) of 10-minute duration were administered as a

possible reference for the video annotator. Throughout the experiment, patients performed a range

of unscripted voluntary activities including walking, sitting, standing, resting, reading, using the

phone, eating, writing, and conversing with researchers and family members. The video camera

operator recorded all the patient activities from a suitable distance which were time synchronized

with the sensor data recordings. The activity type and presence or absence of bradykinesia were

evaluated based upon annotation of video recordings by movement disorder experts using the

standard UPDRS clinical rating scale for Body Bradykinesia (Item 31 of Part III Motor

Examination) (see Appendix A) to identify a truth file for type of activity (sitting, standing,

walking, lying) as well as bradykinesia presence or absence. Those segments of the recordings in

which the presence of bradykinesia overlapped with dyskinesia and freezing were excluded to

avoid possible confounding factors that go beyond the scope of this thesis. The included data had

clear manifestations of bradykinesia as annotated by movement disorder experts.

Manifestation of bradykinesia in unscripted ADL: The impairments caused by bradykinesia

manifest differently by visual observation as well as in the signal data depending on whether there

1 The fluctuating response to levodopa can be broadly divided into "on" and "off" periods. During an "on" period, a

person can move with relative ease often with reduced tremor and stiffness. “Off” periods describe those times when

a person has greater difficulty with movement. A common time for a person with Parkinson's disease to experience

an "off period" is just prior to taking the next dose of levodopa, and this experience is called "wearing off.

12

is a baseline-level of activity present, such as in case of walking, or whether no baseline-level of

activity exists, as is the case for non-walking activities such as sitting or standing. The signals from

lower limb gyroscope (TA muscle) corresponding to walking activities and non-walking activities

are demonstrated in Figure 3. The plots indicated that walking and non-walking signal looked very

different both in presence and absence of bradykinesia. However, on comparing figure 3 (b) and 3

(c) it is observed that due to impairments caused by bradykinesia the signals during walking 3(c)

closely overlap the feature space of normal non-walking activities 3 (b). To eliminate the overlap

of sensor data attributes in feature space, it was pertinent to separate out walking and non-walking

activities.

Figure 3. Sensor data captured during daily activities – Sensor data captured using lower limb

gyroscope during walking and non-walking activities in the presence and absence of bradykinesia.

13

Walking Detection Algorithm Design: The first stage to classify the sensor data into walking and

non-walking subclasses was to accurately detect the individual steps throughout the unscripted

activities. The detection interval of activity type was informed by the time required to capture the

key signal features [23] of the activity subclasses. In the case of step detection, the key feature was

the characteristic swing frequency. It is necessary to capture at least 2 consecutive steps to inform

the swing frequency (Figure 4). As indicated by [24] the average stepping time for PD patients is

1.06 seconds with approximately 2.7% variability.

After analyzing the sensor data during walking activities for different patients, it was

observed that a window with temporal length of 5 second can incorporate 2 consecutive gait cycles

robustly for all severity levels in PD. The detections were provided each second within each 5

second window in which the activity type was determined at the central second and the 2 second

windows on the left and right sides were retained for activity context (Figure 5). The context helped

to correctly capture the transitions between different activity subclasses.

A rigorous signal analysis to detect the stepping activity directed the selection of the

following features: 1. dominant frequency in the axis representative of the step swings (lower

extremity gyrocope); 2. total signal power in the dominant frequency (lower extremity gyroscope);

Figure 4. Normal healthy walking – Sensor data captured using lower limb gyroscope during walking in

the absence of bradykinesia.

14

and 3. total signal power in the voluntary stepping frequency range of 0.3-1.5 Hz in all three axes

(lower extermity accelerometer and gyroscope). These discriminable feature attributes helped to

separate the unscripted activities into walking and non-walking subclasses.

A single-hidden layer Neural Network architecture was designed to classify walking and

non-walking activity subclasses. Input training data for the classifier included feature vectors from

selected training samples incorporating different activity subclasses under the presence and

absence of bradykiensia manifestation. The annotation labels of activity types were used as the

Figure 5: Temporal segmentation of signal data – The first row displays the sensor data captured from

all Trigno sensors concurrently and the window partitions indicate the bradykinesia severity decision

points. For any decision point, the sensor data from 1-min before and 1-min after is evaluated. The second

row displays the segmentation of 2-min sensor data into 5-sec sub windows with 4-sec overlapping and

the third row displays the overlapped sub windows. The walking or non-walking activity decision is made

individually on all these sub windows.

15

truth values for this algorithm. A total of 3000 training samples were used to train the algorithm

and it was tested on unseen 6000 samples.

The trained algorithm was used to determine the activity type within 2-minute windows of

data. This window length was selected to ensure that the slowness characteristics of bradykinesia

can be captured over a long enough duration of time while maintaining sufficient temporal

resolution to track transitions between the different daily activities. Steps detected by the algrotihm

were grouped into walking data and separated from the surrounding non-walking data. If the total

number of seconds of walking activity was greater than 30, then the walking data was used to

assess the presence or absence of bradykinesia during the respective 2-minute window; otherwise

non-walking data was used for the bradykinesia detection.

Bradykinesia Detection Algorithm Design: From the walking and non-walking data, I designed

separate bradykinesia classifiers for walking and non-walking activities that could learn the

relationships between the input features and output clinical annotations of presence/absence of

bradykinesia. The goal of classification was to identify the areas of bradykinesia presence while

Figure 6: Bradykinesia Detection Algorithm Flowchart – Algorithm architecture to capture body

bradykinesia symptoms during unscripted activities of daily living from body worn sensor data.

16

minimizing the misclassifications. Figure 6 demonstrates the overall architecture flow of the

algorithm.

A large set of clinically informed features was designed to capture the movement and muscle

activity characteristics of bradykinesia during the various acivities. Out of these 21 features, 6

were considered most directly related to the objective measures of bradykinesia as defined by the

UPDRS and were therefore also used to provide objective sensor-based measures that can be

used by clinicians to monitor changes in the magnitude of bradykinesia symptoms (please refer to

the section entitled Calculating clinically informed features of bradykinesia manifestations, that

appears later in this thesis). A detail description of all the features is briefly summarized below:

Poverty of Movements: This symptom captures the decrease in the performance of automatic

movements like hand gestures while talking, adjustments in posture while seated, or distinct

temporal sequences in gait. The features designed to capture this symptom include:

Total Active Time Per 2-Minute: Calculated as sum of instances when the absolute value of

low pass filtered gyroscope signal data in any axis is greater than noise threshold value. A low

value of this measurement captured the paucity of movements with in a 2-minute activity

window during non-walking activity.

Degree of Shuffling: Ratio of heel strike and toe-off signal peaks in lower limb GYR data

during walking activities. During normal gait the ratio of toe off peak to heel strike peak is

greater than 1 otherwise it may indicate indistinct toe off and heel strike movements during

walking and thus impoverish movements in the walking during presence of bradykinesia.

Total Activity Peaks: Determines the total number of discrete activity instances above the noise

level. Depending on the total number of peaks present may indicate paucity of movements.

17

This feature is computed for both upper and lower limbs using GYR data during non-walking

activities.

Reduced Velocity of Movements: This symptom captures the inability to produce normal velocity

of voluntary limb movements during ADL.

Mean velocity of voluntary activities in all 3-axis: Identifies voluntary activity peaks in x, y,

z- directions and computes their mean amplitude in GYR data. This feature indicates the speed

of movement with respect to each voluntary activity. Lower value of in this case may indicate

slower average speed of voluntary movements and thus presence of bradykinesia. This feature

is computed both upper and lower limbs for both walking and non-walking activities.

Average toe-off velocity: Captures the average velocity during the toe-off stance of walking.

Lower value of this feature may indicate a slowness during the toe-off stance of gait. This

feature is computed for lower limb using GYR during walking activities.

Dominant arm swing velocity: Identifies the average speed of arm speed during walking

activities and captures the reduced arm swing which is a prevalent symptom of bradykinesia.

This feature is computed for upper limb using GYR during walking activities.

Reduced Range of Movements: This symptom captures the inability to produce the complete

range of voluntary movements in ADL.

The reduced range of movements is captured by integrating the respective velocity features

for both upper and lower limbs.

Hesitancy: This symptom captures the difficulty in initiating a new movement. It is also referred

to as akinesia.

18

Presence of EMG in swing cycle of walking: Captures the EMG signal energy during swing

phase of the walking cycle relative to the energy in toe-off and heel-strike phase. Two distinct

EMG signal peaks during toe-off and heel-strike with relatively lesser energy in the middle for

swing phase may indicate normal gait and absence of bradykinesia. On the other hand, only

one indistinct peak indicates TA activation during the swing phase and thus manifestation of

bradykinesia in terms of lack of coordination during walking.

Power Spectral Density in High Frequency Range: The integrated power spectral density in

frequency range of 7-10 Hz. It provides a measure of overall energy content in high frequency

range which may be used as an indicator of hesitancy. This feature is computed for both upper

and lower limbs using GYR data.

Other Features: These features include objective measurements of overall slowness of movement

during voluntary activities and the measurements capturing the combined effect of the symptoms

described above.

Energy in Dominant Frequency: This feature determines the signal energy of stepping at the

dominant frequency. A low value may indicate (i) consistent but low range stepping (ii) good

range but non-consistent stepping during walking activities.

Dominant Stepping Frequency: Dominant stepping frequency identifies the dominant

frequency value of GYR within the frequency range of walking activities. Lower value of

dominant stepping frequency may indicate inconsistent or arrhythmic walking.

Average zero-crossing of voluntary activity peaks in all directions: Computes the zero-crossing

time of the GYR peaks identified during voluntary activities and provides the average zero

crossing time. Lower value of this feature may indicate faster movements and thus the absence

of bradykinesia. This feature is computed for both upper and lower limbs using GYR data.

19

Ratio of amplitude of peaks to zero crossing distance: This features computes the relative time

of zero crossing with respect to the peak amplitude. It helps to identify the normalized speed

of voluntary movements and presents the speed of movements independent of their range. This

feature is computed for both upper and lower limbs using GYR data.

Total energy in voluntary activity frequency (0.3-1.5 Hz): Computes the total energy in power

spectral density in the range of voluntary activity range in all 3-axis. This feature is computed

for both upper and lower limbs using ACC and GYR data.

To check the robustness of designed features, I identified the discriminability of individual features

using the Bayesian classification that informed the feature selection for next stages of

Table 2. Complete list of identified characteristic features during non-walking activities – The table

represents the list of characteristic features grouped under different symptoms of bradykinesia along with

individual feature discriminability values. The bold red features are the final features selected by

performing the feature selection process.

20

development. Bayesian classification being generative classifiers2 were appropriate choice in this

case to determine individual feature discriminability. The features were ranked with respect to

their individual descriminability. Strating by using the feature demonstrating the highest individual

discreminability as the initial feature, I executed a forward selection process of features. In this

process, the next best discriminative feature was picked iteratively to test the group

discriminability of features. The feature was added to selected list only if it augments the overall

discreminability of group of features. The selection converged when there was no significant

improvement in the overall discreminability of group of features. This method considered the co-

2 Generative classifiers learn a model of the joint probability, p( x, y), of the inputs x and the label y, and make their

predictions by using Bayes rules to calculate p(ylx), and then picking the most likely label y.

Table 3. Complete list of identified characteristic features during walking activities – The table

represents the list of characteristic features grouped under different symptoms of bradykinesia along with

individual feature discriminability values. The bold red features are the final features selected by

performing the feature selection process.

21

dependence in the features and ensured the selection of features that work well together for

classification. This feature selection process was executed for non-walking and walking activities

separately. A complete list of features and their individual separability is presented in Table 2 and

Table 3 respectively.

Classification Algorithms: Once the input features for walking and non-walking classifiers were

finalized, I designed machine learning architectures to robustly classify the presence and absence

of bradykinesia in both walking and non-walking categories. Classification was made on 2-minute

windows of sensor data with 1-minute overlap. Two discriminative models3 were tested: 1)

Logistic Regressions and 2) Neural Networks. The rationale behind choosing these models for

classification was based on the fact that discriminative models learn the (hard or soft) boundary

between classes which made them well suited to discern between presence and absence of

bradykinesia based on different symptoms rather than the generative models which model the

distribution of individual classes and were a better choice to determine individual feature

separability for feature reduction process. Moreover, while generative algorithms make structure

assumptions on the model, the discriminative algorithms make fewer assumptions. For example,

Naive Bayes assumes conditional independence of the features, while logistic regression (the

discriminative "counterpart" of Naive Bayes) does not [25]. As the underlying nature of data is

unknown, wrong modeling assumptions in generative models might have led to higher asymptotic

errors as compared to discriminative models. Therefore, discriminative models were preferred over

the generative models. The motivation behind the choice of Logistic Regressions and Neural

Networks as the starting point was that while former method allows for easier interpretability of

3 Discriminative classifiers model the posterior p(ylx) directly, or learn a direct map from inputs x to the class

labels.

22

model parameters, the latter method allows more flexibility in structuring the model parameters in

terms of the choice of number of hidden layers, hidden units and hyperparameters of the model

[26].

Logistic Regression: Logistic Regressions are discriminative models that allow for easy

interpretability of model parameters. For logistic regression models, it is possible to test the

statistical significance of the model coefficients [26] which can be used to build models

incrementally. Its also easy to update the model to take in new data (using an online gradient

descent method) in logistic regressions. The decision boundary of logistic regressions is linear.

The logistic regression algorithm was implemented as demonstrated in Figure 7.

The mathematical model of logistic regressions can be represented as below.

u = b + x1w1 + x2w2 + … + xmwm

y=eu

1+eu

Here u is the linear regression equation and y is the estimate probability that the given case in a

category. The loss function used in this case was negative log likelihood also known as the

multiclass cross-entropy [27]. Cross-entropy can be used as an error measure when a network's

Figure 7: Flow diagram of logistic regression algorithm

23

outputs can be treated as representing independent hypotheses (i.e. each node stands for a different

concept), and the node activations can be understood as representing the probability (or

confidence) that each hypothesis might be true. In that case, the output vector represents a

probability distribution, and the error measure - cross-entropy - indicates the distance between

network’s belief in the state of bradykinesia manifestation and the true states as annotated by

movement disorder expert.

Neural networks: Neural networks are also discreminiative models considered as non-linear

generalizations of the logistic regression, and thus are theoratically at least as powerful as that

model. However, neural networks are also more flexible than logistic regressions.

One hidden layer neural network was developed (refer to Figure 8)for the classification agorithm

by utilizing ADAM4, an algorithm for first-order gradient-based optimization of stochastic

4 ADAM: an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive

estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little

memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are

large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems

with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require

little tuning.

Figure 8: Flow diagram of artificial neural network algorithm

24

objective functions, based on adaptive estimates of lower-order moments. The method was inbuilt

in Tensorflow (Google Inc.) and was straightforward to implement, computationally efficient, had

little memory requirements, was invariant to diagonal rescaling of the gradients. The advantage of

using ADAM was that the hyper-parameters required little tuning [28] and it comparesd favorably

to other stochastic optimization methods.

For both classifiers a context layer of classification was implemented using a 5-minute

moving average to obtain more robust results and better capture slower progressions of

bradykinesia . For non-walking activities, the amount of movement had a large variability in per

minute samples which might have led to noisy output detections.The context layer helped in this

case to achieve a consistent classification robust against single 1-minute misclassifications.

Algorithm Training For the training stage, I chose a small set of training data from complete data

set of 8 PD subjects (refer to Table 1) which targeted the differences that could closely discern

between presence and absence of bradykinesia. The training samples were the clear examples of

presence or absence of bradykinesia without any overlap of other PD manifestations. To provide

optimal discrimination of subtle differences in presence and absence of bradyinesia, equal number

of samples were selected to capture each symptom within walking and non-walking activities. The

remaining samples from the same 8 PD patients were used as a part of validation set.

Evaluation Metrics: To evaluate the designed machine learning architectures, discrepancies

between the output of the classification algorithm and expert annotation were evaluated based on

sensitivity (i.e., the ability of the algorithm to correctly identify a movement disorder when it is

present), which can be formulated as

𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 =TP

TP + FN

25

and specificity (which describes the ability of the algorithm to correctly identify all instances when

the movement disorder is absent), formulated as

𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 =TN

TN + FP

The overall accuracy is defined as the as the ratio of the number of correct decisions (true positives

and true negatives) to the total number of decisions, accuracy can alternately be written as

𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = (𝐹𝑝 × 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦) + (𝐹𝑎 × 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦)

Here, 𝐹𝑝 is the fraction of the total minutes in the database in which the disorder is present,

and 𝐹𝑎 is the fraction of total minutes in the database in which the disorder is absent. From this

formulation, I could observe the dependence of the accuracy on the proportion of disorder present.

Calculating clinically informed features of bradykinesia manifestations: To quantify the clinical

manifestations of body bradykinesia as described in UPDRS guidelines, I calculated 6 clinically

informed features from gyroscope sensor signals (Table 4). Three metrics were used to quantify

Table 4. Clinically informed metrics – The table represents the list of clinically informed metrics to

quantify the symptoms of bradykinesia during walking and non-walking activities.

26

bradykinetic symptoms during non-walking activities and three 3 were used to quantify

bradykinetic symptom during walking activities. Each metric is described below:

1. Total Active Time Per 2-Minute (GAT)

GAT is the total time within a 2-minute window when any type of voluntary activity is

performed. It was calculated as sum of instances when the absolute value of low pass filtered

gyroscope signal data in any axis is greater than noise threshold value. The signal-dependent

noise thresholds were computed as a factor of 2 times the average range of signal values when

no activity is being performed. A low value of this measurement captured the paucity of

movements with in a 2-minute activity window during non-walking activity. Figure 9

demonstrates the instances of voluntary activities within a 2-minute window.

2. Mean Velocity of Voluntary Activities in all 3-axis (GMV)

GMV was computed by first calculating the mean of absolute instantaneous joint velocities of

the voluntary activities in x, y, z- directions using gyroscope signal data. Then an overall mean

of those values was taken as all voluntary movements were treated equally and the assessment

is independent of type of activities within the non-walking subclass. A low value of GMV

indicated low average speed of voluntary movements and hence possibility of bradykinesia

Figure 9: Total active time in 2-minute window – Voluntary activity instances highlighted within 2 -

minute window gyroscope signal. Blue signal corresponds to activity in x-axis, green signal corresponds to

activity in y-axis and Red signal corresponds to activity in z-axis. The total time corresponding to all

highlighted segments is the total activity time in a 2-minute window.

27

occurrence. This measurement was computed for upper-limb and lower limb position during

non-walking activities.

3. Mean Range of Voluntary Activities in all 3-axis (GMR)

GMR was computed by first calculating the integral of absolute instantaneous joint velocities

gyroscope during all voluntary activities in x, y, z-directions and then taking an overall mean

of those values. The integral was computed using trapezoidal method which performs discrete

integration by using the consecutive data points to create trapezoids, so it was well suited to

handling data sets with discontinuities. This method assumed linear behavior between the data

points. GMR captured mean range of voluntary activities during non-walking activities and was

computed for upper-limb and lower limb location.

4. Mean Leg Swing Velocity (GLSV)

GLSV was computed during walking activities by calculating the mean velocity during the

forward swing duration of gait cycle (shaded region in Figure 10) where 10(a) represents a gait

cycle in absence of bradykinesia and 10(b) represents a gait cycle in the presence of

bradykinesia manifestation. Velocity values from gyroscope sensor placed at lower limb were

captured during the swing phase in the dominant axis of forward movement direction with

respect to the knee joint. GLSV was computed as a mean of these values for all the steps taken.

Figure 10: Leg swing during walking – Highlighted swing phase during (a) step cycle in absence of

bradykinesia and (b) bradykinetic step cycle as captured by gyroscope signal.

(a) (b)

28

Depending on the total number of steps taken during walking instances, an average value

indicated the mean leg swing speed. As demonstrated in Figure 10 a high value of GLSV

indicated higher mean leg swing speed as compared to a low value.

5. Mean Leg Swing Range (GLSR)

GLSR was computed by integrating the lower limb gyroscope data during the forward swing

phase of gait cycle during walking activities. The integration was calculated using the

trapezoidal method as used in GMR but only in the swing phase of the dominant axis of forward

movement direction with respect to knee joint. A lower value of GLSR indicated lower range of

movements during walking activities.

6. Mean Arm Swing Range (GASR)

GASR was computed by extracting the upper limb gyroscope signal data during identified

walking instances and then calculating the integral of absolute instantaneous joint velocities

captured by the upper limb gyroscope sensor in dominant axis of movement. The integral is

computed by trapezoidal method as described earlier. A mean of integrated values provides

GASR. A low value of GASR indicates reduced arm swing during walking activities which in turn

indicates the presence of bradykinesia.

29

RESULTS

The test data, including 1000 minutes of unscripted daily activity from 8 patients,

demonstrated the following results: 1) the walking classifier discriminated walking and non-

walking activities from unscripted movements with 99.5% accuracy; 2) the Bradykinesia detection

algorithm detected occurrences of bradykinesia with 96.5% accuracy; and 3) clinically-informed

metrics were able to quantify the motor symptoms associated with bradykinesia to provide subject

specific indications of wearing-off in PD.

Classification of Walking and Non-walking Activities: The neural network architecture

was able to classify walking and non-walking activities with 99.5 % accuracy throughout the

unscripted activity data. The detailed accuracy measures are provided in Table 5. Of the relatively

few false positive and false negatives that did occur, most were comprised of instances in which

the patients turned around in between durations of continuous walking in one direction.

Figure 11, demonstrates the walking detection results on test subject data for a duration of

10 minutes capturing different unscripted ADL. It demonstrates that the walking activitity detector

was successfully able to partition all the activities into walking and non-walking subclasses, even

when frequent transitions occurred or the walking periods were brief.

Table 5. Classification accuracies of Walking Detector – The table represents the Sensitivity, Specificity

and overall Accuracy of Walking Detector.

30

Bradykinesia Detection: A Neural Network architecture and Logistic regression

architecture subclasses were compared. The Neural Networks designed to classify the occurrences

of bradykinesia during walking and non-walking activity performed with an average accuracy of

97.4% for non-walking activities and 93.8 % for walking activities on unseen test data from the 8

PD patients (Table 6). The overall accuracy for all unscripted ADL was 96.5%.

I compared the neural network classification results with those obtained from logistic

regression classification of bradykinesia, and observed the logistic regressions decreased in overall

accuracy by 8.3%. Most of the errors in logistic regressions occurred during walking activities.

The details of results obtained from logistic regressions are demonstrated in Table 7

Figure 11: Walking detector algorithm results – An example of gyroscope data from lower extremity for a 10-minute

segment of ADL. Results of walking and non-walking classifications from the classification algorithm (red bar) are

compared with truth value from annotator (blue bar)

Table 6. Bradykinesia detection accuracies using Neural Network architecture – The table represents

the Sensitivity, Specificity and overall Accuracy of Bradykinesia Detection during walking as well as non-

walking activities using Neural Network architecture.

31

Clinically informed metrics to Track Motor Symptoms of Bradykinesia: Figure 12

demonstrates the classification results for occurrences of bradykinesia and the progression of

clinically informed metrics for one of the subjects tested. It is observed that the classification

results were able to capture the bradykinesia manifestations as annotated by clinician except for a

few false positive detections. The figure also plots the metrics used to quantify changes in symptom

magnitude, including total active time (sec), mean range of movements (deg) and mean velocity

(deg/sec) of movements as a function of time during non-walking activities. At the start of the

experiment, shortly after the subject took PD medication, the clinically informed metrics provided

relatively high values of movement activity, and the neural network correctly detected

bradykinesia was absent. As the experiment proceeded, and the time since the subject last took

medication increased, all three metrics progressively decreased in magnitude and the neural all

three-metrics increased in magnitude and the neural network correctly detected the transition to a

period in which bradykinesia is absent. network correctly detected the presence of bradykinesia.

This response is consistent with the clinical phenomenon of wearing-off. Several minutes after the

administration of PD medication.

Table 7. Bradykinesia detection accuracies using Logistic Regression architecture – The table

represents the Sensitivity, Specificity and overall Accuracy of Bradykinesia Detection during walking as

well as non-walking activities using Logistic Regression architecture.

32

Another example of the performance of our software platform during non-walking data

from a different PD subject is presented in Figure 13. In this example, the neural network algorithm

correctly detects that bradykinesia is present throughout the duration of the experiment. However,

the clinically-informed metrics indicate that at the start of the experiment, approximately one hour

Figure 12: Clinically Informed features during non-walking activities Plot I – Example of clinically

informed metrics during non-walking activities captured by gyroscope placed at lower extremity. The plot

demonstrates instances of bradykinesia presence as well as absence.

33

prior to taking the next dose of medication, there are clear signs of wearing off as Mean Velocity

of Movements and Mean Range of Movements (lower two plots) decrease. These metrics are also

shown to increase soon after the medication dose was taken, indicating that even though the patient

is bradykinetic, they are showing signs of responding favorably to the medication as the magnitude

Figure 13: Clinically Informed features during non-walking activities Plot II – Example of clinically

informed metrics during non-walking activities captured by gyroscope placed at lower extremity. The plot

demonstrates instances of bradykinesia presence.

34

of their bradykinesia symptoms decline. These results demonstrate the same trend of “wearing off”

and improvement following medication dose as displayed in Figure 12 but without the same

changes in the presence or absence of bradykinesia.

Figure 14: Clinically Informed features during walking activities Plot I – Example of clinically

informed metrics during walking activities captured by gyroscope placed at lower extremity. The plot

demonstrates instances of bradykinesia presence as well as absence.

35

Similarly, results for the software platform are provided for walking activities in Figure 14.

As the experiment started the subject demonstrated symptoms of bradykinesia during first 2

instances of walking which were correctly detected by the neural network algorithm (dark pink

shading). The clinically informed metrics including mean leg swing velocity (deg/sec), mean leg

Figure 15: Clinically Informed features during walking activities Plot II – Example of clinically

informed metrics during walking activities captured by gyroscope placed at lower extremity. The plot

demonstrates instances of bradykinesia presence only.

36

swing range (deg) and the range of arm swing (deg) were in relatively similar range for these

instances. In the next walking instance after taking the medicine, all these clinically informed

metrics showed a clear increase and the neural network correctly detected it as absence of

bradykinesia (light pink area).

Another example of objective measures during walking activities is presented in Figure 15.

The figure demonstrates that the patient is identified as bradykinetic throughout the course of

experiment (all the walking periods are dark pink in the figure). However, the clinically informed

metrics gradually decrease from the starting time of the experiment until just after they take their

medication which indicates a wearing off trend. Approximately 50 minutes after taking the

medication all three features show an increase which displays that the patient is responding to their

medication and transitioning to an “on period”. These trends are same as observed in figure 14,

but without the changes in presence and absence detections of bradykinesia.

37

DISCUSSION

The robustness of results achieved, establishes a proof of concept that this framework can

be used to detect the occurrences of bradykinesia and track its symptoms in the form of clinically

informed metrics throughout the day or over the course of several days without interrupting the

daily activities of PD patients. The magnitude of impairments can inform the clinicians regarding

nuanced changes in symptom magnitude which is not captured by the neural networks. It can also

provide information such as which symptoms were more prevalent than others over the monitoring

period and indicate the trends or transitions in symptoms with respect to the medication

replacement schedule, thereby informing the degree of responsiveness to the medication. The

results can be examined more closely to demonstrate the clinical utility of the metrics.

Comparing Figure 12 and Figure 13, it is observed that as per the bradykinesia detection

algorithm, in the first case the subject responded to the medication by reverting to their “On” period

(bradykinesia no longer present) however in second case they didn’t (bradykinesia still present).

Here the clinically informed metrics help to reveal that in fact the subject in Figure 13 did show

some level of responsiveness to the medicine. Moreover, the change in individual metrics

demonstrated that some symptoms of bradykinesia such as range and velocity of movements

responded better to medicine than others such as poverty of movements. Such comprehensive

details of progression trends in bradykinesia symptoms can help the clinicians to identify tailored

medicine for their patients. The comparison of results presented during walking activities in Figure

14 and Figure 15 also support this argument.

With the option to detect occurrences and quantify the underlying symptoms, it is now

possible to see a complete picture of bradykinesia manifestations. This solution is the first attempt

38

in the literature to classify occurrences of bradykinesia during ADL as well as to provide clinically

informed metrics of the symptoms of bradykinesia based entirely on the sensor measurement rather

than trying to use machine learning classifiers to match sensor data to severity scores from

annotators using clinical scales such as the UPDRS [7]. The UPDRS was not designed for

providing continuous metrics for changes in severity and the specific guidelines for classifying

severity into an arbitrary 0-4 classification scale are poorly outlined and susceptible to high

variability due to differences in clinical interpretation. For example, the guidelines for body

bradykinesia are clear in terms of which symptoms may be associated with presence and absence

of bradykinesia but do not provide specific rules for differentiating the different levels of severity.

This limitation is in fact broadly present across all the movement disorder tasks contained within

the UPDRS. The novelty of our approach adopted in this thesis is to use the classifiers to

continuously monitor presence and absence of dyskinesia based upon which symptoms are present,

and then rely on the actual objective metrics from the sensor to monitor changes in the different

symptoms that underlie the disorder.

The ability to successfully provide a software platform for monitoring bradykinesia during

ADL can also be attributed to 1. the algorithm design and 2. the high-fidelity sensors. Strength of

the algorithm comes from the strategy to classify the walking and non-walking activities

separately. These activity categories align themselves with the UPDRS description of bradykinesia

as it specifically identifies a different set of symptoms for the walking and non-walking periods.

This also connects the algorithm interpretations with the expert annotator who also relies on such

clinical impressions to determine when bradykinesia is present or absent. Overall this architecture

provided a dynamic approach to tackle unscripted activities and mitigated the risks of

misclassifications during bradykinesia detection.

39

Rigorous signal analysis supplemented this architecture with highly discriminable features.

The previous work reported in the literature for bradykinesia and other symptoms of PD has mostly

concentrated on using statistical variables of raw signals as features [29,30,31,32,33] where the

current work focuses on activity specific features where each feature is mathematically designed

to capture specific properties of the movement, for example the reduced range and velocity. These

features can characterize the quality of voluntary movements more accurately by capturing the

clinically guided symptoms of impairment directly.

The input training data to these algorithms were intentionally selected to include all

combinations of symptoms of bradykinesia to represent nuanced examples to discern between

presence and absence. All the input samples presented to the algorithm were targeted to tune the

hyperplane close to the detection boundary. Profusion of severe bradykinetic and completely

healthy input samples was carefully avoided as it might have ended up biasing the algorithm.

The selection of neural networks over other machine learning algorithms used in literature

such as support vector machines [31,29,34] was based on the nature of data. Due to inter-patient

variability, the extracted feature data is prone to be noisy which might have resulted in needing a

lot of basis expressions to get separation in SVMs. Neural networks on the other hand are very

flexible on the account of their large number of degrees of freedom and can be well controlled

against overfitting. Choice of neural networks was crucial to guard against the possibility of

misclassifications in the data. When compared against other algorithms such as logistic

regressions, neural networks turned out to perform better because they can learn non-linear

decision boundaries while logistic regressions cannot. In case of bradykinesia detection, the

features demonstrate dependency on each other and sometimes do not have linear discriminability.

40

Clinically informed feature design was not possible without the sensor technology to

accurately capture the body movements. The sensors used for this research provide zero inter-

channel and inter-sensor delay which was critical to capture time synchronized movement and

muscle activity data from upper and lower limbs. The accuracy specifications of these sensors are

provided in Appendix B. Due to their high accuracy these sensors are valid tools to capture the

differences in body movements associated with the disorder and nuanced changes in symptom

magnitude. Gyroscope data was primarily used to compute the clinically informed metrics of

bradykinesia symptoms as it is well suited to capture the body movements in the form of joint

motion. High precision of the clinically informed metrics obtained from gyroscope established the

viability to measure the symptoms of bradykinesia on a quantitative scale to deliver this long-

standing need in the context of unscripted ADL.

These advantages and careful design decisions provided the opportunity to develop the first

software platform capable of tracking symptoms of movement impairments in bradykinesia during

ADL that equip the clinicians with consistent real-time tracking to better inform the progression

of the disease and provide outcome measures for therapeutic interventions, ultimately improving

their quality of life.

41

FUTURE WORK

Future work will be needed on several fronts to achieve a software platform that provides both

comprehensive monitoring capabilities across the full complement of PD movement disorders as

well as sufficient vetting to demonstrate its clinical usefulness. The following items should

command the highest priority:

Expand the relatively short (3 hour) simulated home setting data collections to long-term

(8 – 12 hour) actual home setting recordings. The expanded study will establish the

robustness of these metrics to more accurately represent the longitudinal behavior of

bradykinesia manifestations. It will also validate robustness of clinically informed metrics

under a variety of other activities that were not a part of this study, such as climbing stairs

or preparing meals. With the availability of long-term continuous data, there will also be a

possibility to experiment with the use of time dependent learning algorithms such as

dynamic neural networks, which have the power to increase classification accuracy by

including an added level of information about patterns of change that occur temporally.

Expand the unilateral body sensor data analysis to bilateral recordings: The current study

only explored the clinically informed metrics captured by sensors located unilaterally at

EDS and TA muscle locations on the more symptomatic side of the PD subjects. However,

this approach can be extended to analyze clinically informed metrics from less

symptomatic side which can provide discriminative measures of symptoms of bradykinesia

in both sides. Bilateral recordings also will help in augmenting the algorithms by capturing

other critical features such as the double support time during walking. In addition, because

PD typically progresses from unilateral to bilateral symptoms, this expanded capability

may provide more clinically relevant metrics.

42

Expanding the development of the software platform by monitoring of bradykinesia while

other movement disorders are present: By expanding the sensor data corpus to include

different disorders of PD such as dyskinesia, tremor or freezing, the algorithms will be

tuned to deal with the presence of more than one PD disorder, such as tremor during

bradykinesia.

Expanding the software platform development to include monitoring of other PD

movement disorder classifications and objective metrics. The framework adopted to

classify body bradykinesia in this thesis can be expanded to encompass other PD movement

disorders such as tremor, freezing, and dyskinesia, which are also known to be activity-

dependent and therefore approachable by adopting separate classifiers for different activity

states such as walking and non-walking or sitting vs standing.

These techniques will lead to significant validation and improvements that will help augmenting

the designed algorithms to include broader range of PD symptoms. As a complete solution for

tracking all PD symptoms, the final software platform will be an instrumental tool to guide PD

assessment for enabling better treatment and ultimately improving the quality of life.

43

APPENDIX A

UPDRS

Motor Examination Section

31. Body Bradykinesia and Hypokinesia (Combining slowness, hesitancy, decreased arm

swing, small amplitude, and poverty of movement in general.)

0 = None.

1 = Minimal slowness, giving movement a deliberate character; could be normal for some

persons. Possibly reduced amplitude.

2 = Mild degree of slowness and poverty of movement which is definitely abnormal.

Alternatively, some reduced amplitude.

3 = Moderate slowness, poverty or small amplitude of movement.

4 = Marked slowness, poverty or small amplitude of movement.

44

APPENDIX B

EMG Performance Specifications

Passband Gain = 300 +/- 10%, mean error +/- 1%.

Filter Topology: Butterworth

HP Corner: 20 Hz +/- 5 Hz, -40 dB/dec

LP Corner: 450 Hz +/-50 Hz, -80dB/dec

Noise Baseline: <3mV r.t.i. (rms), mean value ~750 nV r.t.i. (rms)

Accelerometer Performance Specifications

1G error +/-10%, mean error <1%

Offset error, <0.25G

AC RMS error, <0.010 g, mean value ~<0.005g

Gyroscope Performance Specifications

RPM error +/-10%, mean error <1% across 10rpm – 350 rpm.

Max offset +/-7rpm, mean value <1rpm

Max AC RMS error <1 rpm , mean value <0.03 rpm

45

BIBLIOGRAPHY [1] James Parkinson, "An Essay on the Shaking Palsy ," , Sherwood, Neely, and Jones

(London), 1817.

[2] Alan Godfrey, Claudia Mazza, Sue Lord and Lynn Rochester Silvia Del Din, "Free-Living

Monitoring of Parkinson’s Disease: Lessons From the Field," 2016.

[3] Andersen K, Larsen JP, Lolk A, Nielsen H, Kragh Serensen P. Aarsland D, "Risk of

dementia in Parkinson’s disease: A community based prospective study.," Neurology, vol.

56, pp. 730-736, 2001.

[4] Reuben DB., "What’s wrong with ADLs?," J Am Geriatr Soc, vol. 43, pp. 936-937, 1995.

[5] Britne Shabbott, and Juan Camilo Corte´s Pietro Mazzoni, "Motor Control Abnormalities

in Parkinson’s Disease," New York, 2012.

[6] Joseph Jankovic, "Motor Fluctuations and Dyskinesias in Parkinson’s Disease: Clinical

Manifestations," Movement Disorders, vol. 20, pp. S11 - S16, 2005.

[7] Elton RL Fahn S, "UPDRS Development Committee, Unified Parkinson’s Disease Rating

Scale," Recent Developments in Parkinson’s Disease, pp. 153-163, 1987.

[8] Sheridan MR. Flowers KA, "Movement variability and bradykinesia in Parkinson's

disease," Brain: a journal for neurology, pp. 1149-61, Aug 1990.

[9] Grabowski M, Lindvall O, Hagell P Reimer J, "Use and interpretation of on/off diaries in

Parkinson’s disease," J Neurol Neurosurg Psychiatry, vol. 75, pp. 396-400, 2004.

[10] Frisoni GB, Bianchetti A, Zanetti O, Trabucchi M Rozzini R, "Physical performance test

and activities of daily living scales in the assessment of health status in elderly people," J

Am Geriatr Soc, vol. 41, pp. 1109-1113, 1993.

[11] Branch LG, Cummings SR, Curb JD Guralnik JM, "Physical performance measures in

aging research," J Gerontol, vol. 44, 1989.

[12] Angelo Antonini, Carsten Buhmann, Joseph Classen Christiana Ossig, "Wearable sensor-

based objective assessment of motor symptoms in Parkinson’s disease," J Neural Transm,

vol. 123, pp. 57-64, 2015.

[13] Hyung Park, Paolo Bonato Leighton Chan and Mary Rodgers Shyamal Patel, "A review of

wearable sensors and systems with application in rehabilitation," vol. 9, no. 21, 2012.

[14] Elvis Camacho, Mickey E Abraham, Samuel S Bruce, Emmanuel LP Dumont Geoff

Appelboom, "Smart wearable body sensors for patient self-assessment and monitoring,"

New York, 2014.

46

[15] Serge H. Roy, Carlo J. De Luca, and S. Hamid Nawab Bryan T. Cole, "Dynamical

Learning and Tracking of Tremor and Dyskinesia From Wearable Sensors," IEEE

TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, vol.

22, 2014.

[16] Matjaz B. Juric Sebastijan Sprager, "Intertial Sensor-Based Gait Recognition: A Review,"

Sensors, pp. 22089-22127, August 2015.

[17] C.E. Jacobi, J.J. van Hilten R.J.W. Dunnewold, "Quantitative assessment of bradykinesia

in patients with parkinson’s disease," vol. 74, pp. 107-112, 1997.

[18] Kim et. al., "Quantification of bradykinesia during clinical finger taps using a gyrosensor in

patients with Parkinson’s disease," vol. 49, pp. 365-371, Oct 2010.

[19] Haijun Lin and Tim C Lueth Houde Dai, "Quantitative assessment of parkinsonian

bradykinesia based on an inertial measurement unit," 2015.

[20] Danielle E. Filipkowski, David E. Riley, Christina M. Whitney, Benjamin L Dustin A.

Heldman, "Automated Motion Sensor Quantification of Gait and Lower Extremity

Bradykinesia," , San Diego, California USA, 2012.

[21] Malcohm H. Granat Kaiyu Tong, "A practical gait analysis system using gyroscopes," vol.

21, pp. 87-94, 1999.

[22] Rong-Long Wang, De-Jyun Liou, Jin-Siang Shaw Pei-Hao Chen, "Gait Disorders in

Parkinson’s Disease: Assessment and Management," vol. 7, pp. 189-193, 2013.

[23] Frances Huxham, Jennifer McGinley, Karen Dodd, Robert Iansek Meg E. Morris, "The

biomechanics and motor control of gait in Parkinson Disease," vol. 16, pp. 459-470, 2001.

[24] Justine Lowentha, Talia Herman, Leor Gruendlinger, Chava Peretz and Nir Giladi Jeffrey

M. Hausdorff, "Rhythmic auditory stimulation modulates gait variability in Parkinson’s

disease," European Journal of Neuroscience, vol. 26, pp. 2369-2375, 2007.

[25] Michael I. Jordon Andrew Y. Ng, "On Discriminative vs. Generative classifiers: A

comparison of logistic regression and naive Bayes," in Advances in Neural Information

Processing Systems 14, 2001.

[26] Lucila Ohno-Machado Stephan Dreiseitl, "Logigstic regression and artificial neural

network classification models: a methodology review," Journal of Biomedical Informatics,

vol. 35, pp. 352-359, 2003.

[27] Christopher Bishop, Pattern Recognition and Machine Learning., 2006.

[28] Jimmy Lei Ba Diederik P. Kingma, "ADAM: A Method for Stochastic Optimization," in

International Conference on Learning Representations, 2015.

47

[29] M. Pansera, M.T. Arredondo, J.J. Estrada, M. Pastorino, L. Pastor-Sanz, J.L. Villalar J.

Cancela, "A Comprehensive Motor Symptom Monitoring and Management System: The

Bradykinesia Case," in Annual International Conference of the IEEE EMBS, 2010.

[30] A.A. v/d Plas MSc, E.A.H. Wagemans MSc J.I. Hoff MD, "Accelerometric assessment of

levodopa-induced dyskinesias in Parkinson's disease," Movement Disorders, 2001.

[31] Konrad Lorincz, Richard Hughes, Nancy Huggins, John Growdon, David Standaert

Shyamal Patel, "Monitoring Motor Fluctuations in Patients With Parkinson's Disease Using

Wearable Sensors," IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN

BIOMEDICINE, vol. 13, November 2009.

[32] Martin W.I.M. Horstink MD, PhD,Stan C.A.M. Gielen PhD Noël L.W. Keijsers MSc,

"Automatic assessment of levodopa-induced dyskinesias in daily life by neural networks,"

Movement Disorders , October 2002.

[33] Tjitske Heida, Member, IEEE, Jeroen P. P. van Vugt, Jan A. G. Geelen, and Peter H.

Veltink, Senior Member, IEEE Daphne G. M. Zwartjes*, "Ambulatory Monitoring of

Activities and Motor Symptoms in Parkinson’s Disease," IEEE TRANSACTIONS ON

BIOMEDICAL ENGINEERING, vol. 57, NOVEMBER 2010.

[34] Member, IEEE, Patrick E. McSharry, Eric J. Hunter, Jennifer Spielman, and Lorraine O.

Ramig Max A. Little∗, "Suitability of Dysphonia Measurements for Telemonitoring of

Parkinson’s Disease," IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, vol. 56,

April 2009.

[35] Resul Das, "A comparison of multiple classification methods for diagnosis of Parkinson

disease," vol. 37, pp. 1568-1572, 2010.

[36] Paolo Bonato, Fatta B. Nahab, Walter Maetzler Alberto J. Espay, "Technology in

Parkinson’s Disease: Challenges and Opportunities," 2016.

[37] Jason A. Chen and Max A. Little Ken J. Kubota, "Machine Learning for Large-Scale

Wearable Sensor Data in Parkinson’s Disease: Concepts, Promises, Pitfalls, and Futures,"

2016.

[38] Kenji Mase and Kiyoshi Kogure Seon-Woo Lee, "Detection of Spatio-Temporal Gait

Parameters by Using Wearable Motion Sensors," vol. 27, 2005.

[39] J Jankovic, "Parkinson’s disease: clinical features and diagnosis," Department of

Neurology, vol. 79, pp. 368-376, 2008.

[40] Markku Kankaanpaa, Mika P. Tarvainen, Vera Novak, Pater Novak, Kun Hu, Brad Manor,

Olavi Airaksinen, and Pasi A. Karjalainen Saara M. Rissanen, "Analysis of EMG and

48

Acceleration signals for Quantifying the Effects of Deep Brain Simulation in Parkinson's

Disease," IEEE Trans Biomed Eng, September 2011.