Application on Tele-Rehab - MIPS

B. Tan and O. Tian Paper Presentation: 6 – 8 March 2014, Seoul Page 1

Using BSN for Tele-Health Application in Upper Limb Rehabilitation

Benedict Tan and Oliver Tian

[email protected] [email protected]

HutCabb Consulting Private Limited

Singapore, Singapore

Abstract - Improved upper limb rehabilitation requires careful

and re-constructed information around stroke patients’ muscle

activation characteristics and kinematic features in functional

movement. Body Sensor Networks (BSN) are deployed to

provide an immersive engagement of the rehabilitation

exercise and translation into an augmented reality world for a

higher order of analytics and consultation by medical

consultants. Results of the analysis generate contextual

intelligence to improve therapy programmes in order of an

increased magnitude with derived information on model

schemas, pattern deviation and effectiveness of diagnostics.

Keywords – body sensor networks, augmented reality, data mining

and analytics, user profiling, intelligent systems, software

engineering, computer applications, wireless networking,

embedded systems, multimedia & signal processing, pervasive

computing, personals services, cloud computing, computer control,

and automation.

I. INTRODUCTION

A. Rampant emergence of Internet-of-Things (IoT)

Internet-of-Things (IoT) is a term coined by Kevin Ashton

in 1999 during a marketing presentation made at Procter &

Gamble (P&G) in 1999 about the potential of Radio

Frequency Identification (RFID) global system in

monitoring product movements through the RFID electronic

tagging [1]. Since then, it has quickly caught on to refer to a

society of physical objects being simultaneously connected

to the internet via the same Internet Protocol (IP). This

therefore allows previously disparate devices to be

connected by the individual via the internet.

IoT as a mechanism can be further perpetuated into two

distinct types of communication: thing-to-person and thing-

to-thing communication [2]. This has been made possible by

the diffusion, as well as convergence, of innovations such as

mobile devices, WiFi, Cloud Computing, data analytics,

software applications, and sensor technology. IoT lends a

major role in the realization of the concept known as

’Ubiquitous Computing’, which was founded in late 1980s

by Mark Weiser, together with the birth of the internet [3].

IoT is hence applicable to many currently under-served

industries.

B. IoT in Healthcare

In essence, IoT represents the world of connected devices

which uses data collection and communication technologies

to digital content and context-aware services [4].

This trend brought about the paradigm change which

resulted in a widespread diffusion of information through

computing. With the development of pervasive services, the

invention and widespread proliferation of technologies and

applications in seamless and lower costs of communications

further strengthened at least three domains of pervasive

computing: home networking, automobile network

solutions, and mobile E-business. By the use of such

technology to bring together various devices that were

previously independent of each other, this has became an

extension of the concept of pervasive computing, from

which the basis for the Internet-of-Things was developed.

Today, Internet-of-Things has progressed as a novel

paradigm which bridges the gap between the worlds of the

virtual internet and the reality of objects, by integrating the

functions of “things” in the real world with the virtual world

through software applications [5].

As information systems are the foundation of new

productivity sources, IoT based healthcare systems play a

critical role and have significant contributions in growth of

medical information systems. However, to take advantage of

IoT, it is essential that medical enterprises and community

should embrace such converging technologies in terms of

performance, security, privacy, reliability and return-on-

investment. Tracking, tracing and monitoring of patients and

medical objects are very essential and are challenging

research directions in applying IoT, hence making the

essential role of IoT in healthcare systems dissimilar among

different healthcare components. Hence, the participation of

IoT between useful research and present realistic

applications warrants attention.

In this paper, we discuss the application of remote

rehabilitation services over the cloud infrastructure for post-

stroke patients using technologies for sensor data collection

and analytics, wireless communications, interactive digital

media as well as contextual profiling.


II. MOTIVATION

Critical Need in Rehabilitation

According to World Health Organization (WHO), 15

million people suffer stroke worldwide each year [6]. Of

these, five million die and another five million are

permanently disabled. Within China alone, there are more

than a million stroke cases yearly with a current estimate of

more than seven million stroke patients. Approximately

67%, if any, of survivors become functionally dependent

and a further 10% require long-term institutional care, thus

imposing a great burden on the family and community.

Stroke leads to movement disability and high morbidity.

Recovery mainly depends on rehabilitation. The prognosis

for upper limb recovery following stroke is poor, a

systematic review [7] concluded that complete motor

recovery of the upper extremities occurs in less than 15% of

patients with initial paralysis.

Rehabilitation is a critical enabler that helps stroke survivors

maximize their quality of life physically, cognitively,

emotionally and socially. Recovery from stroke is a long

process that can continue over several years. Most of the

recovery occurs in the first 2-3 years, and especially the first

6 months. Rehabilitation needs to continue in hospitals, at

rehabilitation centers, in home and residential care.

However, due to limited resources (hospitals facilities,

healthcare specialists and appropriate equipments) full

recovery rate can be relatively low [8].

• Approximately one third of stroke patients recovers

their lost functions fully or almost fully, and get back

to their pre-stroke activities within a year.

• About 50% of stroke survivors who are under the age

of 65 may return to work.

• However at one-year anniversary after a stroke, about

two third of stroke survivors will have some level of

disability, ranging from light and moderate to very

severe.

Existing post-stroke rehabilitation relies on specialists’

manual examination and personal judgment, and the training

activity is performed under the specialist’s supervision.

There is still, very much, a lack of qualified rehabilitation

specialists. Rehabilitation training is a long process,

patients and families prefer to be at home or a community

place of convenience; rehabilitation training is painful,

many patients do not have strength to fully cooperate. As

such, there is a dire need for the next generation post-stroke

rehabilitation system, which is ubiquitous, intelligent,

motivating and immersive [9] [10].

III. EMBRACING BSN TECHNOLOGY TO ACCELERATE

STROKE RECOVERY

A. Research Goals

The objectives of the project is to develop the next

generation rehabilitation system, which needs to be

Immersive, Impactful, Informative and Intelligent, for

post-stroke patients by using technologies embracing Body

Sensor Networks (BSN) and Interactive Digital Media

(IDM), to efficiently capture human body motion patterns

and re-construct into a 3D augmented reality world, coupled

with immersive and interactive gaming technologies. The

research goals are as follows:

1. Produce real-time capture the patient motion and

reconstruct it in a 3D virtual training space;

2. Evaluate, quantitatively, the patient function

status and rehabilitation training progress based

on medical knowledge and normal cases,

providing the basis of system intelligence;

3. Visualize the training process in 3D virtual space,

visually guiding the patient in the training,

highlighting the progress and existing issues;

4. Design the system using immersive gaming

philosophy, so as to motivate the patient, and

create an enjoyable rehabilitation training session;

and

5. Provide rehabilitation services ubiquitously, and

interface communication between doctors,

specialists, patients and family members, so that

rehabilitation training can be at ease of the home

and/or community setting, reducing the cost,

bringing convenience to patients and families.

B. Principles of Design Consideration

There are two major categories of design considerations –

User Interactivity and Contextual Intelligence.

User Interactivity

Accuracy: As a product for healthcare, the system must

show high accuracy and reliability in data collection and

data processing in order to produce useful medical

information. For the same reason, sensors should be

sampled at high frequencies to correctly get the

phenomenon being monitored.

Durability: The portable and mobile kit lends itself to

classify as an everyday appliance which must not be

burdensome especially for elderly or impaired patients.


Wear-ability: Rehabilitation must be carried by patient

during exercising. Hence, data collection sensor nodes must

have an attachment to ensure a steady fixation through the

activity routine, accommodating rude movements as well as

interferences at network layers to provide reliable

communications.

Comfort: During physical activity, the comfort factor

becomes an important design consideration to provide the

highest degree of convenience - unobtrusive devices and

small form factor are high on the design scale.

Safety: The product must be safe and easy to use for non-

specialist. Considering the criticality of the application,

the proposed solution should be context aware to in view

of the patient environment and physiological state.

User Interaction: The product must be engaging and

captivating to encourage the patient to “train hard” while

exploring new activities.

Context Intelligence

Context is the background in which an event takes place,

which involves any set of circumstances surrounding an

event. In training, knowing the specific context of an event

is imperative to training effectiveness so that the social

process attains a higher level of retention [11]

“Contextual Intelligence is the ability to quickly and

intuitively recognize and diagnose the dynamic contextual

variables inherent in an event or circumstance and results

in intentional adjustment of behavior in order to exert

appropriate influence in that context “ [12]

Identifying the factors and variables that constitute

contextual ethos becomes an important aspect of the ability

to diagnose. In this research, we explored the following

elements:

i) Physical Context – understanding the physical

attributes of people in the environment such as

time, and location etc.

ii) Social Context – establishing and leveraging

on the relationship and roles of the people and

objects such as ratings, reviews, and social

attention etc.

iii) Behavioral Context – monitoring patterns over

time, including interactions with devices and

services such as recurrence and actions etc.

iv) Content Context – extracting and extrapolating

specific contents from public domains and

practical daily lifestyle examples.

IV. DESCRIPTION OF SYSTEM

A. Functionalities

In the proposed application, sensor units attached to trunk,

upper arm, lower arm and hand, respectively, to capture the

movements of the upper limb. The 3D reconstruction is

shown on the display screen in front, with an avatar to

support the patient in the therapy exercises. The

performance is evaluated based on the trajectory difference

with the normal person. The amount and types of trajectory

diversion reflect various problems, and requiring different

training scheme and efforts.

Figure 1 Upper Limb Motion capture and 3D reconstruction

The patient is first assessed for functional / physical status,

by using the assessment module. The assessment module

establishes a professional assessment measurement to

determine suitable rehabilitation scheme, or training session

module for training session to support continuous

rehabilitation scheme to allow the patient to start therapy

sessions. During training, the assessment module

continuously evaluate patient’s progress by comparing the

patient’s movement with the norm (as ascertained by the

healthcare specialists). The distance measured is then used

to visualize the 3D virtual training space. The 3D virtual

space shall provide patient with an enjoyable 3D game

scenario to get the patient immersive into the training. The

patient, family members and specialists can view the

training process in various forms of game scenario.

As the training progresses, the current rehabilitation schema

may be completed or upgraded. Good examples can be

stored in the schema base for future reference. In that case,

index is created for that scheme to make it accessible.


Figure 2 Block diagram of the application

B. Findings and Achievements of the Research

There are several key issues which were researched and

acceptable results achieved, and kernel functional modules

to be developed into prototype for assessment purposes:

1. Use of Micro-sensor for capture of motion data for

upper limb movements and reconstruction of 3D

motion hardware and software system to establish

and validate algorithms suitable for real time 3D

reconstruction. This system has been customized to

rehabilitation needs.

The key research studied the biomechanical model of

the upper limb based on current understanding of the

stroke damage and motor brain reorganization so that

movement patterns of stroke patients can be better

analyzed and facilitated.

2. In Functional Assessment module, which has two

functions - one to assess functional scale of patient

respective needs and the other a real-time evaluation

of upper limb motion capabilities – the patient is

navigated through a routine according to the

assessment and rehabilitation scheme selected. The

motion capabilities of the patient, which concerns

performance quality and rehabilitation progress, are

also monitored during the training session.

In this research aspect, medical experts have been

consulted to understand, represent and implement the

existing assessment measurement in a digital way.

The research draws an extension and enhancement of

existing assessment measurement, with care taken to

study the principles of rehabilitation, such as the

“dependent functional reorganization” and “motor

relearning theory”.

When developing distance measures to evaluate

performance of rehabilitation training, it is not

enough to compare normal trajectory, but the quality

of trajectory, factors such as smoothness, time taken,

are amongst important considerations. While

working with medical experts, the distance

measurement has been tested with continual

improvements and fine-tuning during its usage.

3. In Rehabilitation Training Management module,

sessions are created for the patient based on status of

assessment results, and previous training records.

During training, rehabilitation scheme parameters,

movement quality measures and training progress

will be recorded, and supervised.

C. Issues and Challenges

The various technologies have been successfully applied

and converged into a seamless hybrid to demonstrate the

viability of the application. The prototype is undergoing

independent practice trials and validation. From the early

results, we were able to distill out common characteristics

which define a robust solution. Unlike conventional

methods, we encountered several issues and challenges and

are taking steps to resolve them [12].

1) Physical mechanics The wearable physical form factor needed to be small,

light-weighted and non-obtrusive. The size and weight of

sensors are predominantly determined by the battery

factor. A careful trade-off between communication and

computation is crucial for an optimal design.

2) Location of sensors For purpose of accurate measurement, sensor location

may be subjective to the physical built of the patient user.

Sensor attachment is also a critical factor, since the

movement of loosely attached sensors creates spurious

oscillations which may be disruptive.

3) Applicable algorithms Application-specific algorithms mostly use digital signal

pre-processing combined with a variety of artificial

intelligence techniques to model user's states and activity

in each activity. Most of the algorithms in the open

literature are not executed in real-time, or require

powerful computing platforms such as laptops for real-

time analysis. Furthermore, there is to singly accepted

protocol for assessing rehabilitation recovery status.

4) Social phsychology Social issues of wearable systems include privacy/security

and legal issues. Due to communication of health-related

information between sensors and servers, all

communications over network should be encrypted to

protect user's privacy. In addition, deriving benefits from

medical automation is also a new-found hurdle.


D. Current Results

In China, the associate project team recruited trial subjects

for clinical trials in renowned hospitals including Beijing

University 1st Hospital, Shanghai Huashan Hospital, and

Nanjing University of Medicine.

The subjects are divided into two groups - post-stroke

patients and healthy subjects who have matching age and

sex profiles to the first group. All of them were tested

through rigorous inspection processes by neurologists. The

steps of such trials include:

i. Basic Data Capture and Recording: Each

subject’s basic data were recorded including: 1)

age, sex, educational level, smoking history,

alcohol history, body weight, history of

hypertension and coronary heart disease; 2)

motion data measured by the system.

ii. Motion Capture and Analysis: The healthy

subjects’ motion of the upper limbs were

captured and analyzed to obtain normal range of

doing some specific movements.

iii. Status Evaluation: Evaluate post-stroke patients’

status using distance measurement between the

patient’s movement/function and one performed

by normal person, and the existing professional

assessment measures, such as Fugl Meyer

assessment of physical performance, and hand-

path ratio parameters.

iv. Personalized Rehabilitation Training: Post-stroke

patients would perform rehabilitation training at

home/community based on personalized training

schemes and their status provided by the system;

the results would be assessed by medical

specialists remotely. Upper limb motion

capabilities of the patients were also evaluated,

which concerns the performance quality and the

rehabilitation progress during the training

scheme.

In Singapore, the project team had engaged a second level

experimentation with selected nursing homes to establish

the viability of the product for beta testing. We had sourced

and identified participating institutions and developed a

similar, yet more practical oriented approach to the Chinese

experience.

Healthy subjects from selected institutions and post-stroke

patients from identified medical institute participated in the

trials and provided valuable feedback. With more valuable

feedback, the product was further fine-tuned and improved.

Additional steps taken included:

i. Interactive Gaming Components: Participants

were streamed through sessions with a more

enriched and light-hearted user interface in the

product, with gaming scenarios reflecting more

practical life home routines.

ii. Analysis and Update on Rehabilitation Plans:

The system should be able to provide an

acceptable level of analysis to update the

roadmap laid out for the rehabilitation progress.

V. POTENTIAL USE CASES

Several Use Cases were detailed with a high potential of deployment – each paving the way for higher productivity gains and deriving a multiplier effect on the recovery rate. In the following section, two such Use Cases are discussed briefly.

Hospital

Internet

Cloud

Servers

Database

IMURSServer

Doctor

travelling

Home

Patient

Internet

WLAN

Office

Family

Figure 3 Use Case – Remote Home Care

The above Use Case depicts the scenario where the patient

is undergoing therapy in the convenience of the home,

possibly with the assistance of a home caregiver. The

patient’s case is being monitored closely by a travelling

medical consultant, who may be in transit between medical

centers. The exercise results will be immediately available

for review by the medical team in the hospital as well as the

travelling consultants. Any feedback can be further relayed

to the family members via mobile devices while they are

working in a remote office.

Such a Use Case can also be extended to create a multiplier

effect to increase the number of touch-points for doctor-

patient ratio, hence increasing the productivity of the

medical consultation. More importantly, more patients are

expected to receive the attention which has been lacking

until now.


Figure 4 Use Case – Step-down Remote Care

The above Use Case depicts the scenario where the patient

is undergoing therapy in the step-down facility of a General

Practitioner (GP) Clinic supported by a nursing home. The

patient’s case is being monitored by a GP doctor in the local

clinic, with relevant support from the medical specialist.

The GP doctor can provide feedback on the patient’s

progress to the nursing home. The exercise results will be

immediately available for review by the therapists in the

supporting nursing centre.

This Use Case can be extended to patients receiving therapy

in any of the affiliated GP Clinics where it is most

convenient for the patient and at more convenient times of

the day. In this way, the patient can continue his/her therapy

with a much better recovery rate. The therapist can continue

to support the patient with much more frequency and touch

points after the patient has discharged from the nursing

home.

VI. CONCLUSION

We have demonstrated that a selection of converging and powerful technologies can form a viable infrastructure to support effective remote consultation in extended rehabilitation. This solution supports a dire need in current step-down care and such a tele-rehabilitation solution has the potential to advance current medical practice many folds. We expect to use this solution to increase the recovery rate of post=stroke patients. This is a practical case study which is expected to derive significant benefits in the healthcare industry, is the result of applying technologies such as sensor data collection and analytics, wireless communications, interactive digital media as well as contextual profiling – a exemplary manifestation of IoT in action.

ACKNOWLEDGMENT

This project and preparation of this paper is derived from efforts from a symbiotic relationship between research teams from two countries – China (GUCAS) and Singapore (HutCabb). In consultation with years of research on both shores and the efforts to converge powerful technologies in light of the imminent IoT era, the authors would like to thank all parties who have contributed in one form or another to the preparation of this paper.

The Chinese Academy of Sciences (CAS), formerly known as

Academia Sinica, is the National Academy for the Natural

Sciences of the People's Republic of China. It is an

institution of the State Council of China. It is headquartered

in Beijing, with institutes all over the People's Republic of

China. It has also created hundreds of commercial

enterprises, Lenovo being one of them. Graduate University

of Chinese Academy of Sciences (GUCAS) was founded in

1978; GUCAS is the first graduate school in China with the

ratification of the State Council. GUCAS boasts a galaxy of

pioneering scientists with lots of achievements. The

research team under Professor J Wu put in tremendous

efforts to validate the applied technology.

REFERENCES

[1] K. Ashton. (2009). That 'Internet of Things' Thing.

http://www.rfidjournal.com/articles/view?4986

[2] N. Dlodlo, T. Foko, P. Mvelase, and S. Mathaba, (2012) “The State of Affairs in Internet of Things Research” The Electronic Journal Information Systems Evaluation Volume 15 Issue 3 2012, (244- 258), available online at www.ejise.com

[3] M. Weiser and R. Gold, (1999).The origins of ubiquitous computing research at PARC in The late 1980s. IBM Systems Journal.

[4] International Telecommunication Union, (2006). ITU Internet Reports 2006: digital.life, Geneva, Switzerland, December 2006.

[5] Y. Wang and X. Zhang, (2012). Internet of Things.

[6] World Health Report – 2002, from the World Health Organization..

[7] P. Langhorne , et al. (2009) Motor recovery after stroke: a systematic review. Lancet Neurol. 2009; 8(8):741-54

[8] World Report on Disability, Chapter 4 Pg 103-108,, from the World Health Organization..

[9] Z Huang and J. Wu, et al. (2013) Upper Limb Function Analysis of Stroke Patients by Fusion of Surface EMG and Motion Data. RehabTech Conference, Poster Session, (27 Feb – 1 Mar 2013), Singapore.

[10] Z Huang and J. Wu, et al. (2011) Motor Impairment Evaluation for Upper Limb in Stroke Patients Based on Micro-senor. The Journal for Advanced Nursing Practice, July/August 2010 Vol 24 No 4, pp 196 – 201, 2010.

[11] J. Lave and E. Wenger. (1991). Situated learning: Legitimate peripheral Participation: Cambridge University Press.

[12] A. Hadjidja, M. Souila, et al. (2013) Wireless Sensor Networks for Rehabilitation Applications: Challenges and Opportunities "Journal of Network and Computer Applications 36 (2013) 1-15" DOI: 10.1016/j.j nca.2012.10.002

Technology

Application on Tele-Rehab - MIPS