17
Applied Bionics and Biomechanics 9 (2012) 157–172 DOI 10.3233/ABB-2011-0047 IOS Press 157 Quantifying upper-arm rehabilitation metrics for children through interaction with a humanoid robot Douglas A. Brooks and Ayanna M. Howard Georgia Institute of Technology, Atlanta, GA, USA Abstract. The objective of this research effort is to integrate therapy instruction with child-robot play interaction in order to better assess upper-arm rehabilitation. Using computer vision techniques such as Motion History Imaging (MHI), edge detection, and Random Sample Consensus (RANSAC), movements can be quantified through robot observation. In addition, incorporating prior knowledge regarding exercise data, physical therapeutic metrics, and novel approaches, a mapping to therapist instructions can be created allowing robotic feedback and intelligent interaction. The results are compared with ground truth data retrieved via the Trimble 5606 Robotic Total Station and visual experts for the purpose of assessing the efficiency of this approach. We performed a series of upper-arm exercises with two male subjects, which were captured via a simple webcam. The specific exercises involved adduction and abduction and lateral and medial movements. The analysis shows that our algorithmic results compare closely to the results obtain from the ground truth data, with an average algorithmic error is less than 9% for the range of motion and less than 8% for the peak angular velocity of each subject. Keywords: Rehabilitation, humanoid, motion history imaging, dynamic time warping, random sample consensus 1. Introduction In the United States, the Individuals with Disabil- ities Education Act (IDEA) states that children with a physical disability are entitled to a free public edu- cation that emphasizes special education and related services designed to meet their unique needs and pre- pare them for further education, employment, and independent living. Unfortunately, access to neces- sary assistive technology remains unequal and persons Corresponding author: Douglas A. Brooks, Georgia Institute of Technology, 85 Fifth Street NW, Atlanta, GA 30308, USA. Tel.: +1 404 385 4066/404 385 4824; E-mail: [email protected]. with severe or multiple physical disabilities are largely overlooked. However, recent successes in commercial robots appear to foreshadow an explosion of promising robotic applications for individuals with disabilities. Mechatronic and robotic systems for neurorehabilita- tion can be generally used to record information about the motor performance (position, trajectory, interaction force/impedance) during active movements [11, 27, 39,]. Sensory-motor rehabilitation techniques based on the use of robotic and mechatronic devices have been applied with stroke patients [6, 23, 26, 28, 30, 31, 34, 37, 41]. Furthermore, being able to objectively assess the performance of a patient through repeatable and quantifiable metrics has shown to be an effective means 1176-2322/12/$27.50 © 2012 – IOS Press and the authors. All rights reserved

Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

Applied Bionics and Biomechanics 9 (2012) 157–172DOI 10.3233/ABB-2011-0047IOS Press

157

Quantifying upper-arm rehabilitation metricsfor children through interactionwith a humanoid robot

Douglas A. Brooks∗ and Ayanna M. HowardGeorgia Institute of Technology, Atlanta, GA, USA

Abstract. The objective of this research effort is to integrate therapy instruction with child-robot play interaction in order tobetter assess upper-arm rehabilitation. Using computer vision techniques such as Motion History Imaging (MHI), edge detection,and Random Sample Consensus (RANSAC), movements can be quantified through robot observation. In addition, incorporatingprior knowledge regarding exercise data, physical therapeutic metrics, and novel approaches, a mapping to therapist instructionscan be created allowing robotic feedback and intelligent interaction. The results are compared with ground truth data retrievedvia the Trimble 5606 Robotic Total Station and visual experts for the purpose of assessing the efficiency of this approach. Weperformed a series of upper-arm exercises with two male subjects, which were captured via a simple webcam. The specificexercises involved adduction and abduction and lateral and medial movements. The analysis shows that our algorithmic resultscompare closely to the results obtain from the ground truth data, with an average algorithmic error is less than 9% for the rangeof motion and less than 8% for the peak angular velocity of each subject.

Keywords: Rehabilitation, humanoid, motion history imaging, dynamic time warping, random sample consensus

1. Introduction

In the United States, the Individuals with Disabil-ities Education Act (IDEA) states that children witha physical disability are entitled to a free public edu-cation that emphasizes special education and relatedservices designed to meet their unique needs and pre-pare them for further education, employment, andindependent living. Unfortunately, access to neces-sary assistive technology remains unequal and persons

∗Corresponding author: Douglas A. Brooks, Georgia Institute ofTechnology, 85 Fifth Street NW, Atlanta, GA 30308, USA. Tel.: +1404 385 4066/404 385 4824; E-mail: [email protected].

with severe or multiple physical disabilities are largelyoverlooked. However, recent successes in commercialrobots appear to foreshadow an explosion of promisingrobotic applications for individuals with disabilities.Mechatronic and robotic systems for neurorehabilita-tion can be generally used to record information aboutthe motor performance (position, trajectory, interactionforce/impedance) during active movements [11, 27,39,]. Sensory-motor rehabilitation techniques based onthe use of robotic and mechatronic devices have beenapplied with stroke patients [6, 23, 26, 28, 30, 31, 34,37, 41]. Furthermore, being able to objectively assessthe performance of a patient through repeatable andquantifiable metrics has shown to be an effective means

1176-2322/12/$27.50 © 2012 – IOS Press and the authors. All rights reserved

Page 2: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics

for rehabilitation therapy [2, 9, 16, 23, 24, 41]. Themajor barrier is that, to date, most assistive roboticdevices are not designed for children.

Studies in child cognitive behavior indicate thatwith repetitive and monotonous conditions over time,performance decrement occurs due to a reduction inarousal [12]. This, in-turn, leads to a decrease in sus-tained attention for the current activity and overallmotivation for continued repetitions of the exercises.Given that physical rehabilitation sessions consists ofmany repetitive and monotonous actions and sustainedattention improves with age [12], it is important thatresearchers incorporate various methods for motivat-ing children during this process.

There are several reasons that may be fueling thelack of necessary technological devices geared towardsaiding children, both with and without disabilities. Ingeneral, researchers must first and foremost obtain theapproval of their Institutional Review Board (IRB)before conducting any studies involving humans. Forthose who wish to conduct studies involving children,researchers must additionally obtain parental consent.This alone may cause a hindrance for collecting datafrom children that could ultimately lead to ground-breaking technology for this particular group. Whenconsidering children with disabilities, parents are typ-ically even more protective and more reluctant toallow their child to participate in such studies. Fur-thermore, because disabilities vary from one individualto the next, it becomes quite challenging to designan assistive aid that is capable of providing assis-tance that is unique to each person. For instance, evenwithin the realm of cerebral palsy there are varioustypes and levels. One may have an issue of spas-ticity, where the muscles are very stiff and causesabnormal or jerky movement. However, a separateindividual may have full motor functionality, but hasan issue with speech. These variations cause a uniquechallenge for deploying robotics for this target demo-graphic.

To overcome this barrier, state-of-the-art techniquesmust be created to facilitate the interaction necessaryto be useful for therapeutic rehabilitation with respectto children. Utilizing the logical fact that animatetoys naturally engage children, this research focuseson the design of a robotic therapeutic playmate thatwill aid children in physical rehabilitation by fus-ing play and rehabilitation techniques that are bothentertaining for the child and effective for upper-armrehabilitation. The robotic playmate, consistent with

prior theory from physical therapy metrics, will incor-porate visual queues that will determine the amount ofmovement that a child makes and his or her consis-tency with respect to the movement of the robot. Ofparticular importance within this proposed work areapproaches that allow therapists to assess the infor-mation obtained by the robot during play scenarios,thus requiring that the data is conveniently decipher-able.

In this article, several successful trials of employingthis novel approach to physical therapy scenarios willbe illustrated. Section 2 gives a brief overview of thecurrent state of the field of therapeutic robots. Section3 provides an overview of the current methods thatare used by physical therapists to administer upper-arm rehabilitation sessions. Section 4 gives a detailedaccount of the methodology used in this research toachieve the aforementioned goals of attaining appro-priate physical therapy metrics via image processing.Sections 5 and 6 are a presentation of the results and theadvantages and disadvantages of this approach, respec-tively. Finally, Section 7 concludes the article with adiscussion of the future direction of this research.

2. Therapeutic robotics

The use of robotics to aid in therapy is an on-going area of research. Scientists continue to designnew methods for administering aid to those in need ofcognitive and physical development. Even though norobot can replicate the knowledge and experience of atherapist in assessing the needs of the patient and theoutcomes of the therapeutic program, robotic technol-ogy can be used to carry out the repetitive practice thatis needed to facilitate functional gains [29]. Whetherusing these new methods to assist with developingcommunication skills or aiding in patient motor devel-opment, researchers have shown that the use of robotscan greatly improve patient therapy sessions.

2.1. Keepon

One robotic platform that has been developed fortherapy and play is Keepon. Introduced by researchersand physicians in Japan, Keepon was designed to getautistic and non-autistic children (ages 2–4) involvedin playful interaction that was generally initiated anddirected by the child (i.e., without any experimentalsetting or instruction) [25]. A point of emphasis for

Page 3: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics 159

the Keepon design was to keep it as simple as pos-sible in order to abstain from overwhelming and/orfrightening the children, a notion that all researchersworking with therapeutic robots should keep in mind.Therefore, the simple structure and function only con-tained two color CCD cameras as eyes, a microphoneas a nose, a small gimbal and four wires, by which thebody is manipulated like a marionette, four motors andtwo circuit boards (an SH2-based PID controller and amotor driver). Since the body is made of silicone rubberand its inside is relatively hollow, Keepons head andbelly deform whenever it changes posture and whenpeople touch it [25].

The resulting information, obtained from a year anda half of data collection, indicated that Keepons simpleappearance and predictable responses gave the autis-tic children a playful and relaxed mood, in which theyspontaneously engaged in dyadic play with Keepon.This in turn expanded into interpersonal communica-tion where Keepon worked as the pivot of triadic playwith adults or other children [25].

2.2. CosmoBot

Another robotic system that has been designed fortherapy, education, and play is the CosmoBotTM. Chil-dren interact with the CosmoBotTM, controlling therobots movements and audio output, using a variety ofgestural sensors and speech recognition, while activelytargeting their therapy goals [3]. The system consists ofthe physical robot, an interface board, interface soft-ware, and plug-in gestural interfaces (joystick, headsensor, and arm sensor).

Researchers recruited six children who were alreadyreceiving outpatient physical and occupational therapyservices with a physical therapist. More specifically,the subjects ranged in age from 4−10 years old,and each subject had a diagnosis of cerebral palsyand received physical and/or occupational therapyfor treatment of upper extremity motor deficits. Thephysical therapist provided feedback regarding thesystem, noting that it was easy to use and that itprovided a motivating factor for client participation.Furthermore, researchers reported improvements inthe physical abilities of patients, reporting patientnovelties such as multi-tasking, improved attentionspan, improved upper extremity strength and coordi-nation, and an improvement in daily activities of living[3].

2.3. Moving forward

These humanoids provide a brief overview of state-of-the-art robotic platforms used in therapy. Whilethese robotic systems are among the first to make sig-nificant progress toward aiding individuals with severedisabilities, there is still much that needs to be solved.First, neither of these designs give a physical assess-ment of the individuals’ motor skills. Even thoughchildren with disabilities may never have full usageof certain body parts, significant improvements can bemade in hopes that these individuals will be able to leadhealthy, self-contained lifestyles. Also, of the systemsthat do incorporate a physical assessment of the indi-vidual, only few, if any, are fully autonomous with acompletely on-board system; each has to connect to acomputer which incorporates its intelligence system oris teleoperated.

As an example, MIME or Mirror-Image MotionEnabler [6], was designed to aid in the physical reha-bilitation of stroke victims. This platform functionsby strapping the subject into a wheel-chair that hasmobile arm supports, which limit horizontal move-ment, and a robotic arm that applies forces andtorques to the arm through those supports. The desireis to monitor patient recovery through parametersobtained via force-feedback. Another robotic platformwith the capability of providing an assessment of apatient’s physical attributes was introduced by Gock-ley and Mataric in 2006. The researchers in this projectattached a laser range finder to a Pioneer-2DX roboticplatform that was used to track and locate a patientwearing a reflector on his or her leg. The researchersactually used the system as a way of encouraging thepatient to continue the rehabilitation session throughpositive reinforcement, which was provided via a seriesof beeps and movements based on the patient’s move-ments. As a control variable, they varied the robot’slevel of engagement (or amount of movement) and theyreported the patients’ level of engagement and comfortwith the robot [21].

With the introduction of these robotic systems, wehave research showing approved attention span and,separately, contact assistive robotics. By bridging thegap between these two research areas, we can design asystem that allows a patient to move freely while keep-ing them engaged and motivated during a continuousrehabilitation session. Furthermore, since the progres-sion of robotics has been to achieve full autonomy,it is ideal to produce systems with the capabilities of

Page 4: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

160 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics

making intelligent decisions and performing actionswithout the aid of a separate system, be it human orcomputer. This work is an attempt to alleviate theseissues.

3. Current physical therapy metrics

Until recently, many of the current physical therapymetrics were not designed with the intent to admin-ister such practices to children with disabilities. Forinstance, the Fugl-Meyer Test [20], an assessment thatconsists of 155 items and is used to determine theimpairment in the upper and lower extremities, wasdesigned specifically for stroke victims. Utilizing ascoring scale of 226, physical therapists administer aseries of tests, which include reflex activity, balance,sensation, position sense, and range of motion (ROM)for evaluating a patient’s degree of impairment [15].However, this scoring scale is based upon the move-ment expectations of a healthy adult subject.

The lack of a legitimate metric of child motordevelopment lead therapists to design scales specif-ically for assessing the aforementioned group. TheGross Motor Function Measure (GMFM) [38] and thePeabody Developmental Motor Scales (PDMS) [8] arethe two well-known motor instruments for children,specifically those diagnosed with cerebral palsy (CP).However, the GMFM measures the gross motor (GM)domain only [8]. For measurement of the fine motor(FM) domain, the GMFM is inadequate as an eval-uative tool [42]. The opposite is true of the originalPDMS.

During the late 1990’s, with the culmination in 2000,the PDMS was revised to the Peabody Developmen-tal Motor Scales-Second Edition (PDMS-2), with newnorms, revised testing materials, more precise scoringcriteria, and more information on norm samples [19].Scores include 1) a Gross Motor Quotient which isa composite of the Reflexes, Stationary, Locomotion,and Object Manipulation subtests, 2) a Fine MotorQuotient, a composite of the Grasping and Visual-Motor Integration subtests, and 3) a Total Quotient,a combination of the gross and motor subtests [19].The new normative data on the PDMS-2 was collectedon 2,003 children both with and without disabilities.There are also more reliability and validity data forthe PDMS-2 than for the PDMS [19]. Therefore, thePDMS-2 is potentially appropriate for investigatingthe progress of the gross and fine motor domains for

children with and without disabilities because itassesses both GM and FM composites and incorporatesboth quantitative and qualitative rating criteria [42].

Since the research presented in this work focuses onnon-contact, upper-arm rehabilitation, specifically forthe shoulder joint, we present the successful analysis ofa typical metric used by physical therapist, ROM. As anextension of ROM, we also quantify the patients’ peakangular velocity (PAV) for the purpose of providinga more accurate quantitative analysis. These types ofmeasurements can coincide with a child’s ability tocatch and throw objects as well as his or her reaction toenvironmental events; each of those items are withinthe object manipulation and reflexes subtests of thePDMS-2. Simultaneously, we desire to provide a newmethod for obtaining this data, namely a way to keepthe child subjects motivated during the process.

4. Approach

In order to achieve the goal of obtaining physicaltherapy metrics by using computer vision as the onlymeans of collecting data, several image processingtechniques are employed. First, the video sequence ispartitioned into separate gray-scale images containinginformation regarding human movement via a processtermed Motion History Imaging. Then, the incomingdata is aligned with the robot’s notion of its own motion(i.e., a set of images representing the robot’s move-ments). Next, a contour extraction process is appliedto each of the remaining images, which creates an idealrepresentation of the two movements. Finally, a Ran-dom Sample Consensus is applied to the ideal contours,which enables the determination of straight line seg-ments ultimately allowing the calculation of the ROM;adding the known factor of the number of frames persecond (fps), the PAV is also calculated.

4.1. Motion history imaging

4.1.1. BackgroundThe initial step in determining the consistency

between robotic movements and child movements isto segment a video sequence into individual imagesthat contain pertinent information as they relate to theoverall representation of recent movement [4]. Onecommon technique for attaining the three-dimensionalinformation from a particular movement is to recoverthe pose of the person at each time instant using a

Page 5: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics 161

three-dimensional model [1]. This generally requiresa strong segmentation of foreground/background andalso of individual body parts to aid the model align-ment process. Furthermore, background uniformityis typically a necessity when processing images formotion estimation [22, 36]. However, in this work, itis desirable to enable child-robot interaction and datacollection immediately rather than require the child towait during an initiation process.

Since the purpose of this work is to analyze themovement of specific body parts, similar to Camp-bell and Bobick [7], the algorithmic approach is touse temporal templates. While some algorithms uti-lize sequences of static configurations, which requirerecognition and segmentation of the person [35], here,a Motion History Image (MHI) to represent howmotion in the image is moving is specifically formed.This essentially allows real-time processing of theinput data.

4.1.2. MethodologyIn a MHI, pixel intensity, Hτ , is a function of the

temporal history of motion at that point in physicalspace [1]. Similar to Bobick and Davis, a replacementand decay operator is used, as shown in Eq. (1), toobtain the MHIs:

Hτ (x , y, t) ={

τ if D(x,y,t)=1

max(0,Hτ (x , y, t − 1 ) − 1) otherwise(1)

where D is a binary image sequence indicating regionsof motion, x and y are the horizontal and vertical direc-tions in the image, respectively, t is the current timestep, and τ is the current intensity value [1]. The result,as illustrated in Fig. 1c, is a scalar-valued image wheremore recently moving pixels are brighter in intensity.

Once the MHI has been determined for a specificframe, a feature vector is created for that frame bydividing the MHI into an 8×8 grid, calculating themean, µ, and standard deviation, σ, from each grid, andthen constructing an 64 length feature vector. Varioussize grids could be used, but it is ideal to adhere to thelaw of diminishing returns by utilizing a feature vectorthat will allow the most features for image recognitionwithout compromising a significant amount of process-ing time (i.e., minimize the classification error usingthe appropriate feature vector while maximizing theclassification rate of the classifier). Figure 2 and Table1 show the results of testing different size grids, whichindicate that a 64 length feature vector is ideal.

A

B

C

Fig. 1. Motion History Image of the left arm movement. (a) Subject’sstarting position. (b) Subject’s ending position. (c) MHI of sequence.

Utilizing the feature vectors of the reference imageand the input image, a normalized Manhattan distanceis calculated, for ease of use, between the two as shownin Eq. (2):

Page 6: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

162 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics

Fig. 2. Graphical representation of feature vector selection.

Table 1Resulting processing times and recognition

rates for various feature vector lengths

Feature vector Processing Recognitionlength time (s) rate(%)

9 76.59 90.4516 80.96 92.1136 83.35 92.4064 86.06 92.5981 89.58 92.59

144 93.17 92.59

d (−→x , −→y ) =n∑

i=1

|xi − yi| (2)

where x and y are the the feature vectors of the referenceand input frames and d is the Manhattan distance. How-ever, the normalized Manhattan distance only givesinformation on a per frame basis. Once the child’smovements have been effectively represented usingMHI, the next step invokes aligning that movementwith the robot’s perception of its own movement.

4.2. Dynamic time warping

Dynamic Time Warping (DTW) is a process typ-ically used in communication protocols for speechrecognition [33]. The overall idea is to align the inputspeech data with data from a reference speech signalby repeating certain portions of a set of data that donot match the current position of the opposing set. Forexample, when someone wishes to place a phone callvia voice dialing, he or she may simply say “Call, JohnDoe”. The next time that the same individual wishesto call John Doe via voice dialing, his or her voicewill undoubtedly have variations from the first timethat voice dialing was initiated. For instance, the sec-ond time, the individual may say “Call, Jooohn Doe”.Obviously, these two statements are not identical; how-

Input frames Frame 1

Frame 1

In sertion Deletion Substitution

Frame 2

Frame 2

Frame 3

Frame 3

Frame 4

Frame 4Reference frames

Fig. 3. Pictorial representation of the basic operations of DynamicTime Warping.

ever, the desire for the user to call John Doe has notchanged. Therefore, DTW would be used to correctlymatch the vocal inflections made during the secondspeech (i.e., the input signal) with those of the pre-viously stored first speech (i.e., the reference signal).Here, the three “o’s” in the input signal would simplymatch with the single “o” in the reference signal, andJohn Doe would receive a phone call.

In this work, DTW is used to align image representa-tions of robotic movements with those of the patient’smovements. Since there will be variations in segmentlengths and duration due to varying velocities betweenthe two movements, this process becomes necessaryfor deciding whether or not two movements are simi-lar. Figure 3 is a pictorial representation of the DTWprocess and is described as follows: 1.) Insertion: Thecurrent reference frame does not match the currentinput frame, based on some threshold value γ , thusthe current reference frame is repeated until a match isfound during the linear search. 2.) Deletion: The cur-rent input frame does not match the current referenceframe, thus the current input frame is repeated untila match is found. 3.) Substitution: The current refer-ence frame matches the current input frame, and thesequence progresses forward.

The Manhattan distances calculated from the fea-ture vectors obtained during the MHI process arestored in matrix form and used to determine the best

Page 7: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics 163

possible choice between the operations for a specificframe comparison. In other words, the DTW matrix ispopulated by performing the following comparison:

dtw(m, n) = min

⎧⎪⎨⎪⎩

WId(m, n) + dtw(m − 1, n)

WDd(m, n) + dtw(m, n − 1)

WSd(m, n) + dtw(m − 1, n − 1)(3)

where WI , WD, and WS are weights for performingan “insertion”, “deletion”, or “substitution”, d is theManhattan distance between each frame, and m andn are the row and column positions in the matrices,respectively. The weights are chosen such that choos-ing to perform an “insertion” or “deletion” has moreof a penalty than choosing to perform a “substitution”because the data has been somewhat skewed in choos-ing the former. It should be noted that the standard formof a DTW implementation assumes a known startingand ending point in the sequence; however, here, it isnot assumed that the starting and ending points of theexercise are known. Thus, DTW is employed in orderto determine a minimal exit point.

Utilizing this method, the least costly path (i.e.,the optimal matching sequence) is calculated and amapping between the input sequence to the reference

sequence is obtained thus minimizing the effects ofvarying velocities. In order to quantify this method forthe purpose of this work, contour color maps were gen-erated from patient exercise data. Figure 4 illustratesthe results of mapping a reference sequence providedby the first participant with a separate input sequenceprovided by the same participant. As shown in thegraph, the input sequence was significantly longer thanthe reference sequence; in essence, the participant per-formed the exercise at a faster velocity in the inputsequence as opposed to the reference sequence. Giventhat darker contours represent points of lower cost, anoptimal path would contain the maximum number ofdark contours reaching the end of each sequence; thebold black line shows the chosen path for mapping thetwo sequences in this specific scenario.

As a comparison, Fig. 5 illustrates the results of ourDTW algorithm mapping a reference sequence pro-vided by the first participant with a separate inputsequence provided by the same participant performingtwo different exercises. There are two important pointsto note from the graph. First, although the DTW algo-rithm did find an “optimal” path for the two sequences,the path contains numerous light colored contourswhich are of high values. Second, only 50 frames of

200

Contour Color Map of DTW

Reference Frames

Inpu

t Fra

mes

180

160

140

120

100

80

60

40

20

10 20 30 40 50 60 70

600

500

400

300

200

100

Fig. 4. Contour color map of DTW illustrating the optimal path for mapping an input sequence to a reference sequence for one articipant witha separate input sequence provided by the same participant.

Page 8: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

164 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics

Reference Frames

Inpu

t Fra

mes

Contour Color Map of DTW

160

600

500

400

300

200

100

140

120

100

80

60

40

20

10 20 30 40 50 60 70 80 90 100

Fig. 5. Contour color map of DTW illustrating the optimal path for mapping a reference sequence provided by the first articipant with a separateinput sequence provided by the same participant performing two different exercises.

the 160 possible frames for the input image providedcomparable feature vectors. In other words, this wouldnot be considered a match for movements.

4.3. Contour extraction

Once the humanoid and child movements have beenaligned using DTW, a contour representing the shapeof the respective movements is then extracted. The firststep in this process is to use a median filter that willremove smaller, unwanted contours in the image typ-ically caused by camera jitter or human inaccuracies(i.e., movements of body parts other than the desiredlimb). The median filter is a sliding-window spatial fil-ter that replaces the center pixel value in the windowwith the median of all the pixel values in the window,and it can be of any central symmetric shape (e.g., around disc, square, rectangle, or cross). Here, a 16 × 16square window was used. Figure 6b illustrates the useof a median filter on image data.

When the smaller contours have been removed, acanny edge detection algorithm [8] is utilized in orderto extract the edges of the contour representing theupper-arm movement, see Fig. 6c. Utilizing the edgedetected shape, a proper representation of the sequenceis then created. Again, due to camera inaccuracies therewill undoubtedly be areas in the image where actualmovement is not properly represented, even after the

initial filtering and edge detection processes. Figure 6aillustrates an inaccuracy caused by camera lag time;even though the subject moved his arm in one com-plete motion, the video sequence shows a gap betweenarm positions. Therefore, the convex hull of the edgedetected image is determined.

The convex hull can be thought of as the boundaryof a minimal convex set of points containing a givennon-empty finite set of points in a plane. The convexhull that is utilized is that of a simple polygon. By look-ing at three consecutive vertices of the polygon, duringa recursive progression around the polygon, this algo-rithm simplifies to determining whether the resultingangle between the three vertices is concave or convex.If the resulting angle is concave, then the middle pointis removed and the next (along the polygon) vertex isadded to the triple to be tested. If the angle is convex,then the each of the points in the triple is shifted by onevertex along the polygon [13]. Melkman’s Algorithmwas employed to ensure correct outcomes [32]. Thisimplementation gives a somewhat ideal outline as seenin Fig. 6d.

4.4. RANSAC

Now that a somewhat ideal outline has beenobtained, determining the best method for finding therange of motion using only the image data is needed.

Page 9: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics 165

A

B

C

D

Fig. 6. Example image processing sequence used to extract an idealcontour from the human’s movements. (a) Original image obtainedfrom the MHI process. (b) Image obtained from the Median Filter.(c) Image obtained from the Canny Edge Detection algorithm. (d)Image obtained from the convex Hull.

Utilizing the major axis as a symmetrical dissection ofthe polygon and employing a Hough Transform [14]on either the upper or lower region of the contour could

Fig. 7. Illustration of the original convex hull image.

enable a determination of upper or lower line for thepurpose of finding the angle between either line andthe major axis. The Hough Transform is a methodused in computer vision to detect simple shapes, suchas straight lines, by using the parameters of a line,y = mx + b and representing the slope and interceptin parameter space (b, m). However, after much delib-eration and testing, it was decided that since the HoughTransform merely makes estimations of the best pos-sible line to fit the upper or lower region a moreaccurate approach would be beneficial. Therefore, itwas decided to use the RANdom SAmple Consensus(RANSAC) algorithm.

RANSAC determines the best possible line fit byiteratively selecting a random subset of the originalinput data and returns points from the original inputdata that are inliers. Given a set of data points, U, thereis an unknown number of data points that are consistentwith a model with unknown parameters from parame-ter space, �. These data points are inliers, and all othersare outliers. The goal is to find model parameters, θ∗,from a parameter space, �, that maximizes a cost func-tion, JS (θ, U, �). In the standard formulation, the costfunction, JS , is the size of the support of the modelwith parameters, θ, i.e., how many data points from Uare consistent with it. Data points with error smallerthan a threshold, �, are considered to be consistentor to support the model; � is an input parameter toRANSAC. An error function, ρ (θ, x), representing adistance of a data point to a model is typically given[10, 17].

The RANSAC algorithm carries out the maximiza-tion of JS by repeatedly executing two steps: (i) ahypothesis generation step, where a hypothesis, θk,of the model parameters is computed from a subset ofpoints, Sk, selected from the input data points, U, atrandom and (ii) the verification step, where the qual-ity of the hypothesized model parameters is calculatedby utilizing a user predefined probability (confidence

Page 10: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

166 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics

40

60

80

100

120

140

16050 60 70 80 90 100 110 120 130 140

Convex Hull

Polygon Fit

Minor Semiaxis

Major Semiaxis

Fig. 8. Illustration of the Major (dashed line) and Minor (dotted line) semiaxes located on a contour obtained from human upper-arm movement.The polygon boundary used to calculate the two semiaxes is shown with square markers.

= 1 – ηo, typically set to 95%) to recover maximum ofthe cost function JS [10, 17].

The algorithm can be described mathematically asfollows: Let P be the probability that a sample of sizem is randomly selected from a set U of N data points

P(I) =

(I

m

)(

N

m

) =m−1∏j=0

I − j

N − j≤ εm, (4)

where ε is the fraction of inliers ε = I/N. The numberof inliers, I, is not known beforehand. Let χ∗

k be thelargest support of a hypothesized model found up tothe k-th sample inclusively, I∗

k = |χ∗k |. The sampling

process is terminated when the likelihood of finding abetter model (with larger support than I∗

k ) falls undera threshold, i.e., when the probability, η, of missing aset of inliers, χ+, of size |χ+| ≤I∗

k within k samplesfalls under a predefined threshold, ηo,

η = (1 − P(I∗k ))k (5)

The number of samples that has to be drawn to satisfyη≤ηo is

kηo (I∗k ) = ln(ηo)

ln(1 − P(I∗k ))

. (6)

This method is a more accurate approach than theHough Transform because it only returns points fromthe original input data (inliers) rather than creating its

own values when predicting the line segment. Also,rather than using the major axis as one of the two linesused to find the range of motion, it was determinedthat a more accurate measure would be to performRANSAC on the upper and lower regions (created bythe major axis’s dissection of the contour), thus cre-ating a measure of the highest and lowest positionsof the subject’s arm. Figures 7, 8, and 9 illustrate thisprocess.

4.5. Range of motion and angular velocity

4.5.1. Range of motionOnce the points that create the upper and lower lines

are recognized, the slopes of each are used to calculatethe angle between the two lines via simple geometry,shown in Eq. 7:

m1 = y2−y1x2−x1

m2 = y4−y3x4−x3

� = arctan( m2−m11+m2m1

)

ROM = |� 180◦π

|

(7)

where xi and yi are the coordinates of points on eachline segment, m1 and m2 are the slopes of each line,and � and ROM are the current angle in radians anddegrees, respectively. The maximum angle found overthe length of the video sequence gives the total ROMof the child’s movements.

Page 11: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics 167

75

80

85

90

95

100

105

110

115

120

125

13075 80 85 90 95 100 105 110 115 120 125

x

y

80

A

B

90

100

110

120

130

70 75 80 85 90 95 100 105 110

x

y

Fig. 9. Lower line, determined by RANSAC, used to find the ROM.(a) Lower line. (b) Upper line.

4.5.2. Peak angular velocityGiven that the frame rate of the camera used to cap-

ture the patient’s movements was 15fps, calculatingthe angular velocity of the arm was trivial. We chose

to use the initial lower line and subsequent upper linesthat were recognized via RANSAC to determine theangular velocity as it relates to each frame. Meaning,the lower line found during the first RANSAC calcu-lation over the convex hull in frame one of the videosequence was used as the initial position of the arm,while the current upper line changed as the subjectmoved his arm upward during the exercise. Using thestandard equation for angular velocity, shown in Eq. 8,the angular velocity of the patient can easily be deter-mined with the angle obtained from each pass of theROM calculation and known frame rate.

ω = dθ

dt(8)

The maximum angular velocity would be consideredthe PAV. Thus, this algorithm gives two physical ther-apeutic metrics that can be easily read by physicaltherapists for the purpose of analyzing a patient’s cur-rent status and overall progress.

5. Results and analysis

For the initial testing of the methodology of thisresearch, two subjects were utilized, both male. Thesubjects were asked to perform a series of upper-armexercises, which were captured via a simple web-cam. The specific exercises involved adduction andabduction, shown in Fig. 1, and lateral and medialmovements, shown in Fig. 10. The images were thenprocessed by our algorithm in order to obtain theROM and PAV, which then was compared to the dataobtained from separate subjects who performed visualobservation techniques (deemed expert users), and datacaptured via the Trimble 5606 Robotic Total Station.The specific task of the expert users was to locatedthe top portion of each patient’s arm in several setsof images using mouse clicks. Using that information,in congruence with a constant location for the patient’sbeginning position, it was trivial to determine the ROMand, by extension, the PAV as viewed by the expertusers. Expert users were adults between the ages of30–35. They were required to have at least five yearsof experience in image-processing research. Recruit-ment of the expert users was done by word-of-mouth.Since physical therapists typically use visual feedbackand estimation as the sole method for determining apatient’s ROM, the author’s deemed that this methodis at least equivalent to such an analysis.

Page 12: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

168 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics

A

B

Fig. 10. Lateral and medial movement demonstrated by participant.(a) Subject’s starting position. (b) Subject’s ending position.

Occasionally, physical therapists use a measurementdevice known as a geniometer, which measures theinitial and final position of a joint during an exer-cise to determine a patient’s ROM. However, theauthor’s decided to utilize a more accurate approachfor retrieving this information. The Trimble 5606 usesa time-of-flight measurement technique based on thepulse measurement principle; it measures the time fora very short transmitted pulse to travel to a targetedprism, held by the subject, and back, thus calculat-ing the position of the subject’s end-effector. Thelong-range 5600 series DR200+ total station allowsmeasurements up to 600 m (1,968 ft) to a 90% reflec-tive Kodak Gray Card and 200 m (656 ft) to an 18%reflective Kodak Gray Card. The range using a sin-gle prism is 5,500 m (18,040 ft) with an accuracy of(3 mm + 2 pixels per meter (ppm)). Coupling that withthe robotic tracker gives a range up to 1,200 m (3,937ft) with 2′′ (0.5 milligon (mgon)) [40]. (A gon is a unitof angle measurement equal to 1/400 circle, 0.01 right

angle, 0.9◦, or 54′). As such, the data obtained from therobotic station represents ground truth. Tables 2 and 3show the ROM comparison between the expert users,algorithmic data, and Trimble 5606 for each subjectper exercise. Tables 4 and 5 show the PAV comparisonbetween the aforementioned data sets.

As shown in the tables, the ROM values calculatedvia our algorithm are closely related to the ground truthdata (overall average expert user error <11% and over-all average algorithmic error <9%). Figure 11a andb show a comparison between the expert users andalgorithm, with respect to the ground truth data, of theaverages across all subjects for the ROM data. For apatient with a limited range of motion, our algorithmcould be used to identify this condition in real-time,given a known standard ROM. This will allow thesystem to monitor patient progress between sessions.

The PAV calculated via our algorithm is related tothe trend of those calculated via ground truth data(average expert user error <16% and average algorith-mic error <8%). Figure 12a and b show a comparisonbetween the expert users and algorithm, with respectto the ground truth data, of the averages across all sub-jects for the ROM data. It should be noted that becausethe Trimble 5606 is a real tracking system, and humanmotion is not ideal, instances where the patient is notmoving may not be conveyed with zero velocity in theground truth data; thus values that are approximatelyzero are categorized as non-movement.

Of particular importance are the comments madeby the expert users when attempting to determine therange of motion of the patients. Both users noted thatit was quite difficult to determine the exact location ofSubject B’s arm due to the fact that he was wearing alarge blue sweater. One advantage that our algorithmhas is that regardless of the clothing that the patient iswearing, we can still track the overall movement andobtain reasonable data.

6. Advantages and disadvantages

When considering this approach for gathering phys-ical therapeutic data, one has to be knowledgeable ofthe limitations or potential concerns. Many physicaltherapists use sensation or force-feedback as a methodof assessing the patient’s current state and deter-mining the effects of the physical therapy sessions.Since our algorithm focuses on non-contact assess-ments, it will not be possible to gather the important

Page 13: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics 169

Table 2Measurements of ROM for adduction/abduction exercise

Repetition Subject A Subject B

1 2 3 1 2 3

Expert user (◦) 90.1 65.5 39.2 86.7 57.0 45.8Algorithm (◦) 86.4 56.5 40.1 88.1 54.9 45.6Ground truth (◦) 90.3 56.5 31.1 90.0 65.0 42.1Expert user difference (◦) 0.2 9.0 8.2 3.3 7.9 3.7Algorithm difference (◦) 3.9 0.0 9.0 1.9 10.1 3.5Expert user error (%) 0.18 15.96 26.33 3.65 12.20 8.81Algorithm error (%) 4.32 0.00 29.07 2.14 15.53 8.43

Table 3Measurements of ROM for lateral/medial exercise

Subject A Subject B

Repetition 1 2 3 1 2 3

Expert user (◦) 86.2 57.3 29.8 84.9 56.0 44.7Algorithm (◦) 80.5 55.5 28.0 84.5 62.8 38.4Ground truth (◦) 90.0 67.2 26.4 88.4 63.2 40.5Expert user difference (◦) 3.8 9.9 3.4 3.5 7.3 4.2Algorithm difference (◦) 9.5 11.7 1.6 3.9 0.4 2.1Expert user error (%) 4.22 14.70 13.07 3.95 11.46 10.40Algorithm error (%) 10.57 17.43 6.25 4.42 0.67 5.11

Table 4Measurements of PAV for adduction/abduction exercise

Subject A Subject B

Repetition 1 2 3 1 2 3

Expert user (ω) 82.1 62.2 57.0 81.9 29.1 26.0Algorithm (ω) 79.6 57.7 67.4 71.3 22.8 52.3Trimble 5606 (ω) 87.8 59.9 67.5 62.4 23.9 54.6Expert user difference (ω) 5.7 2.3 10.5 19.5 5.2 28.6Algorithm difference (ω) 8.2 2.2 0.1 8.9 1.1 2.3Expert user error (%) 6.51 3.77 15.53 31.24 21.72 52.36Algorithm error (%) 9.33 3.66 0.09 14.32 4.41 4.13

Table 5Measurements of PAV for lateral/medial exercise

Subject A Subject B

Repetition 1 2 3 1 2 3

Expert user (ω) 50.8 40.9 47.1 39.7 37.3 30.8Algorithm (ω) 48.0 39.3 65.4 34.1 40.2 27.4Trimble 5606 (ω) 51.3 43.8 65.9 36.5 34.5 31.1Expert user difference (ω) 0.5 2.9 18.8 3.2 2.8 0.3Algorithm difference (ω) 3.3 4.6 0.5 2.4 5.7 3.7Expert user error (%) 1.00 6.60 28.53 8.83 8.05 1.09Algorithm error (%) 6.48 10.42 0.74 6.51 16.62 11.97

information obtained from sensation analysis. Thislimitation in-turn creates a concern as to whether or notthis method will produce comparable effects to roboticrehabilitation devices that incorporate force-feedbackwith subtle, unrecognizable to the patient, changes thatinduce motor movements that the patient believes he or

she is incapable of achieving due to injury or disabil-ity. However, noting the earlier references to sustainedattention, it is believed that this approach will have pos-itive results in that it will provide a motivating factor forindividuals during the sessions. Thus, with an increasein motivation and attention, we hypothesize that this

Page 14: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

170 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics

x>80 50<x<70 25<x<500123456789

A

B

Angle Ranges

Diff

eren

ces

Average Differences Across All Subjects for ROM

Visual Observers

Algorithm

x>80 50<x<70 25<x<500

5

10

15

Angle Ranges

Per

cent

age

Err

or

Average Percentages Across All Subjects for ROM

Visual Observers

Algorithm

Fig. 11. Comparison between the expert users and algorithm, withrespect to the ground truth data, of the averages across all subjectsfor the ROM data. (a) Average Differences. (b) Average PercentageErrors.

x>80 50<x<70 25<x<500

5

10

15A

B

Angle Ranges

Diff

eren

ces

Average Differences Across All Subjects for PAV

Visual Observers

Algorithm

x>80 50<x<70 25<x<500

5

10

15

20

25

Angle Ranges

Per

cent

age

Err

or

Average Percentages Across All Subjects for PAV

Visual Observers

Algorithm

Fig. 12. Comparison between the expert users and algorithm, withrespect to the ground truth data, of the averages across all subjectsfor the PAV data. (a) Average Differences. (b) Average PercentageErrors.

will indeed produce comparable results to robotic reha-bilitation devices that incorporate force-feedback.

The use of a robotic platform for this specific appli-cation also reintroduces the debate of robotics versusunembodied (i.e., simulated) agents. Specifically, whatare the benefits of using a robotic platform ratherthan a simulated agent such as those that currentlyexist within the gaming realm (e.g., Nintendo Wii andMicrosoft Xbox)? For years, the goal of artificial lifehas been to move from simulation to embodied sys-tems. Two main reasons are that 1.) without regularvalidation on real robots there is a great danger thatmuch effort will go into solving problems that sim-ply do not come up in the real world and 2.) thereis a real danger that programs which work well onsimulated robots will completely fail on real robotsbecause of the differences in real world sensing andactuation [5]. These statements were true in the early1990’s and are still prevalent today. If we stay withinthe realm of simulated agents we may never reachfully autonomous robotic placement within our soci-ety. The major role of simulated agents should be theinitial modeling of real robots. Human robot interac-tion, even at the simplest level, enables researchers togain valuable insight regarding robotic acceptance. Forthis specific research, as it was stated earlier, robotshave been shown to be engaging to children both withand without disabilities. We hypothesize that havingan agent that is embodied in the real world increasesthe motivation and enthusiasm for patients.

7. Conclusions and future work

In this paper, an approach to matching child move-ments with robotic movements for the purpose ofevaluating child upper-arm rehabilitation exercises hasbeen discussed. Specifically, the physical therapeuticmetrics range of motion and peak angular velocity havebeen calculated via computer vision techniques andcan be utilized in a robotic system. When analyzingthe data, one should note that the Trimble 5606 cap-tured data at a rate that is equivalent to 4 fps, which is abit on the slow side, while our camera captured data ata rate of 15 fps. Therefore, there is approximately fourtimes the amount of data points for our graphs thenthere are for the ground truth data without any interpo-lation. With this in mind, there is a potential issue incases where the patient is moving at very high veloc-ities (i.e., there may be issues in tracking efficiency

Page 15: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics 171

when using devices with low frame rate capabilities).Ideally, most patients will move at a rate that is capableof being tracked due to injury (i.e., relatively slowly).However, in this pilot study, patients were not injured.This may give some insight into the high variancesin percent errors between the algorithmic and Trim-ble data such as those found in Tables 2 and 3. Assuch, the immediate future work consists of obtainingground truth data from a tracking device with a fasterframe rate, giving us even greater accuracy.

Concurrently, the next step is to incorporate anotherphysical therapy metric, namely movement smooth-ness and apply each of these principles to otheractivities such as reaching. One possible method ofdetermining movement smoothness is to analyze theacceleration of the patient’s motion over time, wheredrastic increases or decreases may signify a jitterymotion. Ultimately, this will enable the final roboticsystem to better assess the physical disability ofthe patient. Another seemingly simplistic task is todecrease the processing time that is required to achieveaccurate results when using RANSAC. Because thealgorithm requires a variable number of iterations toachieve high accuracy (99% is used in this work), somecompromise between the number of iterations allowed,

Fig. 13. Illustration of the robotic platform.

accuracy, and number of images available may have tobe forced in order to speed things up. Another com-ponent that will decrease the processing time is use ofcategorical information. By allowing the humanoid tocategorize movements and store the data representingits own motion into specific locations a prior, only oneset of images (the input from the patient’s exercise) willneed to be captured and processed. The overall desire todecrease time comes from the logical fact that childrenwill become restless when required to wait for extendedperiods when engaged in a turn-taking scenario.

The final step of this research is to equip a roboticplatform, shown in Fig. 13, with a small camera andGumstix OveroTM Earth that will enable the robot toperform its movements, video capture, and image pro-cessing completely on-board [4]. Currently, our visionprocessing is done on a separate host computer. Oncethis phase is complete, the researchers intend to con-duct testing with physically injured children in realisticphysical therapy scenarios.

Acknowledgments

The authors gratefully acknowledge the contribu-tions of the individuals in the HumAnS Lab, Dr.Patricio Vela, Dr. Jochen Teizer, Miguel Serrano,NASA JPFP, UNCFSP, and those who participated inthe therapy sessions.

References

[1] A.F. Bobick and J.W. Davis, The recognition of human move-ment using temporal templates. In: IEEE Transactions onPattern Analysis and Machine Intelligence 23 (2001), pp.257–267.

[2] B. Brewer, S. McDowell and L. Worthen-Chaudhari, ThomasLand; Poststroke upper extremity rehabilitation: A review ofrobotic systems and clinical results, Topics in Stroke Rehabil-itation 14(6) (2007), 22–44.

[3] A. Brisben, C. Safos, A. Lockerd, J. Vice and C. Lathan,The CosmoBotTM System: Evaluating its Usability inTherapy Sessions with Children Diagnosed with CerebralPalsy (2005), http://web.mit.edu/zoz/Public/Anthrotronix-ROMAN2005.pdf

[4] D. Brooks and A.M. Howard, Upper-limb Rehabilitation andEvaluation of Children using a Humanoid RobotIn: ICMI-MLMI09 Workshop on Child, Computer and Interaction,Cambridge, MA, (2009), pp. 22.

[5] R. Brooks, Artificial life and real robots. In: Proceedings of thefirst European Conference on Artificial Life, Citeseer, (1992),3–10.

[6] C. Burgar, P. Lum, P. Shor and M.V. derLoos, Development ofrobots for rehabilitation therapy: The Palo Alto VA/Stanfordexperience, J Rehabil Res Dev 37(6) (2000), 663–673.

Page 16: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

172 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation metrics

[7] L. Campbell and A.F. Bobick, Recognition of human bodymotion using phase space constraints. In: Proc Int’l Confer-ence Computer Vision (1995), pp. 624–630.

[8] J. Canny, A computational approach to edge detection. Read-ings in Computer Vision: Issues, problems, principles andparadigms 8(6) (1986), 184–203.

[9] M. Casadio, P. Giannoni, P. Morasso and V. Sanguineti, Aproof of concept study for the integration of robot therapywith physiotherapy in the treatment of stroke patients, ClinicalRehabilitation 23(3) (2009), 217.

[10] O. Chum, Two-view geometry estimation by random sampleand consensusCzech Technical University, 2005.

[11] R. Colombo, F. Pisano, S. Micera, A. Mazzone, C. Delconte,M. Carrozza, P. Dario and G. Minuco, Upper Limb Rehabil-itation and Evaluation of Stroke Patients Using Robot-AidedTechniquesIn: ICORR 2005 Conference Proceedings, June28 - July 1, (2005), pp. 515–518.

[12] E. Cooley and R. Morris, Psychology Press; Attention in chil-dren: A neuropsychologically based model for assessment,Developmental Neuropsychology 6(3) (1990), 239–274.

[13] T. Cormen, C. Leiserson, R. Rivest and C. Stein, 2001. Intro-duction to Algorithms. The MIT Press.

[14] R. Duda and P. Hart, ACM New York, NY, USA; Use of theHough transformation to detect lines and curves in pictures,(1972).

[15] P. Duncan, M. Propst and S. Nelson, Reliability of the Fugl-Meyer assessment of sensorimotor recovery following cere-brovascular accident, Physical Therapy 63(10) (1983), 1606.

[16] M. Federico Posteraro, S. Mazzoleni, S. Aliboni and B.Cesqui, Robot-mediated therapy for paretic upper limb ofchronic patients following neurological injury. REHABILI-TATION MEDICINE 41 (2009), 976–980.

[17] M. Fischler and R. Bolles, ACM New York, NY, USA; Ran-dom sample consensus: A paradigm for model fitting withapplications to image analysis and automated cartography,(1981).

[18] M. Folio and R. Fewell, Peabody Developmental MotorScales and Activity Cards (Manual): Hingham. MA: TeachingResources (1983).

[19] M. Folio, R. Fewell, Peabody Developmental Motor Scales:Examiner’s Manual. Pro-ed, 2000.

[20] A. Fugl-Meyer, L. Jaasko, I. Leyman, S. Olsson and S.Steglind, The post-stroke hemiplegic patient. 1. a method forevaluation of physical performance, Scandinavian Journal ofRehabilitation Medicine 7(1) (1975), 13.

[21] R. Gockley and M. Mataric, Encouraging physical therapycompliance with a hands-off mobile robot. In: Proceedings ofthe 1st ACM SIGCHI/SIGART conference on Human-robotinteraction ACM (2006), pp. 150–155.

[22] L. Goncalves, E. DiBernardo, E. Ursella and P. Perona,Monocular Tracking of the Human Arm in 3DIn: Proc Int’lConference Computer Vision (1995), pp. 764–770.

[23] S. Hesse, G. Schulte-Tigges, M. Konrad, A. Bardeleben andC. Werner, Robot-assisted arm trainer for the passive andactive practice of bilateral forearm and wrist movements inhemiparetic subjects, Arch Phys Med Rehabil 84(6) (2003),915–920.

[24] M. Hurley, Effectiveness of robot-assisted therapy on strokepatients with upper extremity impair-ment, Physical Function(2009), pp. 14.

[25] H. Kozima, C. Nakagawa and Y. Yasuda, Interactive Robotsfor Communication-Care: A Case Study in Autism Thera-pyIn: ROMAN 2005, (2005), pp. 341-346.

[26] H. Krebs, N. Hogan, M. Aisen and B. Volpe, Robot-aidedNeurorehabilitation, IEEE Trans. Rehab Eng 6(1) (1998),75–87.

[27] G. Kwakkel, B. Kollen and H. Krebs, Am Soc Neurorehabil;Effects of robot-assisted therapy on upper limb recovery afterstroke: A systematic review, Neurorehabilitation and NeuralRepair 22(2) (2008), 111.

[28] R. Loureiro, F. Amirabdollahian, M. Topping, B. Driessenand W. Harwin, Upper Limb Robot Mediated Stroke TherapyGentle/s Approach, Autonomous Robots 15(1) (2003), 35–51.

[29] P. Lum, C. Burgar, P. Shor, M. Majmundar and M.V. derLoos,Robot-assisted movement training compared with conven-tional therapy techniques for the rehabilitation of upper-limbmotor function after stroke, Arch Phys Med Rehabil 83(7)(2002), 952–959.

[30] P. Lum, D. Reinkensmeyer, R. Mahoney, W. Rymer and C.Burgar, Thomas Land; Robotic devices for movement ther-apy after stroke: Current status and challenges to clinicalacceptance, Topics in stroke rehabilitation 8(4)(2002), 40–53.

[31] J. Mehrholz, T. Platz, J. Kugler and M. Pohl, Am HeartAssoc; Electromechanical and robot-assisted arm training forimproving arm function and activities of daily living afterstroke, Stroke 40(5) (2009), e392.

[32] A. Melkman, On-line construction of the convex hull of asimple polyline, Information Processing Letters 25(1) (1987),11–12.

[33] T. Parsons, Voice and Speech Processing, New York:McGraw-Hill. (1987).

[34] J.L. Patton and F.A. Mussa-Ivaldi, Robot-assisted adaptivetraining: Custom force fields for teaching movement patterns,IEEE Trans Biomed Eng 21(4) (2004), 636–646.

[35] R. Polana and R. Nelson, Low Level Recognition of HumanMotionIn, Proc IEEE Workshop Non-Rigid and ArticulatedMotion (1994), pp. 77–82.

[36] J. Rehg and T. Kanade, Model-Based Tracking of Self-Occluding Articulated ObjectsIn, Proc Int’l ConferenceComputer Vision (1995), pp. 612–617.

[37] D.J. Reinkensmeyer, B.D. Schmit and W.Z. Rymer, Assess-ment of active and passive restraint during guided reachingafter chronic brain injury, Ann Biomed Eng 27(6) (1999),805–814.

[38] D. Russell, P. Rosenbaum, D. Cadman, C. Gowland, S. Hardyand S. Jarvis, Wiley Online Library; The gross motor functionmeasure: A means to evaluate the effects of physical therapy,Developmental Medicine & Child Neurology 31(3) (1989),341–352.

[39] C. Takahashi, L. Der-Yeghiaian, V. Le, R. Motiwala and S.Cramer, Oxford Univ Press; Robot-based handmotor therapyafter stroke, Brain (2007).

[40] Trimble. Trimble - 5605 Total Station and LM80 Layout Man-age. Datasheet. (2005).

[41] B.T. Volpe, H.I. Krebs, N. Hogan, L. Edelstein, C. Diels andM. Aisen, A novel approach to stroke rehabilitation: Robot-aided sensorimotor stimulation, Neurology 54(10) (2000),1938–1944.

[42] H. Wang, H. Liao and C. Hsieh, American Physical TherapyAssociation; Reliability, sensitivity to change, and respon-siveness of the Peabody Developmental Motor Scales–secondedition for children with cerebral palsy, Physical Therapy86(10) (2006), 1351.

Page 17: Quantifying upper-arm rehabilitation metrics for children ...downloads.hindawi.com/journals/abb/2012/978498.pdf · 158 D.A. Brooks and A.M. Howard / Quantifying upper-arm rehabilitation

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttp://www.hindawi.com Volume 2010

RoboticsJournal of

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Hindawi Publishing Corporation http://www.hindawi.com

Journal ofEngineeringVolume 2014

Submit your manuscripts athttp://www.hindawi.com

VLSI Design

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

The Scientific World JournalHindawi Publishing Corporation http://www.hindawi.com Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Modelling & Simulation in EngineeringHindawi Publishing Corporation http://www.hindawi.com Volume 2014

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

DistributedSensor Networks

International Journal of