Artificial Organs 19(3):231-237, Blackwell Science, Inc., Boston 0 1995 International Society for Artificial Organs

An Artificial Neural System for Closed Loop Control of Locomotion Produced via Neuromuscular

Electrical Stimulation

Francisco Sepulveda and Albert0 Cliquet, Jr.

Biomedical Engineering Department, State University of Campinas, Campinas, Scio Paulo, Brazil

Abstract: The use of neuromuscular electrical stimulation for restoration of gait in spinal cord injured subjects has been seriously pursued by many investigators for the past 15 years. By and large, however, systems to date require the intervention of a person, be it the patient or an ob- server, and are restricted to control of stimulation onset and termination. Further, existing systems are not adapt- able to environmental and patient variations. This work proposes a system that relies on neural computing to de- termine proper muscle activation patterns from biome- chanical signals. The intelligent system is trained to per- form gait under supervision, after which it can be used to control muscle stimulation in an unknown environment.

Computer simulations suggest that the best neural archi- tecture for control of gait is a neural network including units corresponding to movement history. Separate net- works for the stance and swing phases, respectively, were found to work better than a single neural network trained on the entire gait cycle. The artificial neural de- vice proposed here also includes a voice recognition sys- tem that will allow for voluntary locomotion. A safety circuit has been designed to preclude acceptance of un- wanted vocal commands in the latter system. Key Words: Neural prosthesis-Dynamic neural network control-Auto-adaptive neural control of gait.

Locomotor planning takes place in supraspinal regions which send appropriate signals to lower spi- nal centers. These in turn are directly responsible for the development of suitable activity patterns in the relevant muscle groups. In spinal cord injured patients, communication between supraspinal cen- ters and muscles below the injury level is often completely absent. Yet motor nerve fiber segments distal to the lesion are frequently intact. Electrical stimulation to these fibers thus can re-establish con- trolled muscle contraction.

The use of neuromuscular electrical stimulation (NMES) for reactivating paralyzed muscles can be traced back to the work of L. Galvani in 1761, but it has been seriously pursued by investigators around the world for only the past 40 years. Early attempts include the use of open loop NMES to the phrenic nerve in order to restore controlled ventilation (1). More elaborate systems were developed throughout

Received July 1994. Address correspondence and reprint requests to Dr. F.

Sepulveda, Biomedical Engineering Department, FEE- UNICAMP, P.O. Box 6040, Campinas, SP 13081-970, Brazil.

the 1980s for restoring upper and lower limb move- ments. In particular NMES has been applied to lower limb musculature in attempts to restore pa- tient locomotion. Control of the devices for this case is critical. By and large most systems to date (2,3,4,10) require the intervention of a person, be it the patient or an observer, and are restricted to con- trol of stimulation onset and termination times. Re- gardless, such open-loop systems have limited ap- plication. There are a number of aspects in the motion, in the patient, and in the environment that change from session to session, from gait cycle to gait cycle, and within the gait cycle itself. Thus, an intelligent, adaptable, closed-loop system that is also portable, and, most importantly, one that can be trusted to abide by strict safety rules out of the laboratory environment is long overdue. In short, adequate rehabilitation of spinal patients remains an illusion.

This manuscript introduces a system whose heart lies in an intelligent neural algorithm ( 5 ) that utilizes signals from biomechanical sensors to determine proper muscle activation patterns in gait-related movements. By “proper” is meant stimulation with

231

232 F. SEPULVEDA AND A . CLIQUET, JR.

minimal muscle fatigue, mechanical stability of the man-machine system, and, of course, acceptable execution of movements. Some of the advantages inherent in the use of artificial neural circuits in- clude on-line learning, fast prediction time, and the possibility of producing a microchip that incorpo- rates all of the system’s processing elements. In- deed, the latter would be a major step toward creation of an implantable neural circuit for resto- ration of movements (even voluntary) in spinal patients.

MATERIALS AND METHODS

Fundamentals of the neural network algorithm The heart of the system presented in this paper

consists of a neural network with supervised learn- ing (Fig. 1). The standard structure for such neural networks consists of three layers of “neurons” or processing units: input neurons, output neurons, and, between them, hidden neurons. In order to understand this explanation, the reader may asso-

INPUTS

1 1 1

- I I 1 I c \

OUTPUTS \ \ I

I I

FIG. 1. A schema is presented of the artificial neural network. Traditionally, a three-layer structure with interlayer connec- tivity and sigmoidal transfer functions are used. See text for a detailed explanation.

ciate input units with retinal cells, hidden units with the neocortex, and output units with descending motor fibers. The circles in Fig. 1 represent neuron cell bodies and can be as numerous as one wishes. But be cautioned because the analogy is only illus- trative.

Input units are mainly transducers and deliver in- put signals to hidden units without any processing. Signals from the input layer are traditionally as- sumed to reach all hidden layer units (so called “complete interlayer connectivity”). However, be- fore a hidden unit actually takes incoming signals into account, the signal intensity is multiplied by a weight factor W. This factor is unique for each in- dividual signal reaching a neuron. The weight factor W is analogous to synaptic modulation in that a neg- ative factor results in decreased postsynaptic cell output (inhibition), and a positive factor yields en- hanced postsynaptic cell output (excitation). Thus, the W factor is often called a synaptic weight. Hid- den neurons add up all incoming signals (appro- priately modified by synaptic weights) and produce an output accordingly (see enlarged units in Fig. 1). This is done by the so called transfer function (a sigmoidal function is used in this work). Hid- den units then deliver signals to output neurons. These in turn process the information much as in hidden layer units and yield a final network out- put. The sequence of events discussed in this para- graph is called a feedforward sequence and takes place for each input pattern presented to the net- work.

During supervised learning, the output signals re- sulting from a feedforward pass are to match pre- determined patterns, the so called target values. In- variably outputs at the beginning of the learning process are very different from target values. Whenever the difference between target and actual network output values exceeds a minimum error (called tolerance), all synaptic weights are changed accordingly. This is called a feedback sequence. The particular algorithm used in this work for de- termining synaptic weight variations is the general- ized back-propagation (6,7).

Traditionally synaptic weights are randomized before a network is trained. Then feedforward and feedback passes are repeated until network outputs are within a pre-established tolerance. When this occurs the network is said to have learned, and the acquired synaptic weights are stored. Thereafter, the now trained network can be used on feedfor- ward only mode to predict output patterns in an unknown environment where inputs differ from those used during training.

ArtifOrgans, Vol. 19, No. 3, 1995

A NEURAL SYSTEM FOR NMES-AIDED GAIT 233

The control system The basic architecture of the system for closed-

loop control of NMES-assisted gait is depicted in Fig. 2. Commands will be given verbally by the pa- tient or by an observer. A neural network will be trained to recognize phonemes and to associate them to a given movement sequence (a similar neural network is already under use in our labo- ratory for generating upper limb movements (8)). This will allow greater patient freedom as no man- ual intervention by h idher will be needed during movement production. Stand up (SEQ A) and sit down (SEQ B) sequence generators may utilize pre- determined stimulation sequences and may also receive input from force and angle sensors. The lat- ter angle sensors consist of hip, knee, and ankle electrogoniometers for estimating joint flexion and extension. These, despite the apparent simplicity, require close patient-goniometer coupling, and their production has already become a major problem in this project. Force sensors consist of a foot sole array of pressure sensors as well as sensors placed on the walking aids (e.g., crutches). From the foot sensors, at least vertical ground reaction forces will be estimated and used. Longitudinal and medial/lateral forces may be considered as well.

The walk routine has been divided into a swing and a stance phase. Just when one or the other should be activated will be determined primarily by afoot contact detector (Cmp, Fig. 2). The latter will compare current foot sole force readings (FGR) to a minimum noise-related value (Fmj,,). As implied by the ANN label in Fig. 2, controllers for both phases

FIG. 2. A schema of the closed loop sys- tem for control of NMES-assisted gait and related movements is shown. ANN at the center top represents the neural network for recognition of verbal commands. F denotes vertical ground reaction forces (FGR or GRF) and forces applied to walk- ing aids. Theta indicates reading of hip, knee, and ankle flexion/extension angles. SEQ A and SEQ B denote predetermined sequences for standing up and for sitting down, respectively. A unit for standing still is also included (ST STILL MAINT). Stance and swing ANNs refer to neural networks trained for determining (or modulating) NMES signal characteristics during stance and swing phases, respec- tively. Cmp denotes a simple threshold comparator. Q refers to hip, knee, and an- gles.

L

will consist of artificial neural networks properly trained for the task (see below).

Needless to say a sequence of commands given by the patient cannot be allowed to produce prob- lematic situations. For instance the signal sequence for sitting down cannot be activated while the pa- tient walks! To take such safety issues into account, two elements have to be added to the system. First, early interruption of a movement sequence can only be produced by emergency keys (not shown in Fig. 2). Second, once a movement is concluded only cer- tain commands can be accepted. For example after a patient stands up, it is necessary to activate a routine for standing still until the patient and/or ob- server feels that the subject is ready for stepping. Similarly the walking routine cannot be activated when the patient has just sat down. To avoid such problems, a simple logic circuit has been devised (Fig. 3). The commands at the top of the figure are those recognized by the neural voice recognition system whereas the commands enclosed in boxes, bottom center in Fig. 3, are the routines actually activated. No output will be produced by these to the logic circuit until the movement sequence is concluded. A buffer will be used for storing a com- mand until it can be executed. In addition the whole stimulation process has to begin with the stand up sequence (this may be changed later). The latter routine is the only one allowed to function at an NIL state, which denotes the state where the sys- tem has just been turned on and the command buffer is empty. The safety system described in Fig. 3 is an important part of the command-discriminating neural network (ANN) shown at the top of Fig. 2.

COMMAND

4.

SENSORS F O c ~ ’ FO

Art$ Organs, Vol. 19, No. 3, 1995

234 F . SEPULVEDA AND A . CLIQUET, JR.

RECOGNIZED COMMANDS

SIT WALK $ WALK

Stand UP Stand STILL

.@I t- UI

FIG. 3. A schema of the safety system for discrimination of recognized verbal commands is given. NIL denotes the state when the device has just been turned on. Notice that the first command has to be Stand UP. The boxes around commands indicate separate routines for execution of each command once it is accepted. Only one routine can be enabled at a time.

It is important to mention that all control and neural devices described above will eventually be transferred to a set of microcontroller units. This will drastically reduce system volume and weight.

Neural network training and evaluation The basic structure of the model presented here is

displayed in Fig. 4. Input for the model consists of hip, knee, and ankle angles and vertical ground re- action forces (GRF).

The output signal corresponds to the electrical activity of 5 lower limb muscles (Table 1). Indeed the model was trained to yield values equivalent to

IHIDDEN UNITS! 1

00000 OUTPUT UNITS:

MUSCLE ACllVATlON

I

FIG. 4. A schema of the structure underlying the networks used in this study is shown. Although the stance phase, walk- ing neural network is depicted, the actual type of input and output signals may vary. All units from a layer are connected to all units in the subsequent level. Dashed portions illustrate the process whereby the loop might be closed (walking), yielding data for adjusting NMES signals.

the linear envelope of electromyography (EMG). Insofar as the NMES system cited above is con- cerned, the neural network output is the time inte- gral (time versus signal area) of the electrical stim- ulation signal applied to the muscles (although precise postprocessing of network output has yet to be determined).

The precise constitution of the various configura- tions of neural networks investigated in this study is presented in Table 2. Several aspects are common to all networks: a three-layer structure was used; all neurons from a layer are connected to all elements in the subsequent level only; a sigmoidal neuronal transfer function with outputs ranging from zero to 1 was used; network outputs correspond to EMG linear envelope values for 5 lower limb muscles; and there are 2 hidden units.

The reader will notice that some of the networks tested so far (e.g., nets 2 and 3) require inputs from time periods preceding the current acquisition frame. This is to determine whether to monitor movement history instead of considering instanta- neous motion parameters only.

The networks were trained on normal human gait data for the right leg whereas evaluation of network performance was done by use of known clinical data (Figs. 5 and 6). The latter clinical values con- sisted of force, angle, and EMG data gathered from an adult cerebral palsy subject during walking. All normal and clinical data were obtained from Vaughan et al. (9). Testing consisted of presenting clinical case force and angle data at the networks’ input and later comparing network outputs with ac- tual EMG values for the clinical case. This testing technique is rather drastic, but it shows some of the networks’ predicting abilities in a complex situa- tion. In the future when angle and force data from spinal cord injured patients walking with NMES are made available, more realistic tests may be per- formed.

Although target values included activity patterns for all 5 muscles in Table 1, only hamstring data are

TABLE 1. Muscles included in the system. Linear envelope EMG for these muscles were used for training and testing the artificial neural networks. However, the actual muscle groups used in the final system will vary

from patient to patient

Muscle Name

1 Gluteus maximus 2 Hamstrings 3 Rectus femoris 4 Tibialis anterior 5 Triceps surae

Art$ Organs, Vol. 19, No. 3, 1995

A NEURAL SYSTEM FOR NMES-AIDED GAIT 235

TABLE 2. Neuraf configurations tested in this work

Network Input

1 2 3

4" 5 6 7

8

9

10

Joint angles and GRF, CC Joint angles and GRF + EMG output, CC Joint angles and GRF (current + one

previous frame), CC Joint angles and GRF, CC Joint angles and GRF, ST Joint angles and GRF, SW Joint angles and GRF + Last EMG

Joint angles and GRF + Last EMG

Joint angles and GRF (current + one

Joint angles and GRF (current + one

output, ST

output, sw previous frame), ST

previous frame), SW

CC indicates a complete gait cycle. ST and SW denote stance and swing phases, respectively.

"Indicates network training and testing with time-recurrent back-propagation (7). Other common network characteristics are discussed in the text.

displayed in Fig. 6 for brevity. The gait cycle was divided into 20 frames. Thus, each data set imposed onto the networks represents a 5% progression in the gait cycle. All data were normalized between 0.2 and 0.8 according to minimum and maximum for each variable.

Care should be taken when comparing normal data to clinical data in Fig. 6. The magnitude of the EMG signal is often unimportant whereas the

shape of the EMG curves is a rich source of infor- mation.

All synaptic weights were initially randomized between -0.5 and 0.5. Learning tolerance was set to 0.1, and the learning rate was variable between 1 .O and 0.05, obeying a linear relationship between the learning rate and the number of units yielding an output greater than the tolerance. To keep network output from falling into an unwanted energy mini- mum, the learning rate was reset to 1.0 whenever 1,000 iterations went by without change in the num- ber of neurons with unacceptable outputs. This was found to work better than the traditional synaptic weight resetting (the jump produced by this tech- nique is rather small), taking the network from the unwanted local minimum without wasting all the learning already effected.

RESULTS Of all the networks listed in Table 2, a combina-

tion of Net 9 (stance phase) and net 10 (swing phase) was found to yield best results when clinical data were presented to the network previously trained on normal data.

Hamstring activity, both actual and predicted by networks (presented clinical inputs), is displayed in Fig. 7. Notice how predictions from Net 1 resemble the normal data on which the network was trained rather than the clinical data. All other networks dis-

a

N 0 R 1 M A \ 0.8 z E D 0.6

V E R 0.4 T I

L 0.2

G O

I --.V-. NORMAL -x- CLINICAL I b

1

1

0.6

E 0.2

R -0 20 40 60 80 100 0 20 40 60 80 100 F % GAIT CYCLE 'Yo GAIT CYCLE

FIG. 5. Normal and clinical data presented at the networks' input. Normal data were used for training whereas clinical data were used for testing the control neural networks in a computer simulation. Heel strike occurs at 0% of the gait cycle, toe-off at about 60%, and 100% indicates a new heel strike. (a) Vertical ground reaction forces. (b) Hip, knee, and ankle angles (larger values indicate flexion of the hip and knee, and dorsiflexion of the ankle). Notice that all data were normalized to fall between 0.2 and 0.8. This speeds up network training and permits handling of data whose range falls outside the normal domain.

Artf Organs, Vol. 19, No. 3, 1995

236

S T o.a?

F . SEPULVEDA AND A . CLIQUET, JR.

~~

X .,

I V NORMAL ++CLINICAL I H I

T -0 Y

played similar behavior. The combination of Net 9 and Net 10, on the other hand, departs from normal patterns toward clinical values in the central and final portions of the stance phase, between 20 and 60% of the gait cycle (recall that toe-off, or end of the stance phase occurs at about 60% of the gait cycle).

However, none of the networks made good pre- dictions for the swing phase. This may be a result of the structure of the networks for the swing phase, namely, the same as for the stance phase, including units for vertical GRF. The latter were left with inputs fixed at 0.2 (corresponding to zero detected force, as per the normalization discussed above). It

FIG. 7. Comparison between predictions from net 1 (see Table 2) and a combina- tion of net 9 and net 10 and actual clinical data. In this example clinical data were presented at the networks’ input.

N 0 R M A L I Z E D

M U S C L E

20 40 00 a0 100

% GAIT CYCLE

may be advisable to remove the GRF input unit in all swing-phase networks in the future.

Regardless, major differences between predicted and actual clinical data merely support the idea that training with normal data alone is not enough (5).

DISCUSSION

First and foremost it is clear from the results that training of the networks with actual patient data must take place before the artificial neural system is used in an unknown environment. In this respect, however, the behavior of the networks is consistent with biological fact. Intelligent beings (human or

1

€4

0 0 20 40 60 80 100

% GAIT CYCLE

Artif Organs, Vol. 19, No. 3, 199s

A NEURAL SYSTEM FOR NMES-AIDED GAIT 237

otherwise) trained on very specific tasks tend to behave in a manner biased toward training condi- tions when exposed to new settings.

Furthermore, the results indicate the need for a buffer where some sort of movement history is stored. A buffer with more than one time-frame memory (as in Nets 3, 9, and 10) should be inves- tigated. In addition the need for separate networks for the swing and stance phases, respectively, has become evident. In any event, Net 9 above appears to be suitable for control of NMES during stance.

This document has also proposed a system for closed-loop control of NMES-assisted gain that in- cludes the following features: recognition of vocal commands. This will reduce encumbrance as the patient will no longer need to press switches to pro- duce or interrupt movements. This will also allow for restoration of voluntary motility; a safety sys- tem against improper triggering of movement se- quences; adaptive control. The neural algorithm will permit adaptation of the system to new envi- ronments (via on-line learning).

These are important factors if the system is to become a truly functional, albeit limited, substitute for lost neuromuscular mechanisms in spinal pa- tients.

Acknowledgment: This work has been sponsored by Brazil’s CNPq.

REFERENCES

1 .

2.

3 .

4.

5 .

6.

7.

8.

9.

10.

Sarnoff SJ, Hardenburgh E, Whittenberger JL. Electro- phrenic respiration. Am J Physiol 1948; 155: 1 . Kralj A, Badj T, Turk R. Gait restoration in paraplegic pa- tients: a feasibility demonstration using multichannel surface electrode FES. J Rehab Res Develop 1983;20:3-20. Petrofsky JE, Phillips CA. Closed-loop control of movement of skeletal muscle. CRC Crit Rev Biomed Eng 1985;13:

Marsolais EB, Kobetic R. Functional walking in paralyzed patients by means of electrical stimulation. Clin Orthop 1983 ; 175:26-30. Sepulveda F, Wells DM, Vaughan CL. A neural network representation of electromyography and joint dynamics in human gait. J Biomech 1993;26(2): 101-9. Rumelhart DE, Hinton GE, Williams RJ. Learning represen- tation by back propagation errors. Nature 1986;323:533-6. Werbos PJ. Backpropagation through time: What it does and how to do it. Proc ZEEE 1990;78(10):1550-60. Cliquet A, Jr. et al. A neural network-voice controlled neu- romuscular electrical stimulation system for tetraplegics. In: RESNA International ’92, Toronto, Canada 1992;12:29-31. Vaughan CL, Davis BL, O’Connor JC. Gait analysis iabo- ratory. Champaign, IL: Human Kinetic Publishers, 1992. Cliquet A, Jr. Paraplegic gait restoration through neuromus- cular stimulation based strategies. Med Biol Eng Comp 1 9 9 1 ; 21:711.

35-96.

Artif Organs, Vol. 19, No. 3, 1995