Networked Bio-Inspired Modules For Sensorimotor Control of Wearable Cyber-Physical Devices

Yong-Lae Park1,2, Diana Young1,2, Bor-rong Chen3, Robert J. Wood1,2,, Radhika Nagpal1,2, and Eugene C. Goldfield1,4 1Wyss Institute for Biologically Inspired Engineering, Harvard University, Boston, MA, USA.

2School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA. 3Biosensics LLC, Boston, MA, USA.

4Boston Children’s Hospital, Boston, MA, USA.

Abstract— We present a functioning prototype of a soft, modular, active cyber-physical assistive device comprised of a sealed network of conductive liquid sensors and collectives of miniature pneumatically-driven actuators that serve as artificial muscles. The system is multi-functional, supports large deformation, and operates with its own on-board pneumatics and controllers. When multiple artificial muscles are collectively actuated (contracted), the overall displacement and force produced is scaled to the size, form, and capabilities of the wearer. Each muscle is equipped with a soft strain sensor that detects the muscle contraction. Four muscles with strain sensors are controlled by one micro-controller as one module. The current prototype has four modules with 16 muscles in total. With different combinations of contracted muscles, various shapes may be demonstrated.

Index Terms—cyber-physical system (CPS); assistive device; sensor network; pneumatic artificial muscle (PAM); soft strain sensor; inertial measurement unit (IMU).

I. INTRODUCTION We present a novel, wearable, bio-inspired cyber-physical

assistive device for rehabilitation of injured nervous systems, called the “Second Skin.” The device is characterized by (a) a soft, responsive interface with the body, (b) sensors and actuators that emulate the function of biological components, and (c) a control system based upon distributed networks with modular components. Unlike current exoskeletons [1], [2] that are heavy and structurally similar to motorized braces, our cyber-physical device is made from soft and elastic materials.

There have been various approaches to develop active orthotic devices using different actuation technologies, such as series elastic actuators (SEA) [3], brushless DC motors [4], McKibben pneumatic artificial muscles [5], [6], and shape memory alloy (SMA) wire springs [7]. Many of these devices are bulky and built on rigid frame structures that constrain the user’s natural degrees of freedom of the joint in a manner similar to that of exoskeletons. Although [6] and [7] completely removed rigid frames making the devices entirely flexible, their designs are customized to individual users resulting in difficulty in resizing. The design of Second Skin has an emphasis on modularity as well as the soft structure, making the device easily resizable and reconfigurable maintaining its inherent functionality and wearability.

Fig. 1 Overall design concept of “second skin.”

The sensing, actuation, and control functions employed by Second Skin are each inspired by natural systems. For example, nature uses multiple sensors to transduce in parallel certain body motions at different time scales: a faster time scale that uses mechanical stretch as a signal for regulating individual muscle contractions, and a slower time scale for sensory modulation of the contraction of organized groups of muscles response to changing environmental conditions. And finally, distributed biological networks composed of modular components are the basis for multifunctionality in insects, invertebrates, and vertebrates.

The rest of the paper is organized as follows: an overview of the Second Skin device design; a description of the different functions of the sensors; a discussion of how the device may achieve different shapes through the coordinated control of multiple artificial muscles; a review of the controller network architecture; current research challenges; and conclusions.

II. AN OVERVIEW OF THE SECOND SKIN

A. A soft, responsive interface. The Second Skin (Fig. 1) is designed as a 3D elastomer

cylinder with embedded soft actuators and sensors. By contracting multiple actuators collectively, the device not only changes its overall shape but also produces pulling or bending force. Since the device is made of a highly compressible and stretchable silicone elastomer, it will be easily wearable and conformable to the user’s complicated 3D body contours. Also, the morphological and functional modularity of the device allows being easily resizable and reconfigurable.

Each actuator has its own soft sensor for measuring the local displacement of the actuator. The device is also equipped

978-1-4673-5288-8/13/$31.00 ©2013 IEEE

2013 International Conference on Computing, Networking and Communications, Workshops Cyber Physical System

92

Fig. 2 Sensor-actuator unit embedded in a silicone elastomer. (a) Relaxed

artificial muscle. (b) Contracted artificial muscle. (Front side with strain sensor, backside without strain sensor, and sensor resistance, respectively.)

with multiple inertial measurement units (IMUs) for detecting global shape changes. One microcontroller can control four sensor-actuator units making one module. Fig. 1 describes the overall design concept of the Second Skin with multiple sensor-actuator units.

B. Sensor-actuator functional units. One functional unit is composed of two major components:

a soft actuator [8], [9] and a soft sensor [10], [11]. The soft actuator is a miniature McKibben-type pneumatic artificial muscle that is custom-built. When compressed air is supplied to the artificial muscle, the muscle contracts in its axial direction. This muscle contraction can be measured with the embedded soft strain sensor.

C. Distributed control. The functional units are inter-connected together with a

mesh topology. Each unit has a processor that is capable of executing a local control loop to achieve local actuation goals (i.e., actuating the artificial muscle to stay at a target length). The network of distributed controllers maintains the global shape of the second skin device. In order to change the device shape, a communication protocol is defined to propagate target length values to each of the individual functional units in the network. To drive a desired motion pattern, the network of controllers can be programmed to achieve different target shapes with a pre-defined schedule and speed. Although not shown in this paper, it is also possible to define reactive rules to achieve shape changes driven by environmental inputs.

III. USING MULTIPLE SENSORS FOR DIFFERENT FUNCTIONS

A. Strain sensors for detecting pneumatic muscle contraction. Embedded on top of each pneumatic artificial muscle is a

hyperelastic strain sensor to measure the muscle contraction. The strain sensor is made of silicone elastomer and has embedded micro-channel patterns filled with liquid metal (Eutectic Gallium Indium – EGaIn) [12]. The overall resistance of these channels change as the sensor experiences strain changes. Since the strain sensor is placed on top of the actuator perpendicular to the axial direction of the muscle detecting the muscle’s lateral expansion, its electrical resistance increases with axial contraction of the muscle. Fig. 2 shows a strain

Fig. 3 Strain sensor and functional unit characterization plots. (a) Strain sensor signal when stretched up to 100 % strain before being integrated with a pneumatic artificial muscle. (b) Resistance change of strain sensor integrated

with a pneumatic muscle with the muscle contraction.

sensor integrated pneumatic muscle embedded in the silicone elastomer base material showing increased sensor resistance with muscle contraction. The characterization results of individual strain sensor and sensor-actuator functional units are shown in Fig. 3.

B. Inertial measurement units for modulating contraction of organized groups of muscles. The Second Skin will include an inertial measurement unit

(IMU), capable of measuring 3D acceleration and angular velocity, for each limb segment. By additionally including magnetometer measurements to correct for changes in yaw, these data can be used to estimate the absolute 3D orientation of each limb segment with respect to the frame of the Earth, thus reflecting the overall shape of the device. To accomplish this, over short periods of time, relative orientation may be obtained through direct integration of the gyroscope measurements, and relative velocity/position may be obtained by integrating the accelerometer measurements. The accuracy of integration may be further improved with sensor fusion algorithms that use quaternion-based representation of orientation [13], [14].

Nonlinear Kalman Filters, such as the Extended Kalman Filter (EKF) [15], represent a class of sensor fusion algorithms that can correct for the drift exhibited by commercial inertial sensors, while providing absolute unit estimation. The EKF uses a nonlinear model of the dynamics (including noise) of the state to be estimated. The EKF evolves the state according to this model and then updates the state based on observations from the appropriate sensors to minimize the error of the estimated state. For example, using gyroscopes, accelero-meters and magnetometers, the EKF may reliably reduce the error generated when integrating the angular velocities by using observations of the acceleration and magnetometer measurements (include contributions from gravity and inertial movement), enabling robust estimation of absolute orientation with the lowest error, relative to the absolute reference frame of the Earth. With this absolute orientation and the joint angles determined by a number of soft, flexible strain sensors, all limb segments may be oriented with respect to the absolute reference frame of the Earth.

Fig. 4 illustrates the accuracy of the above method, in comparison to that obtained by traditional motion capture and inverse kinematics, for estimation of lower limb segment

93

Fig. 4 Comparison of knee and ankle sagittal joint angles during adult level

ground walking estimated using IMU data and an EKF to those obtained using optical motion capture and inverse kinematics software OpenSim, from [16].

Fig. 5 Early exploration using acceleration and angular velocity data as inputs to an EKF for estimation of sagittal knee joint angle and individual

limb segment angles during adult supine kicking (trial performed by one of the authors wearing three MicroStrain® 3DM-GX2™ IMUs).

orientations [16]. Using acceleration and gyroscope measurement from three commercial IMUs (MicroStrain®

3DM-GX2™), one mounted to each of the three lower limb segments of one side, joint angles and individual limb segment orientations may be estimated with low error.

In order to begin exploring the performance of this method to other motions of interest to which the Second Skin device may be applied, a preliminary trial featuring adult supine kicking was conducted using the same MicroStrain® IMUs, with one of the authors wearing each of the three IMUs in the same configuration as in the above walking example. Fig. 5 shows the resulting estimated knee joint sagittal angle, as well as the individual upper and lower limb segment angles.

IV. USING DIFFERENT COMBINATIONS OF SYNTHETIC MUSCLES The Second Skin design contains 16 pneumatic muscles,

organized into four modules, as described in Fig. 6(a). It also has embedded soft strain sensors to detect the pneumatic muscle contraction. Fig. 6(b) shows the actual prototype and a single functional unit. In our current prototype, only four muscles in one column are equipped with strain sensors. With different combinations of contracted muscles, the device can achieve various shape changes and force production, as shown in Fig. 6(c). Based on the characterization [8], [9], a single

Fig. 6 (a) Prototype design. (b) Actual prototype. (c) Examples of possible

shapes: left bending, right bending, and whole body contraction.

Fig. 7 Block diagram for controlling Second Skin

pneumatic muscle can produce the maximum contraction rate and force of 25% and 100 N, respectively, under an air pressure of 570 kPa. More details about the design, fabrication, and characterization can be found in [8].

In addition to the strain sensor that measures local displacement of each pneumatic muscle, the IMUs detect the global shape of the device and are used for controlling the individual limb segment and joint angles of the user. Fig. 7 shows an example block diagram describing how the two sensors are combined to achieve a same goal. Once the reference is defined, the shape controller determines the bending angle and direction of the device. Then, each muscle controller determines the contraction rate of the muscle. The muscle control loops are nested in the shape control loop.

V. CONTROLLER NETWORK ARCHITECTURE A network of programmable embedded controllers controls the sensor/actuator modules in the Second Skin device. The controllers are connected in a two-dimensional mesh pattern, both spatially and logically. Global motion patterns of the device are carried out by scheduled tasks that run on the network of controllers. In this topology, each controller has four neighboring controllers with which it can directly communicate. The network forms the computation backbone that provides intelligent processing, interpretation, and timely detection of global patterns from real-time signals. Based on the sensor information, the controller network generates actuation plans in response to certain global patterns. Optionally, the network can also communicate to external devices to integrate with other computing infrastructure.

A. Inter-controller communication and synchronization The most fundamental communication primitive provided

by the Second Skin controller network is inter-controller serial communication. Each pair of directly connected controllers in the mesh can communicate with a two-wire serial protocol.

94

Fig. 8 Second Skin controller platform software architecture.

B. System software architecture Fig. 8 shows the Second Skin software architecture. The

system is divided into two main layers: local service layer, and the global service layer. The local service layer implements fundamental components that manage local resources. The global service layer implements higher level global algorithms. The local service layer includes a clock-driven real-time scheduler, establishes timing, handles inter-module communication with immediate neighboring modules, and issues commands to local sensors and actuators.

The global service layer specifies the application goal with the services provided by the local service layer. The local service layer is general purpose and is shared among multiple functionally different applications of the Second Skin, while the global service layer is customized for each application. We focus on the mechanisms provided by the local service layer.

C. Clock driven scheduling To provide a firm basis for coordinating controller

behavior to achieve various goals using the Second Skin device, we design the controller network to provide predictable timing. For this purpose, we adopt a clock driven scheduling approach to schedule software tasks at fixed time slots. As shown in Fig. 9, the system divides time into fixed width chunks, called rounds. Each round is further broken into five fixed-order stages: Sense, Fetch, Process, Actuate, and Emit. All computational tasks for the controllers are abstracted into these five stages. Such a basic structure for time division allows the simplification of control on timing of sensing, processing, and actuation tasks. The duration of each round and each stage are configurable and can be tuned to fit application-specific goals.

D. Scheduled controller communication Since each controller can only communicate with one neighbor at any instant, the controller network relies on a topology-aware scheduling scheme to carry out network-wide communication. The design leverages the two-dimensional structure of the network, as summarized below: • The controllers are divided into two groups: “odd node”

and “even node”. An odd node’s neighbor can only be an even node and vice versa.

• As shown in Fig. 10, odd nodes and even nodes follow matching schedules to communicate with each neighbor periodically. This method allows non-neighboring controllers to perform data communication in parallel.

Fig. 9 Timing diagram of node layer task scheduler.

Fig. 10 Neighbor-to-neighbor communication scheduling in each time step.

VI. CHALLENGES There are multiple engineering challenges that must be met

before our Second Skin may be implemented for assisting human body motions.

First, a robust control algorithm should be developed, which can combine strain sensor information and IMU information to achieve desired limb segment orientations. Although a simple example was provided in Section IV, further details of the control algorithm need to be defined for practical use because there may exist many different combinations of artificial muscle contractions that result in a single desired shape or joint angle. Also, the controller should be able to identify transition points in the gait cycle and appropriately change the configuration of the Second Skin as needed. Fig. 11 shows possible shape changes during a walking gait cycle.

Second, the device needs to be made of more skin-friendly and breathable materials while providing sufficient contact force to prevent slippage. We are currently developing a composite fabric-elastomer material for the next prototype.

Third, the pneumatic power source needs to be improved to be more portable. Although the current prototype uses a lab-bench air source, the next prototype will have lightweight compressed air canisters that the user can easily carry.

A remaining challenge for a device that shares control of personal mobility is fault-tolerance. Questions about fault tolerance with respect to mobility include (a) how the control network is able to distinguish that the body is "safe" or "in danger of falling" with respect to the behavior of the body center of mass during the gait cycle, (b) how second skin shape changes are limited only to those that the body may perform without injury, e.g., bending according to the anatomical limitations of a joint, and (c) how the ensemble of

95

Fig. 11 Possible shape changes of Second Skin during gait cycle: relax for toe-off, bending for midswing, opposite bending for heel strike, and stiffening

for midstance, respectively.

actuators respond if one or more of their number fails to provide their share of force production. The answer to all three questions is based upon the relation between information from multiple sensors and the parameter settings of what we envision as a higher order "interoperabilty" controller [17]. A fault-tolerant Second Skin will be able to distinguish safe gait from an imminent fall on the basis of sensor information about the location of the body center of mass with respect to parameters defining a "stability region". The device will be designed to keep the body inside the stability region, so thatthe interoperability controller directs the system to provide corrective force to stabilize the body. Shape change limitations are based upon continual sensor-based comparisons of the relation between detected body joint angles and the parameter settings of goal states that define the anatomical limits of the body (e.g., limitations on angular excursions). And finally, failure of an individual actuator may be detected from the strain information that is continually available from each sensor-actuator unit. The ensemble compensates for any loss of power due to a failing actuator by collectively changing their output to achieve the force required to reach the goal state established by the interoperability controller.

Before the Second Skin may be used in any human testing scenarios, it is important that its performance be characterized to confirm that the system unconditionally obeys appropriate constraints for the forces generated and resulting ranges of motion, as described above. Toward this goal, a test apparatus equipped with integrated force sensors, similar to the strain sensors incorporated within the Second Skin, and possessing degrees of freedom similar to those of a human limb, will be employed.

VII. CONCLUSIONS A wearable cyber-physical device with networked bio-

inspired modules, including soft pneumatic actuators and strain sensors embedded within elastomer material, has been developed. The device is equipped with multiple soft sensor-actuator modules that work collectively to produce a desired shape and force. The device is also equipped with IMUs to detect the global shape of the device for shape control.

Although the current prototype is still in a preliminary stage, to the best our knowledge, this is the first approach to build a modularized cyber-physical system that is soft and wearable for human assistance and rehabilitation.

ACKNOWLEDGMENT This work was supported by the Wyss Institute for

Biologically Inspired Engineering at Harvard University and National Science Foundation (NSF) grant CNS 0932015. The authors would like to thank James Niemi for his support and feedback, and Dava Newman for use of equipment used in the early exploration of supine kicking discussed.

REFERENCES [1] A. B. Zoss, H. Kazerooni, and A. Chu, “Biomechanical design of the

Berkeley lower extremity exoskeleton,” IEEE/ASME Trans. Mechatron., vol. 11, no. 2, pp. 128-138, 2006.

[2] J. F. Veneman, R. Kruidhof, E. E. G. Hekman, R. Ekkelenkamp, E. H. F. Van Asseldonk, H. van der Kooij, “Design and evaluation of the LOPES exoskeleton robot for interactive gait rehabilitation,” IEEE Trans. Neural Sys. Rehabil. Eng., vol. 15, no. 3, pp. 379-386, 2007.

[3] H. M. Herr and R. D. Kornbluh, “New horizons for orthotic and prosthetic technology: artificial muscle for ambulation,” in Proc. SPIE, vol. 5385, pp. 1-9, 2004.

[4] A. Roy, I. Kerbs, D. J. Williams, C. T. Bever, L. W. Forrester, R. M. Macko, and N. Hogan, “Robot-aided neurorehabilitation: a novel robot for ankle rehabilitation,” IEEE Trans. Rob., vol. 25, no. 3, pp. 569-582, 2009.

[5] D. P. Ferris, J. M. Czerniecki, and B. Hannaford, “An ankle-foot orthosis powered by artificial pneumatic muscles,” J. Appl. Biomech., vol. 21, pp. 189-197, 2005.

[6] Y.-L. Park, B. Chen, D. Young, L. Stirling, R. J. Wood, E. Goldfield, and R. Nagpal, “Bio-inspired active soft orthotic device for ankle foot pathologies,” in Proc. IEEE/RSJ Int. Conf. Intell. Rob. Syst., San Francisco, CA, September 2011, pp. 4488-4495.

[7] L. Stirling, C. Yu, J. Miller, R. J. Wood, E. Goldfield, and R. Nagpal, “Applicability of shape memory alloy wire for an active, soft orthotic,” J. Mater. Eng. Perform, vol. 20, no. 4-5, pp. 658-662, 2011.

[8] Y.-L. Park, B. Chen, C. Majidi, R. J. Wood, R. Nagpal, and E. Goldfield, “Active modular elastomer sleeve for soft wearable assistance robots,” Proc. IEEE/RSJ Int. Conf. Intell. Rob. Sys., Vilamoura, Portugal, October, 2012, pp. 1595-1602.

[9] M. Wehner, Y.-L. Park, C. Walsh, R. Nagpal, R. J. Wood, T. Moore, and E. Goldfield, “Experimental characterization of components for active soft orthotics,” Proc. IEEE Int. Conf. Biomed. Rob. Biomechatron. Rome, Italy, June 2012.

[10] Y.-L. Park, C. Majidi, R. Kramer, P. Berard, and R. J. Wood, “Hyper elastic pressure sensing with a liquid embedded elastomer,” J. Micromech. Microeng., vol. 20, no. 12, 2010.

[11] Y.-L. Park, B. Chen, and R. J. Wood, “Design and fabrication of soft artificial skin using embedded microchannels and liquid conductors,” IEEE Sens. J., vol. 12, no. 8, pp. 2711-2718, 2012.

[12] M. D. Dickey, R. C. Chiechi, R. J. Larsen, E. A. Weiss, D. A. Weitz, and G. M. Whitesides, “Eutectic gallium-indium (EGaIn): a liquid metal alloy for the formation of stable structures in microchannels at room temperature,” Adv. Funct. Mater., vol. 18, pp. 1097-1194, 2008.

[13] J. Favre, B. M. Jolles, O. Siegrist, and K. Aminian, “Quaternion-based fusion of gyroscopes and accelerometers to improve 3D angle measurement,” Electron. Lett., vol. 42, no. 11, pp. 612-614, 2006.

[14] J. Favre, B. M. Jolles, R. Aissaoui, and K. Aminian, “Ambulatory measurement of 3D knee joint angle,” J. Biomech. Vol. 41, no. 5, pp. 1029-1035, 2008.

[15] R. Brown and P. Hwang, “Introduction to random signals and applied Kalman filtering,” John Willey & Sons, New York, 1997.

[16] D. Young, S. D’Orey, R. Opperman, C. Hainley, D. Newman, “Estimation of lower limb joint angles during walking using extended kalman filtering,” 6th World Congress on Biomechanics (WCB). Singapore, 2010.

[17] I. Lee, O. Sokolsky, S. Chen, J. Hatcliff, E. Jee, B. Kim, A. King, M. Mullen-Fortino, S. Park, A. Roederer, and K. Venkatasubramanian, “Challenges and research directions in medical cyberphysical systems,” Proc. of the IEEE, vol. 100, pp. 75-90, 2012.

96

本文献由“学霸图书馆-文献云下载”收集自网络，仅供学习交流使用。

学霸图书馆（www.xuebalib.com）是一个“整合众多图书馆数据库资源，

提供一站式文献检索和下载服务”的24 小时在线不限IP

图书馆。

图书馆致力于便利、促进学习与科研，提供最强文献下载服务。

图书馆导航：

图书馆首页 文献云下载 图书馆入口 外文数据库大全 疑难文献辅助工具