The TAO Project: Intelligent wheelchairsfor the handicapped

Takashi Gomi

Applied AI Systems, lnc. (Add)340 A4arch Road, Suite 600

Kanata, Ontario, CANADA K2K 2E4E-mail: gomi@dpplied-,4Lcom

Home Page: ht~o://fox.nstn.ca/-aai

Abstract

An R&D project to build a series of intelligent autonomous wheelchairs is discussed. A standardized autonomymanagement system that can be installed on commercially available well-engineered power chairs has been developedand tested. A behavior-based approach was used to establish sufficient on-board autonomy at minimal cost and materialusage, while achieving high efficiency, maximum safety, Iransparency in appearance, and extendability. So far, the add-onsystem has been installed and tried on two power wheelchair models. Initial results are highly encouraging.

1. IntroductionIn recent years, with the concept of applying robots to servicetasks [Gomi, 92] and with the accelerated rate of aging of thepopulation being reported in many post-industrial countries,demand for more robotic assistive systems for people withphysical ailments or loss of mental control is expected toincrease. This is a seemingly major application area of servicerobots in the near future. For the past five years, we havebeen developing a range of autonomous mobile robots andtheir software using the behavior-based approach [Brooks,86] [-Maes, 92]. In my experience the behavior-basedapproach [Brooks 86, 91a][Steels, 93] [Pfeifer and Scheier,96] [Maes, 92] allows developers to generate robot motionswhich are more appropriate for use in assistive technologythan traditional Cartesian intelligent robotic approaches[Gomi, 96a]. In Cartesian robotics, on which mostconventional intelligent robotics approaches are based,planning for the generation of motion sequence andcalculation of kinematics and dynamics for each plannedmotion occupy the center of both theoretical interest andpractice. By adopting a behavior-based approach, I felt,wheelchairs which can operate daily in complex real-worldenvironments with increased performance in efficiency,safety, and flexibility, and greatly reduced computationalrequirements can be built at less cost. In addition,improvements in the robustness and graceful degradationcharacteristics were expected.In the summer of 1995, an autonomy management system fora commercially available Canadian-made power wheelchairwas successfully designed and implemented. The systemlooks after both longitudinal (forward and backward) andangular (left and right) movements of the chair as well limited vocal interactions with the user. The results wereexhibited in August 1995 at the Intelligent Wheelchair Event

organized by David Miller at the International JointConference on Artificial Intelligence (IJCAI’95) held Montreal Despite a very short period (a little over a month),the chair performed remarkably well at the exhibition.Encouraged bythe initial success, I implemented a three year

plan to develop a fully autonomous power wheelchair for useby people with various types and degrees of handicap. Theintelligent wheelchair project, now called TAO Project,intends to establish a methodology to design, implement, andtest an effective add-on autonomy management system for usein conjunction with most common commercially availablepower wheelchairs. In order to demonstrate the principle, theproject will build, during its life, an autonomy managementsystem for four well-established electric wheelchair modelscurrently available on the market throughout North Americaand Japan.In late 1995, a sister R&D company was established in Japanexclusively for the development of intelligent robotictechnologies for the disabled. With the initiative of this newR&D group, the development of TAO-2 autonomouswheelchair began in the spring of 1996.Based on the experience gained in the past year, methods

used and some issues related to the application of thebehavior-based approach to realize an intelligent wheelchairand possibly other assistive technologies are discussed.

2. Desirable characteristics of robots for the

handicapped2.1 BackgroundSince around 1992, AAI began a number of exchanges with

people with various handicaps and Individuals who assistthem. This was proceeded by a few years of on-goingInteractions with the handicapped community through

28

From: AAAI Technical Report FS-96-05. Compilation copyright © 1996, AAAI (www.aaai.org). All rights reserved.

marketing, installing, servicing, and training individuals on aspeech-to-text voice interface system for computers. Thisdevice proved to be effective for people with several types ofhandicap, particularly for individuals who had lost arm/handusage. Since late 1995, voluntary work has been attemptedby members of AAI at two institutions for the mobilityhandicapped in Japan: a senior citizen’s hospice for severephysical/mental problems, and an institution for people withsevere physical handicaps. A considerable amount of timepractising physical assistive work has been carried out bymembers of the R&D team, including the designer involvedin the conceptual design of the robots, engineers and atechnician responsible for the construction of the robots, andthe project manager and administrators of the roboticsprojects. In early 1995, an individual with a severe physicaldisability (a quadriplegie) joined AAI as a regular dataentry/bookkeeping clerk and as a future tester of autonomouswheelchairs.Based on these exposures, as well as earlier volunteer work

by myself and some of my colleagues, a preferable approachto robotics for service tasks [Gomi, 96b] and a tentative list ofdeskable characteristics for future robots built for the purposeof interacting directly with severely handicapped or fullydisabled individuals has been compiled. Some of thedesirable characteristics are discussed below.

2.2 Softness and flexibilityEstablishment of rapport between the handicapped personand the earegiver is essential for a care to be successful. Somuch so, there will be a great deal of anxiety in the mind ofthose treated by future robotized arms, support boards, andwheels. The need for set’mess realized at the physical interfaceof the end effectors of such a robot and the human bodysurface or limbs does not stop at simple padding of otherwisesolid effector surfaces, or use of softer materials, or passive oractive compliance of effectors. The softness must also bearchitectural in that the entire physical support structure mustbe able to alter, reconfigure, and even completely restructuremoment to moment reactions and responses to accommodate,whenever necessary, changes innot only the physical but alsothe perceived psychological situation of the user.The flexibility of the system as a whole, as well as that of theend effectors, must essentially come from this "structuralsoftness". The flexibility must be founded on the openness ofthe design of motions the system can generate so that it doesnot rely on fixed modes of operation or rigid scenariosdefined a priori. Humans in general and in mostcircumstances behave without a prepared set of motionpatterns, and since we are dealing with such an existence, aman-made system itself must not act with a fixed set ofmotions which are algorilhmieally describable. This places theappropriateness of most existing system control methods indoubt as a tool to seriously deal with many types of physieaUyhandicapped people.

Learning has often been hailed as a scheme with which a

system can be made more adaptable. We would also have toquestion this relishable notion as a candidate that wouldsufficiently increase adaptability of systems such as servicerobots dealing directly with humans. Learning schemes,particularly those so far studied to a great extent and depth insymbolic AI community, have failed to make significantcontributions to robotic systems operating in highly dynamicapplication areas. In general, learning research has focused onmethods to improve the chosen performance index of systemsbut variables involved in the scheme are most often notgrounded through sensors or actuators.

2.3 Fail safe and robustnessA robot arm holding a fragile human body cannot drop the

person when a bug is hit for the first time. The concept of failsafe implies readiness of a system against possible failure. Intraditional system engineering disciplines, such as FaultTolerant Computer Systems (FTCS) research and practice, thistypically translates into the preparation of additionalcapabilities in the form of a standby in computer hardware andsoftware. The concepts of hot-standby and cold-standby arecommonly employed in system design. Since it is impossibleto prepare for every possa’ole failure, the provision ofreadiness should exist, however, more in the form ofcapabilities spread across the system in atomic form andmeshed fine grain with the competence structure which alsofunctions in the normal execution of tasks. This is analogousto the way readiness to failure is implemented in life formsfound in nature. If a small animal or an insect temporarilyloses the use of a limb, it tries to adjust to the situation byimmediately enlisting the use of other limbs or even otherportions of the body. The additional capability readied in thisformwould be quickly organized and mobilized the momenta fault is detected.

2.4 Graceful degradationA cousin to the concept of fail safe, graceful degradation is

more important in systems that physically interface withhumans ~han in systems that deal with materials and artifacts.A conlrol system designed as a monolith or components withrelatively larger granularity would have less chance ofrealizing the concept fully. When one loses a limb, theresulting transition is not smooth, causing great suffering tothe individual However, as we lose a large number of braincells every day that we know won’t reproduce, we do notdeteriorate or lose capabilities as drastic as loosing a limb.Systems composed of fmer grain active units seem to offermore desirable results.

2.5 EvolvabilityAnother reason for the failure of learning in symbolic AI

would be the relatively short time the methods have typicallytried to achieve the"resulf’. In fact, we probably do not knowwhat desirable results are as much as we think we do. Both

29

shortcomings, this and the lack of grounding, are due mostlyto the very nature of being symbolic rather than pragmatic.In evolution, changes occur along a much longer time scale.In situated and embodied systems, such as life forms in natureand well-built autonomous robots, a search through a veryhigh dimensional space of the real world for adaptationdemands "experiments" on a vast number of combinations ofdimensional parameters, if such dimensionalization orparameterization make sense at all. Evolutionary Robotics(ER) is an emerging field of science and technology [Harvey,92], where physical or virtual robots’ autonomy structures areevolved to achieve collective trans-generational learning. ERseems to be a scheme that could well be applied to robotsoperating to tend and care for humans because of the opennature of human autonomy and ER’s basic principle that canprovide long term learning. Here, the concept of learningshould probably be replaced by a more comprehensiveconcept of evolution, which implies perpetual adaptation of anautonomous system to a constantly changing operationalenvironment rather than optimization of one or moreperformance indices of such a system.

2.6 The development planThe development of autonomous wheelchairs at AAI is

carried out in the following four phases. Some of the phasesoverlap in their execution.

(1) The basic safety phase,(2) The mobility phase,(3) The human interface phase, and(4) The exploration phase.

Currently, we are in the second phase of the project whichbegan on April 1, 1996. Prior to the start of the project on July20, 1995, a study was conducted to identify variousrequirements by potential users of the autonomous wheelchairboth in Canada and Japan through interactions with peoplewith various types of handicap. Causes of the handicaps wecame across included gradual mobility loss by aging, recentsudden loss of body control due to brain damage, andprolonged motion limitations and bodily contortion due tostroke suffered at ayoung age. The project continues to enjoycooperation from institutions for the handicapped andindividuals with disabilities. The TAO project is scheduled toend in the summer of 1998. For a description of thedevelopment plan, please refer to [Gomi and Ide, 1996].

3. Implementation of the first prototype, TAO-1A regular battery powered wheelchair (a motorized chair)

produced and marketed in Canada (FORTRESS Model 760V)was used as the base of the first implementation of theconcept. A set of sensors, a computerized autonomymanagement unit, and necessary harnesses were built andadded to TAO-1 (Figure 1) through the summer of 1995.

3.1. Planned functions of the chairThe selection of functions to be implemented on TAO-1 wassomewhat influenced by the rules for exhibition set out for theIJCAI’95 robotics contest. However, the later demonstrationsof our prototype and observations made at an institution forthe aged confirmed the guideline was in fact appropriate. Ofthe following functions which we now follow, only the fLrsttwo were attempted at our IJCAI’95 entry. However, all freeof them are currently pursued.

(a) Basic collision avoidanceThis is achieved by behaviors which monitor and respond to

inputs from on-board ted cameras or those which respond toactive infrared (IR) sensors. When the chair encounters obstacle, it fkst reduces its speed, and then depending on thesituation it faces, stops or turns away from the obstacle toavoid hitting it. The obstacle can be inanimate (e.g., a columnin a halhvay, a light pole on the sidewalk, a desk, a standinghuman) or animate (a passerby, a suddenly opened door in itspath, an approaching wheelchair). Encountering a movingobstacle, the chair first hies to steer around it. If it cannot, itstops and backs off if the speed of the advancing obstacle isslow enough (e.g., 20 centimeters per second). Otherwise, stays put until the obstacle passes away. Thus, if the chairencounters another wheelchair, both chairs can pass each othersmoothly as long as there is enough space in the passage fortwo chairs. A fast paced human usually does not affect thechair’s progress and at most causes the chair to temporarilyslow down or steer away.(b) Passage through a narrow corridorWhen surrounded by walls on each side of the path, as in a

hallway, the chair travels autonomously from one end to theother in parallel to the walls.(c) Entry through a narrow doorwayThe chair automatically reduces its speed and cautiously

passes through a narrow doorway which may leave only afew centimeters of space on each side of the chair. Some typesof ailment such as Parkinson’s disease or polio often deprivea human of the ability to adjust the joystick of a powerwheelchair through such a tight passage.(d) Maneuver in a fight comer

Similarly, when the chair is surrounded by obstacles (e.g.,walls, doors, humans), it is often dil~cult to handle thesituation manually. The autonomous chair should try to fmda break in the surroundings and escape the confinement byitself unless instructed otherwise by the user.(e) Landmark-based navigationTwo ccd color cameras on-board the chair are used for

functions explained in (a), (b), and (c) above. They constantlydetect the depth and size of free space ahead of the chair. Thecameras are also used to identify landmarks in theenvironment so that the chair can travel from its presentlocation to a given destination by tracing them. An on-boardtopological map is used to describe the system of landmarks.

3O

3.2 Hardware StructureAs a standard powered wheelchair, model 760V has two

differentially driven wheels and two free front casters.Although they are designed to rotate freely around theirvertical and horizontal axis, these casters typically givefluctuations in delicate maneuvers due to mechanicalhysteresis that exists in them because of design constraints(the rotating vertical shaft of the support structure of the castercannot be at the horizontal center of the caster). Thissometimes causes the chair to wiggle particularly when itsorientation needs to be adjusted f’mely. Such line adjustmentsare necessary typically when a wheelchair tries to enter anarrow opening such as a doorway.The entire mechanical and electrical structure, the electronics,

and the control circuitry of the original power wheelchairwere used without modification. The prototype autonomymanagement system still allows the chair to operate as astandard manually controlled electric wheelchair using thejoystick. The joystick can be used anytime to seamlesslyoverride the control whenever the user wishes even inautonomy mode.Physical additions to the chair were also kept to a minimum.AI components added to the chair were made visually astransparent as possible. Two processor boxes, one for vision-based behavior generation and the other for non-visionbehavior generation are tacked neatly under the chair’s seat,hidden completely by the wheelchair’s original plastic cover.Sensors are hidden under the footrests, inside the battery case,and on other supporting structures. Only two ccd cameras area little more visible: they are attached to the front end of twoarmrests for a good line of sight. A small keypad andminiature television set are installed temporarily over the leftarmrest to enter instructions and for monitoring.The non-vision behavior generator is based on a Motorola

68332 32-bit micro controller. A multi-tasking, real-timeoperating system was developed and installed as the softwareframewolk. This combination gave the system the capabilityto receive real-time signals from a large number of sensorsand to send drive outputs to the two motors which govern thewheels. The chair currently has several bump sensors and 12active infrared (IR) sensors which detect obstacles in closevicinity (less than 1 meter) of the chair. Signals from thecameras are processed by a vision-based behavior generationunit based on a DSP board developed by a group at MIT.Vision processing is discussed in Section 5.5 below.

3.3 Software structureThe over-all behavior structure of TAO- l is shown in Figure

3. Smaller behaviors are lumped up to save space on thediagram~ Software for the vision system is also built accordingto behavior-based principles. The major difference betweenthis and conventional image processing is that it consists ofbehaviors, each of which generates actualbehavior output tothe motors. It can presently detect depth and size of free space,

Figure 1 Autonomous wheelchair TAO-1. Cover is removedto show autonomy unit.

vanishing point, indoor landmarks, and simple motions up to10 meters ahead in its path. Indoor landmarks are a segmentof ordinary office scenery that naturally comes in view of thecameras. No special markings are placed in the environmentfor navigation.There are also a large number of behaviors invoked by IRsand bumpers which collectively generate finer interactionswith the environment Vision-based and non-vision behaviorsjointly allow the chair to proceed cautiously but efficientlythrough complex office spaces. Note that there is no mainprogram to coordinate behaviors.~tly, the autonomy program occupies about 35 KBytes

for all ofthe vision related processing and 32 KBytes for otherbehavior generation and miscellaneous computation. Of the35 KBytes for vision related processing, only about 10KBytes are directly related to behavior generation. The rest areinvolved invarious forms of signal preprocessing: generationofdepthmap, calculation of the size of free space, estimationof the vanishing point, and detection of specific obstaeles inthe immediate front of the chair.Of the remaining 25 KBytes, approximately 20 KBytes are

used in the neuralnetwolk system for detecting landmarks andreferencing atopological map. The current implementation ofthe landmark system consumes only 256 Bytes per landmark,although this figure may change in the future as moresophisticated landmark description might become necessary.The current system has space for up to 64 landmarks but thiscan also be adjusted in the future version.Of the 32 KBytes of non-vision processing (i.e., processing

31

of inputs from IR’s, bump sensors, voice I/o, etc.), again nomore than several KBytes are spent for generating behaviors.Altogether, there are some 150 behaviors in the currentversion of TAO-1. A considerable amount of code has beenwlitten to deal with trivial periphery, such as keypad interface,voice I/o, and LCD display. The comparable inefficiency ofceding is because these non-behavioral processing had to bedescribed in more conventional algorithms.

4. The second prototype, TAO-2Encouraged by the success of TAO-1, in late 1995 a sister

company of AAI (AAI Japan, Inc.) was established innorthern Japan. AAI Japan is dedicated to the development ofadvanced intelligent robotics to aid people with varioushandicaps. In May 1996, AAI Japan purchased anew powerwheelchair (Suzuki MC-13P), which is a model widely usedin Japan. MC-13P has a form of power steering in which thetwo front casters alter their orientation in synchrony with thedrive wheels when a turn is indicated by joystick. The servocontroller also halts the inside turn wheel of the two drivewheels while the chair is making a tight turn. This is asignificant departure from the way the FORTRESS modelmakes a turn. The latter simply turns the two differentiallydriven main wheels in opposite directions, allowing the chairto turn in place. The intent of providing a power steeringfeature on the Suzuki chair is obviously for ease of use, andthe user is freed from the wiggly caster problem descn’bedabove. However, this prevented the chair from making turnsin a tight turn circle. The feature was felt undesirable for anautonomous chair.Immediately following the purchase of the Suzuki chair, the

development team began building an autonomy managementsystem for TAO-2; a new prototype autonomous chair basedon MC-13P. The over-all computer hardware and softwarestructures as well as sensors are almost identical to those forTAO-1, except for a few changes listed below toaccommodate the above mentioned and other minordifferences in characteristics.(1) The behaviors responsible for turning TAO-2 needed

their parameters adjusted.(2) The locations of touch sensors made up of lhin piano

wires needed to be moved forward in order tocompensate for a larger turn circle.

(3) The back bumper was not activated since it was hardlyused. The difference in turning characteristics reducedthe chance of the Suzuki chair performing frequentswitch backs.

(4) Two prominent side bumpers were added to protectbystanders when the chair makes a turn in theirdirections. This was necessitated by the lack of structureon which to mount sensors.

TAO-2 is shown in Figure 2. It was fitted with the autonomymanagement system at AAI in Canada in the span of a week.After two days of testing, it was shipped back to Japan in time

for a public demonstration in the town of Kosaka, AkitaPrefecture.

5. Evaluation of the prototypes

5.1 DemonstrationsWhen TAO-1 was demonstrated at IJCAI’95 in Montreal onthe 22rid of August, itwas the 33rd day of the development ofthe first prototype. Everything from the motherboard, visionsystem, sensor arrangements and their harnessing, operatingsystem (based on an earlier prototype), a large number behaviors (some 60 by that time) were all developed and

Figure 2 TAO-2 autonomous wheelchair

tested in that period. The chair could perform functions (a)and (b) in Section 3.1 well and functions (c) and moderately well although they were not initially targeted.Function (e) was not yet implemented. In all, it performed well as other chairs at the exhibition most of which took muchlonger time to develop. All five functions are nowimplemented on TAO-1 and are undergoing continuousimprovement.TAO-2 was demonstrated on June 4, 1996 at a gymnasium ofa local school in Kosaka. The chair ran smoothly throughoutthe 1 hour demonstration persistently avoiding by-bystanders,other obstacles and the walls. Unsolicited, a severelyhandicapped spectator who could not even reach the joystickvolunteered to test ride the chair. The chair performed to hersatisfaction and excitement as it went through the gymnasiumamong a large number of spectators.The success of the two prototypes suggests that my intention

to build a standardized add-on autonomy unit is a valid one.The concept has at least been proven in two power wheelchairtypes which come from drastdcaI~ different backgrounds. Thedivergence in design philosophy and practical variances in

32

vlslon systemCameras

Vanlshrng po~t

LLCD ~Itor Cameral Keypad

’~"B’left~

"- -I IR~ ,~~Rll 1

B 4

12B3

--IR 10

B 2 -IR 3

B-right IR 9 B 5

Camera2

Sensor arrangement for TAO-1

Voice handler

Depth map

r~s - Front ovoid

~] Speaker

/~ Joystick r con~’o]ed)

~a I Motors I

@’°"’,

j/

¢

Figure 3 TAO-1 behavior structure (Not all behaviors arc shown)

implementation, some fairly significant, of a base powerwheelchair can be absorbed by relatively minor hardware andsoftware alterations made on the standardized add-on unit.TAO-2 also showed that the installation, testing, andadjuslment of a separately built autonomy unit can be made in

a very short period of time. In both TAO-1 and TAO-2, nocooperation fromthe manufactures was sought. In eaoh ease,charaotedsties of the joystick were studied and a seamlessinterface was designed around it.

33

5.2 Development timeThe extremely short development time required for the initial

prototype for both TAO-1 and TAO-2 can largely beattributed to the behavior-based approach. To achieve thedemonstrated level of mobility and flexibility would normallyhave required another several months to a few years inconventional AI-based mobile robotics. In behavior-basedrobotics, the operational characteristics of the sensors need notbe as precisely uniform as in conventional mobile robotics.For example, emission strength and angular coverage of theemitter, and the sensitivity and shape of the reception cone ofthe receptor of on-board IR sensors need not be homogeneousacross all sensors, allowing the use of inexpensive sensors andsimpler testing.All sensors, including the ccd cameras, need not be installedat precise translational and angular coordinates. They also donot need calibration. They were placed on the chair in arelatively ad hoc manner at First, and continually movedaround for better results as the development went on. In fact,the cameras and some of the sensors are attached to the chairby velcro detachable tape, so that their location and orientationcan be adjusted easily. Such loose treatment of sensors is notcommon in conventional robotics where robot’s motions arederived after high-precision measuremeuts of the relationshipsbetween its extremities and environment. The large tolerancefor signal fluctuation is due also to flexibility of processingand greater adaptability inherent in Subsumption Architecture[Brooks, 86].With the absence of "sensor fusion", sensor inputs are

directly linked to motor output only with a simple signaltransformation and amplification (e.g., from sensor outputvoltage to motor drive current). The developer only needs to

adjust the appropriateness of the definition and performanceof the sensor-action pair or behavior in terms of its outputwithout a detailed and precise analysis of input signalcharacteristics and elaborate planning and computation ofoutput signal generation. Readers not familiar with thetheoreticalbasis of behavior-based AI are encouraged to read[Brooks, 91b]. These theories are fuUy put into practice in ourdevelopment.

5.3 Software structureDuring development, sensor-actuator pairs or behaviors aresimply "stacked up". They are added to the system one by onewithout much consideration for the design of the over-allsoftware structure. Our operating system provided anadequate framework for the incremental development processallowing for shorter development time.Thus, software development went totally incrementally side

by side with finer adjustment of the sensors. Only generalfunctions needed assigned to each sensor-actuator pair typefirst. For example, depth map - motor pairs are excellent fordealing with obstacles that suddenly appear in the path of thechair a few meters away. But the same sensor-actuator pairtype is not at all effective for the management of the situationin which the chair has actually made physical contact with anobstacle.Sometimes, competition or contradiction occurs between two

or more behaviors. Such contradicting definitions of behaviorareinmost cases easily observable and corrected quickly. Anexample of more complex contradiction occurs when two IRcollision-detection sensors placed on left and right front sidesof the chair detect an approaching doorway in quicksuccession~ Since the doorway is normally quite narrow, the

.... : of fne bul[d~

m

.... ~iiiiiiii~ii~::~:::::: ::iiiiiiii~iiiii~iiiiiif:~ ....................~iiiiii:J~:: "%~iiiiiiiiiiiiiiiiiiii ........

============================== cow .... ~iiiiii~i ~::::::

.... ::::i::iii::iiiiiiiiiiiiiii::i::i~ ...... ~#i::i::i::i~::~:: o9

%iiiii® =============================================================================================== ....~~ _

Figure 4 The office space which contains the test loop

34

reflection of infrared signalsreceived by these sensors isusually strong enough to causethe chair’s immediate evasiveaction. As both sensors reactalternatingly, the chair can getinto an oscillatory motion,commonly known as"Braitenburg’s oscillation" after[Braitenburg, 84]. In thisspecific situation, otherfrontally-mounted IR sensorstake in "just go ahead" signalsthat invoke behaviors which canbreak the tie.

t~a:

200

0200

0

2O00

200

0

2000

200

0

IR4

".51--~j1 ~jAl~l’~’--°---~e ~----~j~l~ flvI R3

_IR2

IR1I I I

(11 (21 (3) (4) (5) (6)

Figure 5a Output of active infrared OR) sensors

5.4 Priority schemeThe priority arrangement is shown in top fight comer of

Figure 4, where several output lines to the motors are joinedby O nodes or suppression nodes. Input from the left of thenode is suppressed and replaced by one coming in verticallywhenever it is active. Inputs from the joystick take the highestpriority in deciding which action the chair should take. Theelectronically and mechanically seamless interface betweenthe joystick controller and the autonomy management systemallows the chair to nm as a standard power wheelchair simplyby operating the joystick. The second highest priority is givento behaviors which take in signals from left and fight bmnpersand some key frontal IR sensors. Behaviors are bundled upin Figure 4 with implied logical relationships among inputlines to simplify the diagram. After several groups ofbehaviors that mostly depend their invocation on signals fromIR sensors, behaviors invoked by signals from the voice inputsystem are arranged next, followed by vision-driven behaviorsas the lowest priority behavior groups. They are, indescending order of priority, depth map, vanishing point, andfree area.Figure 5a shows IR signals from a test run in which TAO-1

went around the test loop in our office floor shown in Figure3 (shaded area). Note that signals from only 6 of the 12 sensors are plotted here. The x axis in Figures 5athrough 5d shows the passage of time and itslength corresponds to the time required tocomplete the loop from the workshop where thechair began its run and back there counter-clockwise. Note that checkpoints (1) through (6)shown in Figure 3 are also marked on thediagrams. When there is no reflection from anobstacle, output of an IR is kept at 255.Depending on the strength of the reflected signal,a receptor may report lower values, 0 being thelowest. When the value becomes less than athreshold, the sensor would have "detected an obstacle." Thethreshold is set as a function of speed of the chair, and in thisspecific test set at 210, 180, and 150, for when the chair is

running fast, medium, and slow speed, respectively. In anothermode of obstacle detection using an IR, changes in value aremonitored for several sensor cycles. If the change issufficiently large, detection of an obstacle is reported. The IRsensors take invalues at 64Hz and several consecutive valuesare compared. Once invoked, a behavior corresponding to aspecific IR sensor generates a predetermined reactive motion,altering the speed and orientation of the chair.

5.5 Vision processingInputs from 2 cc, d cameras are alternatively processed

through a single flame grabber into two primary vision planesof 256 x 128 pixels each at about 8 frame sets per second.Images in these primary vision buffers are averaged down to64 x 32 pixel secondary vision plane by combining the leftand right vision inputs after dividing each primary plane intoleft, center, and fight. All vision processing described belowoccur using image data in this secondary visual plane.

Figure 5b plots three depth values (Y1, Yo and Yr) in termsof the number of pixels in the secondary visual planedetermined according to HorswiU’s habitat constraint visionprocessing [Horswill, 92]. In the absence of active bumperand IRinvoked behaviors, the parameter set directly dictatesthe orientation and speed of the wheels.

[3 o-5

IIIllll NIlJl

111 121

[] 5-10 ¯ 10-15 [] 15-20 [] 20-25 [3 25-30

liilli llt ll(31 (41 (51

# pixies

(6)

Figure 5b Depth map parameters from the vision subprocess

Output from the vanishing point detector of the vision systemis shown in Figure 5c. The detector attempts to find avanishing point in the secondary visual plane and outputs its

35

X

"6Q.t:De-°_e-toCt~

>

t-O

OIX.

x axis value when it finds one. The value 0 corresponds to theleft most angle in the visual plane, and 63 to the right most.When it fails to come up with a vanishing point, value 99 isoutput. The combined horizontal viewing angle of the left andthe right cameras is approximately 100 degrees.

lOO

9o

8o

70

6O

5O

4O

30

20

lO

0 -

~C~O 00~ ~ O O~

Bestfit lines sloped same way -vanishing point set to appropriate

edge

(1) (2)

Invalid Vanishingpoint- use Area to

~:turn

Mid point @

0 0

O

O O0 ~ zt~D

(3} (4} (5)

domestio tranaportation needs.In April 1996, TAO-1 was brought to a local shopping mall inOttawa to freely roam around for an hour or so. TAO-1skilfully skirted all internal structures of the mall such asesoalators, flower pots, benches, signs, and showcases, as well

as allemoun shoppers. TAO- 1and its rider visited stores as if

¢~> he was window shopping orjust strolling the mall. Virtuallyall fellow shoppers failed tonotice that it was not drivenmanually. This made me feelthat with proper engineering to

0 make the chair furthero ~>o dependable, it can already

OO o~oO serve as an autonomous chair

in limited application areas,

O such as strolling or window~> shopping for the severely

t6~ handioapped.After the modest

demonstration of TAO-2 inJapan, which was reported in

local television news and several looal newspapers, we havereceived inquiries for the chair’s availability. Needless to say,it would be at least a few more years before even a modestlyautonomous chair can be released for use by the handicappedat large and put into daily use only with affordable amount ofsupport. Maintenance would be another issue if we proceed,not to mention various public liability issues that,unfortunately but undoubtedly, foUow. I am not at alloptimlstio about the efforts required to establish aninfrastructure for physical and moral support thatencompasses all these and other yet to be found issues.Nevertheless, I can foresee that we will be able to answer, inthe near future, some of the sincere wishes that already comefrom people who would be most benefitted by the technology.

Figure 5c Output of vanishing point detector

Figure 5d depicts output from the area deteetor. The numberof pixels representing free space in the left, center and rightvisual fields are calculated by the detector, and steering andspeed of the chair are determined by the size of availablespace as in depth map processing. The behaviors associatedwith area detection are invoked only when all other behaviorsare not invoked.As the project proceeds the vision system will be enhancedto detect more objects and events such as outdoor landmarks,indoor landmarks that change in time, more complex anddynamic obstaoles, and traffic signals in the path. In the fallof 1996, a popular American wheelchair will be fitted with theautonomy management system to become TAO-3.

[] 0-200 [] 200-400 ¯ 400-600

11) 12~ 131

# pixels

141 151

~left

Figure 5d Output of the area detector

6 Lessons learned so far from the chair projectAlthough the experience is stiU very limited, I can state that

there is a strong expectation among the population for thedevelopment of an autonomous wheelchair for assisting andeventually fully taking care of the handicapped person’s

Getting into technioal issues, the list of thingsyet to be done is still quite long. Landmarkdetection, for example, requires a lot morework. Although we have su~ed innavigating the chair to go through a series oflandmarks arbitrarily chosen in the chair’spresent operational environment, this is still afar cry from being able to state that it can runfreely in any environment traversable by awheelchair by detecting landmarks.Apart from these and other shortcomings, I feel

the technology as it is, is already useful in realworld applications by individuals with certain types ofhandicap. Persons with bodily contortions such as those whosuffered polio in earlier life, or individuals with involuntaryhand/ann movements such as patients of Parkinson’s disease,now could travelthrough confined and narrow spaces such ascorridors and doorways without assistance. Other interface

36

mechanisms such as neck control and voice recognizer wouldalso make the inlroduction of the autonomous chair easier.Less handicapped users can use the chair as a manual powerwheelchair whenever desired, while autonomy managementcan assist in mundane situations and emergencies.Everybody we have interfaced with so far, from a passer-by

at a shopping center where TAO-1 was tested, fellow roboticsresearchers, several handicapped people and caregivers whoheard about the project and came to see and even volunteeredto try an early prototype, willing investors, to journalists allgave us positive feedback. They agree in principle thatmobility should be provided as much as and as soon aspossible to those who otherwise are not capable of going toplaces by themselves. Although the development is still farfrom complete, TAO-1 and 2 have so far been covered byseveral TV programs and a few dozen newspaper andmagazine articles in Europe, Japan, USA, and Canada,indicating the keen level of interest the public has on thissubject.I wish to report our next interim results at IJCAI’97 (Kyoto,Japan) and other occasions around that time so that resultscan be shared among researchers and the concept of theautonomous wheelchair which is manually overridablebecomes familiar to the population at large. Needs of the usersand capabilities that could be provided by the technology arematched and better understood through frequent disclosuresthrough technical papers, presentations, and serious mediacoverage.

7. ConclusionsTwo prototype autonomous wheelchairs based on

commercially available motorized wheelchairs have been builtusing behavior-based AI. The initial prototyping went veryrapidly and the size of the software is significantly smallerthan control programs for similar vehicles operating in realworld environment implemented using conventional AI androbotics methodologies. One of the chairs is now capable oftravelling to its indoor destinations using landmark-basednavigation~ The performance of the prototypes indicates thereis a cautious possibility today to build a functional intelligentwheelchair that is practical and helpful to people with certaintypes and degrees of handicap.

AcknowledgementsKoichi Ide is responsible for the detailed design and most of

the implementation of both TAO-1 and TAO-2. ReubenMartin and Richard Edwards assisted Ide in implementationand testing of both chairs. Ann Gfifl~th proofread earliermanuscripts of this paper and aided the author in editing.

[Brooks, 86] 1L A. Brooks, "A Robust Layered ControlSystem For a Mobile Robot", 1EEEdournal of Robotics andAutomation, RA-2 (1986).[Brooks, 91a] 1L A. Brooks, "New Approaches to Robotics",Science 253, 1991, pp 1227-1232.[Brooks, 91b] 1LA. Brooks, "Intelligence Without Reason". InProc. IJCAI-91, Sydney, Australia, August 1991.[Gomi, 96a] Takashi Gomi, "Non-Cartesian Robotics",Robotics and Autonomous Systems 18, 1996, Elsevier, pp169-184.[Gomi, 96b] T. Gomi, "Aspects of Non-Cartesian Robotics"in proc. Artificial Life and Robotics (AROB’96), Japan,February 1996.[Gomi, 92] T. Gomi, "Subsumption Robots and theApplication of Intelligent Robots to the Service Industry",Applied AI Systems, Inc. Internal Report, 1992.[Gomi and Ide, 96] T. Gomi, T., and K. Ide, "TheDevelopment of an Intelligent Wheelchair". In proc. IVS’96,September 1996, Tokyo, Japan.[Gomi et al, 94] T. Gomi, K. Ide, H. Matsuo, "TheDevelopment of a Fully Autonomous Ground Vehicle(FAGV), Intelligent Vehicles Symposium (1V’94), Pads,France, October 1994.[Harvey., 92] I. Harvey, "Evolutionary Robotics and SAGA:the case for hill crawling and tournament selection", InArtificial Life 3 Proceedings, C. Langton, editor, Santa FeInstitute Studies in the Sciences of Complexity, Proc. VolXVI, pp 299-326.[HorswiU, 92] I. Horswill, "A simple, cheap, and robust visualnavigation system", Simulation of Adaptive Behavior(SAB ’92), Honolulu, Hawaii, 1992.[Maes, 92] Pattie Maes, "Behavior-based ArtificialIntelligence", Second International Conference on Simulationof Adaptive Behavior (SAB’92), 1992.[Pfeifer and Scheier, 96] 1L Pfeifer, C. Scheier, Introductionto "New Artificial Intelligence", Ins3itut fttr Informatik derUniversit/it ZtLrich, July 1996.[Steels, 93] Luc Steels, "Building agents out of autonomousbehavior systems", The Biology and Technology of IntelligentAutonomous Agents, NATO Advanced Study Institute,Trento, 1993, Lecture Notes.

References[Braitenberg, 84] V. Braitenberg, "Vehicles: Experiments InSynthetic Psychology", MIT Press, Cambridge, MA. 1984.

37