Control of Multiple Small Surveillance Robots at AAAI 2000

Paul E. Rybski, Dean F. Hougen, Sascha A. Stoeter, Maria Gini and Nikolaos PapanikolopoulosCenterfor DistributedRobotics,Departmentof ComputerScienceandEngineering

Universityof Minnesota�rybski,hougen,stoeter,gini,npapas � @cs.umn.edu

Abstract

We describeour experiencesdemonstratinga teamof minia-ture robotsengagedin a surveillancemissionat the AAAI2000Mobile RobotsExhibition. We presentthearchitectureandthemain featuresof our systemanddiscusssomeof theenvironmentalfactorsthataffect its performance.

IntroductionReconnaissanceandsurveillancetaskscanbenefitfrom theuseof multiplesmallyethighly capablerobots.Suchrobotsmustbe easilydeployableandableto move efficiently yettraverseobstaclesor uneven terrain. They mustbe abletosensetheirenvironment,actontheirsensing,andreporttheirfindings.They mustbeableto becontrolledin acoordinatedmanner.

We have developeda setof extremelysmall robotic sys-tems,calledscouts(Hougenet al. 2000a),which werede-signedspecificallyto meettheserequirements.The scoutrobot, shown in Figure 1, is a cylindrical robot 11cm inlengthand4cm in diameter.

Scoutshave two modesof locomotiondesignedto trans-port themoverdifferentkindsof terrainandobstacles.In thefirst mode,the scoutsusetheir wheelsto roll over smoothsurfaces(evenupa20degreeslope).Whenconfrontedwithan obstacletaller that itself, the scoutemploys its secondlocomotionmode,the jump. The spring-loadedtail of thescoutis compressedand releasedto propel the robot overobjectsupwardsof 20cm in height.Figure2 shows a scoutjumpingovera barrier.

Scoutscarry a small video cameraandvideo transmitterwhich they useto captureinformationabouttheir environ-ment. They canalso transmitandreceive digital informa-tion over a separateRF communicationssystemwhich usesa packet-basedcommunicationsprotocol.

The small sizeof the scoutsprovidesmany advantages.They areinexpensiveandeasilytransportable,whichmakesthemideal for usein large teams. This allows themto bepresentthroughoutawidearea,formingamobilesensornet-work. It alsoallowsindividualscoutsto beexpendablewith-out jeopardizinganentiremission.scoutsarewell suitedtoclandestineoperationssincethey can be concealedeasily.

Copyright c�

2000, American Associationfor Artificial Intelli-gence(www.aaai.org). All rightsreserved.

Figure 1: Two scout robots (shown next to a quarterforscale). c

�2000by ACM, appearedin Rybskiet al. (2000).

Their small sizeandcylindrical shapealsoallows themtobedeployedby launchingor throwing by hand.

Related WorkAutomatic security and surveillance systemsusing cam-erasandothersensorsarebecomingmorecommon.Thesetypically use sensorsin fixed locations,either connectedad hoc or, increasingly, through the sharedcommunica-tion linesof “intelligent buildings” (Porteous1995). Thesemaybeportableto allow for rapiddeployment(Pritchardetal. 1998)but still requirehumaninterventionto repositionwhennecessary. Thisshortcomingis exacerbatedin casesinwhichthesurveillanceteamdoesnothavefull controlof theareato beinvestigated,ashappensin many law-enforcementscenarios.Staticsensorshave anotherdisadvantage.Theydo not provide adaptabilityto changesin the environmentor in thetask. In caseof poordataquality, for instance,wemight wanttheagentto move closerto its targetin ordertosenseit better.

Mobile roboticscanovercometheseproblemsby givingthe sensorwheelsand autonomy. This researchhas tra-ditionally focusedon single, large, independentrobotsde-signedto replaceasinglehumansecurityguardashemakeshis rounds(Kajiwaraet al. 1985). Suchsystemsarenowavailablecommerciallyandarein placein, for example,fac-tory, warehouse,andhospitalsettings(Kochan1997),and

Figure2: A scoutjumpingovera barrier(sequencestartsfrom theupperleft corner). c�

2000by IEEE,appearedin Hougenetal. (2000a).

researchcontinuesalongtheselines(Dillmann et al. 1995;Osipov, Kemurdjian,& Safonov 1996). However, the sin-gle mobile agentis unableto be many in placesat once—oneof the reasonswhy securitysystemswere initially de-veloped.Further, largemobilerobotsareunableto concealthemselves,which they mayneedto do in hostileor covertoperations.They mayalsobetoolargeto exploretightareas.

Multiple robots often can do tasks that a single robotwould not be able to do, or can do them fasterand morereliably, asdescribedin theextensive survey by Cao,Fuku-naga,& Kahng (1997). Taskstraditionally accomplishedwith multiple robotsareforaging(Mataric 1997),formationmarching(Balch & Arkin 1998),mapmaking(Burgardetal. 2000),janitorial service(Parker 1996),security(Everett& Gage1999),andexploration(Hougenet al. 2000b).

Traditionally, mobile robots have rangedin size fromroughly dog-sizedto somewhat larger than a human. Forsmall robots,Lego blocksandmicroprocessorboards,suchastheHandyboard(Martin 1998),areoftenusedto quicklyprototyperobots. We have designedmultiple Lego-basedrobotsandusedthemfor avarietyof navigationtasks(Ryb-ski etal. 1998).

Recently a significant interest has arisen in designingeven smallermobile robotsfor exploration and reconnais-sance.TheseincludethepopularKheperarobot(Mondada,Franzi, & Ienne1993) and other prototypes,suchas AL-ICE (Caprarietal. 1998)andourown scouts(Hougenetal.2000a),developedby variousresearchgroupsandnot yetcommerciallyavailable.

The major challengein designingminiaturerobotsis infitting all themechanicalpartsandelectronicsinto a limitedvolume and in designingan adequatemethodof locomo-tion. Most miniaturerobotshave wheels(Baumgartneretal. 1998),but somecanjump (Halme,Schonberg, & Wang1996),roll (Chemel,Mutschler, & Schempf1999),fly (Wu,Schultz,& Agah 1999), or swim (Fukuda,Kawamoto,&Shimojima1996). Our miniaturerobotshave two rollingwheelsbut they canalsojump to gooversmallobstacles,asshown in Figure2.

Due to their small size,mostminiaturerobotsuseproxyprocessing,asin Inabaetal. (1996),andcommunicatevia aradiolink with theunit wherethecomputationis done.Lim-ited communicationbandwidthbecomesa problem whenrobotshave to transmitlarge amountsof data,suchaslivevideo,andwhenmany robotsneedto sharethebandwidth.Power consumptionis anothermajorproblemwhich limitsthemobility, ability to communicate,andlong termsurviv-ability of miniaturerobots.

Scout Control ArchitectureThescout’s smallsizeandlimited power supplygreatlyre-strictsthekindsof on-boardprocessingthatcantake place.The only computationsdoneon the scout’s two 8-bit mi-croprocessorsarewhat’snecessaryfor communicationsandactuation.All high-level decisionmakingandvideo imageprocessingmustberelegatedto anoff-boardworkstationorahumanteleoperator.

Autonomousdecisionprocess,suchasplanners,reactivebehaviors,or teleoperationcontrols,sendlow-level actuatorcommandsto the scoutsthroughan RF transceiver. Eachscoutis assigneda uniquenetwork ID which allows pack-ets to be routedto the correct robot. By interleaving thetransmissionof packetsdestinedfor different robots,mul-tiple scoutscanbe controlledsimultaneously. The currentbottleneckin thesystemis thenumberof packetspersecondthatcanbetransmitted.Thefrequency at which commandscanbebroadcastis currentlybetween5 to 8 persecond.Thelimiting factoris thespeedatwhichthescoutcandecodethemessagesfrom theradio.

Scoutvideo datais capturedby a video receiver. Thiscanbeeitherdisplayedon a monitoror fed into a digitizer.Becausethe video is a continuousanalogstream,only onerobotcanbroadcastat any instant.Signalsbroadcastsimul-taneouslyfrom multiple robotswould interferewith eachother, makingbothsignalsuseless.Scoutsrunningin paral-lel mustinterleave their video transmissionsandlimit theirtransmissionsto shortbursts.

We have developeda distributed,CORBA-based(Group

1998)architectureto coordinateall of the individual hard-ware� resourcesin our system. Accessto robotic hardwareandothercomputationalresourcesis controlledthroughpro-cessescalledRESOURCE CONTROLLERS (or RCs). Everyphysicalresourceis givenits own RC processto manageit.Behaviors and decisionprocessesconnectto theseRCs tosendcommandsandreceive datafrom them.Behavior pro-cessesrunningon differentcomputerscanaccessall of theRCs without evenbeingawarethat they mayberunningondifferentcomputers.

Scout BehaviorsThe scoutsarecapableof operatingautonomouslythroughtheuseof simplebehaviors. Theonly environmentalsensoravailableto the scoutis its video camera,the useof whichpresentsseveral problems. First, the scout’s proximity tothe floor severely restrictsthe areait can view at a time.Secondly, sincethe video is broadcastover an RF link toa larger robot or a workstationfor processing,the qualityof thereceivedvideooftendegradesdueto of rangelimita-tions,proximity of objectsthat interferewith transmission,and poor orientationof the antennas.Figure 3 shows anexampleimagereceived from the scout’s camera.The RFnoisedegradingthis imageincreasesthedifficulty of distin-guishingtheobjectsfrom thebackground.

Figure3: Theworld from thescout’spointof view. Herethescoutis viewing alabbenchandtwo chairsatarangeof 2m.c

�2000by ACM, appearedin Rybskietal. (2000)

A numberof differentvisually-basedbehaviorshavebeenimplemented;seeRybskiet al. (2000)for the full list. Thebehaviors demonstratedat the AAAI 2000 Mobile RobotExhibitionwere:

Drive Towards Goal: Identifying a dark areato move to-wardsis a simplematterof scanningacrossthe imageata fixed level on or aboutthe level of thehorizonandde-termining the horizontalposition of the darkest areainthe image. The meanpixel valuesin a set of overlap-ping windows in the imageare determined. The scoutselectsthe darkest window and drives in that direction.

The scoutknows that it shouldstopwhen its cameraiseitherpressedup againsta darkobject,or thescoutis inshadows.Scoutmotionin thisbehavior is continuousandthescoutdoesnot checkto seethatit hasmovedby com-putingdifferencesbetweenframes.Otherkindsof motionmakesuseof thischeckbecausethescoutdoesnotalwaysreceive the commandssentfor it dueto RF interference.Framedifferencingis nothelpful in thisbehavior becausethescoutis unableto move very quickly. Thedifferencebetweensubsequentframescapturedduringforwardmo-tion is minimal, making it very difficult for the robot todetectforwardmotion.

Detect Motion: Detectingmoving objectsis accomplishedusingframedifferencing.Oncethescouthasbeenplacedin a single location, it mustdeterminethe quality of thevideo andseta thresholdto filter out RF noise. This isaccomplishedby doing imagedifferencingon a streamof videoandincreasingthedifferencethresholduntil RFnoiseis filtered out. The scoutthensubtractssequentialimagesin the video streamand determineswhetherthescenechangesat all (causedby movementin theimage).

Demonstrations at the Exhibition

At the AAAI 2000 Mobile Robot Exhibition, we demon-stratedseveral aspectsof our distributedscoutcontrol sys-tem on two networked laptop computers.The first laptopran Linux and handledall of the scoutcontrol programs.The secondran Windows and handledthe video process-ing hardwareandsoftware.Thescout’scommandradiowasconnectedto the Linux laptop while a video receiver wasconnectedto the Windows laptop. In addition to the au-tonomousbehaviors definedabove, a simple teleoperationclient wasusedwhich alloweda humanoperatorto drive ascoutand operateits sensorpayloadand jumping mecha-nism. Figure4 illustratesthe implementationof the scoutcontrolsystem.

CommandRadio

AutonomousBehavior

TeleoperationConsole

Radio RC

Framegrabber RC

VideoReceiver

PCMCIAFramegrabber

VideoSignal

TransmittedCommands Scout

Robots

Linux Laptop

RS232 Serial Port

Ethernet

Windows Laptop

Figure4: Diagramof thehardwareandsoftwarearchitectureusedto controlthescoutrobots.

Several RCs were usedby the behaviors and teleopera-tion consolesto control the scouts.The first, andarguablythemostimportant,RC is in chargeof controllingaccesstotheradio.Theuserinterfaceconsolesandbehaviorshave toconnectto the radio RC’s CORBA interfaceto sendpacketrequeststo it. The radio RC supportstwo kinds of com-mandpacket requests.In thefirst requesttype,thedecisionprocessmustexplicitly instructtheradioto sendeachcom-mand. If commandsare sentto the radio fasterthan theycanbetransmitted,they arestoredin aqueueuntil theradiocanhandlethem. In the secondrequesttype, the decisionprocessgivestheradioa singlecommandandinstructsit torepeatit asoften asit can. This allows the systemto keepinter-processcommunicationsto aminimum.If multiplede-cision processessetup repeatedcommands,the radio willscheduletransmissionof eachin a round-robinfashion. Ifthequeueof non-repeatedcommandshaspendingrequests,thequeuewill alsobeaddedto theschedule.

ThesecondRC is the framegrabberprocess,which is re-quired for autonomousbehaviors. Imagesarecapturedbytheframegrabberon theWindows laptopandareprocessedbasedon the requestsof the decisionprocess.Several dif-ferent imageoperatorswereimplemented,including framedifferencing,connectedregionextractionandvariousstatis-tical operators.To cutdown on theamountof network over-head,all imageprocessingwasdoneby theframegrabberRCandonly theprocessedimageinformationwaspassedbackto thedecisionprocess.

At the exhibition, the scoutrobotsweredemonstratedinboth teleoperatedandautonomousmodes. In the teleoper-ation demo,we showed the rolling and jumping mobilitymodesof the robot,aswell asdemonstratingits usefulnessasa remotecamera.In the autonomousdemos,the scoutsserved asmotion detectorsby transmittingvideo by doingframedifferencingoperationsandnotifying auser’sconsolewhenevermotionwasperceivedin theimageframe.Wealsodemonstratedthe scout’s ability to servo to a dark object.For this behavior, imageswereanalyzedto find thedarkestmeanvaluein the robot’s visual field. The robot wascom-mandedto drive itself towardsthedarkestobjectsandkeepthemcenteredin its visualfield.

Discussion & Lessons LearnedBecausethe scoutrobotsarecompletelydependenton thetwo RF communicationschannelsfor operation,any inter-ferenceon thosefrequenciescanseriouslydegradeperfor-mance.Becausethe commandRF channelis packet-based(asopposedto streaming),it is somewhat resilient to localinterference.The video channel,on the otherhand,trans-mits continuousanalogdatawhich is verysensitive to inter-ference.Corruptionin the video signalcanmake it nearlyunusableby autonomouscontrol processes.The frequencythatwe usefor ourvideotransmittersis the900MHzindus-trial/medicalbandandis, unfortunately, extremelypopular.Severalothergroupsat theexhibition usedthis frequency totransmitvideo(mostnotablytheteleoperatedblimp usedbytheBotBall competition).We hadto work out a schedulingarrangementwith the organizersof the exhibition to sharethecommunicationsband.

Thesmall physicalsizeandbatterysupplyof our robotsseverely limits thekind of RF transceiversthatwe canuse.We arealsoseverely limited in the kinds of video capturedevicesthatwe canuse.In our currentdesignof therobots,digital cameraswerenot anoptionbecauseof lack of avail-ablehardwaresmall enoughto fit into our robot aswell aslackof computationalpower to processanimageevenif wehadsucha camera.More attentionwill have to be paid tothis issuein future scoutdesigns. Onesolutionwill be tosacrificerobot size in order to include additionalor morepowerful computationalresources.Somesort of a spread-spectrumor frequency-hoppingradio systemwould be ex-tremelyusefulaswell. Not only would thehigherfrequen-ciesinvolvedallow for fasterdatatransmission(critical forthe transmissionof video data), the ability to changefre-quencieswould allow more robotsto operatein the sameareaandwould bemorerobustto interference.

We hopeto beableto addresssomeof theseissuesin thefuture as advancesin technologymake betterand smallerequipmentmoreavailable.Until thishappens,wearefocus-ing oureffortsonalgorithmsfor bettervideosignalprocess-ing as well as decisionprocesseswhich take the inherentfragility of the RF communicationschannelsinto accountwhencontrollingtherobots.

AcknowledgementsWe would like to acknowledge the University of Min-nesota’s DoctoralDissertationFellowship programfor par-tial supportof this researchandthe AmericanAssociationfor Artificial Intelligence for the AAAI Robot Competi-tion andExhibitionScholarshipthatallowedusto bringourrobotsto AAAI 2000in Austin,TX.

This materialis baseduponwork supportedby the De-fenseAdvancedResearchProjectsAgency, MicrosystemsTechnologyOffice(DistributedRobotics),ARPA OrderNo.G155,ProgramCodeNo. 8H20, Issuedby DARPA/CMDunderContract#MDA972-98-C-0008.

ReferencesBalch,T. R.,andArkin, R.C. 1998.Behavior-basedforma-tion controlfor multiagentrobotteams.IEEETransactionson RoboticsandAutomation14(6):926–939.Baumgartner, E.; Wilcox, B.; Welch, R.; and Jones,R.1998.Mobility performanceof asmall-bodyrover. In Int’lSymposiumon Roboticswith Applications,World Automa-tion Congress.Burgard, W.; Moors, M.; Fox, D.; Simmons,R.; andThrun., S. 2000. Collaborative multi-robot exploration.In Proc.of theIEEE Int’l Conferenceon RoboticsandAu-tomation, 476–481.Cao,Y.; Fukunaga,A.; andKahng,A. 1997. Cooperativemobilerobotics:Antecedentsanddirections.AutonomousRobots4(1):7–27.Caprari,G.; Balmer, P.; Piguet,R.; andSiegwart,R. 1998.The autonomousmicro robot ALICE: A platform for sci-entific and commercialapplications. In Proc. of 1998Int’l Symposiumon Micromechatronics and HumanSci-ence(MHS’98).

Chemel,B.; Mutschler, E.; andSchempf,H. 1999. Cy-clops:� Miniature robotic reconnaissancesystem. In Proc.of theIEEE Int’l Conferenceon RoboticsandAutomation,2298–2303.

Dillmann, R.; Kaiser, M.; Wallner, F.; andWeckesser, P.1995. PRIAMOS: An advancedmobile systemfor ser-vice, inspection,andsurveillancetasks. In ModellingandPlanningfor SensorBasedIntelligentRobotSystems, vol-ume21 of Seriesin MachinePerceptionandArtificial In-telligence. Singapore:World Scientific.

Everett,H. R., andGage,D. W. 1999. From laboratoryto warehouse:Securityrobotsmeetthe real world. Int’lJournalof RoboticsResearch 18(7):760–768.

Fukuda,T.; Kawamoto,A.; andShimojima,K. 1996. Ac-quisition of swimming motion by RBF fuzzy neurowithunsupervisedlearning.In ALIFE V, 31–37.

Group,O. M. 1998. TheCommonObjectRequestBro-ker: Architecture and Specification. ObjectManagementGroup.

Halme,A.; Schonberg, T.; andWang,Y. 1996. Motioncontrolof a sphericalmobile robot. In 4th Int’l WorkshoponAdvancedMotion Control.

Hougen,D. F.; Benjaafar, S.; Bonney, J. C.; Budenske,J. R.; Dvorak, M.; Gini, M.; Krantz, D. G.; Li, P. Y.;Malver, F.; Nelson, B.; Papanikolopoulos, N.; Rybski,P. E.; Stoeter, S. A.; Voyles,R.; andYesin,K. B. 2000a.A miniatureroboticsystemfor reconnaissanceandsurveil-lance. In Proc. of the IEEE Int’l Conferenceon RoboticsandAutomation, 501–507.

Hougen,D. F.; Erickson,M. D.; Rybski, P. E.; Stoeter,S. A.; Gini, M.; andPapanikolopoulos,N. 2000b. Au-tonomousmobile robotsanddistributedexploratorymis-sions. In Int’l Symposiumon Distributed AutonomousRoboticSystems.

Inaba,M.; Kagami, S.; Kanechiro,F.; Takeda,K.; Tet-sushi, O.; and Inoue, H. 1996. Vision-basedadaptiveandinteractive behaviors in mechanicalanimalsusingtheremote-brainedapproach.RoboticsandAutonomousSys-tems17:35–52.

Kajiwara,T.; Yamaguchi,J.; Kanayama,Y.; Yuta,S.; andIijima, J. 1985. Developmentof a mobile robot for secu-rity guard. In Proceedingsof the15thInt’l SymposiumonIndustrialRobots, volume1, 271–278.

Kochan,A. 1997. HelpMateto easehospitaldelivery andcollectiontasks,andassistwith security. Industrial Robot24(3):226–228.

Martin,F. G. 1998.TheHandyBoard TechnicalReference.MIT MediaLab,Cambridge,MA.

Mataric,M. 1997.Behavior-basedcontrol:Examplesfromnavigation,learning,andgroupbehavior. Journalof Exper-imentalandTheoreticalArtificial Intelligence9(2–3):323–336.

Mondada,F.; Franzi,E.; andIenne,P. 1993.Mobile robotminiaturisation: A tool for investigationin control algo-rithms. In ExperimentalRoboticsIII, Proc.of the3rd Int’l

Symposiumon ExperimentalRobotics, 501–513. Kyoto,Japan:SpringerVerlag,London.Osipov, A.; Kemurdjian,V.; andSafonov, B. 1996. Mo-bile robotsfor security. In Proceedingsof the 19962ndSpecialtyConferenceon Roboticsfor ChallengingEnvi-ronments,RCE-II, 290–295.Parker, L. E. 1996.Onthedesignof behavior-basedmulti-robotteams.Journalof AdvancedRobotics10(6):547–578.Porteous,J. 1995. Intelligent buildings and their effecton thesecurityindustry. In Sanson,L. D., ed., IEEE Int’lCarnahanConferenceon SecurityTechnology, 186–188.Pritchard,D.; White, R.; Adams,D.; Krause,E.; Fox, E.;Ladd,M.; Heintzleman,R.; Sprauer, P.; andMacEachin,J.1998. Testandevaluationof panoramicimagingsecuritysensorfor force protectionandfacility security. In IEEEInt’l CarnahanConferenceon SecurityTechnology, 190–195. Larry D. Sanson,ed.Rybski,P. E.; Larson,A.; Lindahl,M.; andGini, M. 1998.Performanceevaluationof multiple robotsin a searchandretrieval task. In Workshopon Artificial IntelligenceandManufacturing, 153–160.Rybski, P. E.; Stoeter, S. A.; Erickson,M. D.; Gini, M.;Hougen,D. F.; andPapanikolopoulos,N. 2000.A teamofroboticagentsfor surveillance. In Proc. of the Int’l Conf.on AutonomousAgents, 9–16.Wu, A. S.; Schultz,A. C.; andAgah,A. 1999. Evolvingcontrol for distributedmicro air vehicles. In Proc. IEEEInt’l SymposiumonComputationalIntelligencein RoboticsandAutomation.