1

An Advanced Telereflexive Tactical Response Robot

G.A. Gilbreath, D.A. Ciccimaro, H.R. Everett

SPAWAR Systems Center, San DiegoCode D371

53406 Woodward RoadSan Diego, CA 92152-7383

gag@spawar.navy.milhttp://www.spawar.navy.mil/robots/

Abstract

ROBART III is intended as an advanced demonstrationplatform for non-lethal tactical response, extending theconcepts of reflexive teleoperation into the realm ofcoordinated weapons control (i.e., sensor-aided controlof mobility, camera, and weapon functions) in lawenforcement and urban warfare scenarios. A rich mix ofultrasonic and optical proximity and range sensorsfacilitates remote operation in unstructured andunexplored buildings with minimal operator oversight.Supervised autonomous navigation and mapping ofinterior spaces is significantly enhanced by an innovativealgorithm which exploits the fact that the majority ofman-made structures are characterized by (but notlimited to) parallel and orthogonal walls. This paperpresents a brief overview of the advanced telereflexiveman-machine interface and its associated “human-centered mapping” strategy.

1. Background

From a navigational perspective, the type of controlstrategy employed on a mobile platform runs the fullspectrum defined by teleoperated at the low end throughfully autonomous at the upper extreme. A teleoperatedmachine of the lowest order has no onboard intelligenceand blindly executes the drive and steering commandssent down in real-time by a remote operator. A fullyautonomous mobile platform, on the other hand, keepstrack of its position and orientation and typically usessome type of world modeling scheme to represent thelocation of perceived objects in its surroundings. A verycommon approach is to employ a statistical certainty-gridrepresentation [1], where each cell in the gridcorresponds to a particular “unit square” of floor space.The numerical value assigned to each cell represents theprobability that its associated location in the building isoccupied by some object, with a value of zero indicatingfree space (i.e., no obstacles present).

The existence of an absolute world model allows forautomatic path planning, and subsequent route revisionsin the event a new obstacle is encountered.Unfortunately, however, the autonomous execution ofindoor paths generally requires a priori knowledge of thefloorplan of the operating environment, and in all casesthe robot must maintain an accurate awareness of itsposition and orientation. Accordingly, traditionalautonomous navigation techniques are of limited utilityfor applications where a requirement exists to enterpreviously unexplored structures of opportunity as theneed arises.

Teleoperated systems, on the other hand, permit remoteoperation in such unknown environments, butconventionally place unacceptable demands on theoperator. For example, simply driving a teleoperatedplatform using vehicle-based video feedback is no trivialmatter, and can be stressful and fatiguing even under veryfavorable conditions. If a remote operator has to mastersimultaneous inputs for drive, steering, camera, andweapons control, the chances of successfully performingcoordinated actions in a timely fashion are minimal.

Easing the driving burden on the operator was a majorforce behind the development of the reflexiveteleoperated control scheme employed on ROBART II[2, 3], a prototype security robot capable of bothteleoperated and autonomous operation. The robot’snumerous collision-avoidance sensors, originallyintended to provide an envelope of protection duringautonomous transit, were also called into play duringmanual operation to greatly minimize the possibility ofoperator error. The commanded velocity and direction ofthe platform was altered by the onboard processors tokeep the robot traveling at a safe speed and precluderunning into obstructions. Work on ROBART III (Figure1) now extends this reflexive-teleoperation concept intothe realm of sensor-assisted camera and weapon controlfor indoor tactical systems.

Report Documentation Page Form ApprovedOMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.

1. REPORT DATE APR 2000

2. REPORT TYPE N/A

3. DATES COVERED -

4. TITLE AND SUBTITLE An Advanced Telereflexive Tactical Response Robot

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) G. A. /Gilbreath; D. A. /Ciccimaro; H. R. /Everett

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Space and Naval Warfare Systems Center Code D371 53406 WoodwardRoad San Diego, CA 92152-7383

8. PERFORMING ORGANIZATIONREPORT NUMBER

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)

11. SPONSOR/MONITOR’S REPORT NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release, distribution unlimited

13. SUPPLEMENTARY NOTES The original document contains color images.

14. ABSTRACT

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT

UU

18. NUMBEROF PAGES

6

19a. NAME OFRESPONSIBLE PERSON

a. REPORT unclassified

b. ABSTRACT unclassified

c. THIS PAGE unclassified

Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18

2

Figure 1. ROBART III is a laboratory prototypesupporting the development of enhanced telereflectioncontrol strategies for tactical response robots.

2. Man-Machine Interface

A very simplistic graphical user interface (GUI) has beenimplemented under Visual Basic to support thedevelopment and diagnostic needs of this technology-base effort (Figure 2). For purposes of this discussion,the man-machine interface issues can be subdivided intothree general categories: 1) mobility control, 2) cameracontrol, and 3) non-lethal weapon control.

2.1 Mobility Control

The Mobility Control Window (lower right corner of thescreen) provides a convenient means for the operator toset the desired speed, and if necessary, manually changethe platform’s heading. Each time the operator clicks onthe forward arrow button, for example, the platform’s

velocity is increased one increment. Clicking on eitherthe right- or left-turn arrows imposes a differential turnon the forward velocity, speeding up one wheel andslowing down the other. The more times a turn arrow isclicked, the bigger the differential and hence the fasterthe rate of turn. If the forward (or reverse) speed is zero(i.e., platform stopped), clicking a turn button causes therobot to pivot in place.

Figure 2. Navigation Control Screen, showing the high-level driving icons surrounding the Map Window (lowerleft corner). The robot has been instructed to enter thenext door encountered on the left.

Once the platform is set in motion, the operator caneasily control its subsequent actions by clicking onspecial behavioral icons depicted on the navigationdisplay. For example, selecting a wall-following iconcauses the platform to enter wall-following mode,maintaining its current lateral offset from the indicatedwall using side-looking sonar. The wall-following iconsare implemented as long vertical command buttonssituated on either side of the Map Window in the lowerleft corner. The nine dots displayed in front of therectangular robot icon at the bottom of the map indicatethe measured range to perceived objects in the path.

Two additional wall-segment icons are seen above themap in the form of short-length horizontal commandbuttons. The open spaces between these graphicaldepictions of wall structures represent three potentialdoorways: one directly ahead of the robot and one oneither side. By clicking in one of these doorway icons,the robot is instructed to seek out and enter the nextencountered location of that type of door along its currentpath. For the example illustrated in Figure 2, theplatform is looking for a door off to the left, as indicated

3

by the highlight box shown in the selected doorway icon,and the associated text displayed in the System StatusWindow above the map.

The primary mobility controls shown in Figure 2 aremimicked on a stand-alone hand-held pendent (Figure 3)employing an array of capacitive touch-sensor icons,based on the Quantum Research QProx E6S2 matrixdecoder. A high-resolution 2.5-inch color LCD monitorprovides video output, in addition to selected statusinformation overlaid at the top of the screen. A miniaturemotor-driven eccentric (as is commonly found invibrating pagers) is mounted inside the enclosure toprovide tactile motion feedback to the user [4]. Thespeed of this motor (and hence the vibration of the case)is varied in direct proportion to the velocity of the remoteplatform.

Figure 3. A capacitive touch-panel interface on thehand-held pendent mimics the drive icons shown inFigure 2.

2.2 Camera Control

Manual control of the ROBART’s head-mounted cameracan be accomplished using the slider and button controlswithin the Head Pan Control Window on the right side ofthe display screen. In addition, computer-aided camerapan is provided to support the three system functionalitiesof platform mobility, intruder assessment, and weapontracking. For mobility, the camera-pan commands areembedded within the “seek-door” behaviors. If the robotis instructed to enter the next door on the right, forexample, the camera immediately turns 45 degrees rightof center to acknowledge the behavior request andprovide a better view of the doorway detection process.As soon as the door is detected and the penetrationbehavior invoked, the camera pans to compensate for the

platform’s rate of turn in order to keep the door openingin the center of its field-of-view.

The intruder detection and assessment algorithms operateupon the output from the video motion detection (VMD)system and a 360-degree array of passive-infrared (PIR)sensors configured as a collar just below the head. ThePIR data is used to pan the surveillance camera to thecenter of any zone with suspected intruder activity. TheVMD output is then used to track and keep the intruder inthe center of the visual field, using a combination ofrobot head and body movement.

Whenever the head reaches its maximum pan limit (±100degrees) relative to the robot, the mobility base will pivotin place towards the target. The head meanwhile movesat the same speed in the opposite direction to keep theprimary target in the center of the visual field. Thiscoordinated action provides the robot with unlimited (i.e.,continuous 360-degree) pan coverage.

Automated camera pan for weapon tracking is treated inthe next section.

2.3 Non-Lethal Weapon Control

The principle non-lethal response system incorporated onROBART III is a six-barreled pneumatically-poweredGatling-gun (Figure 4) capable of firing 3/16-inch-diameter simulated tranquilizer darts or plastic bullets.Projectiles are expelled at a high velocity from 12-inchbarrels by a release of compressed air from a pressurizedaccumulator at the rear of the gun assembly. The mainair bottle is automatically recharged by a small 12-voltreciprocating compressor mounted in the robot’s base.

Figure 4. A six-barrel pneumatic tranquilizer gun is usedto demonstrate computer-assisted control of a non-lethalweapon.

4

The operator specifies what type of control strategy (i.e.,manual or automatic) to use when entering weapon-tracking mode by clicking on the appropriate option inthe Track Mode Window shown in the bottom-rightcorner of Figure 5. In manual mode, the firing decisionis made by the operator. A 5-milliwatt 670-nanometervisible-red laser sight facilitates manual training of theweapon using video from the head-mounted surveillancecamera. The operator can slave the surveillance-camerapan to the weapon pan axis by clicking on the “Head”option in the Slave Window (just below the System StatusWindow, upper left corner). The mobility base can alsobe slaved, so the robot turns to face the direction theweapon is aimed. If a forward drive speed is entered atthis point, the operator merely has to keep the weapontrained on the intruder, and the robot will automaticallygive chase.

Figure 5. Interim control and diagnostic screen usedduring development of the computer-assisted-weapon-control software on ROBART III.

In automatic mode, ROBART III is responsible formaking the firing decision, contingent upon a confirmedtarget solution stabilized for a pre-determined timeinterval, and pre-authorization from the operator.Azimuthal and elevation information from the VMD isavailable to the right-shoulder pan-and-tilt controller forpurposes of automated weapon positioning. Whenweapon-tracking is activated in automatic mode, therobot centers its head and turns to face toward the currentthreat. The mobility base then becomes stationary whilethe weapon begins tracking the target.

3. Human-Centered Mapping

The exploration and mapping of unknown structuresbenefits significantly when the interpretation of rawsensor data is augmented by simultaneous supervisoryinput from the human operator. A human-centeredmapping strategy has been developed to ensure validfirst-time interpretation of navigational landmarks as therobot builds its world model (currently on an externalRF-linked desktop PC). In a nutshell, the robot can enterand explore an unknown space, building a valid modelrepresentation on the fly, while dynamically re-referencing itself in the process to null out accumulateddead-reckoning errors.

Upon first entering a previously unexplored building, theoperator guides the robot using typical commands like:“follow the wall on your left,” and “enter the nextdoorway on the left.” Such high-level direction isprovided by clicking on screen icons as previouslydescribed. With this minimal operator input, the robot inthis example doesn’t just think it sees a wall, it knows itsees a wall. In other words, in addition to directing therobot’s immediate behavior, these same commands alsoprovide valuable information to the world modelingalgorithm. The end result of such an approach is a muchfaster and more accurate generation of objectrepresentations (relative to conventional sensor-only datacollections), particularly valuable when there is no apriori information available to the system.

The world model is first initialized as a two-dimensionaldynamic array with all cells marked as unknown. (Anunknown cell is treated as potentially traversable, butmore likely to be occupied than confirmed free space.) Ifsome specific subset of the current sonar data can bepositively identified from the outset as a wall-likestructure, it can be unambiguously modeled as aconfirmed wall without the need for statisticalrepresentation. This makes the resulting worldrepresentation much less ambiguous and therefore lesssubject to error.

In support of this objective, ROBART III has beenmechanically and electronically equipped specifically tosupport supervised operation in previously unexploredinterior structures. Two self-contained ElectroCorporation piezoelectric PCUC-series ultrasonic sensorsoperating at 215 KHz are used to generate range data forthe wall-following algorithm. (These sonar sensorsoperate at a much higher frequency than the 49.4-KHzPolaroid sensors used for collision avoidance, so thereare no problems associated with crosstalk fromsimultaneous operation.)

5

4. Orthogonal Navigation

The Achilles Heel of any world-modeling scheme,however, is accurate positional referencing in real-timeby the moving platform. Since all sensor data is takenrelative to the robot’s location and orientation, theaccuracy (and usefulness) of the model quickly degradesas the robot becomes disoriented. While wall followingis a very powerful tool in and of itself for determining therelative offset and heading of the robot, conventionalschemes normally assume some a priori informationabout the wall in the first place to facilitate its utility as anavigational reference. In short, a relative fix withrespect to an unknown entity does not yield anunambiguous absolute solution, for obvious reasons.

ROBART III uses a new and innovative world modelingtechnique that requires no such a priori information.This navigation scheme, called orthogonal navigation, or“Ortho-Nav,” exploits the orthogonal nature of mostbuilding structures where walls are parallel andconnecting hallways and doors are orthogonal. Ortho-Nav also uses the input from a magnetic compass toaddress the issue of absolute wall orientation. Theaccuracy of the compass need be only good enough toresolve the ambiguity of which of four possible wallorientations the robot has encountered. This informationis stored in the model in conjunction with the wallrepresentation (i.e., wall segment running north-south, orwall segment running east-west), in arbitrary buildingcoordinates. The precise heading of the vehicle (inbuilding coordinates) is then mathematically derivedusing sonar data taken from the wall surface as the robotmoves.

A typical wall-following routine uses a ranging sensor tomaintain a particular distance from a planar object (wall)on one or both sides. Due to sensor inaccuracies and theaccumulation of errors inherent in odometry, the rangedata will appear to drift toward or away from the robot,resulting in a wall plot that is skewed or perhaps evencurved. These errors can be mitigated by assuming thatthe wall is straight and immovable, and any perceivedundulations in the sonar data plot in actuality representirregular motion of the robot. Armed with this heuristic,both the lateral offset and heading of the robot can bedynamically corrected, even while the world model isstill being generated.

In order for this system to work properly, the robot mustfollow a reasonably planar wall surface rather than justblindly reacting to whatever clutter is nearby. This iswhere the human-centered aspect of the scheme comesinto play. By way of example, when the robot enters anunknown space under telereflexive control as illustratedin Figure 6, the operator examines the video and informs

the robot there is a wall it can follow on the left side. Inaddition, the operator also clicks on the left doorway icon(as illustrated earlier in Figure 2) to further instruct therobot to find and enter the next doorway on the left. Theonboard computer then begins acquiring range data fromthe appropriate sensors. When enough points have beenaccumulated for a fit (subject to a quality-of-fit-criteria),the resulting line is examined to determine its orientation.

Figure 6. Initial view of an interior space as seen fromthe robot’s onboard surveillance camera, revealing aclean wall for following on the immediate left.

The majority of buildings are laid out such that all wallsare either parallel or orthogonal to one another, so theorientation of the line is snapped to 0°, 90°, 180°, or 270°in arbitrary building coordinates. The robot’s heading isthen reset to this same value. Once the initial location ofthe wall has been established, an infinitely long potential-wall representation is entered into the model (Figure 7).

Potential Wall

Confirmed Wall

First Wall Fit

: Sensor Reading

Figure 7. After obtaining the first wall fit, a potentialwall is created and indexed to the cardinal heading whichmost closely matches the magnetic compass reading.

6

As the robot continues to follow the actual wall gettingvalid line fits, it incrementally converts the potential wallto a confirmed wall.

As previously discussed, the robot can now correct itslateral position by using the wall as a reference. For thesituation shown above, given that the X coordinate of thewall is Wx and the current range to the wall is r, therobot’s X coordinate is given by Rx = Wx + r. Similarly,the robot’s current heading can be dynamically correctedby subtracting the difference between the orientation ofthe current wall fit and the current wall orientation fromthe robot’s current heading. This is given by thefollowing equation:

( )θθθθ WFRR −−=where:

Rθ = the robot’s current headingFθ = the orientation of the current wall fitWθ = the orientation of the current wall

Potential Wall

Confirmed Wall

Confirmed Wall

Figure 8. As the robot enters the found doorway, themodeling algorithm uses the side-sonar range informationto cut an appropriately-sized opening through the existing(shown vertical) confirmed- and potential-wallrepresentations. The new (horizontal) potential wall isclipped against the previously constructed potential wall.

The robot turns left (as previously instructed) to enter thediscovered doorway, using the ranging sensors on bothsides to determine the size of the opening that must be

cut in the wall it has been constructing (Figure 8), to formthe doorway representation. After transitting thedoorway, the robot next detects and begins to follow awall to its right. Accordingly, it constructs a newpotential wall and snaps it perpendicular to the previousmodel entry. Note this second potential wall (shownhorizontally) is semi-infinite, in that it is clipped againstthe previously constructed potential wall (shownvertically).

Whenever the robot detects a new potential wall, itcompares it to the list of potential and confirmed wallsalready constructed. If the new wall coincides (withinpre-specified orientation and offset tolerances) with apreviously modeled wall, the range data is snapped to theexisting representation, rather than generating a new one.

5. Conclusion

This paper covers the implementation of a prototypetactical/security response robot capable of semi-autonomous exploration in unknown structures. Thesystem is able to confront intruders with a laser-sightedtranquilizer dart gun, and automatically track a movingtarget with the use of various sensors. A human-centeredmapping scheme ensures more accurate first-timeinterpretation of navigational landmarks as the robotbuilds its world model, while orthogonal navigationexploits the fact that the majority of man-made structuresare characterized by parallel and orthogonal walls.

6. References

1. Moravec, H. P., Elfes, A., “High Resolution Mapsfrom Wide Angle Sonar,” Proceedings of the 1985IEEE International Conference on Robotics andAutomation, St. Louis, MO, pp. 116-121, March,1985.

2. H.R. Everett, G.A. Gilbreath, T. Tran, J.M. Nieusma,“Modeling the Environment of a Mobile SecurityRobot”, TD 1835, Naval Ocean Systems Canter, SanDiego, CA, June, 1990.

3. Laird, R.T., Everett, H.R., "Reflexive TeleoperatedControl," Association For Unmanned VehicleSystems, 17th Annual Technical Symposium andExhibition (AUVS '90), Dayton, OH, pp. 280-292,July-August, 1990.

4. H.R. Everett, J.M. Nieusma, “Feedback System forRemotely Operated Vehicles,” Navy Case # 73322,U.S. Patent # 5,309,140, 3 May, 1994.