Mapping, Navigation, and Learning for Off-Road Traversalaravind/documents/konolige08-mapping.pdf · 2009-02-20 · Mapping, Navigation, and Learning for Off-Road Traversal Kurt Konolige

• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •

Mapping, Navigation, andLearning for Off-Road Traversal

Kurt KonoligeWillow GarageMenlo Park, California 94025e-mail: [email protected]

Motilal AgrawalSRI InternationalMenlo Park, California 94025e-mail: [email protected]

Morten Rufus BlasElektro/DTU UniversityLyngby, Denmarke-mail: [email protected]

Robert C. BollesSRI InternationalMenlo Park, California 94025e-mail: [email protected]

Brian GerkeyWillow GarageMenlo Park, California 94025e-mail: [email protected]

Joan SolaLAAS-CNRSToulouse, Francee-mail: [email protected]

Aravind SundaresanSRI InternationalMenlo Park, California 94025e-mail: [email protected]

Received 8 April 2008; accepted 11 November 2008

Journal of Field Robotics 26(1), 88–113 (2009) C© 2008 Wiley Periodicals, Inc.Published online in Wiley InterScience (www.interscience.wiley.com). • DOI: 10.1002/rob.20271

Konolige et al.: Mapping, Navigation, and Learning for Off-Road Traversal • 89

The challenge in the DARPA Learning Applied to Ground Robots (LAGR) project is toautonomously navigate a small robot using stereo vision as the main sensor. Duringthis project, we demonstrated a complete autonomous system for off-road navigation inunstructured environments, using stereo vision as the main sensor. The system is veryrobust—we can typically give it a goal position several hundred meters away and expectit to get there. In this paper we describe the main components that comprise the system,including stereo processing, obstacle and free space interpretation, long-range perception,online terrain traversability learning, visual odometry, map registration, planning, andcontrol. At the end of 3 years, the system we developed outperformed all nine other teamsin final blind tests over previously unseen terrain. C© 2008 Wiley Periodicals, Inc.

1. INTRODUCTION

The DARPA Learning Applied to Ground Robots(LAGR) project began in spring 2005 with the am-bitious goal of achieving vision-only autonomoustraversal of off-road terrain. Further, the participatingteams were to be tested “blind”—sending in code tobe run on a robot at a remote, unseen site. The hopewas that by using learning algorithms developed bythe teams, significant progress could be made in ro-bust navigation in difficult off-road environments,where tall grass, shadows, deadfall, and other obsta-cles predominate. The ultimate goal was to achievebetter than 2 × performance over a Baseline systemalready developed at the National Robotics Engineer-ing Center (NREC) in Pittsburgh, Pennsylvania. Allparticipant teams used the same robotic hardwareprovided by NREC [Figure 1(a)]; testing was per-formed by an independent team on a monthly basisat sites in Florida, New Hampshire, Maryland, andTexas.

Figure 1. (a) LAGR robot with two stereo sensors. (b) Typical outdoor scene as a montage from the left cameras of the twostereo devices.

Although work in outdoor navigation has prefer-entially used laser range finders (Bellutta, Manduchi,Matthies, Owens, & Rankin, 2000; Guivant, Nebot, &Baiker, 2000; Montemerlo & Thrun, 2004), LAGR usesstereo vision as the main sensor. One characteristic ofthe vision hardware is that depth perception is goodonly at fairly short range: its precision deterioratesrapidly after 7 m or so. Even when good stereo in-formation is available, it is often impossible to judgetraversability on the basis of three-dimensional (3D)form. For example, tall grass that is compressiblecan be traversed, but small bushes cannot, and theymight have similar 3D signatures. The robots wouldoften slip on sand or leaves and be unable to climbeven small grades if they were slippery. These con-ditions could not be determined even at close rangewith stereo vision.

Another area that the testing team was keen ondeveloping was the ability of the robots to make de-cisions at a distance. Many of the tests had extensive

Journal of Field Robotics DOI 10.1002/rob

90 • Journal of Field Robotics—2009

cul-de-sacs, dead ends, or paths that initially led to-ward the goal but then turned away. Here, the robotcould not rely on local information to find a good wayout. The expectation was that the teams would copewith such situations using long-range vision sensing,that is, be able to tell from the appearance of the ter-rain whether it was traversable.

Throughout the life of the project, we evaluatedthe potential of learning methods and appearance-based recognition. The emphasis was always on gen-eral methods that would work well in all situations,not just artificial ones designed to test a particularability, like bright orange fencing that could easilybe recognized by its distinctive color. In the end, themost useful and novel technique we developed wasan online method for path finding based on color andtexture. Although we also developed algorithms forclassifying obstacles at a distance, they did not workreliably enough to be included in a final system.

In addition to appearance-based learning, wehad to build improved algorithms for many differ-ent aspects of vision-based off-road navigation. Theparagraphs below summarize the methods that dis-tinguished the SRI system and contributed to its over-all performance.

Online Color and Texture SegmentationIt became clear from the early stages of the projectthat color-only methods for recognizing vegetation orterrain were not sufficient. We concentrated on de-veloping fast combined color/texture methods thatcould be used online to learn segmentations of theimage. These methods advance the state of the artin appearance-based segmentation and are the keyto our online path-finding method. They reliably findpaths such as the one in Figure 1(b), even when theparticular appearance of the path is new.

Precisely Registered MapsIf the robot’s reconstruction of the global environ-ment is faulty, the robot cannot make good plansto get to its goal. After noticing navigation fail-ures from the very noisy registration provided byglobal positioning system (GPS), we decided to givehigh priority to precise registration of local mapinformation into a global map. Here, we developedreal-time visual odometry (VO) methods that aremore precise than existing ones while still beingcomputable at frame rates. To our knowledge, this isthe first use of VO as the main registration methodin an autonomous navigation system. VO enabled

us to learn precise maps during a run and so escapeefficiently from cul-de-sacs. In the last stage of theproject, we also discovered that the precision of VOmade it possible to reuse maps from a previous run,thereby avoiding problem areas completely. Thisrun-to-run learning, or map reuse, was unique amongthe teams and on average halved the time it took tocomplete a course.

Efficient Planner and ControllerThe LAGR robot was provided with a “Baseline” sys-tem that used implementations of D* (Stentz, 1994)for global planning and the Dynamic Window Ap-proach (DWA) (Fox, Burgard, & Thrun, 1997) for localcontrol. These proved inadequate for real-time con-trol: for example, the planner could take several sec-onds to compute a path. We developed an efficientglobal planner based on previous gradient techniques(Konolige, 2000), as well as a novel local controllerthat takes into account robot dynamics and searchesa large space of robot motions. These technqiues en-abled the robot to compute optimal global paths atframe rates and to average 85% of its top speed overmost courses.

1.1. Performance

It is hard to overemphasize the contribution of consis-tent map construction and map reuse. Without well-registered maps, the robot would often spend largeamounts of time getting cornered as badly remem-bered obstacles filled in open spaces, or it would reen-ter dead-end areas that had shifted in the map. Be-cause the testing team emphasized cul-de-sacs andgarden path scenarios, it was critical to have accuraterepresentations of areas that were no longer withinthe short stereo range. As it turned out, an accuratemap was a more robust way to deal with these scenar-ios than unreliable long-range sensing. Further, oncethe map was constructed, map reuse led to very effi-cient runs: if you memorize the route, there is no needto repeat your mistakes. Although the idea is simple,the execution was difficult, requiring very precise lo-calization based on VO.

At the end of the project, the teams were testedin a series of courses (Tests 25–27) with a variety ofchallenges (see Section 6.2). We chose these last testsfor inclusion here because our system was complete,having just added the map reuse feature. Over thesetests, we averaged about 4 × the score of Baseline, thebest of any team. In each of these tests, our score beat



or tied that of the best other team, and in the aggre-gate, we scored 60% higher than the best other team.These results validate the applicability of our tech-niques to autonomous navigation.

In this paper we show how we built a system forautonomous off-road navigation that embodies themethods described above and in particular performsonline path learning and run-to-run map learning toincrease its performance. In the following sections,we first discuss local map creation from visual input,with a separate section on learning color models forpaths and traversable regions. Then we examine VOand registration in detail and show how consistentglobal maps are created and reused. The next sectiondiscusses the global planner and local controller. Fi-nally, we present performance results for the last se-ries of tests at the end of the project.

1.2. Related Work

There has been an explosion of work in simultane-ous localization and mapping (SLAM), most of itconcentrating on indoor environments (Gutmann &Konolige, 1999; Leonard & Newman, 2003). Muchof the recent research on outdoor navigation hasbeen driven by DARPA projects on mobile vehicles(Bellutta et al., 2000). The sensor of choice is a laserrange finder, augmented with monocular or stereovision. In much of this work, high-accuracy GPS isused to register sensor scans; exceptions are Guivantet al. (2000) and Montemerlo and Thrun (2004). Incontrast, we forgo laser range finders and explicitlyuse image-based registration to build accurate maps.Other approaches to mapping with vision are thoseof Rankin, Huertas, and Matthies (2005) and Speroand Jarvis (2002), although they are not orientedtoward real-time implementations. Obstacle detec-tion using stereo has also received some attention(Rankin et al., 2005).

VO systems use structure-from-motion methodsto estimate the relative position of two or morecamera frames, based on matching features betweenthose frames. There have been a number of recentapproaches to VO (Johnson, Goldberg, Cheng, &Matthies, 2008; Maimone, Cheng, & Matthies, 2007;Nister, Naroditsky, & Bergen, 2006), including mo-tion estimation on the Mars vehicles (Matthies etal., 2007). Other teams in LAGR also developed VOsystems to aid in navigation (Howard, 2008). Oursystem (Agrawal & Konolige, 2006, 2007; Konolige,Agrawal, & Sola, 2007) is most similar to the recent

work of Mouragnon, Lhuillier, Dhome, Dekeyser,and Sayd (2006) and Sunderhauf, Konolige, Lacroix,and Protzel (2005), which exploits bundle adjust-ment techniques to obtain increased precision. Onedifference is the introduction of a new, more sta-ble keypoint detector and the integration of an iner-tial measurement unit (IMU) to maintain global poseconsistency. Our system is also distinguished by real-time implementation and high accuracy using a smallbaseline in realistic terrain. It has been in regular usein demonstrations for more then 2 years as the pri-mary mode of localization and map registration. Inaddition, the system has been tested over trajectoriesof up to 9 km with a ground truth RTK-GPS data setand has achieved accuraces of under 1% error (seeSection 3.4).

Our segmentation algorithm uses a compact de-scriptor to represent color and texture. In a seminalpaper, Leung and Malik (2001) showed that manytextures could be represented and recreated usinga small number of basis vectors extracted from thelocal descriptors; they called the basis vectors tex-tons. Whereas Leung and Malik used a filter bank,Varma and Zisserman (2003) later showed that smalllocal texture neighborhoods may be better than us-ing large filter banks. In addition, a small local neigh-borhood vector can be much faster to compute thanmultichannel filtering such as Gabor filters over largeneighborhoods.

Our planning approach is an enhanced reimple-mentation of the gradient technique (Konolige, 2000),which computes a global navigation function overthe cost map. A similar approach is used in wave-front planning (Latombe, 1991), although wavefrontplanners usually minimize Manhattan or diagonaldistance, whereas we minimize Euclidean distance.Level sets (Kimmel & Sethian 1998) offer an equiva-lent method for computing paths that minimize Eu-clidean distance. The underlying computation for allsuch planners is a variation on dynamic program-ming (Bellman, 1957). For reasons of efficiency, ourplanner treats the robot as a holonomic cylinder withno kinodynamic constraints. These constraints couldbe incorporated into the planner by use of sampling-based algorithms such as rapidly exploring randomtrees (RRTs) (LaValle, 2006).

We enforce kinodynamic constraints in our lo-cal controller. Control algorithms such as DWA (Foxet al., 1997) compute local controls by first determin-ing a target trajectory in position or velocity space(usually a circular arc or other simple curve) and then



inverting the robot’s dynamics to find the desiredvelocity commands that will produce that trajectory.We instead explore the control space directly andsimulate and evaluate the resulting trajectories, in amanner reminiscent of the controller used in theRANGER system (Kelly, 1994), with the key differ-ences being the definition of the state space andthe trajectory evaluation function. The Stanley con-troller (Thrun et al., 2006) also rolls out and evaluatespossible trajectories but divides them into two cate-gories (“nudges” and “swerves”), based on their ex-pected lateral acceleration. Howard, Green, and Kelly(2007) present a more general approach to constrain-ing the search for controls by first sampling directlyin the vehicle’s state space.

2. LOCAL MAP CONSTRUCTION

The object of the local map algorithms is to deter-mine, from the visual information, which areas arefree space and which are obstacles for the robot: thelocal map. Note that this is not simply a matter of geo-

metric analysis: for example, a log and a row of grassmay have similar geometric shapes, but the robot cantraverse the grass but not the log.

Figure 2(a) is an outline of visual processing,from image to local map. There are four basic trajecto-ries. From the stereo disparity, we compute a nominalground plane, which yields free space near the robot.We also analyze height differences from the ground tofind obstacles. Via the technique of sight lines we caninfer free space to more distant points. Finally, fromcolor, texture, and path analysis, coupled with theground plane, we determine paths and traversabilityof the terrain.

All of the processing of local cost maps, with theexception of the color/texture learning, takes placevery efficiently. We can run the full algorithm, includ-ing stereo computation and obstacle detection, in un-der 70 ms (15 Hz), enabling very quick response tonew features in the environment. Each stereo pair hadan associated computer with dual 2 GHz core proces-sors; we used one processor for stereo and cost mapanalysis, another for visual odometry.

Points for estimatingground plane

Space declared emptybased on sight line

"Sight line" marking clear spacealong azimuth

Distant pointsabove ground plane

Fitted ground plane

(b) Stereo point interpretation

analysiscolor

(1/d) imagedisparity

points3D

linessight

planeground

analysisheight

analysispath

3Dimage

traversibility

obstacles

freespace

local map

(a) Processing Diagram

Figure 2. Visual processing. In (a), the paths from visual input depict the processing flow in constructing the local map.The interpretation of stereo data points is in (b): nearby points (out to 6 m) contribute to the ground plane and obstacledetection; farther points can be analyzed to yield probable free space (“sight lines”) and extended ground planes.



Figure 3. (a) Disparity image from the left stereo pair of the robot in Figure 1. Closer pixels are lighter. (b) Extracted groundplane, in green overlay. Limit of ground plane is shown by green bar; sight line has a red bar. (c) Ground plane overlaid onoriginal image, in green. Obstacles are indicated in purple.

2.1. Stereo Analysis and GroundPlane Extraction

We use a fast stereo algorithm (Konolige, 1997) tocompute a disparity image at 512 × 384 resolutionin less than 40 ms1 [Figure 3(a)]. In typical outdoorscenes, it should be possible to achieve very densestereo results, The high resolution gives very de-tailed 3D information for finding the ground planeand obstacles. Each disparity image point [u, v, d]�

corresponds to a 3D point in the robot’s frame([x, y, z,w]� = R[u, v, d, 1]� in homogenous coordi-nates, where R is the reprojection matrix) (Konolige& Beymer, 2007). The matrix multiplication is donefor each disparity point as part of stereo processing.

Output from the stereo process is used in a num-ber of ways: the diagram in Figure 2(b) summarizesthem. Most of our analysis is biased toward findingfree space, especially in areas that are farther from therobot. This strategy stems from the high cost of see-ing false obstacles, closing off promising paths for therobot.

The most important geometric analysis is findingthe ground plane. Although it is possible to detectobstacles using local variation in height (Happold,Ollis, & Johnson, 2006), using a ground plane sim-plifies processing and yields more stable results. Toextract a ground plane, we use a RANSAC technique(Fischler & Bolles, 1981), choosing sets of threenoncollinear points. Hypothesized planes are rankedby the number of points that are close to the plane.Figure 3 shows an example, with a green overlay in-dicating the inliers. Points that lie too high above the

1All processing times referenced in this paper are on a 2-GHz IntelCPU.

ground plane, but lower than the robot’s height, arelabeled as obstacles. This method is extremely simplebut has proven to work well in practice, even whenthe ground has modest dips and rises; one reasonis that it looks out only to 6 m around the robot. Amore sophisticated analysis would break the groundplane into several segments or model more complexshapes. To compute the ground plane efficiently, wesubsample the image to 10,000 points and apply theRANSAC algorithm above. Hypothesizing a planeand finding inliers are simple matrix operations, andtypical running time is 5 ms.

To find obstacles, a typical algorithm would clus-ter the 3D points into grid cells on the ground plane.Then, by analyzing the points in each cell, it wouldbe declared ground, obstacle, or unknown. The prob-lem is that there is a geometric mismatch between the3D cells and the image point density: as the pointsprojected on farther cells become sparse, it is diffi-cult to determine obstacle boundaries. Instead, wefind obstacles using algorithms in the disparity plane.The ground plane is projected back onto the dispar-ity points, which are shown in green in Figure 3(b).The disparity image is divided into thin columns, andeach column is traversed from the bottom of the im-age (red line in the image). When the ground planeends and enough disparity points are found, theremust be an obstacle at that endpoint in the groundplane. In practice this technique is much faster andmore reliable than ground plane projection, typicallyconsuming only 5 ms.

2.2. Sight Lines

Although we cannot precisely locate obstacles past6–8 m, we can determine whether there is free



space, using the following observation. Consider theinterpreted image of Figure 3(c). There is a path thatgoes around the bushes and extends out a good dis-tance. The ground plane extends over most of thisarea and then ends in a distant line of trees. The treesare too far to place with any precision, but we cansay that there is no obstacle along the line of sight to thetrees. Given a conservative estimate for the distanceof the trees, we can add free space up to this estimate;typically we would add free space to at most 25 m.The computation of sight lines is most efficiently ac-complished in disparity space, by finding columns ofground plane pixels that lead up to a distant obsta-cle [red vertical line in Figure 3(b)]. Note that the ex-ample sight line follows the obvious path out of thebushes.

2.3. Learning Color and Texture Models

Our learning algorithm uses an online unsupervisedsegmentation algorithm that uses color and textureto group and cluster similar regions. This segmentedimage is then used to learn a color and texture modelfor path-like regions in outdoor images. Our segmen-tation algorithm is based on textons (Leung & Malik,2001) and is accomplished in two stages. The maindesign issues have been speed (for real-time segmen-tation) and robustness (to minimize false-positives).

In the first stage, we cluster color and texture vec-tors over small local neighborhoods to find a smallset of basis vectors (also known as textons) that char-acterize different scene textures. For reasons of speed,this vector should be as compact as possible withoutlosing appearance characteristics of the region.Angelova, Matthies, Helmick, and Perona (2007)use the three color components over a 5 × 5 re-gion centered on each pixel. This results in a large75-dimensional feature vector for each pixel. Forspeed, we use a 3 × 3 neighborhood and use thepixel intensity gradients between the surroundingpixels relative to the center pixel to represent texturecompactly. We also augment this eight-dimensionalfeature vector with the 3D color vector of the centerpixel in the CIELAB color space. CIELAB has theproperty that colors are perceptually uniform. Theresulting 11-dimensional color/texture feature vectoris very compact, and we have found that it stillretains the crucial appearance properties to discrim-inate and segment regions. A detailed comparisonwith other representations can be found in Blas,Agrawal, Konolige, and Sundaresan (2008).

In the second stage, we cluster histograms ofthese textons over larger 32 × 32 regions (which isdependent on the scale of the image) to find morecoherent regions with the same mixture of textonsusing k means as our clustering algorithm. These his-tograms can be constructed efficiently (irrespectiveof window size) using integral images (Viola & Jones,2004). The algorithm is generally set to oversegmentthe image slightly, as in our case oversegmentationcan be dealt with by a subsequent geometricalanalysis of the image. Undersegmentation is harderto deal with, as considerable information is lost. Thenumber of clusters was set to eight in the secondstage. There are other ways of doing the second stagethat provide better results by specifically dealingwith boundary conditions, but they are much slowerand are thus currently ill-suited for our real-timeneeds. [See graph-cut (Martin, Fowlkes, & Malik,2004) and level-sets (Liapis, Sifakis, & Tziritas, 2004).]

2.3.1. Segmentation Results

The University of Southern California (USC) hoststhe Brodatz texture database and also providestexture mosaics that are a number of Brodatz texturesstitched together in a jigsaw-type pattern. texmos3was selected as the texture mosaic for benchmarkingour texton descriptors. Figure 4 shows this mosaicalong with the ground truth segmentation. Thismosaic has eight textures and does not contain color,which tests the descriptors’ ability to discriminatetextures. Four basic descriptors are tested: a 48-dimensional descriptor composed of the responsesfrom the Leung–Malik (Leung & Malik, 2001) filterbank (LM, 32); a 75-dimensional descriptor of 5 × 5raw red–green–blue (RGB, 5 × 5, 32) values as

Figure 4. Synthetic texture mosaic used (provided by USCvia its website). The left image is the texture mosaic. Theright image shows which texture regions belong to whichtexture.



Figure 5. Results for the synthetic texture segmentation. Each color represents a different histogram cluster. An overlayshows which regions should have homogeneous colors. (a) LM filter, 32; (b) RGB 5 × 5, 32; (c) LBP 3 × 3, 32; (d) SRI 3 × 3,32; and (e) SRI 5 × 5, 64.

used in Angelova et al. (2007) (which in effect is25-dimensional on gray-scale images); the localbinary pattern (Maenpaa & Pietikainen, 2005) (LBP)in a 3 × 3 neighborhood (LBP, 3 × 3, 32); and twoversions of our descriptor: the 3 × 3 neighborhood(11-dimensional with the L,a,b color components setto zero) (SRI, 3 × 3, 32) and a 5 × 5 neighborhood(SRI, 5 × 5, 64) with the descriptor components stillbeing the intensities minus the center intensity.

For the test, the LM filter bank is the only one inwhich the descriptors are not learned on the imageitself. For all other descriptors, 32 textons are learnedfrom the image itself. Our 5 × 5 version used 64textons illustrating our best possible result. The lackof color information meant that more textons wereneeded to discriminate the textures. The second stageof clustering is then applied to give the segmentationresults. It is important to note that the underlyingsegmentation algorithm is the same for each of thesedescriptors.

The actual segmentations obtained for each de-scriptor can be seen in Figure 5. Each descriptor isthen scored using two scores: the detection rate andthe confusion rate. The detection rate gives a measureof how much of a given texture it managed to clas-sify correctly. The confusion rate gives a measure ofhow many correct versus false detections to expect.A good segmentation will have a high detection rateand a low confusion rate.

The total confusion and detection rates are shownin Table I. The LM filter bank performs the worst, as ithas higher confusion and lower detection rates thanall the other descriptors. The raw intensity value de-scriptor also performs poorly. LBP has problems dis-criminating between textures 2 and 8 but is otherwiseclearly better than the raw intensities and LM filter

Table I. Total confusion and detection rates for differenttypes of descriptors.

LM, SRI, SRI,% 32 RGB LBP 3 × 3 5 × 5

Total confusion rate 50 56 46 38 34Total detection rate 40 53 68 79 68

bank. Our descriptors do a much better job at dis-criminating between textures 2 and 8, which indicatesthat the intensity gradients are necessary to do thisand that it is not enough to rely just on the gradientdirection. All the methods find it hard to discriminatebetween textures 3 and 4 except the LBP, which aids itgreatly in the total scores. The results for our descrip-tor are on average better than those of the other meth-ods on this data set. Interestingly, for our descriptorsthe 3 × 3 version actually gets a better total detectionrate than the 5 × 5 version at the cost of a higher totalconfusion score.

2.3.2. Learning Paths

We use our segmentation algorithm to learn andsubsequently recognize both natural and man-madepaths in outdoor images. Paths are characterized bytheir color, texture, and geometrical properties. Train-ing samples for a path can come from teleoperation orfrom a priori knowledge that the robot is starting ona path. The robot can also search for paths by tryingto identify image clusters that have the geometry ofa path. We deal with oversegmentation of the path(wherein a path is split into multiple segments due topossibly differing textures) by grouping multiple seg-ments based on their overall geometry. We compute



geometrical properties of the path that could be com-posed of a single or multiple segments. The prop-erties include width, length, and spatial continuityof the path in order to verify whether it geometri-cally resembles a path. These geometrical propertiesare computed in three dimensions using the groundplane information available from the stereo camerasand are hence not affected by the perspective projec-tion. We assume a fixed path width (but allow a cer-tain deviation from this assumption). Once a path isidentified, the robot learns the texton histograms ofthe component segments as a model for the path. Thismodel can be used to identify even paths that are par-tially outside the field of view (FOV) by allowing thepath to end prematurely at the image boundary.

For classification, each pixel is first labeled usingthe shortest Euclidean distance on the color/texturevector at this pixel to the clustered textons. Likewisehistograms of textons at each pixel are classified us-ing Euclidean distance to the clustered histograms.The texton histograms from our training provide pos-itive examples (the histograms that belong to a path)as well as negative examples (the histograms thatdo not belong to a path). This helps prevent false-positives. A final geometrical analysis of the labeledhistograms makes sure that potential path regionshave the right geometry.

The learning process runs at 1 Hz for training ona single image and is typically performed at the be-ginning of a run (although it could be performed atregular intervals to update the path model). Classi-fication based on the learned model runs at around5 Hz. Figure 6 shows the various steps of our al-gorithm on one of our test runs. The path betweenbushes is identified in yellow in Figure 6(d). Detailson this algorithm can be found in Blas et al. (2008).

2.4. Results of Local Map Construction

The combined visual processing results in local mapsthat represent traversability with a high degree of fi-delity. Figure 7 shows the results of an autonomousrun of about 130 m, over a span of 150 s. We usedoffline learning of mulch paths on a test site and thenused the learned models on the autonomous run. Thefirst part of the run was along a mulch path underheavy tree cover, with mixed sunlight and deep shad-ows. Cells categorized as path are shown in yellow;black is free space. Obstacles are indicated by purple(for absolute certainty) and white-to-gray for decreas-ing certainty. We did not use sight lines for this run.

The path did not lead directly to the goal, andthere were many opportunities for the robot to headcross country. About two-thirds of the way throughthe run, no more paths were available, and the robotwent through heavy grass and brush to the goal. Therobot’s pose, as estimated from filtered VO (see Sec-tion 3.2), is in green; the filtered GPS path is in yel-low. Because of the tree cover, GPS suffered from highvariance at times.

A benefit of using VO is that wheel slips andstalls are easily detected, with no false positives (Sec-tion 5.4). For example, at the end of the run, the robotwas caught on a tree branch, spinning its wheels. Thefiltered GPS, using wheel odometry, moved far off theglobal pose, while the filtered visual odometry posestayed put.

3. CONSTRUCTING CONSISTENTGLOBAL MAPS

In this section we provide solutions to two problems:representing and fusing the information provided byvisual analysis, and registering local maps into a con-sistent global map.

3.1. Map Representation

For indoor work, a standard map representation isa two-dimensional (2D) occupancy grid (Moravec &Elfes, 1985), which gives the probability of each cell inthe map being occupied by an obstacle. Alternativesfor outdoor environments include 2.5-dimensionalelevation maps and full 3D voxel maps (Iagnemma,Genot, & Dubowsky, 1999). These representationscan be used to determine allowable kinematic anddynamic paths for an outdoor robot in rough terrain.We choose to keep the simpler 2D occupancy grid,foregoing any complex calculation of the robot’sinteraction with the terrain. Instead, we abstract thegeometrical characteristics of terrain into a set ofcategories and fuse information from these categoriesto create a cost of movement.

We use a grid of 20 × 20 cm cells to representthe global map. Each cell has a probability of be-longing to each of the four categories derived fromvisual analysis (Section 2): obstacle, ground planefree space, sight line free space, and path free space.Note that these categories are not mutually exclusivebecause, for example, a cell under an overhangingbranch could have both path and obstacle properties.We are interested in converting these probabilities



Figure 6. Various steps of our segmentation algorithm on a typical outdoor image. (a) The image from one of the stereocameras. (b) Each pixel assigned to a texton. (c) Each histogram of textons gets assigned to a histogram profile. In thisparticular example, the path is composed of two segments (green and yellow). (d) A path is recognized (in yellow).

into a cost of traversing the cell. If the probabilitieswere mutually exclusive, we would simply form thecost function as a weighted sum. With nonexclusivecategories, we chose a simple prioritization sched-ule to determine the cost. Obstacles have the highestpriority, followed by ground plane, sight lines, andpaths. Each category has its own threshold for sig-nificance: for example, if the probability of an obsta-cle is low enough, it will be ignored in favor of oneof the other categories. The combination of prioritiesand thresholds yields a very flexible method for de-termining costs. Figure 7 shows a color-coded versionof computed costs.

3.2. Registration and Visual Odometry

The LAGR robot is equipped with a GPS that is ac-curate to within 3–10 m in good situations. GPS in-formation is filtered by the IMU and wheel encodersto produce a more stable position estimate. However,because GPS drifts and jumps over time, it is impossi-ble to differentiate GPS errors from other errors suchas wheel slippage, and the result is that local mapscannot be reconstructed accurately. Consider the sit-uation of Figure 8 and 9. Here the robot goes throughtwo loops of 10-m diameter. There is a long linearfeature (a low wall) that is seen as an obstacle at the



Figure 7. Reconstruction on a 130-m autonomous run. Yellow is recognized path, black is free space, and white and grayare obstacles.

Figure 8. Three stages during a run using GPS filteredpose. Obstacle points are shown in white, free space inblack, and the yellow line is the robot’s path. The linear fea-ture is marked by hand in red in all three maps, in its initialpose. Map extent is 35 m on a side.

beginning and end of the loops. Using the filteredGPS pose, the position of the wall shifts almost 2 mduring the run, and obstacles cover the robot’s previ-ous tracks.

Figure 9. VO in the same sequence as Figure 8. GPS fil-tered path in yellow; VO filtered path is in green.

Our solution to the registration problem is to useVO to ensure local consistency in map registration.Over larger regions, filtering VO with GPS informa-tion provides the necessary corrections to keep er-rors from growing without bounds. We describe thesetechniques in the next two sections.



The LAGR robot presents a challenging situa-tion for visual odometry: wide FOV and short base-line make distance errors large, and a small offsetfrom the ground plane makes it difficult to trackpoints over longer distances. We have developed arobust VO solution that functions well under theseconditions. We briefly describe it here; for moredetails consult Agrawal and Konolige (2006) andKonolige et al. (2007).

For each new frame, we perform the followingprocess.

1. Distinctive features are extracted from eachnew frame in the left image. Standard stereomethods are used to find the correspondingpoint in the right image.

2. Left-image features are matched to the fea-tures extracted in the previous frame usingour descriptor. We use a large area, usuallyaround 1/5 of the image, to search for match-ing features.

3. From these uncertain matches, we recover aconsensus pose estimate using a RANSACmethod (Fischler & Bolles, 1981). Several thou-sand relative pose hypotheses are generatedby randomly selecting three matched non-collinear features and then scored using pixelreprojection errors.

4. If the motion estimate is small and the per-centage of inliers is large enough, we discardthe frame, because composing such small mo-tions increases error. A kept frame is called akey frame. The larger the distance between keyframes, the better the estimate will be.

5. The pose estimate is refined further in a sparsebundle adjustment (SBA) framework (Engels,Stewnius, & Nister, 2006; Triggs, McLauchlan,Hartley, & Fitzgibbon, 2000). SBA is a nonlin-

ear batch optimization over camera poses andtracked features. An incremental form of SBAcan reduce the error in VO by a large factor atvery little computational overhead. A featurethat is long lived, that is, that can be trackedover more frames, will give better results.

Precise VO depends on features that can betracked over longer sequences. Hence, the choice ofa feature detector can have a large impact in the per-formance of such a VO system. Harris corner featuresare widely used for VO. We have found that althoughHarris corners give good results and are very efficientto compute, they fail in many situations in outdoorenvironments. In addition, these features are not verystable, resulting in very short track lengths. Otherwidely used feature detectors such as SIFT (Lowe,2004) and SURF (Bay, Tuytelaars, & Gool, 2006) workwell but are not suitable for a real-time system. Wehave developed a novel feature (named CenSurE)(Agrawal, Konolige, & Blas, 2008) that has improvedstability and is inexpensive to compute. Whereasthe basic idea of CenSurE features is similar to thatof SIFT, the implementation is extremely efficient,comparable to Harris. Just as SIFT approximates theLaplacian of Gaussian with difference of Gaussians,CenSurE features approximate the LOG with bilevelcenter-surround filters. The extreme simplicity ofthese filters makes them extremely fast to computebut without sacrificing performance. Figure 10 showsa progression of bilevel filters with various degreesof symmetry. The circular filter is the most faithfulto the Laplacian but hardest to compute. The otherfilters can be computed rapidly with integral imageswith decreasing cost from octagon to hexagon to boxfilter. Further details of our CenSurE feature detectorare described in Agrawal et al. (2008). Figure 11shows the CenSurE features tracked over severalframes.

Figure 10. Progression of center-surround bilevel filters. (a) Circular symmetric BLOG (bilevel LOG) filter. Successivefilters (octagon, hexagon, box) have less symmetry.



Figure 11. CenSurE features tracked over several frames.

The IMU and the wheel encoders are used to fillin the relative poses when VO fails. This happens dueto sudden lighting changes, fast turns of the robot, orlack of good features in the scene (e.g., blank wall).

3.3. Global Consistency

Bundle-adjusted incremental motions between con-secutive frames are chained together to obtain the ab-solute pose at each frame. Obviously, this is bound toresult in accumulation of errors and drifting. We useGPS and the IMU to correct the pose of the vehicle.We perform two types of filtering:

1. Gravity normal: The IMU’s accelerometersmeasure the gravity normal in vehicle frametogether with vehicle accelerations. Vehicle ac-celerations have zero mean in the long run andcan therefore be considered as white perturba-tions of the gravity measurements. We applyregular extended Kalman filter (EKF) correc-tions to the tilt and roll angles. By assigningvery large noise values to the perturbing ac-celerations (we used 10g standard deviation),the effect is imperceptible in the short termbut sufficient to cancel the long-term angulardrifts, otherwise unbounded.

2. GPS yaw: The IMU yaw data are very badand cannot be used for filtering (for exam-ple, over the 150-m run, they can be off by60 deg). Instead, we used the yaw estimateavailable from the LAGR GPS. These yaw esti-

mates are comparable to a good-quality IMU.Over a very long run, the GPS yaw does nothave an unbounded error, as would an IMU,because it is globally corrected.

To maintain globally consistent maps, we haveturned off any position filtering based on GPS. Wecompletely ignore position estimates from the GPSin calculating our pose. In addition, to limit the ef-fect of velocity noise from GPS on the heading esti-mate, GPS yaw is used only when the GPS receiverhas at least a 3D position fix and the vehicle is travel-ing 0.5 m/s or faster. Our filter is a simple linear filterthat nudges the tilt/roll (for gravity normal) and yaw(for GPS yaw) toward global consistency, while main-taining local consistency.

The quality of the registration from filtered VO,shown in Figure 9, can be compared to the filteredGPS of Figure 8. The low wall, which moved almost2 m over the short loops when using GPS, is muchmore consistent when VO is employed. And in casesin which GPS is blocked or degraded, such as underheavy tree cover in Figure 7, VO still produces mapsthat are locally consistent. It also allows us to deter-mine wheel slips and stalls with almost no false pos-itives; note the end of the run in Figure 7, where therobot was hung up and the wheels were slipping andwheel odometry produced a large error.

3.4. Results of Visual Odometry

In Test 17, the testing team surveyed a course usingan accurate RTK-GPS receiver. The “Canopy Course”was under tree cover, but the RTK GPS and theLAGR robot GPS functioned well. Sixteen waypointswere surveyed, all of which were within 10-cm er-ror according to the RTK readout. (One waypointwas deemed inaccurate and not included.) The totallength of the course was about 150 m. Subsequently,the LAGR robot was joysticked over the course, stop-ping at the surveyed points. The robot was run for-ward over the course and then was turned aroundand sent backward to the original starting position.

The course itself was flat, with many smallbushes, cacti, downed tree branches, and other smallobstacles. Notable for VO was the sun angle, whichwas low and somewhat direct into the cameras onseveral portions of the course. Figure 12 shows twoimages acquired by the robot. The left image shows agood scene in the shadow of the trees, and the rightimage shows a poor image where the sun washes



Figure 12. Images from the Canopy data set.

out a large percentage of the scene. (The lines in theimages are horizon lines taken from VO and fromground plane analysis.) The uneven image qualitymakes it a good test of the ability of VO under re-alistic conditions.

Because the initial heading of the robot is un-known, we used an alignment strategy that assumesthat there is an initial alignment error and corrects itby rotating the forward VO path rigidly to align theendpoint as best as possible. This strategy minimizes

VO errors on the forward path and may underesti-mate them. However, for the return path, the errorswill be caused only by VO and can be taken as a moreaccurate estimate of the error.

For this test, our CenSurE features were notready, and we were able to match frames along thewhole route using Harris corners. Figure 13(a) showsthe RMS error between VO (with different filters) andthe RTK waypoints, on the return path. As notedabove, the forward VO path of the robot has been

Figure 13. Results of VO on the Canopy data set. (a) RMS error between VO (with different filters) and the RTK waypoints,on the return path. (b) Trajectory of bundle-adjusted VO (without any filtering) compared to RTK groundtruth.



aligned with the RTK path. As can be seen, the bestresults are obtained using bundle-adjusted VO withgravity normal and GPS yaw filtering. In this case,the errors between waypoints is very small, amount-ing to <1% of distance traveled. Without filtering, theresults are worse [(Figure 13(b)], amounting to about3% of distance traveled. At some points in the middleof the return trip, the VO angle starts to drift, and atthe end of the backward trip there is about a 10-mgap. Note that this effect is almost entirely causedby the error in the yaw angle, which is corrected byGPS yaw. It is also worth mentioning that the use ofCenSurE features substantially improves the perfor-mance of VO, although we do not have results of us-ing CenSurE on this data set.

We present results of VO with CenSurE featureson two other large outdoor data sets. These data setswere collected using a larger tank-like vehicle calledCrusher [also developed by the National RoboticsEngineering Center (NREC), Pittsburgh, underDARPA’s Unmanned Ground Vehicle–PerceptORIntegration (UPI) program]. They have frame-registered ground truth from RTK GPS, which isaccurate to several centimeters in XY and 10 cm inZ. For these data sets, the camera FOV is 35 deg,the baseline is 50 cm, and the frame rate is 10 Hz(512 × 384), so there is often large image motion. Wetook data sets from Little Bit (9-km trajectory, 47,000frames) in Pennsylvania and Ft. Carson (4 km, 20,000frames) in Colorado, to get variety in imagery. TheFt. Carson data set is more difficult for matching,with larger motions and less textured images. In theexperiments, we use only CenSurE features, whichfailed the fewest times (0.17% for Little Bit, 4.0% forFt. Carson).

The VO angular errors contribute nonlinearly totrajectory error. On the two data sets, we comparedRMS and max XYZ trajectory errors. In the case ofmatching failure, we substituted IMU data for the an-gles and set the distance to the previous value. InTable II, the effects of bundle adjustment and IMU fil-tering are compared.

In both data sets, IMU filtering plays the largestrole in bringing down error rates. This is not surpris-ing, because angular drift leads to large errors overdistance. Even with a noisy IMU, global gravity nor-mal will keep Z errors low. The extent of XY errorsdepends on how much the IMU yaw angle drifts overthe trajectory: in our case, a navigation-grade IMUhas 1 deg/h of drift. Noisier IMU yaw data wouldlead to higher XY errors.

Table II. Trajectory error statistics, in meters and percentof trajectory.

RMS error Max errorin XYZ in XYZ

Little VO No SBA 97.41 (1.0%) 295.77 (3.2%)Bit VO SBA 45.74 (0.49%) 137.76 (1.5%)(9 km) VO No SBA + IMU 7.83 (0.08%) 13.89 (0.15%)

VO SBA + IMU 4.09 (0.04%) 7.06 (0.08%)Ft. VO No SBA 263.70 (6.9%) 526.34 (13.8%)Carson VO SBA 101.43 (2.7%) 176.99 (4.6%)(4 km) VO No SBA + IMU 19.38 (0.50%) 28.72 (0.75%)

VO SBA + IMU 13.90 (0.36%) 20.48 (0.54%)

The secondary effect is from SBA. With or with-out IMU filtering, SBA can lower error rates by onehalf or more, especially in the Ft. Carson data set, inwhich the matching is less certain.

3.5. Map Reuse

VO and IMU/GPS filtering enable us to constructconsistent maps on a single run. These maps are use-ful for getting out of traps and cul-de-sacs in the envi-ronment, which occurred quite frequently. In fact, thetesting team was interested in long-range sensing ca-pabilities and would use natural or constructed trapsas a way of rewarding robots that could detect themfrom a distance. Unfortunately, the vision sensors onthe robots were not very capable at a distance [seeSection 6 and Figure 18(a)]. So, our strategy was touse map information learned in the first run to com-pute an optimal path for the second and subsequentruns. This type of learning, run-to-run learning, turnedout to be the most powerful form of learning for thetests and the key to performing better than any otherLAGR team.

Our first successful test of map learning andreuse was in Test 25 at the end of the project [Figure 14and later in Figure 18(a)]. The direct line to the goalwas through a small copse of trees, where there werebarriers of deadfall and tall grass. In the first run, therobot wandered through this area, eventually findinga way out to the goal. In the second run, the robotstarted with the map constructed on the first run andheaded around the problem area. Note that the robotactually started into the cul-de-sac and then decidedto go around. The planner had a finite horizon ofabout 40 m and recognized the blockage only at that



Figure 14. Map reuse during Test 25. The global map in (a) shows the first run: black is free space (including long sight-lines), and white and gray are obstacles. The robot path estimated from VO is the yellow line. The starting position of therobot is the left side of the screen; the goal is on the right at about 80 m. Note the many extended concave obstacles and cul-de-sacs. Image (b) shows the robot’s trajectory for the second run in green, bypassing the cul-de-sac obstacles and headingaround to the right. The original run is superimposed.

point. In subsequent tests we extended the horizon ofthe planner to the goal.

Our map-reuse technique is simple: at the startof a run, match the robot’s view to the start of theprevious run, using the same method as for match-ing frames in VO. If a good match is found, the mapfrom the previous run is brought in and adjustedto the robot’s current position. From this point therobot’s position on the old map is “open loop,” thatis, there is no reregistration or localization of the robotwithin the map. Because VO performance is generallywithin 1% over 100 m, this strategy was overwhelm-

ingly successful during the tests. Still, a true visualSLAM algorithm would work better in more difficultconditions, and we have made significant progresshere, closing loops over 5-km data sets (Konolige &Agrawal, 2008), but unfortunately this research wasdone too late to incorporate into the LAGR system.

4. PLANNING

The LAGR robot was provided with a “Baseline” sys-tem that used implementations of D* (Stentz, 1994)for global planning DWA (Fox et al., 1997) for local



control. Using this system, we (as well as other teams)had frequent crashes and undesirable motion. Themain causes were the slowness of the planner andthe failure of the controller to sufficiently accountfor the robot’s dynamics. The D* planner is opti-mized for very large-scale environments. It uses dy-namic programming to compute the minimum-costpotential to the goal at each cell; it needs significantresources to maintain the indices necessary to un-ravel the minimum-cost computations incrementally.In our environments (100 × 200 m, 20 cm2 cells) itwould take many seconds to compute a plan, evenwhen only a small portion of the map was filled.For large-scale maps this may be acceptable, but weneed much faster response to tactical maneuvers oversmaller scales (e.g., cul-de-sacs).

Instead, we reimplemented a gradient planner(Konolige, 2000; Philippsen & Siegwart, 2005) thatcomputes optimal paths from the goal to the robot,given a cost map. The gradient planner is a wavefrontplanner that computes the cost of getting to a goal orgoals at every cell in the workspace. It works by us-ing a local neighborhood to update the cost of a cell.If the cell’s cost is higher than the cost of a neighborcell plus the local transit cost, then it is updated withthe new cost. The overall algorithm starts by initializ-ing the goal with a zero cost and everything else witha very large cost. All goal cells are put onto an “open”list. The algorithm runs by popping a cell of the openlist and updating each of the cell’s neighbors. Anyneighbor that has a lowered cost is put back onto theopen list. The algorithm finishes when the open list isempty.

There are many variations on this algorithm thatlead to different performance efficiences. Our algo-rithm has several unique modifications:

• Unlike other implementations, it uses a trueEuclidean metric, rather than a Manhattanor diagonal metric, in performing the updatestep (Kimmel & Sethian 1998). The update canbe performed on the four nearest neighbors ofa cell. Generally speaking, the two lowest-costneighbors can be used to determine the direc-tion of propagation of the cost potential andthe cell updated with an appropriate distancebased on this direction.

• The algorithm computes the configurationspace for a circular robot and includes safetydistances to obstacles. This is one of the inter-esting parts of the gradient method. Because

there is already a method for computing thedistance transform from a set of points, theconfiguration space can be computed effi-ciently. The obstacle points are entered as goalpoints, and the update algorithm is run overeach of these points, generating a new openlist. Each open list is processed fully, leadingto a sequence of open lists. At the end of n

cycles, the distance to obstacles has been de-termined up to n ∗ c, where c is the cell size.Usually this is done to a distance of three orfour times the robot radius, enough to estab-lish a safety cushion to the obstacle. Finally, acost is associated with the distance: an infinitecost within the robot radius to an obstacle anda decreasing cost moving away from this.

• The queue handling is extremely efficient,using threshold-based queues, rather than abest-first update, which has high overheadfor sorting. Instead, we use a two-priority-queue method. A threshold shuttles newcells to one queue or the other, dependingon whether their cost is greater or less thanthe threshold. The low-cost queue is alwaysprocessed first. When no more cells remainin it, the threshold is increased, the secondqueue becomes the low-cost queue, and anew high-cost queue is initialized. This queuestrategy is the key to the good performanceof the algorithm: each update step happensvery rapidly. Although the complexity ofthe algorithm is the order of the area to becovered, and there is no “best-first” searchfrom the goal to the robot position, still theextreme rapidity of each step makes it possi-ble to cover reasonable areas (e.g., 80 × 80 m)in several tens of milliseconds.

• Rapid switching of global paths is avoidedby including hysteresis: lowering the costalong the path. There is a trade-off betweensticking to the current path and exploringsome new path if current readings indicateit might be better. We lower the cost enoughso that it takes a significant amount of newinformation to turn the path aside.

Typically we run the global planner within a sub-region of the whole map because the robot is con-tinuously moving toward the goal and encounter-ing new areas. On longer runs, up to 200 m, we usean 80 × 80 m area; the global planner runs in about



Figure 15. Line goals for a robot in a 200-m environment. The line goal is placed 60 m ahead of the robot, and its extentvaries with the distance to the goal.

30 ms in this region. Unless there is a large cul-de-sac,longer than 80 m, this area is sufficient to maneuverthe robot tactically around obstacles. For more globalplanning, which occurs when starting a run with apreviously made map, we run the planner over thewhole area, which can take up to 100 ms for a large100 × 200 m map.

The global planner is optimistic in assuming therobot to be circular, with a diameter equal to thewidth of the robot. Also, it does not take into accountthe nonholonomic nature of the robot’s motion. In-stead, we rely on a local controller to produce feasibledriving motions (Section 5).

One of the problems encountered in directing therobot toward a point goal is that the plans tend to con-stantly urge the robot toward the center of the map.This is not necessarily an efficient strategy because,for example, the robot will prefer to run near vegeta-tion on the side of a path that does not point directlytoward the goal. Instead, when the robot is far fromthe goal, we posit a relaxed virtual goal line that al-lows the robot to pursue more indirect paths to thegoal (Figure 15). In a line goal, any point on the line isconsidered to be a goal, and the robot navigates to thenearest (lowest-cost path) position on the line. For ex-ample, in Figure 15 the robot is shown with the direc-tion of travel straight ahead to its line goal, whereaswith the goal point at the end it would want to movediagonally.

The line goal is easily implemented in the gradi-ent planner by simply adding all points on the line asgoal points. The navigation function then computes

the lowest-cost path to any point on the line. The linegoal is always placed about 60 m ahead of the robot,and its extent grows in the middle of the run and con-tracts as it gets nearer to the goal. In experiments, therobot is able to navigate more than 50 m off the centerline to the goal and consequently find easily traversedpaths that would have been difficult to find if it hadheaded directly to the goal (Figure 7).

5. CONTROL

Given the global cost information produced by thegradient planner, we must decide what local controlsto apply to the robot to drive it toward the goal.

5.1. Trajectory Generation

We take an approach that is opposite to techniquessuch as DWA. Instead of searching the space offeasible trajectories, we search the space of feasiblecontrols. As is the case with most differentially drivenplatforms, the LAGR robot is commanded by apair (x, θ ) of desired translational and rotationalvelocities.2 Thus we have a 2D space of possiblecommands to consider.

This space is bounded in each dimension by ve-locity limits that reflect the vehicle’s capabilities. Be-cause we are seeking good, as opposed to optimal,

2We could instead work in terms of left and right wheel velocities;the two velocity spaces are equivalent, being related by a simplegeometric transformation.



control, we sample, rather than exhaustively search,this rectangular region of allowed velocities. We takea regular sampling (∼25 in each dimension, ∼625total) and for each sample simulate the effect ofapplying those controls to the robot over a shorttime horizon (∼2 s). The simulation predicts therobot’s trajectory as a sequence of five-dimensional(x, y, θ, x, θ ) states with a discrete-time approxima-tion of the vehicle’s dynamics.

Of significant importance in this simulation arethe vehicle’s acceleration limits. Although the LAGRrobot can achieve a speed of 1.3 m/s, its low-levelmotor controller (which we cannot modify) followsa trapezoidal velocity profile that limits the transla-tional acceleration to approximately 0.5 m/s2 (we de-termined this value empirically). Thus more than 2 smay elapse between commanding and achieving adesired velocity. We found that the ability to accu-rately predict the LAGR robot’s future state dependsvitally on appropriate integration of these accelera-tion limits. We expect this to be the case for any vehi-cle with a similarly large ratio of maximum velocityto maximum acceleration.

The generated trajectories, projected into the(x, y) plane, are smooth, continuous 2D curves that,depending on the acceleration limits, may not be eas-ily parameterizable. For the LAGR robot, the trajecto-ries are generally not circular arcs (Figure 16).

5.2. Trajectory Evaluation

Each simulated trajectory t is evaluated by the follow-ing weighted cost:

C(t) = αObs + βGdist + γ Pdist + δ1x2 , (1)

where Obs is the sum of grid cell costs through whichthe trajectory passes (taking account of the robot’sactual footprint in the grid); Gdist and Pdist arethe estimated shortest distances from the endpointof the trajectory to the goal and the optimal path,respectively; and x is the translational component ofthe velocity command that produces the trajectory.We choose the trajectory for which the cost (1) is min-imized, which leads our controller to prefer trajecto-ries that (a) remain far from obstacles, (b) go towardthe goal, (c) remain near the optimal path, and (d)drive fast. Trajectories that bring any part of the robotinto collision with a lethal obstacle are discarded asillegal.

Note that we can compute C(t) with minimaloverhead: Obs is a simple summation over grid cellcosts, Gdist and Pdist were already computed by theplanner for all map cells, and x is a known constantfor each trajectory.

Figure 16. The controller generates trajectories by sampling feasible velocities and simulating their application over ashort time horizon. Generated trajectories are purple, the chosen trajectory is yellow, the desired global path is cyan, andobstacles are white. As shown in (a) and (b), the trajectories are smooth but not easily parameterizable as they depend onthe vehicle’s current velocity and its acceleration limits. When forward motion is not possible, backward trajectories areconsidered (c): robot is facing down toward obstacles.



5.3. Supervisory Control

We could generate, evaluate, and compare all poten-tial trajectories. However, given the kinematic design(driven wheels in front, passive casters behind) andsensor configuration (forward-facing cameras andforward-mounted bumper) of the LAGR robot, wefound it useful to add supervisory logic to direct theorder in which candidate velocities are simulated andevaluated.

All forward velocities (x > 0) are tried first; if anylegal forward trajectory is found, the best one is se-lected. If there are no legal forward velocities, thenthe controller tries in-place rotations (x = 0), and thenbackward velocities (x < 0). This preference order-ing encourages the robot to make forward progresswhenever possible and discourages driving back-ward (during which the robot is essentially blind). Ifno legal trajectory is found, the default behavior ofthe robot is to move slowly backward.

5.4. Slip Handling

Because the robot may have to traverse rough, steepterrain, it is necessary to detect and react to condi-tions in which the wheels slip or become stuck. Weemploy two mechanisms to handle these situations.In both cases, we are comparing the motion reportedby the wheels to the motion estimated by VO, whichis sufficiently accurate to be treated as ground truth(Section 3.2).

First, the controller continuously compensatesfor the slip in each wheel by reducing its maximumspeed. Our approach is similar to automotive trac-tion control. For each wheel, we monitor the slip ratios, defined as (Angelova, Matthies, Helmick, Sibley, &Perona, 2006):

s = ωr − v

ωr∈ [0, 1], (2)

where ω is the measured angular velocity of thewheel, r is the wheel radius, and v is the actual lin-ear velocity of the wheel. We obtain ω directly fromthe wheel encoders. To compute v, we difference se-quential VO poses to produce translational and rota-tional velocities for the vehicle and then use the ve-hicle geometry to distribute these velocities betweenthe two wheels. When the slip ratio s for a wheel ex-ceeds a minimum threshold (∼0.25), we compensateby proportionally reducing the maximum allowablespeed for that wheel, which produces better traction

on most terrain. Importantly, the controller takes ac-count of the current speed limits, ensuring that pre-dicted trajectories will be achievable under these lim-its. The slip ratios and speed limits are recomputed atthe frequency of VO pose estimation (∼15 Hz).

Although continuous slip compensation im-proves performance, there are situations in whichthe robot can become truly stuck and require ex-plicit escape mechanisms. The robot usually becomesstuck because of extremely slippery soil (e.g., sand) orground clutter (e.g., fallen branches). We detect theseconditions by looking for significant, time-extendeddisparities among the velocities that are commandedby the controller, reported by wheel odometry, andestimated by VO (we maintain a running window ofeach velocity). If a slip or stall is detected, or if thefront bumper is triggered, the robot enters a stochas-tic finite state machine of preprogrammed escapemaneuvers (e.g., drive forward, turn in place, drivebackward). These maneuvers are executed blindly, onthe assumption that the vision system failed to iden-tify the terrain as dangerous and so is unlikely toyield good advice on how to escape it.

One indication of how well slip detection per-formed was on Test 18, about 2 years into the pro-gram. This was a difficult test around small sanddunes, and the robots would spin easily on the sandon small inclines. The slip detection code and es-cape maneuvers, coupled with VO for localization,allowed us to finish this challenging course, the onlyteam to do so.

6. PERFORMANCE

For the LAGR program, the government testinggroup (LGT) ran monthly blind demos of the percep-tion and control software developed by the teams andcompared their performance to that of a baseline sys-tem. Teams were encouraged but not required to sendcode for each test; they could also send the same codeon successive tests. In general the tests would changeeach month, to expose the teams to different environ-mental conditions.

There were two major checkpoints for all theteams, the first at the end of Phase I of the programafter 1.5 years (Tests 12 and 13) and the second at theend of the program (3 years, Test 27). The goal was tobeat the baseline system at the end of Phase I and todo better by a factor of 2 for the second. In this sectionwe show results from all of these tests and addition-ally the penultimate Tests 25 and 26. At this point, our



system was essentially complete, and Tests 25 and 26presented interesting terrain challenges, whereas Test27 was somewhat artificial and designed to isolatespecific learning capabilities.

On each test, the robot was given four runs, andthe best three were taken to give a combined score.The highest achievable score is 1.0, calculated by mea-suring the shortest possible path to the goal and mea-suring the time it took a skilled operator to manuallydrive the robot. There were also penalties for not get-ting to the goal within a cutoff time.

6.1. Phase I Tests

The tests at the end of Phase I were designed to testthe overall ability of the system to handle typical out-door terrain (make a consistent map, recover fromslips, etc.), while focusing on perceptual skills andlearning. For these tests, we had completed a basicVO system, the global planner, and the controller.All of the main stereo interpretation algorithms werealso in place, including sightlines. However, ourappearance-based learning was only a simple super-vised color classifier, and there was no map reuse, be-cause the VO system was not precise enough.

In Test 12 [Figure 17(a)], a dark mulch pathformed the easiest way to the goal, and teams were

invited to submit automatic supervised learning codethat would be trained on a similar path from log files.The minimum times for a skilled operator were 69 salong the path (103 m) and 77 s through the maze(distance not measured).

Several teams managed to follow the path af-ter training and generally had good times. We didnot; the dark color of the path confused our color-based learner and led us to develop the combinedcolor/texture model described in Section 2.3. Wecompleted the maze course three times, with timesof 106, 110, and 112 s. Our best time equaled that ofthe best team following the mulch path, and our av-erage score of 0.73 was the highest of any team, threetimes the baseline score (which was also through themaze). Although the LGT expected the maze to be asignificant problem for the teams (as it was for thebaseline), our combination of consistent map making,fast global planning, and rollout controller combinedto make the robot zip through the maze with no hes-itation.

In Test 13, the objective was to stay along an olddirt and asphalt road for much of the course, untilthere was an opening in brush to the left of the roadtoward the goal [see Figure 17(b)]. The easiest way(most open route) to the goal was along the thirdroute from the left, which required the robot to stray

Figure 17. Aerial views of the Phase I tests. In (a), the haybale maze is drawn in blue, and the mulch path leading to thegoal is in red. The tree copse leading directly to the goal was not traversable. In (b), four possible routes to the goal aredrawn, with the third from the left being the easiest (154 m).



Figure 18. Views of three final tests. In (a), a robot’s-eye view of the beginning of Test 25. The copse in the distance could beavoided on the right or left. The yellow line is the robot’s horizon from a noisy INS, and the green line is the VO-stabilizedhorizon. In (b), a pipe corridor from Test 26b: note the blocked left corridor. In (c), Test 27a shows the Jersey barrier, withthe goal immediately behind.

far from the direct route to goal, following the openroad. The second route was actually the shortest, re-quiring a skilled operator just 90 s, and the third routetook 95 s.

In all three scoring runs, we followed the thirdroute, utilizing sight lines (Section 2.2) to find openspace along the road and then the open space alongthe third route. The times for the three runs were 132,132, and 148 s. All of our times were faster than thebest time of any other team, and our overall score,0.86, was just under 3× the score of the baseline. Weexpect that the better online color/texture path learn-ing of Section 2.3 would have found the second route,and a shorter time, but we did not develop this tech-nique until the end of the project.

6.2. End-of-Project Tests

The end-of-project tests were through different typesof terrain and with different degrees of difficulty. Forthese tests, our full system was operational, includingonline color/texture learning of paths and map reuse.Here is a summary of the courses (Figure 18):

Test 25 83-m straight-line distance to the goal,through a copse of trees with a cul-de-sacand tall grass [Figure 18(a)]. Ideal behav-ior was to go around the copse, followinga mulch path as in Test 12.

Test 26a (93 m) Narrow paths through tall bushes,with several false turns that might leadmore directly to the goal. Desired behav-ior was to avoid the false turns.

Test 26b (106 m) A challenging course with man-made and natural obstacles, including acul-de-sac of parked cars, stacked pipes,hay bales, and rock piles [Figure 18(b)].The course to the goal was indirect and in-volved narrow passageways, and findingit was a challenge.

Test 27a (34 m) A simple course on a grassyfield with Jersey barriers stretched directlyacross the route [Figure 18(c)]. Ideal be-havior would be to avoid the barrier with-out getting close.

Test 27b (34 m) Similar to 27a, but using low haybales for obstacles, with two gaps in thebarrier containing tall grass. The objectwas to identify the tall grass and pushthrough it directly to the goal.

The first four tests were designed to reward be-havior that could avoid routes that were temptinglydirect, but ultimately dead ends. There were twomethods of doing this: long-range perception (>10 m)and map memorization and reuse. For Test 26a, thenarrow routes through the bushes were easily de-tected by our online learning algorithms, and thepath planner moved the robot quickly along the cen-ter of the path. On the first run, the robot turned twiceto look briefly at side paths that could have beenmore direct but then turned back to the main route.Figure 19 shows the scores for this run. The Baselinescore is 0.23, and SRI’s score is 0.83, which is better bya factor of 3.6. In this test, because the long-range per-ception of paths worked well, the first run was very



Figure 19. Summary of results from the last three LAGRtests. Raw scores are given for the Baseline software andthe SRI system, where 1 is a perfect score (as fast as therobot can go). The other scores are presented as a factorover Baseline; the target performance for the project was2 × Baseline.

good (2.9 × Baseline), and subsequent map reuse con-tributed only a modest amount, by not turning to ex-amine the dead-end paths. In fact, our score couldhave been higher, but the fourth run failed becauseof a map registration error in the middle of the run,closing off the narrow path.

In the other three tests (25, 26b, and 27a), mapreuse is the primary enabler of good performance: itimproved by almost a factor of 2 from the first run.For example, in Test 25, after wandering through thecopse and encountering the cul-de-sac and tall grassobstacles, the robot made its way to the goal. Onthe second run, the robot avoided the copse entirely,choosing a path around it as less costly.

Test 27b was a learning-by-example test. Therobots were shown samples of the hay bales and tallgrass. Operators would drive the robots into the haybales and over the grass, to give the robot an ideaof the traversability of each. Our online learning al-gorithms correctly picked out the grass as drivable,based primarily on its texture, because the color wassimilar to that of the hay bales. We also learned thathay bales were obstacles; however, we had set thesuppression of obstacles by drivable objects a littletoo high, and the robot bumped the hay bales next tothe grass area. After a few bumps, it drove throughthe grass and onto the goal. In subsequent runs, ofcourse, map reuse allowed an optimal plan directlythrough the grass.

6.3. Analysis

There is no doubt that our system achieves both ro-bustness and good performance on a wide variety ofoutdoor, unstructured terrain. Map building relies onVO to provide good localization, efficient real-timestereo and robust ground-plane analysis for obstacledetection, and sight lines to identify distant regionsthat are likely to be navigable. Online path learninghelps in the very common situation of tracks throughvegetation, or man-made dirt and asphalt roads. To-gether these techniques allow us to construct well-registered, precise maps that serve well during thefirst run to get the robot reliably to the goal. Evenmore importantly, on subsequent runs, the path plan-ner is able to construct an optimal path to the goalfrom the start of the run.

Moving quickly is very important to achievinggood performance, especially because many smallobstacles such as branches could be traversed atspeed but might hang up the robot if it was movingmore slowly. As described in Section 5, the path plan-ner and local controller combined to give the robot avery agile feeling. Our average speed was more than1.1 m/s, even while exploring unknown terrain (topspeed of the robot is 1.3 m/s).

The government team was very interested in cre-ating scenarios to test the long-range perception ofthe robot. Unfortunately, the robot’s vision sensorshad very little resolution at distance. Depth informa-tion from stereo was very uncertain after about 7 m.Even using monocular information, very few pixelswere available for long-range sensing. In Figure 18(a),a high-resolution camera with a longer focal lengthclearly shows routes around the copse of trees to theleft. But looking through the robot cameras, there isvery little to show that the copse of trees could beavoided to the left: perhaps there are a few more ver-tical pixels of brown-colored grass on that side. Butthis information is insufficient to reliably navigatefrom the robot’s perspective, and teams that tried todo this would as often pick a bad way as a good one.

What we could reliably learn is the map structurefrom the first run. With this in hand, subsequent runscould be much more efficient. We had this techniqueworking reliably only in the last tests (25–27), and itwas difficult for the government team to react and setup tests that would allow long-range perception todo as well as map learning and reuse. It was also dif-ficult for other teams to adopt our technique, becauseit required very good map registration, and a badly



registered map is worse than no map at all. In Test26a, the narrow paths (≈2 m wide) meant that evensmall registration errors could cause a prior map toclose off the current path, which happened to us inthe fourth run. Note that the map reuse was run openloop: after registering with an initial image at the be-ginning of the run, we relied on VO to keep the robotlocalized.

We compared our results with the publishedresults of the other teams, both the average and thebest for each test (Figure 19). In all these tests, wehad the best score (or tied for the best). Typicallywe outperformed the average team by a factor oftwo. In the most difficult test, 26b, even our first-runscore was almost as good as the best overall teamscore; map reuse enabled us to do even better. Thecontroller, planner, and VO system were used in thebest-in-class NIST system, and in fact NIST was ourclosest competition in two of the tests, includingthe difficult Test 26b. We also surpassed the targetof 2 × baseline performance, achieving almost a4× improvement, better than the 3× improvement ofthe Phase I tests. There is no doubt that map reusewas the primary enabler for this performance.

Whereas many teams concentrated on findingobstacles at a distance using color-based learning, wedecided that the risk of using this technique in com-plicated environments was not worth the results. Insome simple (and contrived) scenarios, such as Test27a [Figure 27(c)], it could indeed help, but if it wereused all the time, it would be as likely to lead to badchoices as good (dark green areas could be shadowsinstead of trees) and cause poor overall behavior. Ouronline path learning, by contrast, used both geomet-ric and color/texture cues to reliably find good paths,with almost no false positives.

7. CONCLUSION

We have demonstrated a complete autonomous sys-tem for off-road navigation in unstructured environ-ments, using stereo vision as the main sensor. Thesystem is very robust—we can typically give it a goalposition several hundred meters away and expect itto get there. It is also one of the first to demonstratethe practical use of VO as the primary method of reg-istration, with extremely good results. The precisionof VO is such that maps can be reused on subsequentruns, doubling the performance of the system.

To be sure, there are hazards that are not dealtwith by the methods discussed in this paper: water

and ditches are two robot killers. We also were re-stricted to running open loop in reusing maps; wewould like to use visual landmarks to reregister theposition of the robot in the map, but this work wasnot ready at the time of the last tests.

REFERENCES

Agrawal, M., & Konolige, K. (2006). Real-time localizationin outdoor environments using stereo vision and inex-pensive GPS. In Proceedings of the International Con-ference of Pattern Recognition (ICPR) (pp. 1063–1068).Washington, DC: IEEE Computer Society.

Agrawal, M., & Konolige, K. (2007). Rough terrain visualodometry. In Proceedings of the International Confer-ence on Advanced Robotics (ICAR), Jeju Island, SouthKorea.

Agrawal, M., Konolige, K., & Blas, M. R. (2008). Cen-SurE: Center surround extremas for realtime fea-ture detection and matching. In Proceedings of theEuropean Conference on Computer Vision (ECCV),Marseille, France. Lecture Notes in Computer Science5305 (pp. 102–115). Springer.

Angelova, A., Matthies, L., Helmick, D., & Perona, P. (2007).Learning and prediction of slip from visual informa-tion. Journal of Field Robotics, 24(3), 205–231.

Angelova, A., Matthies, L., Helmick, D., Sibley, G., &Perona, P. (2006). Learning to predict slip for groundrobots. In Proceedings of the IEEE International Con-ference on Robotics and Automation (ICRA), Orlando,FL (pp. 3324–3331).

Bay, H., Tuytelaars, T., & Gool, L. V. (2006). SURF:Speeded up robust features. In Proceedings of the Eu-ropean Conference on Computer Vision (ECCV), Graz,Austria (pp. 404–417).

Bellman, R. (1957). Dynamic programming. Princeton, NJ:Princeton University Press.

Bellutta, P., Manduchi, R., Matthies, L., Owens, K., &Rankin, A. (2000). Terrain perception for DEMO III.In Proceedings of the IEEE Intelligent Vehicles Sym-posium, Dearborn, MI (pp. 326–331). IEEE ComputerSociety.

Blas, M. R., Agrawal, M., Konolige, K., & Sundaresan, A.(2008). Fast color/texture segmentation for outdoorrobots. In Proceedings of the IEEE/RSJ InternationalConference on Intelligent Robots and Systems, Nice,France (IROS) (pp. 4078–4085).

Engels, C., Stewnius, H., & Nister, D. (2006). Bundle ad-justment rules. In Proceedings of the Conference onPhotogrammatic Computer Vision, Bom, Germany(pp. 266–271).

Fischler, M., & Bolles, R. (1981). Random sample consensus:A paradigm for model fitting with application to imageanalysis and automated cartography. Communicationsof the ACM, 24, 381–395.

Fox, D., Burgard, W., & Thrun, S. (1997). The dynamic win-dow approach to collision avoidance. IEEE Roboticsand Automation Magazine, 4(1), 23–33.



Guivant, J., Nebot, E., & Baiker, S. (2000). High accu-racy navigation using laser range sensors in outdoorapplications. In Proceedings of the IEEE InternationalConference on Robotics and Automation (ICRA), SanFrancisco, CA (pp. 3817–3822).

Gutmann, J. S., & Konolige, K. (1999). Incremental map-ping of large cyclic environments. In Proceedingsof the IEEE International Symposium on Computa-tional Intelligence in Robotics and Automation (CIRA),Monterey, CA (pp. 318–325).

Happold, M., Ollis, M., & Johnson, N. (2006). Enhancingsupervised terrain classification with predictive unsu-pervised learning. In Proceedings of Robotics: Scienceand Systems, Philadelphia, PA (RSS) (pp. 901–914).

Howard, A. (2008). Real-time stereo visual odometryfor autonomous ground vehicles. In Proceedings ofthe IEEE/RSJ International Conference on IntelligentRobots and Systems, Nice, France (IROS) (pp. 3946–3952).

Howard, T., Green, C., & Kelly, A. (2007). State space sam-pling of feasible motions for high performance mo-bile robot navigation in highly constrained environ-ments. In Proceedings of the International Conferenceon Field and Service Robotics, Chamonix, France.

Iagnemma, K., Genot, F., & Dubowsky, S. (1999). Rapidphysics-based rough-terrain rover planning with sen-sor and control uncertainty. In Proceedings of the IEEEInternational Conference on Robotics and Automation(ICRA), Detroit, MI (pp. 2286–2291).

Johnson, A. E., Goldberg, S. B., Cheng, Y., & Matthies, L.H. (2008). Robust and efficient stereo feature trackingfor visual odometry. In Proceedings of the IEEE In-ternational Conference on Robotics and Automation(ICRA), Pasadena, CA (pp. 39–46).

Kelly, A. (1994). A feedforward control approach to the lo-cal navigation problem for autonomous vehicles (Tech.Rep. CMU-RI-TR-94-17). Pittsburgh, PA: Robotics In-stitute, Carnegie Mellon University.

Kimmel, R., & Sethian, J. A. (1998). Computing geodesicpaths on manifolds. Proceedings of the NationalAcademy of Science, 95, 8431–8435.

Konolige, K. (1997). Small vision systems: Hardware andimplementation. In Proceedings of the InternationalSymposium on Robotics Research, Nagoya, Japan (pp.111–116).

Konolige, K. (2000). A gradient method for realtime robotcontrol. In Proceedings of the IEEE/RSJ InternationalConference on Intelligent Robots and Systems (IROS),Tokyo, Japan (pp. 639–646).

Konolige, K., & Agrawal, M. (2008). FrameSLAM: Frombundle adjustment to realtime visual mappping. IEEETransactions on Robotics, 24(5), 1066–1077.

Konolige, K., Agrawal, M., & Sola, J. (2007). Large scalevisual odometry for rough terrain. In Proceedings ofthe International Symposium on Robotics Research,Hiroshima, Japan.

Konolige, K., & Beymer, D. (2007). SVS reference man-ual (Tech. Rep.) SRI International. http://www.videredesign.com/docs/smallv4.4d.pdf; accessed July 2007.

Latombe, J.-C. (1991). Robot motion planning. Norwell,MA: Kluwer Academic Publishers.

LaValle, S. (2006). Planning algorithms. New York: Cam-bridge University Press.

Leonard, J. J., & Newman, P. (2003). Consistent, conver-gent, & constant-time SLAM. In Proceedings of theInternational Joint Conference on Artificial Intelligence(IJCAI), Acapulco, Mexico (pp. 1143–1150).

Leung, T., & Malik, J. (2001). Representing and recogniz-ing the visual appearance of materials using three-dimensional textons. International Journal of Com-puter Vision 43(1), 29–44.

Liapis, S., Sifakis, E., & Tziritas, G. (2004). Color and/or tex-ture segmentation using deterministic relaxation andfast marching algorithms. Journal of Visual Communi-cation and Image Representation, 15, 1–26.

Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Com-puter Vision, 60(2), 91–110.

Maenpaa, T., & Pietikainen, M. (2005). Texture analysis withlocal binary patterns. In C. Chen & P. Wang (Eds.)Handbook of pattern recognition and computer vision,3rd ed. (pp. 197–216). Singapore: World Scientific.

Maimone, M., Cheng, Y., & Matthies, L. (2007). Two years ofvisual odometry on the Mars exploration rovers. Jour-nal of Field Robotics, 24(3), 169–186.

Martin, D., Fowlkes, C., & Malik, J. (2004). Learning to de-tect natural image boundaries using local brightness,color, and texture cues. IEEE Transactions on PatternAnalysis and Machine Intelligence, 26(5), 530–549.

Matthies, L., Maimone, M., Johnson, A., Cheng, Y.,Willson, R., Villalpando, C., Goldberg, S., Huertas, A.,Stein, A., & Angelova, A. (2007). Computer vision onMars. International Journal of Computer Vision, 75(1),67–92.

Montemerlo, M., & Thrun, S. (2004). Large-scale robotic3-D mapping of urban structures. In Proceedings of theInternational Symposium on Experimental Robotics(ISER), Singapore (pp. 141–150).

Moravec, H., & Elfes, A. (1985). High resolution maps fromwide angle sonar. In Proceedings of the IEEE Interna-tional Conference on Robotics and Automation (ICRA)(pp. 116–121).

Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F.,& Sayd, P. (2006). Real time localization and 3D re-construction. In Proceedings of Computer Vision andPattern Recognition Conference (CVPR), New York(vol. 1, pp. 363–370).

Nister, D., Naroditsky, O., & Bergen, J. (2006). Visual odom-etry for ground vehicle applications. Journal of FieldRobotics, 23(1), 3–20.

Philippsen, R., & Siegwart, R. (2005). An interpolated dy-namic navigation function. In Proceedings of the IEEEInternational Conference on Robotics and Automation(ICRA), Barcelona, Spain (pp. 3782–3789).

Rankin, A., Huertas, A., & Matthies, L. (2005). Evaluation ofstereo vision obstacle detection algorithms for off-roadautonomous navigation. In Proceedings of the AUVSISymposium on Unmanned Systems.

Spero, D. J., & Jarvis, R.A. (2002). 3D vision for large-scale outdoor environments. In Proceedings of theAustralasian Conference on Robotics and Automation(ACRA), Auckland, New Zealand (pp. 228–234).



Stentz, A. (1994). Optimal and efficient path planning forpartially-known environments. In Proceedings of theIEEE International Conference on Robotics and Au-tomation (ICRA), San Diego, CA (vol. 4, pp. 3310–3317).

Sunderhauf, N., Konolige, K., Lacroix, S., & Protzel,P. (2005). Visual odometry using sparse bundleadjustment on an autonomous outdoor vehicle. InM. Schanz, R., Lafrenz, P. L., & V. Avrutin (Eds.),Tagungsband autonome mobile systeme (pp. 157–163).Springer Verlag.

Thrun, S., Montemerlo, M., Dahlkamp, H., Stavens, D.,Aron, A., Diebel, J., Fong, P., Gale, J., Halpenny, M.,Hoffmann, G., Lau, K., Oakley, C., Palatucci, M., Pratt,V., Stang, P., Strohband, S., Dupont, C., Jendrossek, L.-E., Koelen, C., Markey, C., Rummel, C., van Niekerk,

J., Jensen, E., Alessandrini, P., Bradski, G., Davies, B.,Ettinger, S., Kaehler, A., Nefian, A., & Mahoney, P.(2006). Stanley: The robot that won the DARPA GrandChallenge. Journal of Field Robotics, 23(9), 661–692.

Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon,A. W. (2000). Bundle adjustment—A modern synthesis.In B. Triggs, A. Zisserman, & R. Szeliski, (Eds.), Visionalgorithms: Theory and practice, LNCS (pp. 298–375).Springer Verlag.

Varma, M., & Zisserman, A. (2003). Texture classifica-tion: Are filter banks necessary? In Proceedings ofComputer Vision and Pattern Recognition Conference(CVPR), Madison, WI (pp. 691–696).

Viola, P., & Jones, M. (2004). Robust real-time face detection.International Journal of Computer Vision, 57(2), 137–154.


Documents

Mapping, Navigation, and Learning for Off-Road Traversalaravind/documents/konolige08-mapping.pdf · 2009-02-20 · Mapping, Navigation, and Learning for Off-Road Traversal Kurt Konolige