Chun-Hung Chen", Vijay Kumar"':', and Yuh-Chyun Luo·i.

Motion Planning of Walking Robots Using Ordinal Optimization

A general approach for coordinating the legs of a multi-legged statically stable walking machine on an uneven terrain. The method of ordinal optimization is used to find a good motion plan for low travel time that specifies the footholds and the gait of the walking machine. Numerical simulations illustrate the approach and demonstrate that we can find a good motion plan for uneven terrain with high com­plexity in a reasonably short time.

Keywords: Motion planning, walking robot, optimization, simulation

Walking machines have the potential to emulate the superior off-road mobility that biological systems

exhibit. However, in order to realize this potential, a walking machine must be able to adapt its gait to the terrain and to select footholds intelligently. In contrast, much of the litera­ture in this area addresses symmetric, periodic and regular gaits that are optimal from the point of view of static stability [37], [25], but do not lend themselves to navigation on uneven terrain. A detailed analysis and classification of the different gaits for quadrupeds and hexapods is found in Song and Waldron [34].

Studies on locomotion in biological systems date back to 1957 when Muybridge[27] used photographs to study animal locomotion. Particularly relevant to this paper is the work by Wilson [39], who quantitatively described gaits of insects

walking on smooth horizontal surfaces. Cruse [8] discovered that stick insects place their

uneven terrain. This strategy reduces the problem of foothold selection to selecting the footholds for the two front legs and has been adopted for walking robots by OzgUner et al.[28] Finally, we mention the work of Pearson and Franklin[31] who performed a cinematic analysis of locusts on uneven ter­rain. This work shows that biological systems employ many different strategies to navigate uneven terrain and many of these strategies do not lend themselves to quantitative engi­neering models.

In walking robots, motion planning for walking on unstructured terrain primarily involves [ZO] (a) the determi­nation of the trajectory (direction) of the vehicle; (b) the selection of footholds in space; and (c) the sequencing of the footsteps in time. A great deal depends on the sensory infor­mation that can be obtained. The Adaptive Suspension Vehicle (ASV) built at the Ohio State University[38],[35] is equipped

with an optical scanning posterior legs close to the supporl sites of the immedi­ately anterior ipsilateral legs. This follow-the-leader behav­ior has been observed in hors­es by Hildebrand[12] and in goats[30], especially in

'iDepartment uf Systems Engineering, University of Pennsylvania, Philadel­

phia. /chchen}{ychluo/@seas.upennedu.'i'·'Department of Mechanical Enyi­neering, University of PennsyilJania, Philadelphia. kumar�'c entral.

cis. upenn edu. This work has been sllpported in part by N:,'F grants DMf-9732173 and eMS .91-57156, ARO yrant DMH04-96-J-()OO7, and the Ifniners;­

ty ofPe,msyilJQnia Research Poundat;rJlZ.

range finder with a range of approximately 30 feet, a field of view of 80 degrees in the azimuth and 60 degrees in elevation, and a resolution of 6 inches. The Ambler at the Carnegie Mellon University[l]

22 • IEEE Robotics & Automation Magazine 1070-9932/9Sf$10.00©1998 IEEE June 1998

z

Figure 1. A Model of a Six-legged Walking Machine.

a six-legged walking machine designed for planetary explo­ration, is equipped with a laser range scanner that provides 256x256 pixel images of the Lerrain at a rate of 2 Hz with a resolution of lcm, and a field of view of 60 degrees in the azimuth and elevation. Zuk and Dell'Eva[41] and Krotkov and Hoffman[181 describe three-dimensional laser range finders that can be used to obtain terrain elevation maps. However, as pointed out by Krotkov and Hoffman [18], even when such a terrain mapping system is available for constructing elevation maps at the appropriate resolution in the correct coordinate system, the available terrain map is incomplete (for example, because of occlusions) or uncertain (for example, because of image defects).

The first study of aperiodic gaits for walking robots that can be adapted to varying terrain conditions was by Kugushev and laroshevskij[19] McGhee and Iswandhi[26] built on this work and developed an algorithm for a free gait. The basic approach involved di!icretizing the terrain into cells, which were then classified based on sensory information into GO or NO-GO cells. The free gait sequences the legs and selects footholds from the GO cells by maximizing the stability of the vehicle as well as the 'availability" of each leg. A support leg is considered to be unav.lilable if the leg is close to its workspace boundary and the progress of the vehicle causes the leg to reach its workspace boundary. However, the free gait was not designed for locomotion on uneven terrain. Another disad­vantage was the substantial computational load placed on the on-board computers. Hirose [13] proposes a hierarchical con­troller that consists of a local path planner responsible for planning the motion of the vehicle, and a gait controller that sequences the legs while avoiding forbidden footholds. How­ever, he does not offer specifics for either module. Lee and Orin [24] addressed the automatic regulation of the body atti­tude and height during locomotion over uneven terrain but the gait was assumed to be known. Kumar and Waldron [21] proposed an adaptive gait control algorithm that strives to maintain a periodic, symmetric optimal wave gait while allow­ing deviations to asymmetric, aperiodic gaits to circumvent obstacles and to adapt to uneven terrain. The selection of footholds was based on a quasi-static analysis of the vehicle and force distribution considerations. However, their

June 1998

approach relies on an accurate geometric model of the terrain and is completely deterministic.

There is a lot of related work in the area of robot motion planning [23]. Much of this work treats the problem of plan­ning a collision-free path in an environment with obstacles [3],[lO]. Of particular interest is the work on mobile robots navigating uneven terrain. Pai and Reissell[29] describe a mul­tiresolution model for rough terrain that lends itself to motion planning of walking vehicles. Simeon and Dacre-Wright[33] consider a robot with wheels and a spring suspension on uneven terrain and find paths that do not violate stability con­straints. Shiller and Chen[3fi] treat the time-optimal control prublem of a puint robot with stability constraints. The work of Barres et al.[2], Gat et al.[ll] and Iagolnitzer et al.[17] is also relevant to the work presented in this paper. However, none of these papers deal with the problems of foothold selection and planning gaits for legged vehicles.

Chen and Kumar[5] propose the use of ordinal optimiza­tion to find optimal motion plans. A motion plan is defined as a sequence of vehicle positions and corresponding footholds that guarantee that the vehicle can be made statically stable. They consider the problem of planning the gait for several characteristic vehicle lengths in a given direction. They dis­cretize the configuration space and the leg workspace, and formulate the motion planning problem as a discrete combi­natorial optimization problem. The method of ordinal opti­mization[16] is applied to find optimal motion plans. Instead of finding the best motion plan, ordinal optimization concen­trates on finding a reasonably good plan with a significantly reduced computation time. The optimality of the plan as well as the confidence level associated with the plan can be quanti­fied by suitable measures as explained later in the paper.

In this paper, we adopt our previous framework[5] to solve the continuous motion planning problem with obstacles. We assume that we have a terrain map that is segmented into obstacles and free space. Obstacles can be positive or negative. Positive obstacles, such as rocks and trees, cannot be pene­trated and must be avoided, while negative obstacles are areas that do not offer viable footholds but can be stepped over. Examples of negative obstacles include ditches, bushes, pud­dles and soft, muddy spots. The motion planning problem includes the determination of: (a) the trajectory of the vehicle including its position and orientation; (b) the stepping sequence or lhe gait; and (c) the placement of the feet on appropriate locations on the terrain. The methods of ordinal optimization are used to generate a plan that is guaranteed to be "near" the best motion plan. Specifically, we can find a plan that is in a desired percentile (in the set of all motion plans) at a given confidence level. Depending on the available computational resources we can improve our confidence level and/or get closer to the optimal gait.

In the next section we formulate the motion planning problem for statically-stable machines walking on three­dimensional terrain. We then we describe the method of ordi­nal optimization and its advantages. Our motion planning problem is then used in our simulations to demonstrate the application of ordinal optimization and the adaptation of the vehicle's gait to uneven terrain. Finally, we discuss the appli­cation of the proposed methods to more complex problems.

IEEE Robotics & Automation Magazine • 23

PROBLEM FORMULATION

Geometric and Dynamic Modeling Consider the six-legged walking vehicle in Figure 1 walking on uneven terrain. The geometric and dynamic modeling is borrowed from previous work [34], [21]. All legs are assumed to be identical and the vehicle is symmetric about the x and y axes. The pitch, P, is the distance between a pair of adjacent contralateral legs and the width, W, is the distance between a pair of ipsilateral legs in their home (mid-stride) positions (see Figure 2). The length, R, is used to characterize the maxi­mum stroke or stride length of a leg.

We assume for simplicity that the only external force acting on the vehicle is the gravitational force and that the legs are controlled so that the vehicle body is kept horizontal. We also aSSUffile that the inertia of the legs can be neglected in compari­son to the rest of the vehicle which can be treated as a rigid body. The contact of the feet with the terrain can be modeled as point contacts. The support plane is defined as a best-fit (in the least squares sense) hori:wntal plane on which the point con­tacts can be considered to lie. The support polygon is a polygon on this plane formed by the convex hull of the projections of the support points as shown in Figure 2. These assumptions allow us to consider a simplified static model in which the vehi­cle is in thex-y plane and hence in a three-dimensional config­uration space (two translational freedoms and one rotational freedom). Further, the three-dimensional terrain can be mod­eled by a z=f(;r,Y) elevation map. In order for the vehicle to be statically stable, the projection of the center of mass (CM) along the vertical onto the support plane must be within the support polygon. The shortest distance between the projection of CM and the support polygon is a measure of static stability and is called lhe stability margin [34],[35]), denoted by SM in Figure 2. Thus the feet must be planted to guarantee a positive stabili­ty margin at each time instant.

The generalization of the model to a six-dimensional con­figuration space in which the height of the vehicle and the roll and pitch angles are included is discussed by [21]. They also discuss the use of a quasi-slalic model lo include inertial effects. The application of the methods of this paper to a six­dimensional model with inertial effects is straightforward and is not considered here.

The Motion Planning Problem We define an inertial reference frame, X-Y-Z, and a vehicle­fixed reference frame x-y-z. We consider a crab-like gait in which the velocity of the center of mass, v,, need not be paral­lel to the longitudinal axis (x) of the vehide. The longitudinal axis makes an angle e with the X axis of the inertial frame, as shown in Figure 3. The coordinates of the projection of the center of mass, :C, in the inertial frame are denoted by (X" Ycl. (Xi' Y;) and (Xi' yJ are the inertial frame and vehicle frame coordinates, respectively, of the position at which foot i (i = 1, 2, .. , 6) is placed. The elevation of the foothold depends on the elevation of the terrain at that point. All these coordinates are functions of time, denoted by t.

We discretize the time interval so that t belongs to the set of non-negative integers. The set of 7 coordinate pairs and the direction of progression at time t completely describe the

24 • IEEE Robotics & Automation Magazine

�������-""���"

I _ p "�-f-oot---p 2�1 r-- - "I- �

o Cer;t&( {home) pOSition of each leg • Contact be�we€n a supporting foot and the ground

Center of gravity. C

Figure 2. Plan View of a Six-legged Locomotion System.

y

I_R� I-R_I 1_

� p ---J>.i- p

(a)

x

(b)

I N

x

Figure 3. (a) The Courdinate System fur the Vehicle; (b) The Leg Work­spaces and the Dimensions of the Vehicle. The shaded overlapping rec­tangles show the workspace for each leg (foot). C denotes the projection

of the center of mass on the support plane. In the simulation, W=!5; H=4, R=6, and P=5 units.

June 1998

• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •

state of the system at that time. Thus, a motion plan for T time intervals is a sequence {Xc(t), Yc(t), a(l), Xi(t), Yi(t), i=I,2, ... ,6}, f = O,I, ... ,r.

Definition 1. SM(t,t+ 1) is the stability margin in the time interval (t,t+ 1) according to the definition in Figure 2. The stability margin of a motion plan, <P, is defined as:

'I> = min SM(t ,t + 1) O<t<;T-l

Motion planning reduces to the problem of finding such a sequence from an initial state at t=O to a final, possibly incompletely specified, state at t=T, where T may or may not be given. For example, given the initial state [Xc(O), Yc(O),

_. (J\\ � . (f\\l ......... ,1 4-L.. ..... . _,... ...... ;"" .... 'I'V\fl ...... + A:n +hn fin".ll ct':lto +h"::lt

In the next two sections we will illustrate the basic idea of ordinal optimization and show a "no-knowledge" ordinal opti­mization based approach yields reasonably good motion plans, while simple heuristics can significantly improve the performance of our algorithm.

ORDINAL OPTIMIZATION Ordinal optimization was proposed by Ho et al. [l6J and has been successfully applied to many stochastic decision prob­lems [4],[14]'[15]. The basic idea is very simple. Instead of finding the best design or plan, ordinal optimization concen­trates on finding a good enough design with a significantly reduced computation time. In this section, we demonstrate the application of ordinal optimization to deterministic com­binatorial optimization problems. In a later section we will <Alup thp mAtinn nl�nnind nrnhlf'm llsing the method of ordi-

ROBOTICS & AUTOMATION MAGAZINE Reader Service Card

'lease send me more information on advertisements and new products: Instructions:

iociated metric which we � of the set is not neces­:lOther set G c;;; S is a sat­:he best design in S, the ;kly. Denote

I 2 3 4 5 6 7 l) 10 :1 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 11 32 33 34 35 36 37 38 39 40 'lease enter my subscription to Robotics & A.utomation:

] IEEE Member $10.00 1:1 Individual Subscription $50.00

\fame

ritle _____________________ _

::ompany or University

I\ddress __________________________________ __ _

City

State/Zip _____________ _ Country ______________ _

Telephone ___________ _ FAX ________ _

Signature ______________ ______ Dale

Note the reader service number at the bottom of

each advertisement or new product, circle the cor· responding number on this card, and mail to our

service center.

For faster response contact IEEE Member Services. 800-678-IEEE; FAX (908) 9819667; email: application.request@ieee.org

Membership in the IEEE Robotics and Automation Society includes subscriptions to the IEEE Transactions on Robotics and Automation.

and the IEEE Robotics and Automulion Muguzine

Please send me more information about: o Regular IEEE membership in R&A Society o Associate Membership in IEEE R&A Society

o IEEE student membership in R&A Society

o Individual Subscription to IEEE Robotics & Automation MagaZine

June 1998 Issue, Valid to October 1, 199B; International Valid to November 1,1998 RIA 06/98

ion of good designs

)blems in which the total large. If we randomly get t this sample is in G is 0:. th replacement, igns is inC} :signs is in G}

least one of the selected east as high asp, Im(l)

In(1-0:)(2) .bability that the selected (99.9 percentile) to be as ve need only 1) = 4602.8", 4603

--�cep,n�t�errnoJfnm�a�s<S"ln1T1umn1l1t�t"lm�e�a�sTfooIIT.IomwN·s�:------------------�S�a�lI�Ip�I�e�s�lITO�dTCnIl�Ie�'�e�'TIUrCTIII�a�CTv�II�Ij dence level for a good

X,(t)::; X,(t + 1)::; X,(t)+ 11x' Yc(t)-Tlq ::;r;(t+l)::;r;(t)+1ly, e(t)-1111 ::;a(t+l)::;e(t)+1le,

where llx and lly are the maximum change of the eM position in the X and Y directions in one time interval, and 118 is the maximum change of a(t) in one unit time.

Finally, in order to avoid collisions between the legs we prevent a posterior II�g from being placed ahead of the imme­diately anterior ipsilaieral leg:

June 1998

Xj(t) > x3(t) > x5(t) x2(t} > x4(t) > X6(t)

design. Note that this number is independent of the size of the sample space lSI. More generally, we have a formal defi­nition of optimality [5].

Table 1: Required Number of Samples for Obtaining an (CJ, p )-Optimal Design.

p tup 1% top 0.1% top 0.01%

50% 69 693 6,932

90% 230 2302 23.025

99% 459 4,603 46,050

99.999% 1,146 11506 115,110

IEEE Robotics & Automation Magazine • 25

Definition 2. An (ex, p)-optimal design is a design which is within the top ex percent with probability p.

Table 1 shows the required number of samples for obtain­ing an (a, p )-optimal design. Note that to obtain these results, we assume no prior knowledge of the physics of the problem and make no assumption on the structure of the domain while sampling the design space other than the existence of an appropriate metric. In robotics applications, we either have some prior knowledge of the design space based on the physics of the problem, or this knowledge can be acquired over a period of time.

The results in Table 1 were obtained by using a "blind" search strategy. Tn most applications, we have some prior knowledge of the set of motion plans based on the physics of the problem. If this knowledge is used to guide our search or sampling strategy, we can obtain better resuJt� with fewer samples. In most problems, the knowledge or physical intu­ition can be used to develop simple heuristics. For example, in our example of statically stable walking, it is easy to see that a longer stride length is preferred if the goal is to maximize velocity. On the other hand, if stability is to be maximized, it is desirable to keep as many legs on the ground as possible. A more detailed analysis of the effect of heuristics on the sam­pling quality can be found elsewhere[7l.

In the motion planning problem defined earlier, at any time t, we need to decide where the eM will move, which legs will be lifted, and on where to place the legs. Chen and Kumar [6J discretize the working space and formulate the motion planning problem as a discrete combinatorial optimization problem. The number of possible selections in such a discrete problem can be extremely huge (e.g., over 10184 for a problem with 25 time steps). In this paper, instead, we will use ordinal optimizaLion in the continuous framework. Thus the only dis­crete countable selection is the decision of which legs to lift. At each time instant t, if L legs are lifted and placed (clearly L ::; 3 for static stability), the number of choices for liftingL legs and then placing them is

6CL> where L = 0, 1, 2, 3, and

3 I sCL =42. i,==(j

For the complete motion plan, we already have (42)7 alterna­tives [or legs selection. For example, T=25, then (42)25 is about 3.81x1040. This is already a very large number without considering the sdection of the center of milSS movement ilnd leg placement. Since the working space is continuous, the number of selections is infinitely large. It is impossible to search the whole design space to find an optimal motion plan. However, as shown in Table 1, we can have a good motion plan with very limited number of samples if ordinal optimiza­tion is applied. In the next section, we will show ordinal opti­mization can effectively obtain a good motion plan within a reasonably short simulation time.

SIMULATION In this section we apply our ordinal optimization approach to three different motion planning problems. We will use the units of feet for distance and seconds for time. In each prob-

26 • IEEE Robotics & Automation Magazine

,------------'----------------,

5 6 7 8 9 10 11 Xi

Figure 4. A Prohability Density FWlction for Foothold Selection in Heuristic Approach.

lem, the parameters of this walking machine are, as shown in Figure 3, P = 5, R = 6, W = 6, H = 4, 11x c= 3, Tly = 1, and Tl8 =

100. This translates to a peak forward velocity of 3 feet/sec, a peak lateral velocity of 1 feet/sec, and a maximum angular velocity of 10o/second. In all the simulations, the initial state is [Xc(O), Yc(O), e(O)J = [5, 6, OOJ and ([X;(O) , Y,(O)], i=l, . . , 6} =

{[J 0,9], [10, :1], [5,9], [5,3], [0, 9], [0, :m.

Strategies for Sampling the Set of Motion Plans Ordinal optimization requires sampling the set of motion plans and evaluating each motion plan using a designated cost function. There are three basic approaches to sample the set of motion 'plans:

Unbiased Search If we do not possess any knowledge of the physics of the prob­lem or the constraints, we want to sample the set of motion plans or sample possible footholds using a uniform distribu­tion. For example, when selecting which leg to lift at any time t, the probabilities of lifting any combination of legs, or of not lifling any leg are all the same. D uring the decision of eM movement, when we have to determine how far eM has to move on the x-axis and the y-axis, we generate two random numbers using uniform distributions.

Heuristic Search We can use our intuition or our knowledge of the physics of the problem to bias our sampling of the set of motion plans. For example, in the example of foothold selection, since we want to minimize the travel time to the goal, we may discriminate against footholds that are not toward the front of the vehicle (in the direction of progression). At the same time, we do not want to exclude other footholds. Thus, we may use a simple heuristic to sample the footholds (x is between 5-11 in the example) according to the density function shown in Figure 4.

The biased sampling idea is also applied to the eM move­ment determination. When determining th e direction of machine progression e, a simp le triangular distribution is used to bias our sampling strategy. To reduce travel time, it is a good idea to orient the vehicle so that it is pointing the des­tination since its longitudinal velocity (Tlx) is much higger than its lateral velocity (11,,). We adopt the following heuris­tics. First of ali, the angle between the current x-direction and the direction to the destination, say .6.El*, is calculated. While trying to increase the orientation by .6.8', we don't want to exclude other possibilities for .6.8'. We want to hias our sam­pling strategy toward .6.8' but allow other sampling values so

June 1998

Figure 5. A Probability De�sity Function for Progression Direction

Determination in Heuristic Approach when -lj, < /lS' < 118'

that we can negotiate terrain constraints and obstacles. If -118 < �e' < 119' the triangular distribution shown in Figure 5 is used. If I�e'l > 11e, a similar triangular distribution with a peak at 118 or -118 is used.

Off-line Search Finally. we may need to determine the optimal motion plan in order to evaluate the performance of the unbiased or the heuristic approach by comparing the result with the optimal plan. However. finding the optimal motion can be very diffi­cult if not impossible because of the large search space even if we pursue the search off-line. The best we can do is to find a very good plan with a high confidence level off-line. From Table 1, we know that a top-om % plan with 99.999% confi­dence level can be obtained with only 115,000 random sam­ples. We expect the heuristic approach to have better performance (we will show this is true). Thus, we can apply the heuristic approach with a large number (greater than a billion) samples to arrive at a very good plan which is at least as good if not better than a (10-8, yy.gYY%)-optimal plan.

Example 1: Smooth Terrain This is a well studied problem and analytical solutions for steady-state optimal gaits can be found in the literature [20]'(34). We use this special case to test our algorithms with the unbiased and the heuristic approaches. The terrain is known to be smooth with no obstacles. Our objective is to minimize the travel ti me for the machine to reach the destina­tion [30,30] while ket:ping the vehicle stable. The final orienta­tion is left unspecified. Thus we choose to minimize T, subject to <I>:2: 1 and [Xc(T), r�(T)] � [30, 30].

It is well known that the optimal gait (for minimizing the travel time for a given distance) for straight line locomotion is a tripod gait. [A tripod gait is a periodic gait in which, at any given time, the front and rear feet on one side and the middle foot on the other side provide support. The gait is symmetric in the sense that the motion of contralateral legs on the same segment (front, middle or rear) are half a cycle out of phase. Further, the gait is a. wave gait. In other words, each leg acts as a support Ie!! for exactly half the time period and the time interval between the deployment of any pair of adjacent ipsi­lateral legs is equal to half the time period. The tripod gait can be shown to be optimal in terms of maximizing speed [34]. It is commonly seen in cockroaches [9].] However, there is no known solution for the optimal gait starting from an arbitrary initial gait or state (for example, a stand-still positiun). All known results are for periodic, steady-state gaits [20]. There-

June 1998

Table 2: Locomotion on Smooth Terrain.

Case 1 2 3 4 5 () Average

Unbiased 42 47 45 41 46 44 44.1

Heuristic 13 13 13 13 13 13 13

fore, we use the off-line approach to find a (10-8, 99.999%)­optimal plan that yields T* = 12. This plan is shown in Figure 6. The path of the center of mass is nominally a straight line and the orientation of the vehicle changes monotonicaIIy from its initial orientation to a final orientation. On closer inspection, the path of the center of mass shows an undulato­ry, side-to-side sway. This can be easily explained if we look at the gait (not shown in the figure). The vehicle transitions from a stationary position with six support legs to a tripod gait with only three support legs. The center of mass sways from side to side in an effort to maximize the stability margin by moving toward the side with two supporting feet.

Finally, note that even if we consider the optimal solution for a periodic tripod gait [in other words, we assume that the vehicle immediately adopts a steady state, periodic tripod gait that is described as being optimal in the literature [34]], the minimum travel time from [5, 6] to [30, 301 is (with a maxi­mum of 3 units per time interval) is 11.6 time units, very close to the 12 time units for the motion plan found by ordi­nal optimization. This further reinforces our very high degree

0 9 2

28 crt 2. 25 3 i

24 2 22 2 20 i9 is 17 i. is " 13

i i2 1 10 9 •

o 1 , 2 , 4 5 , 7

�

1 1 l' 1 1 24' 2627282930 B 9 011 1? 31415 0171892022223 25

Figure 6. A (10.8, 99.999%)-optimal Plan for Smooth-Terrain Locomotion.

The dots are the pOSitions of the center of mass at different t. In addition, the

starting position and the final position of the vehicle are shown.

IEEE Robotics & Automation Magazine • 27

Table 3: Locomotion on Simple Non-smoolh Terrain.

Case 1 2 3 4 5 6 Average

Heuristic 14 14 14 14 14 14 14

of confidence with OLlr (10-8, 99.999%)-optimal plan that yields T* = 12.

We can now explore the performance of the unbiased approach and the heuristic approach and compare the result­ing solution with the (10-8, 99.999%) optimal solution. Tahle 2 contains the results after testing 1000 motion plans for differ­ent random seeds. We observe that with only 1000 samples, on average, the unbiased approach finds a candidate plan with T =

44.1. Recall that the unbiased approach does not assume any knowledge of the physics of the problem. In contrast, the sim­ple heuristic approach can reduce travel time T down to 13 with only 1000 samples. The computation time on a Pentium PC (100 MHz) is only 10 seconds. Given that the total number of candidate motion plans lSI is infinitely large, this is a tremendous savings in terms of computational load. The travel time was 13 units in all our trials and it is only 8% more than the (lO-R, 99.999%) optimal solution of 12 time units.

Example 2: Uneven Terrain with Low Complexity In this and the next example, we consider a terrain with obstacles. The vehicle must plan a path around a positive obstacle and avoid possible collisions. In contrast, the vehicle

Figure 7. A (10-8, .9.9.9.9.9%)-optima! Plan in a Terrain with Obstacles. The

light patterned area represents a ditch and the dark shaded area represents a

positive obstacle which the vehicle cannot cross. The dots denote the posi­

tians of the cenler of milSS for different times.

28 • IEEE Robotics & Automation Magazine

can step over a negative obstacle and "go through it." It is simply a region that does not offer any viable footholds.

In this example (see Figure 7), we consider a terrain map with a ditch-like negative obstacle (shown by the light pat­terned area in the figure) and a positive obstacle (shown by the dark shaded square). The objective, once again, is to mini­mize the travel time to the destination [30, 30], while requir­ing stability (<'I> � 1). But now the vehicle cannot step on the negative obstacle and must avoid the positive obstacle.

A (lOb, 99.999%)-optimal plan obtained using the off-line approach is shown in Figure 7 in which T* = 14. Comparison with the (10-8, 99.999%)-optimal plan for locomotion on smooth terrain reveals that the presence of the positive obsta­cle forces the machine to take a longer path around the posi­tive obstacle, resulting in longer travel time. In addition to being able to traverse the ditch, the algorithm also minimizes the travel time without hitting the obstacle. Note that the final orientation of the vehicle is different from what it was in the example with the smooth terrain (Figure 6).

The results of six experiments with different random seeds using 1,000 heuristic samples are shown in Table 3. Our sim­ple heuristic approach returned successful plans that arrive at the destination in all cases with T = 14, the same as the (10-8, 99.999%)-optimal plan.

Example 3: Uneven Terrain with High Complexity This example is similar to Example 2, except that the terrain has more positive and negative obstacles. In the terrain shown in Figure 8, the light patterned area shows negative obstacles while the dark shaded areas represent positive obstacles. A (10-8, 99.999%)-optimal plan obtained using the off-line approach is shown in the figure in which 1'* = 14. Figure 9 provides intermediate snapshots to illustrate the selection of footholds and the planning of the trajectory around obstacles and over ditches.

If we look at the positions of the center of mass at succes­sive time steps, it is easy to see that the velocity of the vehicle is not significantly affected by the presence of obstacles. Fur­ther, because of the ability to select footholds anywhere in the leg workspace, it is possible to maintain the required stability margin even with a tripod gait. (While it is not shown in this example, by increasing the requirements on the sjability mar­gin, or by increasing the area with negative obstacles, we can force the vehicle into a four- or five-legged gait.) Finally, note that the final orientation of the vehicle is very different from lliose in Figures 6 and 7. This is because the presence of addi­

tional obstacles forces a change in the path and the vehicle must come around the final circular obstacle to get to the final destination.

The results of six experiments with different random seeds using 1000 samples are shown in Table 4. Our simple heuris­lie approach returned successful plans that arrive at the desti­nation at average T = 14.6, which is only 4% more than the (108, 99.999%)-optimal plan.

DISCUSSION

Ordinal Optimization Ordinal oplimization allows us to quickly isolate a good

June 1998

••••••••••••••••• • ••••••••••••••••••••••••••••• •••••••••••••••

motion plan with a high probability. However we' have not investigated the relationship between the (a, p)-opti­mal plan and the globally optimal plan. In fact, we have not even bothered to topologize the set of all motion plans. Further, in general, the measure of optimality of an (a, p)-optimal plan is not guaranteed to be close to that of the optimal plan. With some knowledge of the problem domain it may be possible to establish bounds on the deviation betw.:en the optimality measures of the selected plan and the optimal plan in terms of a. But in general, this is very difficult. Chen et al. [7] consider a simple motion planning problem and explore its relative­ly small search space �xhaustively. They found that it is very difficult to get an extremely good or an extremely poor plan using a unbiased sampling approach, but it is very easy to obtain a plan which is reasonably close to the optimal plan.

Terrain Uncertainty Chen and Kumar [5] provide an effective way to incorpo­rate uncertainty in the world model into motion plan­ning for walking vehicles. This approach can be directly applied to our planning problems without any change. Based on the sensory information on the terrain, the probability of failure associated with placing a foot on any point can be obtained. Denote bx.y as the probability of failure of a footholj at point [X, Yj. In other words,

���������� 29 H-+-+++++ 281-++-1+-1-++ 27 1-+-++++++ 25H-+-++++ " 2. 23 22 21 20 19 18 17 16 15 14 13 12 11 H-++-HH-+++-I++-+-10

Figure 8. A (l08, 99.999%)-optimal Plan in a Terrain with Obstacles. The light pat­

terned area represents a ditch and the dark shaded area represents a positive

obstacle which the vehicle cannot cross. The dots denote the positions of the center

the probability that point [X, Y] can provide the required of mass for different times.

support and traction is (1 - bxy). These probabilities illustrate the uncertainty in the

'environment. In a determin­

istic environment, these probabilities are either zero or unity. If the bx yare all independent, the probability of success of a motion

'plan {Xc(t) , Vc(t), e(t), XJt), YM), i=I,2,,,,6}, t =

O,I, ... ,T, is:

Based on this, we can have a stability metric [5] for the stabil­ity of a vehicle executing a given motion plan in a fairly obvi­ous fashion as the product of the stability margin and the probability of success for a motion:

'I' "<P fUl[ 1- bx, (t) y,(t) ] . t:::l j:::l

With the measurement of stability, we can trade off vehicle speed with safety in an environment with uncertainty. For illustration purpose, we include a simple example presented [5]. This simple example can be obtained by constraining the vehicle to move in a planar gait where all pairs of ipsilateral legs move together. In other words, the center of gravity

Table 4: Locomotion on Complicated Non-smooth Terrain.

Case 1 �� 3 4 5 6 Average

Heuristic 15 15 14 15 14 15 14.6

June 1998

moves in a straight line; Legs 1 and 2 are placed and retracted together, as are Legs 3 and 4, and Legs 5 and 6. These con­straints can be incorporated using a three legged model with Legs 1,3 and 5 where:

(1) If Leg i is lifted, Leg i + 1 will be lifted too, (2)Xm =Xi+1(t), (3) Y;(t) = 3, Y,+1 (t) = -3, and (4) 8(t) = 0°.

The motion planning problem reduces to planning the gait of a planar walking machine with three legs (numbered 1 through 3) moving in a straight line as shown in Figure 10.

We show an example of uneven terrain in Figure 11. Based on the sensory information on the terrain, we assume that the failure probabilities to place a leg at X, bx. for all X are avail­able: b(145.15.5]=0.05, b(15.5.16.5]=0.IO, b(16.5.17.5:=1.00, b \17.5.18.5]= 1.00, b(18.5,19.5]=0.09, b (19.5."0.5]=0.04, b(20.5.21.5]=0.02; otherwise bx=O. Note that no leg can be placed between 16.5 and 18.5. Given the physical limitations of the machine, it is impossible to avoid all the points with a non zero bx.

OUf objective in this example is to maximize the distance traveled while keeping the vehicle stable. Thus we choose to maximize Xc(25). Table 5 shows the results of the (10-8, 99.999%)-optimal plans with different limits on the stability metriC, 't'. By sacrificing the requirements on the stability

metric we can achiever faster speeds. The tradeoff between speed and stability is evident.

System Dynamics In the motion planning problem, we have completely ignored the dynamics of the system and this is perhaps the biggest

IEEE Robotics & Automation Magazine • 29

t = 5

t = 7

t = 9

, t t

!

, t t

!

20

t = 6

t = 8

t = 10

, 1 t

!

, 1 t

Figure 9. Excepts o( a (10.8, 99.999%)-optimal Plan (or the Complicated Terrain Maneuver. The top view of the motion for t = 5, 6, . . ,10, is shown with 6 snapshots. In each frame, the rectangle is the boundary of the vehicle, The "." at the center of vehicle denotes the projection of the center of mass and

other dots denote the footprints. In the frame t = i, the darker iriangfes curre'pond Iv footholds a/ / = i, " Ii, iJ), an insi(mt after time � while the gray

triangles depict footholds at t = i." (U, ij, an instant before time i. Note that the stability margin is always greater than or equal to 1 unit.

30 • IEEE Robotics & Automation Magazine June 1998

C leg 3 leg 2 leg 1 �--��----------��----------

Figure 10. A Planar Simplified Model.

Direction of Progression

'II! !J

x

plan at a given confidence level (p); and (c) a method to incorporate a terrain map into the motion planning. The basic ideas were il lustrated using three-dimensional six­legged models of walking vehicles. The main advantage of this approach is that we can very quickly identify a set of good motion plans for unstructured environ­ments regardless of the complexity of the terrain. The disadvantages include the diffi­culty i n incorporating dynamic considera­tions and the inability to identify the best possihle motion plan. tbX

1 .0

L

_--,-:

_--,JL

=-__ .o::::=� _

_ •

·" 1 0 11 12 1 3 1415 1617 18 19 2021 2223 ··· x ·· ·, 1 0 11 12 1 3 1415 1 6 17 18 1 9 20 21 2223 "'" x

REFERENCES II} Barres, J. and Whitaker, W. 1993. Configuration

of autonomous walkers for extreme terrain. International Journal of Robotics Research.

Figure 11. Uneven Terrain and its Associated Failure Probability.

drawback of the proposed approach. We would like to formu­late the motion planning problem as a variational problem in which the system dynamics, state and input constraints, and the appropriate metrics are incorporated and let the solution determine the gait and the trajectory of the vehicle. This is of course a very complicated problem. A partial solution to this is offered by Lhe wurk of Zefran, Desai and Kumar [40]. If we know the sequence in which legs are lifted or placed it is pos­sihle to optimize the motion and force distribution. Thus one approach to solving the continuous problem would be to solve the discrete problem with appropriate heuristics and use the best motion plan to generate solutions tu the cuntinuous problem. This is an is!,ue for future research.

Multifingered Grasping There are many simil arities hetween the problem of motion planning for legged vehicles and p l anning grasping and manipulation tasks for multifingered grippers. The methods describe d here can be easily app l i ed to d eterm i n e the sequence in which a finger is removed and replaced by anoth­er finger wh(;n an object is manipulated by multiple fingers. This problem is discussed in greater detail in [7] .

CONCLUSIONS We have presented: (a) a formulation of the motion planning problem for statically stable walking vehicles; (b) an ordinal approach to opLimiza1.ion thaL allows us to find (a, p)-optimal motion plans for given a and p, that is, plans that are guaran­teed to be in a desired percentile (I-a) of the optimal motion

Table .1: Motion Plans that Maximize Xc(25) with Different Value,. of the Stability Metric, If'

Case 1 2 3 4 5

Ac (25) 40 39 38 32 31

'I' .786 .813 .858 1 .24 1.28

June 1998

[2] Barres, J., Hebert, M., Kanade, T., Krotkov, E.,

Mitchell, T., Simmons, R., and Whitaker, W. 1989. Ambler: An autonomous robot for plane­

tary exploration. IEEE Computers 22(6): 18-26.

[3J Brooks. RA. and Lozano-Perez, T. 1983. Solving the find-path prob­lem by good representation of free space. IEEE Trans. Systems, Man and Cybernetics. SMC-13 (3): 191-197.

[41 Chen, C.H. 1995. An effective approach to smartly allocate computing budget for discrete event simulation. Proc. 34th Conf. Decision Con­trol: 2598-2605.

[51 Chen, C.H. and Kumar, V. 1996. Motion planning of walking robots in environments with uncertainty. Proc. of the 1996 IEEE International Conf. on Robotics and Automation: 3277-3282.

[61 Chen, C.H., and Kumar, V. 1997. Ordinal Optimization for Multifin­gered Grasping Planning Problems. In preparation.

[71 Chen, C.H., Kumar, V, and Luo, Y.C. 1997. Motion Planning of Walk­ing Robots in Environments with Uncertainty, submitted to The International Journal of Robotics Research.

[81 Cruse, H. 1976. The control of body position in the stick insect (carau­sius morosus), when walking over uneven terrain. Biological Cyber­netics. No. 24: 25-33.

[9J Delcomyn, F. 1981. Insect locomotion on land. Locomotion and Ener­getic in Arthropods. Eds. C.F. Herreid II and c.R. Fourtne!". Plenum Press. New York.

[ 101 Donald, B.R. 1987. A search algorithm for motion planning with six degrees of freedom. Artificial Intelligence. Vol . 31: 295-353.

[111 Gat, E., Slack, M.G., Miller, D.P., and Firby, RJ. 1990. Path planning

and execution monitoring for a planetary rover. Proc. of the IEEE Int. Conf. Robotics and Automation: 20-25.

[12J Hildebrand, M. 1 965. Symmetrical gaits of horses. Science. Vol. 150.

[131 Hirose, S. 1984. A study of design and control of a quadruped walk­ing vehicle. International Journal of Robotics Research . Vol. 3, No. 2: 113-133.

1141 Ho, Y.C. and Deng, M. 1994. The problem of large search space in stochastic optimization. Proceedings of the 33rd Conference of Deci­sion and Control.

[1 "I Ho, ye. and Larson, M. 1995. Ordinal optimization approach to rare event probability problems. Submitted to J. of Discrete Event Dynam­ic Systems, 5:281-301 .

[ 1 6 1 Ho, Y.C., Sreenivas, R.S., and Vakili, P. 1992. Ordinal optimization of DEDS. J. of Discrete Event Dynamic Systems, 2, #2: 61-88.

[ 1 7] Iagolnitzer, M., Richard, F., Samson, J.F., and Tourna.<soud, P. 1992. Locomotion of an all-terrain mohile robot. Proceedings of the IEEE Int. Conf Robotics andAulumatiun: 104-109.

IEEE Robotics & Automation Magazine • 31

[18] Krotkov, E. and Hoffman, R 1993. Results in terrain mapping for a walking planetary rover. International Con!. on Advanced Robotics. Tokyo: 103-108.

[19] Kugushev, E.!. and Jaroshevskij, V.S. 1975. Problems of selecting a gait for a locomotion robot. Proc. 4th IJC4I, Tilisi, Georgia, USSR: 789-793.

[20] Kumar, V. 1987. Motion p lanning in legged locomotion systems. Ph.D. Dissertation. Ohio State Cniversity.

l2 1 1 Kumar, V. and Waldron, K.I. 1989. Adaptive gait control for a walk­ing robot. 1. Robotics Systems. Vol. 6, No. 1 : 49-75.

[22] Kumar, V. and Waldron, KJ. 1990. Force distribution in walking vehicles on uneven terrain. ASME 1. of Mechanisms, Trammissions and Automation in Design, Vol. 1 12, No. 1 : 90-99.

[23] Latombe, J.-C. 1991. Robot motion planning. Kluwer Academic Pub­lishers, Boston.

[24] Lee, WJ. and Orin, D.E. 1988. The kinematics of motion planning for multilegged vehicles over uneven terrain. IEEE 1. Robotics and Automation 4(2): 204·212.

[25] McGhee, RB. 1968. Finite state control of quadruped locomotion. Mathematical Riosciences. No. 2: :07·66.

[26] McGhee, R.B. and Iswandhi, G. 1979. Adaptive locomotion of a mul­tilegged robot over rough terrain. IEEE Trans. Sys. Man and Cyber­netics. 9(4): 176·182.

[27] Muybridge, E. 1957. Animals in Molion, J)over, New York.

[28] Ozguner, F., T:;ai, S.I., and McGhee, RB. 1984. An approach La the use of terrain-preview information in rough-Terrain locomotion by 'a hexapod walking machine. Int. J. Robotics Research 3(2): 134-146.

[29] Pai, D. K and Reissel, L. M. 1994. ]\1ultiresolution rough terrain motion planning. Tech. Report 94-33, Computer Science, University of Ilritish Columbia.

[30] Pandy, M.G., Kumar, V., Berme, N., and Waldron, K.J. 1988. The dynamics of quadrupedal locomotion. 1. Biomechanical Engineering 110(3): 230·237.

[31J Pearson, KG., and Franklin, R 1984. Characteristics of leg move­ments and patterns of coordination in locusts walking on rough ter­rain. Inl. 1. Robotics Research Vol. 3. #2: 101-112.

[32] Poulos, D.D. 1986. Range image processing for local navigation of an autonomous land vehicle. fv1.Sc. Thesis. Naval Postgraduate Seh!. Monterey, California.

[33] Simeon, T. and Dacre-Wright, B. 1 99;). A practical motion planner for all-terrain mobile robots. Proc. IEEE Int. Can! on Intelligent Robots and Systems: 1357-1363.

[34] Song, S.M. and Waldron, K.J. 1987. An analytical approach for gait study and its application on wave gaits. Int. J. Robotics Research, Vol. 6. No. 2.

[35] Song, S.M. and Waldron, KJ. 1989. Machines that Walk. M.I.T. Press, Cambridge, MA.

[36] Shiller, Z. and Chen, 1.C. 1990. Optimal motion planning of autonomous vehicles in three-dimensional terrain. Proceedings of the 1990 IEEE International Conference on Robotics am! Automation: EJB-2m.

[37] Tomovic, R. 1 961 . A general theoretical model of creeping displace­ment. Cybernelica IV: 98·107.

[38] Waldron, K.J., .\<IcGhee, RB. 1986. The adaptive suspension vehicle. IEEE Conti. Sys. Mag. 6(6).

[39] Wilson, D.M. 1966. Insect Walking. Annual Review Entomology. Vol. 1 1 . [40] Zefran, M . , Desai. J . and Kumar, V. 1996. Continuous motion plans for

robotics systems with changing dynamic behavior. 2nd Int. Workshop on Algorithmic Foundations of Robotics, Toulouse, France, July 2·5, 1 996.

[41] Zuk, D. M. and Dell'Eva, M. L. 1983. A 3·d vision system for the adap-tive suspensive vehicle. Final Report, Environmental Research Insti­tute of Michigan, Ann Arbor, Michigan.

32 • IEEE Robotics & Automation Magazine

Chun-Hung Chen received the B.S. degree in Control Engine ering from National Chiao-Tung University, Taiwan, in 1987, the M.S. degree in Electrical Engineering from National Taiwan University, Taiwan, in 1989, and his Ph.D. degree in Simula­tion and Decision from Harvard University, Cambridge, MA, in 1994.

During 1989-1991, he participated in a OI project while per­forming his obligatory service in the Taiwan military. Since 1994, Dr. Chen has been Assistant Professor of Systems Engi­neering at the University of Pennsylvania, Philadelphia, PA. His interests cover a wide range of areas in stochastic simulation, optimal control, stochastic decision proces-ses, ordinal optimiza­tion, and robot motion planning. Recently, he has been engaged in the development of efficient approaches for discrete event simulation and decision problems, and in their application to manufacturing systems and robot motion planning problems.

Dr. Chen won the 1994 Harvard University Eliahu 1. Jury Award for the Best Thesis in the field of Control. I Ie is also one of the recipients of the 1992 MasPar Parallel Computer Challenge Award.

Vijay Kumar received his M.Sc. and Ph.D. in Mechanical Engineering from The Ohio State University in 1985 and 1987, respec­tively. He has been on the Faculty in the Department of Mechanical Engineering and App]ied Mechanics at the University of Penn­sylvania since 1987. He is currently an Asso­ciate Professor and also holds a secondary

appointment in the Department of Computer and Information Science and the Department of Systems Engineering. He is a member of the American Society of Mechanical Engineers, Institution of Electrical and Electronic Engineers, and Robotics International, Society of Manufacturing Engineers. He has served on the editorial board of the IEEE Transactions on Robotics and Automation. He is currently on the Editorial Board of the Journal of Franklin Institute and an Associate Edi­tor of the ASME Journal of Mechanical Design. He has co­authored several book chapters and has numerous journal and conferences papers. He is the recipient of the 1991 National Science Foundation Presidential Young Investigator award. His research interests include Kinematics, Manufacturing, Robot­ics, Mechanism Design and Control.

Yuh-Chyun Luo received his B.S. degree in Computer Science imd the M.S. degree in Electronic Engineering from Chung-Cheng Institute of Technology, Taiwan, in 1990 and 1993, respectively. Also he received an M.S. degree in Systems Engineering from the University of Pennsylvania in 1997. Cur­rently, he is a Ph.D. candidate at the Depart­

ment of Systems Engineering, University of Pennsylvania. His research int�resLs include ordinal optimization, simulation, and robotics.

June 1998