Adaptive behaviors of reactive mobile robot with Bayesian

Appl IntellDOI 10.1007/s10489-009-0164-0

Adaptive behaviors of reactive mobile robot with Bayesianinference in nonstationary environments

Hyeun-Jeong Min · Sung-Bae Cho

© Springer Science+Business Media, LLC 2009

Abstract This paper presents a technique for a reactive mo-bile robot to adaptively behave in unforeseen and dynamiccircumstances. A robot in nonstationary environments needsto infer how to adaptively behave to the changing envi-ronment. Behavior-based approach manages the interactionsbetween the robot and its environment for generating be-haviors, but in spite of its strengths of fast response, it hasnot been applied much to more complex problems for high-level behaviors. For that reason many researchers employa behavior-based deliberative architecture. This paper pro-poses a 2-layer control architecture for generating adaptivebehaviors to perceive and avoid moving obstacles as well asstationary obstacles. The first layer is to generate reflexiveand autonomous behaviors with behavior network, and thesecond layer is to infer dynamic situations of the mobile ro-bot with Bayesian network. These two levels facilitate a tightintegration between high-level inference and low-level be-haviors. Experimental results with various simulations and areal robot have shown that the robot reaches the goal pointswhile avoiding stationary or moving obstacles with the pro-posed architecture.

Keywords Behavior network · Bayesian network ·Avoiding moving obstacles · Adaptive behaviors

H.-J. Min · S.-B. Cho (�)Computer Science Department, Yonsei University,262 Seongsanro, Sudaemoon-gu, Seoul 120-749, South Koreae-mail: [email protected]

H.-J. Mine-mail: [email protected]

1 Introduction

Autonomous robots have operated reliably and adaptivelyin real-life situations. To perform complex activities effec-tively, robots must possess rich perceptual capabilities torecognize objects. Robot must also reason about dynamicconditions. Conventional artificial intelligence deals withcomplex agents in simple environments whereas behavior-based approaches have handled simple agents in noisyand dynamic situations [1]. Behavior network takes ad-vantage of the generation of higher abstract behaviors thanbehavior-based system in addition to react instantly on thestimuli like behavior-based system. The central idea ofbehavior network is that the different types of links en-code various relationships [2]. Often the robot has multi-ple goals [3]. It may try to reach a goal point ahead of itwhile avoiding local obstacles because behavior networkhas goals.

It has been attempted to avoid static obstacles by pathplanning or map building. The planning method can solvethe problem of collision avoidance in the case of known en-vironments. Planners relying heavily on world models canreadily integrate world knowledge. The top-down delibera-tive approach [4] uses a global world model to generate themost appropriate behaviors to avoid static obstacles. If a mo-bile robot has no map about the environment, the problemof avoiding static obstacles can also be solved by behavior-based system [5]. However, it cannot avoid moving obsta-cles using behavior-based system alone because the move-ment of obstacles is difficult to be predicted. The processof predicting what will happen when a behavior is exe-cuted is essential for autonomous robots to adaptively be-have. Therefore, there are many researchers for dealing withvarious aspects of dynamic collision avoidance in uncertainsituations.

mailto:[email protected]

mailto:[email protected]

H.-J. Min, S.-B. Cho

It is difficult to avoid moving obstacles because mobilerobot has to perceive situations by only utilizing its sen-sors in real-time. The predictability is crucial to the collisionavoidance in nonstationary conditions. If there is no needto predict, we can rely entirely on what is sensed, resultingin a purely reactive approach [6]. However, it is difficult topredict in the real world. In MIT AI Lab., they proposed amethod of learning each circumstance for avoiding movingobstacles [7]. Hashimoto utilized the evolutionary compu-tation and fuzzy system for avoiding moving obstacles [8],and Inoue presented a behavior learning method based onBayesian networks and experience of interaction betweenhuman and robots of avoiding moving obstacles [9]. Nico-lescu and Mataric dealt with changing situations by learningand constructing hierarchical structure of previous behaviorsfor solving the problem of avoiding moving obstacles [10].

Although behavior network is proposed for solving prob-lems having goals, it is not enough for adaptive behaviorsin changing situations. To cope with this problem we pro-pose a control architecture in which Bayesian network di-rects behavior network. This architecture selects a behaviorof the highest weight which is computed by the inference ofBayesian network in behavior network. Bayesian networktakes advantage that it is independently applied to an envi-ronment because it is designed by only using the informa-tion from sensors. It differs from existing hybrid architec-tures in three aspects: (1) behavior network performs con-tinuous response encoding, (2) the inference module infersunforeseen and dynamic circumstances, and (3) it also op-erates in other environments with different dynamics. Ex-periments have confirmed that the robot reaches the goalwhile avoiding static or moving obstacles with the proposedmethod, whereas only using behavior network is not enoughfor generating adaptive behaviors in dynamic situations. Wehave also performed the experiments with a real mobile ro-bot.

The next section explains the related works of generat-ing adaptive behaviors in dynamic situations, and Sect. 3describes how the proposed method of integrating behav-ior network and Bayesian network is constructed. FinallySects. 4 and 5 provide the experimental results, discussion,and suggestions for the future work.

2 Related works

Intelligent behaviors of agents have been studied for gen-erating autonomous and adaptive behaviors within dynamicenvironments. The applications with behavior-based systeminclude a guide robot for museum tour using the behavior-based planning by Burgard and Thrun and autonomous un-derwater vehicle by Bennett [11, 12]. Behavior-based ap-proach is based on the notion of behavior, and behaviors are

real-time processes taking inputs from sensors and other be-haviors and sending outputs to the robot’s actuators [13].The behavior network proposed by Maes can acquire globalgoals as well as autonomously select behaviors as bestowinggoals on the behavior-based system.

Among the related works for behavior network, Nico-lescu and Mataric proposed the peculiar behavior networksystem expanding behavior-based architecture for solvingcomplex or sequential problem [14]. Khoo and Zubek con-structed a stationary behavior network system for generat-ing behaviors of a character in computer game, and com-pared those systems with the defined rule-based system [15].Weigel et al. applied an action selection mechanism basedon behavior network for generating behaviors of soccer ro-bot and extended the system to multi-robot problem [16].Matsuura presented the formative behavior network for gen-erating walking behaviors of bipedal robot from the stimuliof sensors [17].

The problems of avoiding obstacles have been activelystudied, but most of them deal with static obstacles. It is animportant issue to generate autonomous and adaptive behav-iors in real world, and the researches for avoiding movingobstacles are contributed to mobile robots. Mucientes et al.described the fuzzy controller system for avoiding movingobstacles using fuzzy temporal reasoning, which is calledas fuzzy temporal rules [18]. Fujimori and Tani presentedthe collision avoidance in cooperative multi-robot systemfor avoiding moving obstacles and for navigating to thegoal [19]. Mbede et al. combined a probabilistic map asa global planner and fuzzy reactive (local) planner for au-tonomous navigation and avoiding obstacles [20]. Sabourinand Madani presented obstacle avoidance strategy for bipedrobot using fuzzy Q-learning method [21].

The hybrid of behavior-based system and high-level con-troller is variously investigated in the problem of avoidingmoving obstacles as shown in Table 1. In this paper we uti-lize Bayesian network for inferring dynamic situations withthe information based on sensors within agent and behav-ior network for generating autonomous and reflexive behav-iors in nonstationary conditions and for acquiring the globalgoals. The controllers by fuzzy inference have limitationssuch that fuzzy rules are defined for each situation and thesystem must prepare for many fuzzy rules as there are var-ious situations in the environment of robots. Bayesian in-ference has concerned in various applications such as therobotics in which the method of inference for map building[26] and the dialogue agents for emotional inference [27].

3 Bayesian-network driven behavior network

Behavior network does not have to be re-designed to chang-ing environments if it has been well designed with appropri-

Adaptive behaviors of reactive mobile robot with Bayesian inference in nonstationary environments

Table 1 Applications of behavior-based system

Type Method Researcher Year Feature

Behavior-based Behavior-basedplanning

Burgard andThrun

2001 A guide robot to generate robust and reflexive behaviors by behavior-basedplanning [11]

Behavior-basedlearning

Smart andKaelbling

2002 Introduce a framework for reinforcement learning to reach the goal [7]

Behavior-basedcontrol

Bennett 2000 Autonomous exploration of underwater vehicle in unknown environment[12]

Behavior network Hierarchicalabstract behavior

Nicolescuand Mataric

2001 System for solving complex or sequential problem in behavior-based system[10]

Behavior network Khooa andZubek

2002 Apply pre-designed behavior network [15]

Behavior network Wigel 2002 Apply behavior network for generating behavior of soccer robot [16]

Formative behav-ior network

Matsuura 2000 Generate walking behavior of bipedal robot [17]

Fuzzy Fuzzy control sys-tem

Mucientesand Barro

2001 Generate adaptive behaviors using fuzzy and fuzzy inference, require toplenty of fuzzy rules [18]

Behavior-based +fuzzy

Thongchai 2002 Avoid moving obstacles using fuzzy rules and reinforcement, localize tolearned behaviors [22]

Neuro-fuzzy +Planning

Mbede, et al. 2005 Combine a global planner and fuzzy reactive (local) planner for autonomousnavigation and avoiding obstacle [20]

GA GA + Inference Inoue 2000 Generate behaviors using Bayesian network and genetic algorithm [9]

MDP Planning Lane, Kael-bling

2001 Hierarchical decomposition for partial plans of partitioning state spaces [22]

NN NN learning Sabourin andBruneau

2005 Learn joint trajectories of biped robot to increase the robustness of its’ dy-namic walking control [23]

Neural Q-learning Carreras,et al.

2007 Learn reactive robot behaviors in real-time using the semi-online neuralQ-learning [24]

Fuzzy-CMAC Sabourin,et al.

2007 Generate autonomous gait pattern of biped robot based on Fuzzy-CMACneural networks approach [25]

Fuzzy Q-learning Sabourin andMadani

2008 Make footstep planning to avoid obstacle for biped robot using fuzzy Q-learning [21]

ate preconditions, goals, and behaviors. However it is dif-ficult to generate adaptive behaviors in some dynamic en-vironments. The system of Bayesian network driven behav-ior network takes advantages of the fast generation of au-tonomous and adaptive behaviors as well as the independentoperations within environments. Figure 1 shows the over-all architecture of the proposed method. It consists of 2-layer controllers in which the layer 1 is for reflexive con-trol of behaviors and the layer 2 is for deliberative infer-ence.

3.1 Mathematical notations

The variables required for the mathematical notations of theproposed method using Bayesian inference driven behaviornetwork are as follows. These variables are also applied tothe function for the Bayesian inference, the computation of

an activation level of each node, and the selection of activenode in the behavior modules.

• α: Activation level• θ : The initial value of the global threshold which is re-

duced by 10% if no executable node has an activationgreater than it

• φ: An amount of the activation energy of environmentalsensor inputs and successor links

• γ : An amount of the activation energy of goal inputs andpredecessor links

• δ: An amount of the activation energy of protected goalinputs and conflictor links

• t : Current time• B: A set of behavior nodes• BPS : A set of preconditions of behavior node b

• BAS : A set of add lists of behavior node b

• BDS : A set of delete list of behavior node b


Fig. 1 The architecture ofBayesian-inference drivenbehavior network

The function for selecting the activation node bi in timet is defined as follows.

bi(t) =

⎧⎪⎪⎪⎨

⎪⎪⎪⎩

1 if

⎧⎪⎨

⎪⎩

αbi(t) ≥ θ,

executable(bi, t) = 1,

αbi(t) ≥ αbj

(t), ∀j · � · (1) and (2)

0 otherwise

(1)

executable(bi, t) ={

1 if bi is executable at time t,

0 otherwise.(2)

The function for the activation level αbiin behavior node bi

is defined as follows.

αbi(t) =

{0 if t = 0,

D(bi, t) + β(bi, t) otherwise.

The activation value of each behavior D(bi, t) is definedas follows:

D(bi, t) = S(bi, t) + G(bi, t) − P(bi, t)

+∑

bj ,bk

(SPEW(bj , bk, t) + SPFW (bj , bk, t)

− SPPG(bj , bk, t)), (3)

SPEW(bj , bk, t) ={

1 if bj ∈ BPS , bk ∈ BAS,

0 otherwise,

SPFW (bj , bk, t) ={

1 if bj ∈ BAS , bk ∈ BPS,

0 otherwise,

SPPG(bj , bk, t) ={

1 if bj ∈ BDS , bk ∈ BPS,

0 otherwise.

β(bi, t), the weight affecting the activation in behaviornodes after the agent infers the situation by Bayesian net-work, is defined as follows:

β(bi, t) ={

ξ if bi ∈ rk,i , i ∈ I , (k, i) = σ ,

0 otherwise,(4)

rk,i = {s | s ∈ effect_node(k), s : ith state},I = {x | x ∈ SI (t),1 ≤ x ≤ #(Effect Nodes),

SI (t) : causes at time t},(k, i) = P(bki |c1j · · · cmi) : conditional probability,

m : #(cause nodes),

σ = Max{(k, i) | : conditional probability,

1 ≤ i ≤ #(states in ri), k : effect node}.In (3), S(bi, t) is the activation value from sensors at

time t , G(bi, t) is the activation value from goals at time t ,and P(bi, t) is the activation value to be deleted fromprotected goal at time t . In addition to these equations,SPEW(bj , bk, t) is the backward spreading from node bj tobk which means both the preconditions of bj and the addlist of bk , and SPFW (bj , bk, t) is the forward spreading fromnode bj to bk , which means both the add list of bj and theprecondition of bk . SPPG(bj , bk, t) specifies which node bj

takes away from node bk that means both the delete list ofbj and the precondition of bk .

In (4), rk,i means the ith state in the kth effect node ofBayesian network, and I is the set of indexes of result nodeswhere preconditions are satisfied at time t . (k, i) is theconditional probability at the ith state in the kth result node,and σ is the state of the highest probability in the kth resultnode. That is, β(bi, t)is set by ξ if the ith state of the kth


Fig. 2 An example ofbehavior-network architecture

node in Bayesian network has the highest probability in thatnode and the preconditions of behavior node, bi , are satis-fied.

3.2 Behavior network

Figure 2 shows an example of behavior network architec-ture that has primitive behaviors such as ‘Follow Light’, ‘GoStraight’, ‘Turn Left’, and ‘Turn Right’. In behavior net-work, each node has preconditions, add lists, delete lists, ac-tivation level, and execution code, and also has the connec-tion between nodes such as internal links and external links[2]. The internal links are specified as predecessor links, suc-cessor links, and conflictor links, and the external links pro-viding input to the network are specified from sensors andgoals. Using these links, an excitation can be fed from thecurrent situation and goals, and a behavior will then, afterseveral iterations, come to settle in the node representing themost appropriate behavior.

For constructing behavior network, we have to first defineprimitive behaviors, information from sensors, and goalswhich are tightly coupled in the problem. In Fig. 2 the in-formation from sensors are ‘Near Light’, ‘Nothing’, ‘NearObstacle’, and ‘Shade Area’, and the goals are ‘Reach Goal’and ‘Avoid Obstacle’. All behavior nodes contain the fiveitems as Table 2. There are preconditions, add list, deletelist, activation level and execution codes for activating be-havior nodes. Here, delete list is optional. Execution codesinto each behavior node are made by manual programmingor learning methods, and activation levels are recalculated ineach time according to the relations between behavior nodesand sensors. Table 2 shows the information of each nodeconsisting of behavior network. After defining the informa-tion of each node, we have to make some connection withsensors, primitive behaviors, and goals through the external

Table 2 The information of behavior nodes within the behavior net-work

Conditions Information

in behavior node

Precondition Logical conditions that are required to be true

Add List Conditions that the node is likely to be true

Delete List Conditions that are likely to be made false

Activation Level Activation value for selecting the best behavior

Execution Code The code for the execution of behavior

links and internal links. In Fig. 2, behavior network has theexternal links from sensors and goals and the internal linksto connect behavior nodes with each other.

Figure 3 shows the internal structure in nodes and links inbehavior network. Behavior nodes are activated by informa-tion excited from internal and external links, and the excita-tion is spread by the predecessor, successor, and conflictorlinks. In Fig. 2, the predecessor links of ‘Follow Light’ arefrom ‘Go Straight’ and ‘Near Light’, and the successor linkis to ‘Reach Goal’. The predecessor links of ‘Go Straight’is also from ‘Nothing’, ‘Turn Left’, and ‘Turn Right’, andthe successor links are to ‘Reach Goal’, ‘Avoid Obstacle’,and ‘Follow Light’. The predecessor links of ‘Turn Right’are from ‘Shade Area’, ‘Near Obstacle’, and ‘Go Straight’,and the successor links are to ‘Avoid Obstacle’ and ‘GoStraight’. Table 3 shows the conditions for activating theinternal links and external links. In this table A and B arebehavior nodes that construct a behavior network. For acti-vating an appropriate behavior, behavior network has to bewell constructed by the information of preconditions, addlist, and delete list.

YAKS simulation robot and Khepera II real robot used inthe experiments have 8 infra-red proximity and light sensors


Fig. 3 The internal structure inbehavior network

Table 3 Conditions for activation of behaviors

Internal Link

Predecessor Link (ρ = false) (ρ ∈ preconditions of node A) (ρ ∈add list of B)

Successor Link (ρ = false) (ρ ∈ add list of node A) (A is exe-cutable) (ρ ∈ preconditions of B)

Conflictor Link (ρ = true) (ρ ∈ preconditions of node A) (ρ ∈delete list of B)

External Link

Sensors (ρ = true) (ρ ∈ preconditions of node A)

Goals (ρ ≥ 0) (ρ ∈ add list of node A)

with up to 100 mm range and 2 DC brushed servo motorswith incremental encoders. For avoiding moving obstacles,we use the directions and locations by previous behaviorsas well as the values from distance sensors. Since layer 1is for reflexive and autonomous behaviors of a mobile ro-bot, it is sufficient to use the information of current sensors.However, layer 2 has to infer dynamic situations so that weneed to analyze the past behaviors and sensors as well as thecurrent sensors.

3.3 Bayesian inference

Bayesian network is represented as a DAG (Directed AcyclicGraph) where each node corresponds to the probabilisticvariable and the arches among the variables correspond tothe probabilistic dependency [28, 29]. It has been known as

a useful tool for modeling and reasoning in various uncer-tain domains last two decades [30]. A structure of Bayesiannetwork represents the relations of cause (parent) and ef-fect (child) from their probability, and it can infer the resultbased on the conditional probability of cause nodes. Figure 4shows an example of a Bayesian network and its conditionalprobability table. With these values in conditional proba-bility tables, the robot can infer the probability of a certaindecision node given observed evidences.

Figure 5 shows the Bayesian network designed for avoid-ing moving obstacles as well as stationary obstacles. Thecause nodes from eight proximity sensors are ‘distance 0’,‘distance 1’, ‘distance 2’, ‘distance 3’, ‘distance 4’, ‘dis-tance 5’, ‘distance 6’, and ‘distance 7’, and the cause nodesfor inferring the changing situations from previous be-havior are ‘distance of obstacles’, ‘position of obstacles’,‘previous behavior’, and the directional change of obsta-cles such as ‘rear_object’, ‘front_object’, ‘left_object’, and‘right_object’. ‘distance 0’ through ‘distance 7’ are the dis-tances between the robot and obstacles measured by eightinfra red sensors in Khepera. The initial probability of eachnode is defined as 1/n for the n conditions in each node.According to this the initial probability of ‘distance 0’ isP(Near0) = 0.5 and P(Far0) = 0.5. There are some im-portant factors to the nodes named ‘distance of obstacles’,‘position of obstacles’, ‘previous behavior’, ‘left_object’,‘right_object’, ‘front_object’, and ‘rear_object’ for inferringdynamic situations.

For example, if the left sensors of robot detect an ob-stacle in current state, the probabilities P(Near0) andP(Near1) in the nodes of ‘distance 0’ and ‘distance 1’get higher. The probabilities P(Approach) and P(GoAway)


Fig. 4 Bayesian network andconditional probability table forBayesian network

Fig. 5 Bayesian network for avoiding moving obstacles

in the node ‘distance of obstacles’ and P(Front 2Left),

P(Left), and P(Rear 2Left) in the node ‘left_object’ are de-

termined from the previous information. If P(Approach) >

P (GoAway) in the node ‘distance of obstacles’, the node

of the highest probability among P(NoTurn),P (TurnLeft),

and P(Rear 2Left) in the effect node is selected. The

node ‘left_object’ consists of P(Front 2Left),P (Left), and

P(Rear 2Left) which mean the change of obstacle from front

to left, from left to left, and from rear to left, respectively.

The conditional probability of the effect node ‘left_turn’ is

determined from the above probabilities.

3.4 Inference in dynamic situations

Current sensors and previous situations are the importantfactors for selecting the optimal behaviors, and Bayesiannetwork can independently apply to their environment us-ing inference of nonstationary conditions. In this paper thefactors affecting Bayesian inference are eight infra-red sen-sors, the change of distance between obstacle and robot, thechange of position from previous to current state, the previ-ous behavior of the robot, and the direction of goals. Therobot perceives the direction of obstacles using the eight


Fig. 6 The process of state inference in the proposed method

Fig. 7 Experimentalenvironment in the simulator (a)and a Khepera II mobile robot(b)

infra-red sensors, and may select different behaviors as thechanges of direction, position, and previous behaviors.

Figure 6 shows the process of state inference in the pro-posed method which has reflexive and autonomous behaviorof behavior network and the inferred behavior for dynamicenvironment using Bayesian inference. The activation valueof nodes is renewable in case the inference by the condi-tional probability table affects the behavior node. After thebehavior network examines the executable behavior nodeshaving the highest activation value and satisfying the pre-conditions, the robot executes the behavior node. If the nodeis not satisfied with the preconditions or the activation valueis less than the threshold, the behavior network decreases thethreshold and repeats the process.

Behavior network is mediated by the inference of currentstate from conditional probability table in Bayesian infer-ence in this process, and also reflects the effect of the in-ference by updating the activation value in behavior nodes.The result nodes having the probability in Bayesian network

are ‘right_turn’, ‘left_turn’, ‘front_turn’, and ‘rear_turn’,and each node has the probability variables, ‘No_Turn’,‘Left_Turn’, and ‘Right_Turn’. These variables are updatedby the probability of the probability variables in the parent’snodes.

4 Experimental results

For experiments we use the YAKS, the 3D robot simula-tor, and a real Khepera II mobile robot as shown in Fig. 7.The environment of the simulator has the mobile robot(number 0) which generates behaviors using the proposedmethod and two robots (numbers 1 and 2) which act as themoving obstacles. These two robot-like obstacles can onlydetect and avoid the walls of the fence, and cannot avoida robot when they collide with other robots. This simula-tor may change the angle of the view. In Fig. 7(a) the whitecylinder represents the static obstacles, and the number of


them changes as the experiments. The robot randomly startsfrom a position for the comparative analysis of behavior net-work and the proposed method, and also starts from the fixedpositions for various analyses of the proposed method.

The goals in these experiments are to avoid movingobstacles and to reach the goal position marked with thelight area in the corner of the fence. The basic behaviorsin behavior-network are ‘Go Straight’, ‘Turn Left’, ‘TurnRight’, and ‘Follow Light’, and the input sensors in a robotare for detecting distance and light. We put randomly 5% ofnoise in the value from sensors in this simulation.

4.1 Comparative analysis

After the experiments of avoiding moving obstacles usingonly behavior network and the proposed method respec-tively, we have observed that the robot may not avoid mov-ing obstacles when a moving obstacle is directly in front ofthe robot. Since the avoidance may succeed or not as thedirection and position of an obstacle from the robot, ‘TurnLeft’ or ‘Turn Right’ is an important behavior. When start-ing at the same position and direction in the behavior net-work and the proposed method respectively, the robot col-lides with a moving obstacle with the behavior network, butavoids the obstacle and reaches the goal with the proposedmethod.

We have analyzed the behavior for avoiding moving ob-stacles which get nearer from various positions and direc-tions using the behavior network and the proposed method.We have verified the success of 52% and 90% among 60 tri-als using the behavior network and the proposed method, re-spectively. The cases of failure with the proposed method arewhen the obstacle gets nearer on the front side or changes itsdirection abruptly. Figure 8 shows the success rate for avoid-ing moving obstacles in the behavior network and the pro-posed method. In this figure y-axis is the success rate whichranges from 0 to 1.

Figure 9 shows the comparison of the behavior networkwith the proposed method. (a) is the comparison of the be-havior sequences and (b) is that of the angles of movementcaused by the behaviors. In this figure the solid line repre-sents the behaviors and angles with the behavior networkand the dashed line represents the behaviors and angles withthe proposed method. In the y-axis, 1 is ‘Go Straight’, 2 is‘Turn Left’, and 3 is ‘Turn Right’. In Fig. 9, the robot doesrarely change its’ behavior with the conventional behaviornetwork while it selects ‘Turn Left’ and ‘Turn Right’ fre-quently and appropriately in the situation with the proposedmethod. It means that the robot can select behaviors moreadaptively with the proposed method than with the conven-tional behavior network.

4.2 Analysis for avoiding moving obstacles

As mentioned before, we have analyzed the behavior selec-tions of the robot as the moved angles between the robot

Fig. 8 The success rate for avoiding moving obstacles

Fig. 9 The behaviors (a) and angles (b) in the behavior network and the proposed method


Fig. 10 The trajectory (a) and each behavior (b) by the time of avoiding moving obstacles with the proposed method

and obstacle using the proposed method. Figure 10 showsthat the robot reaches the goal while avoiding moving obsta-cles, fence, and static obstacles of 2, 3, and 2 times, respec-tively. In this figure, (a) shows the trajectories of the robotand moving obstacles, and (b) shows the behavior selectionsof the robot to reach the goal and has the marked circlesrepresenting the collision avoidances with moving obstacles,static obstacles, and fence.

We have subsequently analyzed the behaviors of a robotin cases of the same direction and the opposite direction ofa robot for comparing behavior selections as the changesof the angles between the robot and obstacles. Figure 11shows the result of avoiding moving obstacles. In this fig-ure (a) and (b) are the trajectories in the cases of collidingwith an obstacle 2 in the same direction at the left and at theright of the robot, and (c) represents the behavior sequencesof (a) according to time. Lastly (d) represents the behaviorsequences of (b) with respect to time. In (c) and (d), x-axis istime and y-axis is the selected behaviors by time. The num-bers of y-axis represent that 1 is ‘Go Straight’, 2 is ‘TurnLeft’, and 3 is ‘Turn Right’. Table 4 shows the results ofadaptive behaviors with respect to the direction and posi-tion between robot and moving obstacles. There are someresults of colliding with obstacles in various angles of therobot.

Figure 12 shows the trajectories and behavior sequenceswhen the robot avoids an obstacle coming from the oppositedirection. There are three cases of the approach at the left,right, and front of the robot. In this figure (a) and (b) are theresults of avoiding obstacle 2 at the left of the robot, (c) and(d) are the results of avoiding obstacle 2 at the right of the

robot, and (e) and (f) are the results of avoiding obstacle 2at the front of the robot. (a), (c), and (e) represent the trajec-tories of the robot, obstacle 1 and obstacle 2, and each axisis the x- and y-coordinates. (b), (d), and (f) represent thebehavior sequences at the time during the experiment, andx-axis is the time and y-axis is the behavior defined that 1 is‘Go Straight’, 2 is ‘Turn Left’, and 3 is ‘Turn Right’.

Figure 13 shows the results to avoid moving obstacle witha real Khepera II mobile robot. In this figure (a) and (b) showthe situations before and after avoiding a white box in frontof the mobile robot, (c), (d) and (e), (f) show those at theleft side and at the right side of the mobile robot, respec-tively.

5 Concluding remarks

Through the experiments by robot simulator, we have val-idated that mobile robot goes to the goal with the gener-ation of an avoidance behavior on moving obstacles us-ing Bayesian inference driven behavior network. An au-tonomous behavior of a mobile robot is generated by theinter-relations of sensors, goals and primitive behaviors inbehavior network, and adaptive behaviors of a mobile ro-bot are also generated by extending the behavior networkthrough the inference of each situation using Bayesian net-work. The proposed method does not have the limitation ofre-constructing the system as the changes of environmentunlike the systems by hybrid learning or planning, and copessuccessfully with the limitation to re-define many rulessuch as the systems using fuzzy rules and fuzzy inference[7, 8].


Fig. 11 Avoiding moving obstacles as the same direction with the robot using the proposed method. Each shows the avoidance at the right side(a), (c) and at the left side (b), (d) of the robot

Table 4 The results of the adaptive behaviors

Robot Obstacle1 Obstacle2 Direction Position Result

(to Robot)Start Angle Start Angle Start Angle

(800,200) 135° (400,900) 270° (600,100) 90° Same direction toobst2

Right Turn left

(300,200) 45° (400,900) 270° (600,100) 90° Left 45° Turn right

(300,700) 90° (400,900) 270° (600,100) 90° Different directionto obst1

Head-on Turn opp. direc-tion to obst2

(200,500) 60° (400,900) 270° (600,100) 90° Left of obst1 Turn opp. direc-tion to obst1

(800,400) 240° (400,900) 270° (600,100) 90° Different directionto obst2

Right of obst2 Turn opp. direc-tion to obst2


Fig. 12 Avoiding moving obstacles as the same direction with the robot using the proposed method. Each shows the avoidance at the right side(a), (b) and at the left side (c), (d), and in the middle (e), (f) of the robot


Fig. 13 Avoiding movingobstacle with the real robotusing the proposed method.Each shows the avoidance in themiddle (a), (b), at the left side(c), (d), and at the right side (e),(f) of the robot

We have compared the proposed method with the con-ventional behavior network in the experiments. It has beenproved to outperform the conventional behavior networkeven though it has a higher decision complexity than con-ventional method due to the inference module of Bayesiannetwork. However, it is not that significant because Bayesiannetwork works fast enough for the robot to avoid obstaclesunless the structure is too complicated.

In the future, we will extend this method for the robotin dynamic speed for generating more adaptive behaviors incollision avoidance. In this expansion, we will consider theefficiency of the proposed method because the complexity ofBayesian networks grows according to the difficulty of theproblem. Also, we have to investigate the hybrid systemsof inference and planning for generating autonomous andadaptive behaviors of the service robots.

Acknowledgements This work is supported by Ubiquitous Comput-ing and Network Project of Korea Ministry of Knowledge Economy(MKE) grant (No. 09C1-T3-11T) and Korea Science and EngineeringFoundation (KOSEF) grant (No. R01-2008-000-20801-0). The authorswould like to thank Mr. Han-Saem Park for this help to revise the man-uscripts.

References

1. Mataric MJ (1994) Interaction and intelligent behavior. PhD the-sis, MIT

2. Tyrrell T (1994) An evaluation of Maes’s bottom-up mechanismfor behavior selection. Adapt Behav 2(4):307–348

3. Kim K-J, Cho S-B (2006) A unified architecture for agent behav-iors with selection of evolved neural network modules. Appl Intell25(3):253–268

4. Hu H, Brady M, Probert P (1993) A decision theoretic approachto real-time obstacle avoidance for a mobile robot. Intell RobotsSyst 1457–1464

5. Gomi T, Volpe P (1993) Collision avoidance using behavioral-based AI techniques. In: Proceedings of intelligent vehicles ’93symposium, pp 141–145

6. Arkin RC (1998) Behavior-based robotics. MIT Press, Cambridge7. Smart WD (2002) Making reinforcement learning work on real

robots. PhD thesis, Brown Univ8. Hashimoto S, Kojima F, Kubota N (2003) Perceptual system for a

mobile robot under a dynamic environment. In: Proceedings 2003IEEE international symposium on computational intelligence inrobotics and automation, pp 747–752

9. Inamura T, Inaba M, Inoue H (2000) User adaptation of human-robot interaction model based on Bayesian network and intro-spection of interaction experience. In: Proceedings of the 2000IEEE/RSJ international conference on intelligent robots and sys-tems, pp 2139–2144


10. Nicolescu MN, Mataric MJ (2002) A hierarchical architecture forbehavior-based robots. In: Proceedings of autonomous agents andmulti-agent systems, pp 227–233

11. Beetz M, Arbuckle T, Belker T, Cremers AB, Schulz D, BennewitzM, Burgard W, Hahnel D, Fox D, Grosskreutz H (2001) Integrated,plan-based control of autonomous robot in human environments.IEEE Intell Syst 16(5):56–65

12. Bennett AA (2000) A behavior-based approach to adaptive fea-ture detection and following with autonomous underwater vehi-cles. IEEE Ocean Eng 25(2):213–226

13. Sukhatme GS, Mataric MJ (2000) Embedding robots into the in-ternet. Commun ACM 43(5):67–73

14. Nicolescu MN, Mataric MJ (2000) Extending behavior-basedsystems capabilities using an abstract behavior representation.In: Proceedings of AAAI fall symposium on parallel cognition,pp 27–34

15. Khoo A, Zubek R (2002) Applying inexpensive AI techniques tocomputer games. IEEE Intell Syst 17(4):48–53

16. Weigel T, Gutmann J-S, Dietl M, Kleiner A, Nebel B, Freiburg CS(2002) Coordinating robots for successful soccer playing. IEEETrans Robot Autom 19(5):685–699

17. Matsuura M, Wada M (2000) Formative behavior network for abiped robot: A control system in consideration of motor develop-ment. Robot Hum Interact Commun 101–106

18. Mucientes M, Iglesias R, Regueiro CV, Bugarin A, Carinena P,Barro S (2001) Fuzzy temporal rules for mobile robot guidance indynamic environments. IEEE Trans Syst Man Cybern 31(3):391–398

19. Fujimori A, Tani S (2002) A navigation of mobile robots with col-lision avoidance for moving obstacles. In: Proceedings of interna-tional conference on industrial technology, pp. 1–6

20. Mbede JB, Ele P, Mveh-Abia C-M, Toure Y, Graefe V, Ma S(2005) Intelligent mobile manipulator navigation using adaptiveneuro-fuzzy system. Inf Sci 171(4):447–474

21. Sabourin C, Madani K (2008) Obstacle avoidance strategy forbiped robot based on fuzzy Q-learning. In: Proceedings of interna-tional conference on climbing and walking robots and the supporttechnologies for mobile machines, pp 695–702

22. Lane T, Kaelbling LP (2001) Toward hierarchical decompositionfor planning in uncertain environments. In: Proceedings of the2001 IJCAI workshop on planning under uncertainty and incom-plete information, pp 1–7

23. Sabourin C, Bruneau O (2005) Robustness of the dynamic walkof a biped robot subjected to disturbing external forces by usingCMAC neural networks. Robot Auton Syst 51(2–3):81–99

24. Carreras M, Yuh J, Batlle J, Ridao P (2007) Application ofSONQL for real-time learning of robot behaviors. Robot AutonSyst 55(8):628–642

25. Sabourin C, Madani K, Bruneau O (2007) Autonomous biped gaitpattern based on fuzzy-CMAC neural networks. Integr Comput-Aided Eng 14(2):173–186

26. Basye K, Dean T, Kirman J, Lejter M (1992) A decision-theoretic approach to planning, perception, and control. IEEE Ex-pert 7(4):58–65

27. Akiba T, Tanaka H (1994) A Bayesian approach for user model-ing in dialogue systems. Technical Report of Tokyo Institute ofTechnology

28. Neapolitan RE (2003) Learning Bayesian network. Prentice Hallseries in artificial intelligence

29. Pearl J (1988) Probabilistic reasoning in intelligent systems: net-works of plausible inference. Morgan Kauffman, San Mateo

30. Haider S, Levis AH (2008) Modeling time-varying uncertain situ-ations using dynamic influence net. Int J Approx Reas 42(2):488–502

Documents

Adaptive behaviors of reactive mobile robot with Bayesian