Multi-Robot Systems Collision Avoidance and Coordination

Multi Robot Systems CourseBar Ilan UniversityMor Vered

MotivationIn a multi-robot environment path planning or

collision avoidance is an important problem.Multi-robot systems researchers have been

investigating distributed coordination methods for improving spatial coordination in teams – a.k.a collision avoidance.

Such methods adapt the coordination method to the dynamic changes in density of the robots.

Basically, the goal is to help avoid collision from any static obstacles or other dynamic objects, such as moving robots.

MotivationSpatial conflicts can cause the team’s

productivity to drop with the addition of robots.

We will see that this phenomenon is impacted by the coordination methods used by the team-members, as different coordination methods yield radically different productivity results.

Heuristic Coordination ApproachesNoise method - If I am on a given trajectory

that is danger of colliding with another agents', add random noise to my direction vector. ”Behavior Based Formation Control for Multi-robot Teams”. Balch, Tucker and Arkin, Ronald C

Heuristic Coordination ApproachesAggression method - Describes a controller which breaks

deadlocks in favour of the most ‘aggressive’ robot. The robots compete and only one gains access to the

resource. When robots come to close to each other, each of the robots chooses an aggression level (randomly); the robot with the lower level concedes its position, preventing a collision. Later showed that it might be best to choose aggression level proportional to the robot's task.

For every cycle a robot found itself within 2 radii of a teammate, it selected either an aggressive or timid behavior, with probability of 0.5. If the robot selected to become timid, it backed away for 100 cycles (10 simulated seconds). Otherwise it proceeded forward, executing the aggressive behavior. As robots chose to continue being “aggressive” or to become “timid” every cycle, the probability that two robots would collide in this implementation was near zero. ”Go ahead, Make my day: Robot Conflict Resolution by Aggressive Competition”. Vaughan, Richard and St{\o}y, Kasper and Sukhatme, Gaurav and Mataric, Maja

Heuristic Coordination ApproachesRepel method - When sensing a possible

collision on course the Repel group backtracked for 500 cycles (50 seconds) but mutually repelled using a direction of 180 degrees away from the closest robot.

Heuristic Coordination ApproachesTimeRand method - This method contained

no repulsion methods to directly avoid collision but in case of a collision could take care of the immobile robot.

When robots sensed that they did not significantly move for 100 cycles (10 seconds), they proceeded to move with a random walk for 150 cycles (15 seconds).

Heuristic Coordination ApproachesTimeRepel method - This method also

contained no repulsion methods to directly avoid collision but only reacted after the fact to collisions.

Once these robots did not move for 150 cycles (15 seconds), they then moved backwards for 50 cycles (5 seconds).

All methods taken from “ A Study of Mechanisms for Improving Robotic Group Performance “Rosenfeld, Avi and Kaminka, Gal A and Kraus, Sarit and Shehory, Onn

Heuristic Coordination ApproachesHomework

Come up with your own heuristic coordination approach by next week’s lesson and e-mail it to me.

Selecting the Best ApproachAssuming we have come up with several solutions.

Another problem arising from that is how to select the best coordination method.

As we stated before spatial conflicts can cause the team’s productivity to drop with the addition of robots. This phenomenon is impacted by the coordination methods used by the team-members, as different coordination methods yield radically different productivity results.

No one collision avoidance method is best in all domain and group size settings.

The effectiveness of coordination methods in a given context is not known in advance.

Selecting the Best Approach

CCC - Combined Coordination Cost MethodHow to select the best coordination methodCCC -quantifies the production resources spent on

coordination conflicts ; quantify the cost of group interactions.

Multi-attribute cost measure to quantify resources such as time and fuel each group member spends in coordination behaviors during task execution.

Facilitates comparison between different group methods.

“ A Study of Mechanisms for Improving Robotic Group Performance “Rosenfeld, Avi and Kaminka, Gal A and Kraus, Sarit and Shehory, Onn

CCC - Combined Coordination Cost MethodContended that if robots dynamically reduce their

CCC, group productivity will be improved. To demonstrate this, they created robotic groups

which dynamically adapt their coordination techniques based on each robot’s CCC estimate.

Problem with this method – it ignores the gains accumulated from long periods of no coordination needs. The next method tries to fix that

“ A Study of Mechanisms for Improving Robotic Group Performance “Rosenfeld, Avi and Kaminka, Gal A and Kraus, Sarit and Shehory, Onn

Adaptive Multi-Robot Coordination: A Game-Theoretic Perspective

Used a reinforcement-learning approach to coordination algorithm selection.

Proved it both on experiments (foraging) and empirically by mathematical equations.

“Adaptive Multi-Robot Coordination: A Game-Theoretic Perspective“Gal A. Kaminka, Dan Erusalimchik and Sarit Kraus

Problem DefinitionThe normal routine of a robot's operation is

to carry out its primary task until interrupted by a conflict with another robot which must be resolved by a coordination algorithm. This is called a conflict event.

The event triggers a coordination algorithm to handle the conflict.

Once it successfully finishes, the robots involved go back to their primary task.

Problem DefinitionDefined several kinds of tasks :1) Loose-coordination between the robots ( only

occasional need for spatial or temporal coordination ). For example; multi-robot foraging.

2) Cooperative task – the robots seek to maximize group utility. For example; exploration.

3) Timed tasks – the task is bound in time. For example; exploration – completely explore a new area as quickly as possible, or patrolling.

Problem DefinitionDivided time into :1)Active interval – in which the robot was

actively investing resources in coordination.2)Passive interval – in which the robot no

longer requires investing in coordination.3)The robot has a nonempty set of coordination

algorithms to select from. The choice of c coordination algorithm selection effects the duration of the active and passive intervals.

Reinforcement Learning

Computes how an agent ought to take actions in an environment so as to maximize some definition of cumulative reward.

Basic reinforcement model consists of :1) A set of environment states - S.2) A set of actions - A.3) Rules of transitioning between states.4) Rules that determine the immediate reward of a

transition.5) Rules that describe what the agent observes.

Reinforcement Learning

The trick is - how to define the reward.They introduced a reward function EI

effectiveness index , that reduces time and resources spent coordinating and maximizes the time between conflicts that require coordination.

Took into consideration time spent on coordination and time not spent on coordination.

Reinforcement Learning

Instead of reward functions – used Q-Learning. ( a variant of reinforcement learning ).

A technique that works by learning an action-value function that gives the expected utility of taking a given action in a given state and following a fixed policy thereafter.

Takes into account an accumulative value - of all results taken up until now.

“Q-learning”, Watkins, Christopher JCH and Dayan, Peter

General reward

The general reward is comprised of : 1) The total cost of coordination – cost of

internal resources such as battery life and fuel.

2) Time spent on coordinating.3) Frequency of coordinating.