MULTI-AGENT COORDINATION FOR MULTI-ROBOT TASK ALLOCATION AND AREA COVERAGE

Raj Dasgupta

C-MANTIC Group

Computer Science Department

University of Nebraska at Omaha

Presentation at INAOE

August 13, 2012

04

/19

/20

23

2

INA

OE 2

01

2 - R

aj D

asg

upta

OUTLINE

Introduction and Preliminaries Multi-robot Coverage

Robotic Team Formation Flocking Coalition Game

Multi-robot Task Allocation Swarm-based Auction-based

Ongoing and Future Work

04

/19

/20

23

3

INA

OE 2

01

2 - R

aj D

asg

upta

RESEARCH PROBLEM (1)

How to coordinate a set of agents (each agent is situated on a robot) to perform a set of complex tasks in a collaborative manner Complex task: single robot does not have

resources to complete the task individually Coordination can be synchronous or

asynchronous Robots might or might not have to perform the task at

the same time Distributed Autonomous

04

/19

/20

23

4

INA

OE 2

01

2 - R

aj D

asg

upta

RESEARCH PROBLEM (2)

Performance metric(s) need to be optimized while performing tasks Time to complete tasks, distance traveled,

energy expended Robots are able to communicate with each

other Bluetooth, Wi-fi IR

04

/19

/20

23

5

INA

OE 2

01

2 - R

aj D

asg

upta

APPLICATIONS

Humanitarian de-mining (COMRADES) Autonomous exploration for planetary

surfaces (ModRED) Automatic Target Recognition (ATR) for

search and recon (COMSTAR) Unmanned Search and Rescue Civlian and domestic applications like

agriculture, vaccum cleaning, etc.

04

/19

/20

23

6

INA

OE 2

01

2 - R

aj D

asg

upta

ROBOT PLATFORMS

E-puck mini robot - suitable for table-top experiments for proof-of-concept

Coroware Corobot (indoor robot)- suitable for experiments in indoor arena within lab; hardware and software compatible with Coroware Explorer robot Coroware Explorer

(outdoor robot) – all terrain robot for outdoor experiments

All techniques are first verified on Webots simulator using simulated models of e-puck and

Corobot robots

04

/19

/20

23

INA

OE 2

01

2 - R

aj D

asg

upta

7

SOLUTIONS Swarming or emergent

computing for low-level coordination Fast, easy to implement No guarantee of achieving

desired outcome always When swarming-based

coordination fails... Low-level

coordinationSwarming Layer

04

/19

/20

23

INA

OE 2

01

2 - R

aj D

asg

upta

8

SOLUTIONS Swarming or emergent

computing for low-level coordination Fast, easy to implement No guarantee of achieving

desired outcome always When swarming-based

coordination fails...use a higher-level coordination mechanism

Game theory Branch of micro-economics

that gives rules of encounter between humans

Low-level coordination

Swarming Layer

Rules of Encounter Game Theoretic

Layer

04

/19

/20

23

9

INA

OE 2

01

2 - R

aj D

asg

upta

OUTLINE

Introduction and Preliminaries Multi-robot Coverage

Robotic Team Formation Flocking Coalition Game

04

/19

/20

23

10

INA

OE 2

01

2 - R

aj D

asg

upta

DISTRIBUTED MULTI-ROBOT COVERAGE

Efficiency is measured in time and spaceTime: reduce the time required to cover the

environmentSpace: avoid repeated coverage of regions that

have already been covered

• Use a set of robots to perform complete coverage of an initially unknown environment in an efficient mannerThe region of the environment that passes under the swathe of the robot’s coverage tool is considered as covered

Tradeoff in achieving both simultaneously

Source: Manuel Mazo Jr. and Karl Henrik Johansson, “Robust area coverage using hybrid control,”, TELEC'04, Santiago de Cuba, Cuba, 2004

FLOCKING-BASED CONTROLLER FOR MULTI-ROBOT TEAMS

04/19/2023 11INAOE 2012 - Raj Dasgupta

Works with physical characteristics such

as wheel speed, sensor reading,

pose, etc.

ControllerLayer (uses

flocking)

04

/19

/20

23

12

INA

OE 2

01

2 - R

aj D

asg

upta

MULTI-ROBOT TEAMS FOR AREA COVERAGE

Flocking based formation using Reynolds’ flocking model

Theoretical analysis: Forming teams gives a significant speed-up in terms of coverage efficiency

Simulation Results: The speed-up decreases from the theoretical case but still there is some speed-up as compared to not forming teams

04

/19

/20

23

13

INA

OE 2

01

2 - R

aj D

asg

upta

COVERAGE WITH MULTI-ROBOT TEAMS

Square

Corridor

Office

20 robots in different sized teams, in different environments over 2 hours

P. Dasgupta, T. Whipple, and K. Cheng, "Effects of Multi-Robot Team Formation on Distributed Area Coverage," International Journal on Swarm Intelligence Research (IJSIR), vol. 2, no. 1, 2011, pp. 44-69. 

04

/19

/20

23

14

INA

OE 2

01

2 - R

aj D

asg

upta

DYNAMIC RECONFIGURATIONS OF MULTI-ROBOT TEAMS

Having teams of robots is efficient for coverage Having large teams of robots doing frequent

reformations is inefficient for coverage Can we make the robot teams change their

configurations dynamically Split and merge teams based on their recent

performance

LAYERED CONTROLLER FOR DYNAMICALLY REFORMING MULTI-ROBOT TEAMS

04/19/2023 15INAOE 2012 - Raj Dasgupta

Works with agent utility, agent strategies,

equilibrium points, etc.

Works with physical characteristics such

as wheel speed, sensor reading,

pose, etc.

Coalition GameLayer (uses

WVG)

ControllerLayer (uses

flocking)

Mediator

Map from agent strategy to robot

action, sensor reading to agent

utility, maintain data structure for

mappingLow-level

coordinationSwarming Layer

Rules of Encounter

Game Theoretic Layer

04

/19

/20

23

16

INA

OE 2

01

2 - R

aj D

asg

upta

COALITION GAME-BASED TEAM RECONFIGURATION

Coalition games provide a theory to divide a set of players into smaller subsets or teams

We used a form of coalition games called weighted voting games (WVG) R: set of players (robots) Each player i is assigned a weight wi

q: threshold value called quota Solution concept: What is the minimum set of players

whose weights taken together can reach q

subject to S wi >=q for all S subset of R

minimize |S|

i e S

04

/19

/20

23

17

INA

OE 2

01

2 - R

aj D

asg

upta

COALITION GAME-BASED TEAM RECONFIGURATION

Coalition games provide a theory to divide a set of players into smaller subsets or teams

We used a form of coalition games called weighted voting games (WVG) R: set of players (robots) Each player i is assigned a weight wi

q: threshold value called quota Solution concept: What is the minimum set of players

whose weights taken together can reach q

subject to S wi >=q for all S subset of R

minimize |S|

i e S

Gives each robot’s coverage

performance

Minimum coverage quality reqd. by a

team

04

/19

/20

23

18

INA

OE 2

01

2 - R

aj D

asg

upta

COALITION GAME-BASED TEAM RECONFIGURATION

Coalition games provide a theory to divide a set of players into smaller subsets or teams

We used a form of coalition games called weighted voting games (WVG) R: set of players (robots) Each player i is assigned a weight wi

q: threshold value called quota Solution concept: What is the minimum set of players

whose weights taken together can reach q

subject to S wi >=q for all S subset of R

minimize |S|

i e S

Minimum Winning

Coalition (MWC)

Gives each robot’s coverage

performance

Minimum coverage quality reqd. by a

team

04

/19

/20

23

19

INA

OE 2

01

2 - R

aj D

asg

upta

quota=3.5

OUTLINE OF WVG ALGORITHM FOR ROBOT TEAM FORMATION

Each follower robot in a team reports its coverage ratio in last T time steps as its ‘weight’ in the WVG to leader robot Represented as a ratio; low values correspond to bad

(repeated area), higher values to good (new area) coverage

Two heuristics proposed to calculate MWC BMWC: Enumerates all MWCs and finds the one with

most robots that have closest pose and shortest distance to form a team with leader – O (R2)

Greedy: Stops as soon as it finds first MWC – O (R log R)

P. Dasgupta and K. Cheng, “Distributed Multi-robot Team Reconfiguration usingWeighted Voting Game,” DARS 2010 + forthcoming journal publication

04

/19

/20

23

20

INA

OE 2

01

2 - R

aj D

asg

upta

WVG TEAM FORMATION DEMO

04

/19

/20

23

21

INA

OE 2

01

2 - R

aj D

asg

upta

WVG TEAM FORMATION RESULTS

Time spent by 5-robot team in different reconfiguration

operations

Percentage of a 4 m2 environment covered by 5-robot team in 30 mins for

different percentage of free space in env.

04

/19

/20

23

22

INA

OE 2

01

2 - R

aj D

asg

upta

CONCLUSIONS AND LESSONS LEARNED

Proposed a new concept of using coalition game for dynamic team reconfiguration of robots

Stablity and convergence verified analytically and experimentally

Generates fewer coalitions (lesser running time) than existing multi-robot coalition generation algorithm by Vig et al [IEEE TRO 2006]

Current technique: If the team is getting obstacles, retain q% (e.g., 70%)

of the team How to adapt this value of q?

Transfer learning: store patterns of obstacles encountered in the past and learn a mapping from obstacle pattern to best possible action of team (preliminary work in ARMS 2011)

04

/19

/20

23

23

INA

OE 2

01

2 - R

aj D

asg

upta

OUTLINE

Introduction and Preliminaries Multi-robot Coverage

Robotic Team Formation Flocking Coalition Game

Multi-robot Task Allocation Auction-based Swarm-based heuristics

04

/19

/20

23

INA

OE 2

01

2 - R

aj D

asg

upta

24

MULTI-ROBOT TASK ALLOCATION

Our solutions: Swarming heuristics-based: Simple

to implement, high comm. overhead.

Market (auction)-based: Guarantees per task completion

time, time-out when a task cannot be completed, trade-off (efficiency loss) in waiting for better solutions

Almost 90% lower comm. overhead than heuristics approaches

T tasks, R robots, T >> R Constraints:

Each robot can communicate with a subset r of other robots (can change over time)

Robots do not know T, has to be discovered online by robots

Each task has to be done by rcap robots

Each task has time constraint Problem: How to find an

allocation from 2R -> T while minimizing the time (distance traveled) and communication overhead to all robots NP-complete problem

TaskRobot

Direction of movement

Legend

04

/19

/20

23

25

INA

OE 2

01

2 - R

aj D

asg

upta

AUCTION-BASED MRTA An agent discovers a task and begins an

auction Other agents in communication range hear

about the auction If an agent does not have a full task list, it

bids in the auction with its distance to the task, via any tasks on its task list

R

4R’s task list

4

04

/19

/20

23

26

INA

OE 2

01

2 - R

aj D

asg

upta

AUCTION-BASED MRTA: FIXEDBIDS An agent discovers a task and begins an

auction Other agents in communication range hear

about the auction If an agent does not have a full task list, it

bids in the auction with its distance to the task, via any tasks on its task list – because it can’t decommit from a previously committed/won auction

R

4 3

5

R’s task list

4

= 7 = 4+3R’s bid on

04

/19

/20

23

27

INA

OE 2

01

2 - R

aj D

asg

upta

FIXEDBIDS BIDDING STRATEGY PROBLEM

Previous strategy: Bid with distance from current location to the task’s location, via any tasks on its task list (No de-commitment!) Constraint: Newly arriving tasks added at end of

task list Problem: Creates very inefficient routes

R

4 3R’s task list

4

7 = 4+3

R’s bid on = 7+6=13

6

04

/19

/20

23

28

INA

OE 2

01

2 - R

aj D

asg

upta

DYNAMICBIDS AUCTION

Soft bids: Bid with a lower bound and an upper bound value (instead of a single bid)

Agent can then replan its path while keeping its edges (to different tasks already on its task list) between bidl and bidu

R

4 3

R’s task list

(4, 8)

(7, 12)

04

/19

/20

23

29

INA

OE 2

01

2 - R

aj D

asg

upta

DYNAMICBIDS AUCTION

Soft bids: Bid with a lower bound and an upper bound value (instead of a single bid)

Agent can then replan its path while keeping its edges (to different tasks already on its task list) between bidl and bidu

R

4 3

R’s task list

(4, 8)

(7, 12)

04

/19

/20

23

30

INA

OE 2

01

2 - R

aj D

asg

upta

DYNAMICBIDS AUCTION

Soft bids: Bid with a lower bound and an upper bound value (instead of a single bid)

Agent can then replan its path while keeping its edges (to different tasks already on its task list) between bidl and bidu

R

33

R’s task list

(4, 8)

(7, 12)

R’s actual cost

5

8

(2, 5) 2

2

04

/19

/20

23

31

INA

OE 2

01

2 - R

aj D

asg

upta

BIDDER ALGORITHM WITH DYNAMICBIDS

On receiving information about a new task (request for bid)Solve a TSP (approximation algorithm) with

nodes corresponding to the existing tasks in task list and new task

If bidu is not exceeded for any task already committed to (existing in task list) as per TSP solution Insert new task into task list at position given by TSP Send the cumulative path cost of new task as bidl

Calculate bidu = bidl + ~N(rc/2,1.0) Send (bidl, bidu) to auctioneer as soft bid for new task

Else don’t bid on new task

04

/19

/20

23

32

INA

OE 2

01

2 - R

aj D

asg

upta

DYNAMICBIDS AUCTIONEER SIDE

Two criteria for selecting winners from the set of biddersWho has the least cost (distance) to get here

(bidl)What could be the delay in executing the task

(and consequent decay in pheromone) if the bidder revises its bid to bidu later on Called “Loss in efficiency” Given by (1-e(bidl-bidu)/v)/(nbid e bidl/v+1)

Selects bidder based on a weighted product of these two criteria

04

/19

/20

23

33

INA

OE 2

01

2 - R

aj D

asg

upta

ANALYTICAL RESULTS: AUCTIONEER

DynamicBids algorithm is robust Handles livelocks gracefully

Analytically proven upper bounds on How many bids an auctioneer should wait for How much time it should wait for getting those

bids How much can auctioneer “lose” if a bidder

revises its bids Is DynamicBids always better? Theorem:

Marginal cost for every task using the DynamicBids algorithm < Marginal cost of the task using the FixedBids algorithm.

04

/19

/20

23

INA

OE 2

01

2 - R

aj D

asg

upta

34

EXPERIMENTAL SETUP Webots robotic

simulation platform 4 X 4 m2

environment 9-27 e-pucks, 5-60

tasks placed randomly, averaged over 10 runs

E-puck sensors IR Bluetooth Camera Overhead camera

based localization

04

/19

/20

23

35

INA

OE 2

01

2 - R

aj D

asg

upta

DYNAMICBIDS AUCTION MRTA RESULTS

04

/19

/20

23

36

INA

OE 2

01

2 - R

aj D

asg

upta

DYNAMICBIDS AUCTION MRTA RESULTS

04

/19

/20

23

37

INA

OE 2

01

2 - R

aj D

asg

upta

SWARM-BASED HEURISTICS FOR MRTA

Distance-based heuristic Select a task that is “closest to me and has highest amount

of pheromone” Robot density-based heuristic

Each robot selects a task that has least number of robots in its vicinity, lowest pheromone (starved tasks first)

Preference-based heuristic Density-based heuristic + amount of task outstanding

(starved tasks nearing completion first) Proximity-based heuristic

Density-based heuristic + effect of other robots - how many other robots are likely to be headed (ahead of me) to the task?• D. Miller, P. Dasgupta, T. Judkins, "Distributed Task Selection in Multi-agent based Swarms using Heuristic

Strategies,“ LNCS vol. 4433 (Proc. 2nd Swarm Robotics Workshop, Rome, Italy), 2006, pp. 158-172.• P. Dasgupta, "Multi-Robot Task Allocation for Performing Cooperative Foraging Tasks in an Initially Unknown Environment,“  Innovations in Defense Support Systems - 2 , Springer, Studies in Computational Intelligence, vol. 338, 2011, pp. 5-20.

Swarm-based heuristics• Require flooding of task updates to all robots in communication

range• Robots make decisions asynchronously

04

/19

/20

23

38

INA

OE 2

01

2 - R

aj D

asg

upta

AUCTION VS. SWARM-BASED HEURISTICS FOR MRTA

04

/19

/20

23

39

INA

OE 2

01

2 - R

aj D

asg

upta

AUCTION VS. SWARM-BASED HEURISTICS FOR MRTA

0

2000

4000

6000

8000

10000

12000

14000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Robot Number

To

tal

no

. o

f b

yte

s s

en

t o

r re

ceiv

ed

Auction Based

Market Based DF

Comparison of total number of bytes between auction protocoland swarming-based heuristic protocols

04

/19

/20

23

40

INA

OE 2

01

2 - R

aj D

asg

upta

AUCTION-BASED MRTA ON PHYSICAL E-PUCK ROBOTS

04

/19

/20

23

41

INA

OE 2

01

2 - R

aj D

asg

upta

CONCLUSIONS AND LESSONS LEARNED

Market-based MRTA offers an inherently distributed and robust technique for MRTA Communication overhead is significantly lower

than swarm-heuristic based techniques Soft guarantees can be made about time (no. of

rounds) required to complete tasks Market-based MRTA algorithms are not the

silver bullet Open Problem: Is there a relationship

between spatial and temporal distribution of tasks and type of MRTA algorithm used?

MRTA results (time) are very susceptible to underlying robot path planning algorithm

04

/19

/20

23

42

INA

OE 2

01

2 - R

aj D

asg

upta

USER INTERFACE FOR ROBOT CONTROL

04

/19

/20

23

INA

OE 2

01

2 - R

aj D

asg

upta

43

EXPLORER ROBOTS

04

/19

/20

23

44

INA

OE 2

01

2 - R

aj D

asg

upta

ONGOING WORK

Coverage Dynamic coverage information compression

(ICINCO 2011) Using Voronoi Partitions for Coverage (ACODS

2012, SPIE 2012, ICRA 2012, ARMS 2012) MRTA

Spatial Queueing Theory for MRTA (ICINCO 2012) Reconfiguration Planning for modular robots

(ModRED project) Distributed Information Aggregation (Foretell

project)

04

/19

/20

23

46

INA

OE 2

01

2 - R

aj D

asg

upta

CONCLUSION

Multi-agent and multi-robot systems appear to be made for each other Collaboration through coordination is cornerstone

of both MAS and MRS Bridging the gap

Abstractions, assumptions vs. crisp definitions Robustness in operation, data (sensor error) Computational overhead vs. fast operation Simulations vs. physical experiments Scientists vs. engineers

04

/19

/20

23

47

INA

OE 2

01

2 - R

aj D

asg

upta

ACKNOWLEDGEMENTS

We thank our sponsors DoD Navair US Office of Naval Research NASA (ESPCoR program)

C-MANTIC Research Group Members Coverage: Taylor Whipple, Dr. Ke Cheng, Task Allocation: Taylor Whipple, Matthew Hoeing,

David Miller, Timothy Judkins http://cmantic.unomaha.edu