Jos e M Vidaljmvidal.cse.sc.edu/talks/negotiation-slides.pdf · 2010. 3. 3. · Zeuthen Strategy Zeuthen Characteristics It is not guaranteed to maximize social welfare. It is guaranteed

Negotiation

Negotiation

Jose M Vidal

Department of Computer Science and Engineering University of South Carolina.

March 3, 2010

Abstract

We describe automated negotiation as it applies to multiagentsystems. Chapter 6.

Negotiation

Introduction

1 Introduction

2 The Bargaining ProblemAxiomatic Solution ConceptsStrategic Solution Concepts

3 Monotonic Concession ProtocolZeuthen StrategyOne Step Protocol

4 Negotiation as Distributed Search

5 Ad-hoc Negotiation Strategies

6 Task Allocation ProblemPaymentsLying About TasksContracts

7 Complex DealsAnnealing Over Complex Deals

8 Argumentation-Based Negotiation

9 Negotiation NetworksNetwork Exchange Theory

Negotiation

Introduction

Why Negotiate?

Coordinate selfish interests.

Aggregate distributed conflicting knowledge.

Solve characteristic form games and more complex versions.

For example: NASA missions, capitol hill?

Negotiation

Introduction

Why Negotiate?

Coordinate selfish interests.

Aggregate distributed conflicting knowledge.

Solve characteristic form games and more complex versions.

For example: NASA missions, capitol hill?

Negotiation

The Bargaining Problem

1 Introduction









Negotiation


Bargaining Problem

ui : ∆→ < where ∆ is the set of deals.

δ− is the no-deal deal.

Assume that for all agents ui (δ−) = 0

Negotiation


Axiomatic Solution Concepts

1 Introduction









Negotiation



Pareto

Definition (Pareto optimal)

A deal δ is Pareto optimal if there is no other deal such thateveryone prefers it over δ. That is, there is no δ′ such that

∀iui (δ′) > ui (δ).

Negotiation



Pareto Frontier

ui (δ)

uj(δ)

Negotiation



What do we want?

We will want a Pareto deal, but which one?

Idea: Come up with some requirements first then see if asolution that meets those requirements exists.

Negotiation



What do we want?

We will want a Pareto deal, but which one?

Idea: Come up with some requirements first then see if asolution that meets those requirements exists.

Negotiation



Independence from Units

Definition (Independence of utility units)

A negotiation protocol is independent of utility units if when givenU it chooses δ and when given U ′ = {(β1u1, . . . , βI uI ) : u ∈ U} itchooses δ′ where

∀i ui (δ′) = βiui (δ).

Negotiation



Symmetry

Definition (Symmetry)

A negotiation protocol is symmetric if the solution remains thesame as long as the set of utility functions U is the same,regardless of which agent has which utility.

Negotiation



Individual Rationality

Definition (Individual rationality)

A deal δ is individually rational if

∀i ui (δ) ≥ ui (δ−).

Which means that ui (δ) ≥ 0 since we will be assuming thatui (δ

−) = 0. A deal is individually rational if all the agents prefer itover not reaching an agreement.

Negotiation



Independence from Irrelevant Alternatives

Definition (Independece of irrelavant alternatives)

A negotiation protocol is independent of irrelevant alternatives if itis true that when given ∆ it chooses δ and when given ∆′ ⊂ ∆where δ ∈ ∆′ it again chooses δ, assuming U stays constant.

That is, a protocol is independent of irrelevant alternative is thedeal it chooses does not change after we remove a deal that lost.Only removal of the winning deal changes the deal the protocolchooses.

Negotiation



Egalitarian

δ = arg maxδ′∈E

∑i

ui (δ′)

where E is the set of all deals where all agents receive the sameutility, namely

E = {δ | ∀i ,jui (δ) = uj(δ)}.

Negotiation



Egalitarian Social Welfare

Find the closest approximation:

δ = arg maxδ

mini

ui (δ)

Negotiation



Egalitarian

ui (δ)

uj(δ)

y = x

Egalitarian deal

Egalitarian social welfare deal

Negotiation



Utilitarian Solution

δ = arg max∑

i

ui (δ).

Negotiation



Utilitarian Solution

ui (δ)

uj(δ)

y = x

Utilitarian deal

Negotiation



Nash Bargaining Solution

δ = arg maxδ′

∏ui (δ

′).

Negotiation



Nash Bargaining Solution

ui (δ)

uj(δ)

y = x

Nash bargaining deal

1

5

10

Negotiation



Nice Nash

Nash bargaining solution is the only one that satisfies:

1 Pareto efficient

2 Independent of utility units

3 Independent of irrelevant alternatives

4 Symmetric

Negotiation



Kalai-Smorodinsky

Let u∗i be the maximum utility that i could get from the set ofall deals in the Pareto frontier.

Then, find the deal that lies in the line between the point δ−

and the point (u∗i , u∗j )

Negotiation



Kalai-Smorodinsky

ui (δ)

uj(δ)

u∗i

u∗i , u∗ju∗j

Kalai-Smorodinsky deal

Negotiation


Strategic Solution Concepts

1 Introduction









Negotiation



Strategic Solutions

Idea:

1 Formalize the bargaining process

2 Assume rational agents

3 Determine their equilibrium strategies for their bargainingprocess.

Negotiation



Rubinstein’s Alternating Offers

1 Two agents i and j

2 At each time step t one agent proposes a deal δ

3 The other can either accept or reject δ

4 Utilities decrease over time ui = λti ui (δ)

Negotiation



Theorem (Alternating Offers Bargaining Strategy)

The Rubinstein’s alternating offers game where the agents havecomplimentary linear utilities (ui (δ) = δ and uj(δ) = 1− ui (δ)) hasa unique subgame perfect equilibrium strategy where

agent i proposes a deal

δ∗i =1− λj

1− λiλj

and accepts the offer δj from j only if ui (δj) ≤ ui (δ∗j ),

agent j proposes a deal

δ∗j =1− λi

1− λiλj

and accepts the offer δi from i only if uj(δi ) ≤ uj(δ∗i ).

Negotiation



Alternating Offers Strategy

The theorem tells us that the best strategy for these agents ispropose a bid on the first time step which will be accepted by theother agent.

Negotiation

Monotonic Concession Protocol

1 Introduction









Negotiation


monotonic-concession

1 δi ← arg maxδ ui (δ)2 Propose δi3 Receive δj proposal4 if ui (δj) ≥ ui (δi )5 then Accept δj6 else δi ← δ′i such that uj(δ

′i ) ≥ ε+ uj(δi ) and ui (δ

′i ) ≥ ui (δ

−)7 goto 2

Negotiation


Monotonic Concession

δ

Utility

ui (δ)

uj(δ)

δ1i

ε δ2i

ε δ3i

ε

δ4i

δ1j

εδ2j

εδ3j

εδ4j

Negotiation


Monotonic Concession Summary

Slow

Agents know others’ utility functions

Tricky last step: both might want other’s offer

Negotiation


Zeuthen Strategy

1 Introduction









Negotiation


Zeuthen Strategy

Zeuthen Strategy

1 Propose my best deal.

2 Let willingness to risk conflict for i be the utility i loses byaccepting j ’s offer divided by the utility i loses by notconceding and causing conflict. That is:

riski =ui (δi )− ui (δj)

ui (δi )

3 If riski < riskj then I must concede just enough so that in thenext round I do not have to concede again.

Negotiation


Zeuthen Strategy

Zeuthen Strategy

zeuthen-monotonic-concession

1 δi ← arg maxδ ui (δ)2 Propose δi3 Receive δj proposal4 if ui (δj) ≥ ui (δi )5 then Accept δj

6 riski ←ui (δi )−ui (δj )

ui (δi )

7 riskj ←uj (δj )−uj (δi )

uj (δj )

8 if riski < riskj

9 then δi ← δ′i such that riski (δ′i ) > riskj

10 goto 211 goto 3

Negotiation


Zeuthen Strategy

Zeuthen Strategy

Deals

ui (δ)

uj(δ)

δi = 0 δj = 6

ui (δ) = 5− δ,uj(δ) = 2

3δδ = {0 . . . 6}δi = 0, δj = 6

riski = 5−(−1)5 = 6

5 ,riskj = 4−0

4 = 1

Negotiation


Zeuthen Strategy

Zeuthen Strategy

Deals

ui (δ)

uj(δ)

δi = 0

δj = 4.9

ui (δ) = 5− δ,uj(δ) = 2

3δδ = {0 . . . 6}δi = 0, δj = 6

riski = 5−(−1)5 = 6

5 ,riskj = 4−0

4 = 1 jmust concede, morethan 1.δj < 5

Negotiation


Zeuthen Strategy

Zeuthen Characteristics

It is not guaranteed to maximize social welfare.

It is guaranteed to terminate, and any agreement it reacheswill be individually rational and Pareto optimal.

It is also in Nash equilibrium–if the other guy is using it thenyou have nothing to gain by not using it. Allows agents topublish their strategy.

But, sometimes risks are equal.

Requires agents to know eachother’s utility functions.

Negotiation


One Step Protocol

1 Introduction









Negotiation


One Step Protocol

One Step Protocol

one-step-negotiation

1 E ← {δ | ∀δ′ui (δ)uj(δ) ≥ ui (δ′)uj(δ

′)}2 δi ← arg maxδ∈E ui (δ)3 Propose δi4 Receive δj5 if ui (δj)uj(δj) < ui (δi )uj(δi )6 then Report error, j is not following strategy.7 Coordinate with j to choose randomly between δi and δj .

Negotiation


One Step Protocol

One Step Protocol

Algorithm is in Nash equilibrium.

Negotiation

Negotiation as Distributed Search

1 Introduction









Negotiation

Negotiation as Distributed Search

Hill Climbing

ui (δ)

uj(δ)

δ0

Deals that Pareto dominate δ0

δ1

Negotiation

Ad-hoc Negotiation Strategies

1 Introduction









Negotiation



A linear discounts utility linearly.

A conceder concedes a lot initially.

An impatient demands a lot initially.

Negotiation

Task Allocation Problem

1 Introduction









Negotiation



The task allocation problem consists of:

T : tasksA: agentsci : s → < cost that i incurs in carrying out tasks s ⊆ T .δ represents allocation of tasks to agents.δ− is initial allocation

The cost function is monotonic.

The cost of doing nothing is 0.

Negotiation



δ si (δ) sj(δ) ci (δ) cj(δ) ui (δ) uj(δ)

δ1 ∅ {t1, t2, t3} 0 8 8 0δ2 {t1} {t2, t3} 1 4 7 4δ3 {t2} {t1, t3} 2 5 6 3δ4 {t3} {t1, t2} 4 7 4 1δ5 {t2, t3} {t1} 6 4 2 4δ6 {t1, t3} {t2} 5 3 3 5δ7 {t1, t2} {t3} 3 1 5 7δ8 {t1, t2, t3} ∅ 7 0 1 8

Negotiation


ui (δ)

uj(δ)

δ1

δ2

δ3

δ4

δ5

δ6

δ7

δ8

Negotiation


Payments

1 Introduction









Negotiation


Payments

Payments

1 Enable more deals by allowing payments.

2 This was the idea behind the original contract net protocol(Smith and Davis, 1981).

Negotiation


Payments

Payments

1 Enable more deals by allowing payments.

2 This was the idea behind the original contract net protocol(Smith and Davis, 1981).

Negotiation


Payments

Contract Net Protocol

manager

manager

contractor

contractor

contractor

Negotiation


Payments


manager

manager

contractor

contractor

contractor

task announcement

Eligibilityspecification.

Task abstraction.

Bid specification.

Expiration time.

Negotiation


Payments


manager

manager

contractor

contractor

contractor

bid

bid

Negotiation


Payments


manager

manager

contractor

contractor

contractor

award

Negotiation


Payments


manager

manager

contractor

contractor

contractor

contract

Negotiation


Payments

Payments Create Deals

ui (δ)

uj(δ)

δ0

δ1

New dominant deals

Negotiation


Payments

Payments Create Deals

ui (δ)

uj(δ)

δ0

δ1

New dominant deals

Negotiation


Payments

Additive Cost Functions

More formally,

Definition

A function c(s) is an additive cost function if for all s ⊆ T it istrue that

c(s) =∑t∈s

c(t).

They are easier to analyze.

Negotiation


Payments

Additive + Payments

Theorem

In a task allocation problem with an additive cost function wherewe only allow exchange of one task at a time, any protocol thatallows payments and always moves to dominant deals willeventually converge to the utilitarian solution .

Negotiation


Payments

ui (δ)

uj(δ)

δ1

δ2

δ3

δ4

δ5

δ6

δ7

δ8

Negotiation


Payments

Arbitrary Cost Functions

In general, not much we can say.

If any deal can be reached from any other deal (fullyconnected) then hill climbing will again reach the utilitariansolution.

Negotiation


Payments

Arbitrary Cost Functions

In general, not much we can say.

If any deal can be reached from any other deal (fullyconnected) then hill climbing will again reach the utilitariansolution.

Negotiation


Lying About Tasks

1 Introduction









Negotiation


Lying About Tasks

Lying About Tasks

Possible Lies

Not tell others about some tasks I have.

Make up tasks and hope I end up having to do them.

Make up tasks and create them if needed.

Assume known final deal. For example, Nash bargaining solution.

Negotiation


Lying About Tasks

Task Creation Example

δ si (δ) sj(δ) ui (δ) uj(δ)

δ1 ∅ {t1} 1 3δ2 {t1} ∅ 2 1

Create phony t2.


δ1 ∅ {t1, t2} 1 5δ2 {t1} {t2} 2 3δ3 {t2} {t1} 2 3δ4 {t1, t2} ∅ 8 1

Negotiation


Lying About Tasks



δ1 ∅ {t1} 1 3δ2 {t1} ∅ 2 1

Create phony t2.


δ1 ∅ {t1, t2} 1 5δ2 {t1} {t2} 2 3δ3 {t2} {t1} 2 3δ4 {t1, t2} ∅ 8 1

Negotiation


Lying About Tasks



δ1 ∅ {t1} 1 3δ2 {t1} ∅ 2 1

Create phony t2.


δ1 ∅ {t1, t2} 1 5δ2 {t1} {t2} 2 3δ3 {t2} {t1} 2 3δ4 {t1, t2} ∅ 8 1

Negotiation


Contracts

1 Introduction









Negotiation


Contracts

Contracts

Agents might want to de-commit on a contract.

Negotiation


Contracts

ui (δ)

uj(δ)

δ0

δ1 : i does task and j and pays nothing.

δ3 : i does nothing, j pays $2.

δ4 : i does task and j pays penalty of $1.

δ5 : i idle, pays $1 penalty, j pays $2.

δ2 : i does task and j and pays $2.

δ0: j does task and i is idle.

Negotiation


Contracts

ui (δ)

uj(δ)

δ0







Negotiation


Contracts

ui (δ)

uj(δ)

δ0







Negotiation


Contracts

ui (δ)

uj(δ)

δ0







Negotiation


Contracts

Contract Penalties

Penalties reduce risks.

But, if we can enforce penalties, why not just enforce originalcontracts?

Negotiation


Contracts

Contract Penalties

Penalties reduce risks.

But, if we can enforce penalties, why not just enforce originalcontracts?

Negotiation

Complex Deals

1 Introduction









Negotiation

Complex Deals

Complex Deals

A multi-dimensional deal is composed of a set of variablesx1, x2, . . . , xn with domains D1,D2, . . .Dn.

ui (δ)

Or, ui (δ) = c1u1i (x1) + c2u2

i (x2) + · · ·+ cnuni (xn)

Yes, this is a constraint optimization problem! But nowagents do not own the variables.

Negotiation

Complex Deals

Complex Deals

A multi-dimensional deal is composed of a set of variablesx1, x2, . . . , xn with domains D1,D2, . . .Dn.

ui (δ)

Or, ui (δ) = c1u1i (x1) + c2u2

i (x2) + · · ·+ cnuni (xn)

Yes, this is a constraint optimization problem! But nowagents do not own the variables.

Negotiation

Complex Deals

Convergence

δ

ui (δ)uj(δ)

δ1i

δ2i

δ1j

δ2j

δ3i ,j

Pareto domi-nate δ3

i ,j

Negotiation

Complex Deals

Annealing Over Complex Deals

1 Introduction









Negotiation

Complex Deals


Negotiation with Mediator

annealing-mediator

1 Generate random deal δ.2 δaccepted ← δ

3 Present δ to agents.4 if both accept5 then δaccepted ← δ

6 δ ← mutate(δ)7 goto 38 if one or more reject9 then δ ← mutate(δaccepted)

10 goto 3

Negotiation

Complex Deals


Hill Climbers and Annealers

Hill Climber Accepts a deal only if it has utility higher than itsreservation price ui (δ

−) and higher than that of thelast deal it accepted. That is, it monotonicallyincreases it reservation price as it accepts deals withhigher utility.

Annealer Use a simulated annealing algorithm. That is, theymaintain a temperature T and accept deals worsethan the last accepted deal with probability

max(1, e−∆UT ), where ∆U is the utility change

between the contracts.

Negotiation

Complex Deals


Hill-Climbers and Annealers

Deals

Ui (δ) Uj(δ)

Negotiation

Complex Deals



Deals

Ui (δ) Uj(δ)

δ1

Negotiation

Complex Deals



Deals

Ui (δ) Uj(δ)

δ1

Hill Climber

Negotiation

Complex Deals



Annealermax(1, e−

∆UT )

Deals

Ui (δ) Uj(δ)

δ1

Hill Climber

Negotiation

Complex Deals



Annealermax(1, e−

∆UT )

Deals

Ui (δ) Uj(δ)

Hill Climber

δ2

Negotiation

Complex Deals


Prisoner’s Dilemma, again!

Hill Climber Annealer

Hill Climber .73, .74 .99, .51

Annealer .51, .99 .84, .84

Negotiation

Complex Deals


Adding Tit-for-Tat

Hill Climber Annealer T4T

Hill Climber 400, 400 700, 180 500, 340

Annealer 180, 700 550, 550 550, 550

T4T 340, 500 550, 550 550, 550

Negotiation

Argumentation-Based Negotiation

1 Introduction









Negotiation


Argument-Based Negotiations

Critique

Counter-proposal

Justify

Persuade

There are also threats, rewards, and appeals.

Negotiation



Critique the proposal.

A: I propose that you provide me with service Xunder conditions P.

B: I am happy with the price of X but the deliverydate is too late.

A: I propose that I will provide you with service Yif you provide me with X .

B: I don’t want Y .

Counter-proposal

Justify

Persuade


Negotiation



Critique

Counter-proposal

A: I propose that you provide me with service X .B: I propose that I provide you with service X if

you provide me with service Z .A: I propose that I provide you with service Y if

you provide me with service X .B: I propose that I provide you with service X if

you provide me with service Z .

Justify

Persuade


Negotiation



Critique

Counter-proposal

Justify his reason for adopting a particular negotiation stance.

A: I don’t have the software for delivering service X .

Persuade


Negotiation



Critique

Counter-proposal

Justify

Persuade the other agent to change its negotiation stance.

A: Service X is much better than you think, look atthis report.


Negotiation



Critique

Counter-proposal

Justify

Persuade


These techniques help

build model of opponent’s utility function,

eliminate whole sets of deals,

change the other agent’s utility function,

change my utility function.

Negotiation

Negotiation Networks

1 Introduction









Negotiation



Definition

A negotiation network problem involves a set of agents A and setof sets of deals. Each set of deals ∆i involves only a subset ofagents ∆a

i ⊆ A and always includes the no-deal deal δ−. A solution~δ to the problem is a set of deals, one from each ∆i set, such thatall the deals that each agent is involved in are compatible witheach other. We thus define

ci (δ, δ′) =

{1 if δ and δ′ are compatible0 otherwise

Negotiation


Negotiation Network

j

i

k

∆1 ∆2

∆3

Negotiation


Network Exchange Theory

1 Introduction









Negotiation




i j

−10−1

1 −1

The coercion network.

Negotiation



Equi-Resistance

i j10

Negotiation



Equi-Resistance

i j10

i ’s resistance to payment p is given by

ri =pmaxi − pi

pi − pconi

wherepmaxi = Maximum i could get, 10

andpconi = Conflict deal, 0

Negotiation



Equi-Resistance

i j10

NET tells us that exchange happens at equi-resistance:

ri =pmaxi − pi

pi − pconi

=pmaxj − pj

pj − pconj

= rj .

We can represent this graphically by simply replacing pj with10− pi in j ’s resistance equation rj and plotting the two curves riand rj . The point at which the curves cross is the point ofexchange.

Negotiation



Equi-Resistance

i j10

p

ri (p) rj(p)

Negotiation



Iterated Equi-Resistance

i j k10 10

Negotiation




i j k10 10

1 Apply Equi-resistance to i j10

.

2 Apply Equi-resistance to j k10

.

3 Repeat until quiescence.

Negotiation




i j k10 10


. Gives us pj = 5.


.


Negotiation




i j k10 10


. Gives us pj = 5.


. Let pconj = 5 and apply

equi-resistance again.


Negotiation



NET Limitations

Only tested on small networks.

Multiple equilibriums.

Might never settle down.

Still, viable descriptive solution.

Documents

Jos e M Vidaljmvidal.cse.sc.edu/talks/negotiation-slides.pdf · 2010. 3. 3. · Zeuthen Strategy Zeuthen Characteristics It is not guaranteed to maximize social welfare. It is guaranteed