On the Endogenous Formation of Energy EfficientCooperative Wireless Networks

Fatemeh Fazel and D. Richard Brown III{fazel,drb}@ece.wpi.edu

Electrical and Computer Engineering DepartmentWorcester Polytechnic Institute

Worcester, MA 01609

Abstract—In wireless networks, it is well-known that inter-mediate nodes can be used as cooperative relays to reduce thetransmission energy required to reliably deliver a messagetoan intended destination. When the network is under a centralauthority, energy allocations and cooperative pairings can beassigned to optimize the overall energy efficiency of the network.In networks with autonomous selfish nodes, however, nodesmay not be willing to expend energy to relay messages forothers. This problem has been previously addressed throughthe development of extrinsic incentive mechanisms, e.g. virtualcurrency, or the insertion of altruistic nodes in the network toenforce cooperative behavior. This paper considers the problem ofhow selfish nodes can decide on an efficient energy allocationandendogenously form cooperative partnerships in wireless networkswithout extrinsic incentive mechanisms or altruistic nodes. Usingtools from both cooperative and non-cooperative game theory, thethree main contributions of this paper are (i) the development ofPareto-efficient cooperative energy allocations that can be agreedupon by selfish nodes, based on axiomatic bargaining techniques,(ii) the development of necessary and sufficient conditionsunderwhich “natural” cooperation is possible in systems with fadingand non-fading channels without extrinsic incentive mechanismsor altruistic nodes, and (iii) the development of techniquesto endogenously form cooperative partnerships without centralcontrol. Numerical results with orthogonal amplify-and-forward(OAF) cooperation are also provided to quantify the energyefficiency of a wireless network with sources selfishly allocatingtransmission/relaying energy and endogenously forming coop-erative partnerships with respect to a network with centrallyoptimized energy allocations and pairing assignments.

I. I NTRODUCTION

Multihop or cooperative transmission is often used in wire-less ad hoc networks to increase energy efficiency by allowingpackets to be delivered over several short links [1]. One ormore intermediate nodes between the source and destinationcan assist in the transmission by forwarding or relaying thepacket along the route to the destination. Autonomous nodesacting in their own self-interest, however, may refuse to usetheir limited resources to forward packets for other nodes.This can lead to inefficient use of the network resources sincemessages may have to be retransmitted or re-routed throughdifferent paths to the destination node [2].

Several techniques have been proposed to encourage co-operation and improve the efficiency of wireless ad hocnetworks with selfish autonomous nodes. A comprehensivestudy of these techniques can be found in [3]. One well-studied

technique to encourage cooperation among selfish nodes is thedevelopment of extrinsic incentive mechanisms, e.g. virtualcurrency [4], [5], where nodes are reimbursed for cooperation.The idea of virtual currency is intuitively appealing in manyscenarios, but the use of virtual currency has the potentialforfraud and/or collusion as discussed in [6]. Another techniquethat can induce cooperation is the introduction of altruisticnodes into the network [7] that punish misbehaving nodes.While both of these techniques have been shown to encouragecooperation among selfish nodes, they both require some levelof central authority in the network to perform accounting orto strategically insert altruistic nodes. They also implicitlyassume that the near-term costs and benefits of cooperativebehavior are one-sided, hence remuneration is necessary toenable cooperation. While this assumption is true in somecases, recent studies, e.g. [8], have shown that the benefitsofcooperation can be two-sided and have considered the questionof when “natural” cooperation is possible in large networkswithout any central authority. In [9], a two-player relayinggame based on the orthogonal amplify-and-forward (OAF)cooperative transmission protocol [10] was analyzed in a non-cooperative game-theoretic framework and it was shown thatnatural cooperation without extrinsic incentive mechanismsor altruistic nodes can emerge under certain conditions onthe channels. Two limitations of this work, however, are thatit used centrally-controlled energy allocations and did notconsider networks with more than two source nodes.

This paper considers the problem of how selfish nodes canlocally decide on an efficient energy allocation and endoge-nously form cooperative partnerships in wireless networkswith two or more source nodes and without any sort ofextrinsic incentive mechanisms, altruistic nodes, or centralauthority. We first use bargaining tools from cooperative gametheory to determine efficient energy allocations that can belocally computed and agreed to by a pair of selfish nodes. Wethen develop a repeated-game framework and employ toolsfrom non-cooperative game theory to describe necessary andsufficient conditions under which natural cooperation betweena pair of nodes is possible. To extend our results to networkswith more than two source nodes, we then consider thequestion of how to endogenously form cooperative partner-ships in general networks and propose the use of the “stable

roommates” algorithm [11], [12] to form partnerships that arestable with respect to unilateral or bilateral deviations.Finally,numerical results with orthogonal amplify-and-forward (OAF)cooperation are provided to quantify the energy efficiency ofa wireless network with sources selfishly allocating transmis-sion/relaying energy and endogenously forming cooperativepartnerships with respect to a network with centrally optimizedenergy allocations and pairing assignments.

Unlike the previous studies on this subject, the noveltyof the approach in this paper is that the nodes in the net-work behave selfishly without any form of central authority,community enforcement, or extrinsic incentive mechanisms.Selfish autonomous nodes endogenously form cooperativepartnerships, locally determine efficient energy allocations,and cooperate by relaying messages during transmission ses-sions with multiple frames. Throughout this paper, we assumethat nodes exhibit rational individual choice behavior, meaningthat each individual source node has a consistent preferencerelation over all possible energy allocations and partners, andalways chooses the most preferred feasible alternative. Wealsoassume that nodes can always refuse to cooperate if it is intheir best interest to do so.

The rest of the paper is organized as follows. Section IIintroduces the system model used throughout the paper. InSection III we present a two-player relaying game in stage-game formulation, develop axiomatic bargaining solutionsto determine an optimum energy allocation that two selfishplayers can agree upon, and then develop necessary andsufficient conditions under which selfish nodes will cooperateand not defect under a repeated-game formulation for bothfading and non-fading channels. Section V extends these two-player results to networks withK > 2 sources and describes atechnique in which the sources can endogenously form stablecooperative pairings. Section VI provides numerical energyefficiency examples based on OAF cooperative transmissionand concluding remarks are made in Section VII.

The rest of the paper is organized as follows. Section IIintroduces the system model used throughout the paper. InSection III we present a two-player relaying game in stagegame and repeated game formulation for both fading and non-fading channel conditions and Section IV provides severalaxiomatic bargaining solutions to determine an optimum en-ergy allocation that the two selfish players can agree upon.Section V extends the two-player game to more than twoplayers by allowing the transmitting sources to form stablecooperative pairs. Section VI provides a numerical examplebased on OAF cooperative relaying. Concluding remarks aremade in Section VII.

II. SYSTEM MODEL

We consider an ad hoc wireless network withL half-duplexnodes and a discrete model of time where nodes transmitinformation to other nodes in the network intransmissionsessionsof variable duration. The sets of source nodes anddestination nodes in a given transmission session are denotedasS andD, respectively, where|S| = |D| = K ≤ L/2 and

S ∩ D = ∅. The destination node for source nodei ∈ S isdenoted asdi ∈ D. It is assumed that the number of nodesand/or the amount of offered network traffic is sufficientlylarge such that, in any given transmission session,K ≥ 2source nodes wish to transmit independent information todistinct destination nodes in the network.

In each transmission session, theK source nodes involvedin the transmission session take turns transmitting using time-division multiple access (TDMA). A transmission session iscomposed ofN ≥ 1 framesand each frame is composed of2K timeslotsas shown in Figure 1 for the case whenK = 2.The channelhij [n] between nodei ∈ S and nodej ∈ D ∪S\i in framen is assumed to be frequency non-selective. Thesquared channel magnitude between nodei and j in framen, normalized with respect to the power of the additive whiteGaussian noise (AWGN) in the channel, is denoted asHij [n].

In the first K timeslots of each frame, each source nodetransmits a packet to its destination. Due to the undirectednature of wireless transmission, the TDMA transmissions inthe first K timeslots are also overheard by the other sourcenodes in the transmission session. The signal received by nodej from nodei in framen is given as

yij [n] =√

Hij [n]xi[n] + uij [n].

for all i ∈ S and all j ∈ D ∪ S\i in the current transmissionsession wherexi[n] is the packet transmitted by source nodei in framen anduij [n] is zero-mean unit-variance AWGN. Inthe remainingK timeslots of the frame, each source node canpotentially help one other source node by relaying a packetto its intended destination. When source nodej ∈ S\i electsto relay a packet for source nodei, it transmits a function ofthe observation received from source nodei in the first halfof the frame. Destination nodedi ∈ D receives

zjdi[n] =

√

Hjdi[n]f(yij [n]) + vjdi

[n]

where the relaying functionf depends on the cooperativeprotocol andvjdi

[n] is zero-mean unit-variance AWGN. Allnoise terms are assumed to be spatially and temporally white.Note that each destinationdi ∈ D always receives at least oneobservation in each frame, i.e. the direct transmissionyidi

[n],and may receive two observations if another source node electsto relay the packet from source nodei.

It is assumed that the normalized channel magnitudesH[n] = {Hij [n]} are quasi-static and fading in the sensethat H[n] is constant over the duration of each frame, butare independent and identically distributed (i.i.d.) in differentframes of the transmission session. The current channel stateis assumed to be known by theK source nodes involved in thecurrent transmission session. Channel phases are only assumedto be known at the respective receivers.

III. T WO-PLAYER RELAYING GAME

This section considers the scenario when there are twosource nodes, denoted asS = {1, 2}, that wish to communi-cate independent information to two distinct destination nodes,

timeslot 11 transmits

timeslot 22 transmits

timeslot 31 relays

timeslot 42 relays

timeslot 11 transmits

timeslot 22 transmits

timeslot 31 relays

timeslot 42 relays

time

transmission session

frame 0 frameN − 1

Fig. 1. A transmission session composed ofN frames withS = {1, 2}.

denoted asD = {3, 4}. We extend the ideas developed in thissection to the case withK > 2 source nodes in Section V.

A. Stage Game Formulation

A stage gameis defined in terms of the players, availableactions, and payoffs received by each player as a consequenceof the actions for one frame of the current transmissionsession. The players in the game are the source nodes. In thefirst two timeslots of framen, each source node transmits to itsdestination using transmit energyE1[n] andE2[n], respectively.If a source node does not request relaying from the othersource node, i.e. it uses direct transmission to its destination,it will transmit with sufficient energy to satisfy a minimumquality-of-service (QoS) constraint, e.g. SNR or rate, at itsdestination based on the current channel state. For example, ifthe required signal-to-noise ratio at destination nodedi is ρ,then source nodei will transmit with energyEi[n] = ρ/Hidi

[n]in frame n when it uses direct transmission. We denote therequired direct transmission energy for source nodei in framen as Edt

i [n]. If a source node requests relaying from theother source node in framen, it will transmit with energy0 < Ei[n] < Edt

i [n].Referring to the timeslot schedule in Figure 1, if node 2

has requested relaying, then node 1 must decide in timeslot 3whether to fulfill this relaying request. Since both source nodesknow the channel state, node 1 can determine the minimumrelaying energyErmin

1 [n] required to ensure the QoS constraintis satisfied at destination node 4 [13]. Although node 1 canchoose any non-negative relaying energyEr

1 [n], it will neverrationally choose a relaying energy larger than the minimumrequired relaying energy since there is no benefit to eithersource node if a node expends excess relaying energy. If sourcenode 1 transmits with relaying energy less than the minimumrequired relaying energy, then the packet will not be receivedat destination node 4 with the required QoS and node 2 willneed to transmit with additional energy at the end of the frameto ensure the QoS constraint is satisfied. For these reasons,weassume that node 1 chooses from the discrete set of actions“do not relay” (a1[n] = DNR ⇔ Er

1 [n] = 0) and “relay withminimum required relaying energy” (a1[n] = R ⇔ Er

1 [n] =Ermin

1 [n]) in framen. If source node 1 chooses the actionDNRwhenErmin

1 [n] > 0, then node 2 will transmit at the end of theframe with the remaining energy required to ensure the QoSconstraint is satisfied at node 4. Since the channel magnitudesare assumed to be constant over the duration of the frame,the total transmission energy expended by node 2 in this casewill be the same as if node 2 had used direct transmission intimeslot 2. If source node 1 chooses the actionR, the packetwill be received by node 4 at the required QoS level withoutany additional transmission energy from node 2.

In timeslot 4, if node 1 has requested relaying, node 2 mustalso decide between the actionsDNR andR. The situation isthe same in this case as when node 1 relays for node 2 exceptthat node 2 has the advantage of having just observed whetheror not node 1 fulfilled its relaying request and can choose itsaction accordingly.

Since packets from both source nodes are always deliveredto each destination irrespective of whether relaying requestsare fulfilled or not, we define thestage-game payoffas thetransmission energy saved in the current frame with respecttodirect transmission. The payoff received by source nodei inframen for actions

a[n]=(a1[n], a2[n])∈{(DNR, DNR),(DNR, R),(R, DNR),(R, R)}

is denoted asπi(a[n], n). Note that whenevera[n] =(DNR, DNR), both source nodes receive a payoff of zero. Ifsource nodei choosesR and nodej choosesDNR, then nodeireceives a payoff ofπi(a[n], n) = −Ermin

i and nodej receivesa payoff ofπj(a[n], n) = E∗

j where

E∗j [n] := Edt

j [n] − Ej [n] > 0

is defined as the energy saved by nodej with respect todirect transmission if nodej requests relaying and nodei 6= jfulfills the relaying request by relaying with sufficient energyto ensure the QoS constraint is satisfied at destination nodedj . Finally, if a[n] = (R, R), the source nodes receive payoffs(π1(a[n], n), π2(a[n], n)) = (E∗

1 − Ermin

1 , E∗2 − Ermin

2 ). Fig-ure 2 summarizes the two-player relaying game in extensiveform [14] and shows the payoffs received by each source nodeas a function of the actions chosen by the players in the currentframe.

start of frame

1:DT 1:RR

2:DT 2:RR 2:DT 2:RR

1:DNR

2:DNR

1:R

2:DNR 2:DNR

1:DNR 1:DNR

2:R2:DNR

1:R1:DNR

2:R2:DNR 2:R2:DNR

timeslot 1

timeslot 2

timeslot 3

timeslot 4

(0, 0)(−Ermin

1 , E∗2 )(0, 0) (E∗

1 ,−Ermin

2 )(0, 0) (E∗1 − Ermin

1 , E∗2 − Ermin

2 )(−Ermin

1 , E∗2 )(E∗

1 ,−Ermin

2 )(0, 0)

Fig. 2. Two-player relaying game in extensive form with source nodesS =

{1, 2}. The actionsDT, RR, R, andDNR correspond to “direct transmission(no relay request)”, “request relay”, “relay”, and “do not relay”, respectively.The pairs at the bottom of the tree correspond to the payoffs of nodes 1 and 2,respectively, at the end of the frame.

1) Stage Game Analysis:Inspection of the payoff pairs inFigure 2 shows that, when node 2 is requested to relay suchthat Ermin

2 > 0, node 2 will choose the actionDNR becauseits payoff of choosingDNR is always better than choosingR, irrespective of node 1’s actions. Knowing this, node 1 willalso choose the actionDNR for the same reasons. Since bothsource nodes know that any relay requests will always berejected, both source nodes will choose to communicate withtheir respective destinations by direct transmission and receive

the payoff pair(π1(a[n], n), π2(a[n], n)) = (0, 0).To formalize this result, we briefly review the concept of a

Nash Equilibrium (NE) [14]. In ak-player game, the actionprofile(a∗

1, ..., a∗k) is an NE if, for each playeri, a∗

i is playeri’sbest response toa∗

−i, wherea−i denotes the actions of all theplayers except playeri. In framen, this can be expressed as

πi({a∗i [n], a∗

−i[n]}, n) ≥ πi({ai[n], a∗−i[n]}, n)

for all ai in the set of available actions for playeri and whereπi is the payoff function playeri. Intuitively, if all of theplayers are choosing NE actions, no player can increase theirpayoff by unilaterally deviating from the NE action profile.It is not difficult to show that the only NE of the two-playerrelaying stage game isa[n] = (DNR, DNR).

The dilemma in this result is that both nodes could poten-tially receive a payoff better than(0, 0) by accepting relayrequests whenE∗

1 − Ermin

1 > 0 and E∗2 − Ermin

2 > 0. Inother words, if the channel state is such that both nodes couldsave energy through mutual cooperation, both nodes would dobetter by choosinga[n] = (R, R) than a[n] = (DNR, DNR).Nevertheless, analysis of the stage game yields only one NEaction profile for selfish players: mutual non-cooperation.Inthe following section, we extend this stage-game analysis to arepeated game formulation and show that, unlike a single-stagegame, a repeated game with uncertain ending can include amutually cooperative NE for selfish players.

B. Repeated Game Formulation

Since each transmission session is composed ofN ≥ 1frames, the stage game formulation developed in the priorsection can be extended to arepeated gamemodel where theplayers interact over multiple stage games. IfN is knownto both source nodes in the current transmission session,backward induction arguments can be used to show that bothplayers will choosea[n] = (DNR, DNR) in each stage game.To see this, first consider the last stage game. Since there isno possibility of gain from future cooperation, node 2 willrationally chooseDNR to maximize its payoff. Knowing this,node 1 will also chooseDNR for the same reasons. Sinceeach node knows that the other node will chooseDNR in thelast stage game, they will also choosea[n] = (DNR, DNR)in the second to last stage game, and so on, ensuring that theonly rational strategy for both source nodes is to reject relayrequests in all of the stage games [15, p.10].

Now consider the scenario when the number of framesin the current transmission session is not known by thesource nodes. Specifically, we consider the case when thetransmission session continues after the current frame withfixed probabilityδ, whereδ is known to both source nodes.In this case, the number of framesN is a geometricallydistributed random variable with probability mass functionpN (n) = (1 − δ)δn−1 for n = 1, 2, . . . . Since the number offrames in the current transmission session is not known to thesource nodes (and, as will be discussed in Section III-B1, thepayoffs in future frames may also be unknown), both sourcenodes seek to maximize theirexpectedtotal payoff in the

transmission session. We define the expected total payoff ofnodei as

Πi := E

{

N−1∑

n=0

πi(a[n], n)

}

whereN is random andπi(a[n], n) may be random. Underthe assumption that the stage-game payoffs are independentof N , the expected total payoff can be shown to be equivalentto a repeated game having an infinite number of stages withfuture payoffs discounted according to the expected durationof the game [15]. For playeri, this can be expressed as

Πi =

∞∑

n=0

δnE{πi(a[n], n)}

whereδ is called thediscount factorandπi(a[n], n) is theith

player’s payoff in framen given action profilea[n].

In repeated games, players use astrategyto specify theiractions in each stage game as a function of the channel state,cooperative protocol, QoS constraint, and previous actionsof the other players. We define aTRIGGER strategy in therepeated two-player relaying game as follows: ifErmin

i [n] > 0,player i chooses the actionai[n] = R unless the other playerhas previously chosenDNR when relaying was requested. Ifplayeri choosesDNR whenErmin

i [n] > 0, then playeri is saidto defect. If either player defects, the other player “triggers”punishment by choosingDNR in all future stage games (notethat, since node 2 chooses its action after it observes theaction of node 1 in the current stage game, node 2 will triggerpunishment by playingDNR in the current stage game).

1) Repeated Game Analysis:In our analysis of the re-peated game scenario, the channel states in the current andprevious frames are assumed to be known to both sources.The channel states in future frames are not known; onlytheir distribution is known. The energy allocationE [n] ={E1[n], E2[n], Ermin

1 [n], Ermin

2 [n]} is assumed to dynamicallydetermined in each framen = 0, 1, . . . according to the knownchannel state, the cooperative protocol, and the QoS constraint.The following proposition establishes necessary and suffi-cient conditions under which the (TRIGGER,TRIGGER) strategyprofile is an NE of the repeated two-player relaying gamewith uncertain ending in systems with quasi-static i.i.d. fadingchannels.

Proposition 1. In a system with quasi-static i.i.d. fadingchannels, the strategy profile (TRIGGER,TRIGGER) is an NE ofthe repeated two-player relaying game with uncertain ending ifand only ifErmin

1 [n] ≤ E∗1 [n]+ δ

1−δ E1 andErmin

2 [n] ≤ δ1−δ E2

for all n = 0, 1, . . . whereEi := E{E∗i [n] − Ermin

i [n]}.

Proof: In framen′, if both nodes have faithfully playedand continue to play the strategy profile (TRIGGER,TRIGGER),

they will receive an expected total payoff of

Πi =

n′−1

∑

n=0

δn(E∗i [n] − Ermin

i [n])

+ δn′

(E∗i [n′] − Ermin

i [n′]) +

∞∑

n=n′+1

δnEi.

The first and second terms in this expression correspondto the known total payoff of the previous frames and theknown payoff of the current frame, respectively. The final termin this expression corresponds to the expected total payofffrom mutual cooperation in future stage games where thei.i.d. channel state assumption has been used to remove thedependence of the mean onn.

If node 1 deviates from theTRIGGER strategy by defectingin stage gamen′, it will receive a total payoff of

Π1 =

n′−1

∑

n=0

δn(E∗1 [n] − Ermin

1 [n])

because node 2 will punish node 1 immediately for itsdefection in the current stage game. Note that this totalexpected payoff does not exceed the total expected payoff fromfaithfully playing the TRIGGER strategy whenδn′

(E∗1 [n′] −

Ermin

1 [n′])+∑∞

n=n′+1 δnE1 ≥ 0. Hence, node 1 has no incen-tive to deviate from the strategy profile (TRIGGER,TRIGGER)whenErmin

1 [n′] ≤ E∗1 [n′] + δ

1−δ E1.If node 2 deviates from theTRIGGER strategy by defecting

in stage gamen′, it is punished by node 1 in the next stagegame and receives a total expected payoff of

Π2 =

n′−1

∑

n=0

δn(E∗2 [n] − Ermin

2 [n]) + δn′

E∗2 [n′].

The second term here corresponds to the payoff received bynode 2 in stage gamen′ when its packet is forwarded bynode 1 but it does not reciprocate. This total expected payoffdoes not exceed the total expected payoff from faithfullyplaying theTRIGGER strategy whenErmin

2 [n′] ≤ δ1−δ E2.

Proposition 1 implies that, as long as both sources can findan energy allocation such that they are not requested to expend“too much” relaying energy in the current frame, then mutualcooperation (with the threat of punishment for defection) is anNE of the repeated two-player relaying game with uncertainending. In other words, both source nodes have no incentive todefect when they can expect to receive more long-term benefitfrom cooperation than short-term benefit from defection. Notethat the strategy profile (ALWAYS DEFECT, ALWAYS DEFECT)is also an NE of the repeated two-player relaying game withuncertain ending since neither player stands to gain fromcooperation with an opponent that always defects.

As a special case of Proposition 1, we can also considera system with non-fading channels where the channel stateis the same over all of the frames in the transmission ses-sion, i.e. H [n] ≡ {H12, H13, H14, H21, H23, H24} for alln = 0, 1, . . . , and the sources use fixed energy allocations,

i.e. E [n] ≡ {E1, E2, Ermin

1 , Ermin

2 } for all n = 0, 1, . . . . Thefollowing lemma establishes necessary and sufficient condi-tions under which the (TRIGGER,TRIGGER) strategy profile isan NE of the repeated two-player relaying game with uncertainending in systems with non-fading channels.

Lemma 1. In a system with non-fading channels, the strategyprofile (TRIGGER,TRIGGER) is an NE of the repeated two-player relaying game with uncertain ending if and only ifErmin

1 ≤ E∗1 and Ermin

2 ≤ δE∗2 .

The proof of Lemma 1 follows from Proposition 1 bysubstituting Ei = E∗

i − Ermin

i for i ∈ {1, 2}. Unlike thecase with fading channels, all of the future payoffs are knownwhen the channels are non-fading; only the duration of thetransmission session is unknown. Both source nodes have noincentive to defect when they expect to receive more long-term benefit from cooperation than short-term benefit fromdefection.

In each framen = 0, 1, . . . , Proposition 1 and Lemma 1identify a set of feasible energy allocations under which selfishnodes will rationally choose mutual cooperation. This setmight be empty, depending on the channel state, cooperativeprotocol, and QoS constraint, in which case the only NE isfor both sources to use direct transmission. When this set isnot empty, there remains the question of how much relayingenergy the sources should demand of each other. The initialtransmit energies0 < E1[n] ≤ Edt

1 [n] and0 < E2[n] ≤ Edt2 [n],

when combined with the channel stateH[n], the cooperativeprotocol, and the QoS constraint, imply the minimum requiredrelaying energiesErmin

1 [n] and Ermin

2 [n]. Since the channelstate is known to both source nodes, one possible strategywould be for both source nodes to chooseE1[n] and E2[n]such that they request the maximum relaying energy from theother source under the conditions of Proposition 1, i.e. node 2selectsE2[n] such thatErmin

1 [n] = E∗1 [n] + δ

1−δ E1 and node 1selectsE1[n] such thatErmin

2 [n] = δ1−δ E2. Under this energy

allocation, the (TRIGGER,TRIGGER) strategy profile is an NEof the repeated game and the total expected payoff for bothsource nodes can be calculated asΠ1 = 0 andΠ2 = E{E∗

2 [n]}.This energy allocation, however, is likely to be inefficientin the sense that there may be other energy allocations thatresult in a better total expected payoff for one orboth sourcenodes. The question of how to select an efficient and mutuallyagreeable energy allocation is considered in the followingsection.

IV. EFFICIENT COOPERATIVE ENERGY ALLOCATION

When the setU\(0, 0) is not empty, selfish nodes willattempt to arrive at a unique mutually agreeable payoff pair(and, consequently, a unique energy allocation) through “bar-gaining”. The bargaining problem is one of the paradigmsof cooperative game theory in which a group of two ormore participants are faced with a set of feasible outcomes,any of which can be the bargaining solution if agreed tounanimously. Our use of the term bargaining here is somewhatmisleading in the sense that the nodes do not actually bargain

by communicating offers and counteroffers to each other.Rather, since the channel state is known to both source nodesand each node knows how the other will bargain, each nodecan determine the bargaining solution locally without any ad-ditional communication. The technique of uniquely dividing asurplus among selfish players is commonly called “bargaining”in the cooperative game-theory literature, however, and wewilluse this term here for consistency.

Let us define the pair(U , ∆) as the bargaining problem,where U is the set of all feasible stage-game payoff pairsand ∆ = (0, 0) is the disagreement payoff. If both sourcesfail to reach an agreement, they use direct transmission todeliver their packets to their intended destinations and receivethe disagreement payoff in the current stage game. Note that∆is always inU because direct transmission is always feasible.It is assumed thatU is a convex and closed set, bounded fromabove. Given the definition of the bargaining problem and theset of Pareto-efficient payoff pairsU , an axiomatic bargainingsolution is a functionB, based on a set of “reasonable”axioms, that maps every(U , ∆) to a unique member ofU .Specifying these axioms serves to characterize the solutionuniquely from among the set of Pareto-efficient points.

The most commonly used axiomatic bargaining solutionis the Nash Bargaining Solution (NBS) which is based onfour simple and well-accepted axioms and has been shown tohave close connections to subgame-perfect equilibria in infinitehorizon games [14]. These axioms can be briefly described as

1) (Pareto-efficiency) the bargaining solutionB(U , ∆)must be Pareto-efficient

2) (independence of linear transformations) ifT =ax + b with a > 0, then the bargaining solutionB(T (U), T (∆)) = T (B(U , ∆)), i.e. the bargainingsolution must be independent of the utility scales of theplayers

3) (symmetry) the bargaining solutionB(U , ∆) will giveequal payoffs to both players if the setU is symmetricin the sense that(u1, u2) ∈ U implies (u2, u1) ∈ U

4) (independence of irrelevant alternatives) ifV ⊆ U andB(U , ∆) ∈ V , then B(V , ∆) = B(U , ∆), i.e. theaddition of irrelevant alternatives does not affect thebargaining solution

Other axiomatic bargaining solutions based on different setsof axioms include the Raiffa (Kalai-Smorodinsky) [16] (RBS)and the Modified Thomson (MTBS) bargaining solutions. Aunified view of all these axiomatic models is presented in [17].

The NBS, RBS, and MTBS bargaining solutions can all beexpressed as

Bβ(U , ∆) = arg max(w1,w2)∈U

w1w2 (1)

where β is a scalar parameter specified by the bargainingsolution (β = 0 for NBS, β = 1 for RBS, andβ = −1for MTBS) and

wi :=πi

mi+ β

(

1 −πj

mj

)

is the preference functionof source nodei, with j ∈ {1, 2},j 6= i, andmi is the maximum stage-game payoff of sourcei over U . The value ofβ in the preference function impliesa tradeoff between a player’s own gain and the other player’slosses, normalized by each player’s maximum gain. Whenplayers use the NBS (β = 0), the bargaining solution issuch that players only maximize their own payoff withoutconsideration of the losses incurred by the other player. Whenplayers use the RBS (β = 1), the bargaining solution issuch that each player’s payoff is proportional to its maximum.Finally, when players use the MTBS (β = −1), each playerhas the same preference function and the bargaining solutionis such that the sum of the normalized payoffsπ1/m1+π2/m2

is maximized.To illustrate the feasible payoff set, the Pareto-efficient

subset, as well as the different bargaining solutions, Figure 3shows the positive quadrant of the feasible stage-game payoffsas well as the NBS, RBS, and MTBS for a two-playerrelaying game using orthogonal amplify-and-forward (OAF)cooperation with channel stateH12 = H21 = 5, H13 = 0.5,H14 = 5, H23 = 3, andH24 = 0.9 and an SNR=10dB QoSconstraint. The maximization in (1) is performed numericallyfor β ∈ {−1, 0, 1} to obtain the three different bargainingsolutions. The centrally controlled maximum total payoff(or, equivalently, minimum total energy), is also plotted forcomparison.

0 2 4 6 8 10 12 14 160

1

2

3

4

5

6

7

8

9

10

payoff player 1

payo

ff pl

ayer

2

Feasible Payoff AllocationRKS Bargaining Solution (β=1)Nash Bargaining Solution (β=0)Modified Thomson Bargaining Solution (β=−1)Maximum Sum Payoff Solution

Fig. 3. Illustration of NBS, RBS, and MTBS two-player bargaining solutionsover the set of feasible stage-game payoffs,U .

As a final comment on the use of bargaining solutions to findan efficient resource allocation to which both selfish sourceswill agree, it should be reiterated any bargaining solutionwillimply an energy allocation that must satisfy the NE criteriadeveloped in Proposition 1 (or Lemma 1 in the case of non-fading channels). If the bargaining solution implies an energyallocation that does not satisfy the NE criteria, both nodeswillknow this and will use direct transmission to avoid defectionand triggering punishment.

V. K > 2 PLAYER RELAYING GAME

In the case when there areK > 2 source nodes in agiven transmission session, we restrict our attention to theparticular scenario in which the source nodes formfixed two-player partnershipsfor the duration of a transmission session.The central problem is then the assignment of partners toeach source node (except one, whenK is odd). Specifically,we consider the problem of how to form stable partnershipsendogenously by selfish source nodes.

We define apairing instanceP as a set of two-playerpartnerships in which all but at most one source nodes aredisjointly paired. It is not difficult to show that each pairinginstance in aK > 2 player relaying game is an equilib-rium with respect to unilateral deviations when the energyallocations are determined by a bargaining solution and theNE conditions are satisfied. Pairing instances may not be anequilibrium with respect to multi-player deviation, however,where two or more players leave their current partners andform different partnerships. As an example, consider a networkwith K = 4 source nodes denoted asS = {1, 2, 3, 4} and apairing instanceP = {{1, 2}, {3, 4}}. Suppose that all nodesreceive an identical expected payoff ofπP > 0 under thispairing instance. Suppose further that, under pairing instanceQ = {{1, 3}, {2, 4}}, nodes 1 and 3 each receive a payoffof πQ > πP while nodes 2 and 4 receive a payoff of zero.It is clear that nodes 1 and 2 both improve their payoff bydeviating from pairing instanceP to Q, and they can do sowithout any consent (or repercussions) from nodes 3 and 4.Hence, although pairing instanceP is an equilibrium withrespect to unilateral deviation, it is not an equilibrium withrespect to multi-player deviations.

While there are many notions of equilibrium inK > 2player games, we restrict our attention here to the notion ofapairwise-stable network [19]. A pairwise-stable network is apairing instance that is immune to any improving two-playerdeviations, where an improving two-player deviation in ourcontext is a deviation in which two players sever their currentpartnerships and form a new partnership such that at leastone player in the new partnership receives a strictly greaterexpected payoff while the other player in the new partnershipreceives an expected payoff no worse than before. The pairinginstanceP in the previous paragraph is clearly not a pairwisestable.

The problem of how to endogenously form a pairwise-stablenetwork among selfish nodes has been studied extensivelyunder the title of stable matching problems[11]. Stablematching problems are generally divided into two categories:two-sided matching and one-sided matching. In a two-sidedmatching problem, also referred to as themarriage problem,there are two sets of participants and the matching is a one-to-one mapping between the two sets. This is the classicmatching problem discussed in [11], where it is shown thatevery instance of the two-sided matching problem alwaysadmits at least one stable solution. In the one-sided matchingproblem, a matching results in a partition of the single set of

participants into disjoint pairs. This is a generalizationof themarriage problem and is known as theroommateproblem. Amajor difference between a two-sided (marriage) problem anda one-sided (roommate) problem is that the roommate problemmay not necessarily have a stable matching.

In the context of aK > 2 player relaying game, thematching problem is one-sided since the source nodes arehomogeneous. In the absence of central control, the sourcenodes can endogenously attempt to form a stable matchingby first computing the bargaining payoffs (and checking theNE conditions) for each of theK − 1 possible partners inthe network. These payoffs then imply a preference table,known to each node, that is used as the input to the StableRoommate (SR) algorithm [12] computed locally at each nodeto determine a pairwise-stable matching, if one exists. If apairwise-stable matching exists, the nodes then cooperateinthe transmission session with these pairings using transmis-sion/relaying energies specified by the appropriate bargainingsolution (or direct transmission if the bargaining solution doesnot satisfy the NE conditions). If a pair-wise stable matchingdoes not exist, one approach is to resort to direct transmissionin the current transmission session. Another approach is tolocally compute the centrally controlled pairings and formpartnerships based on these pairing. Both of these approachesare demonstrated in the numerical results in the followingsection.

VI. N UMERICAL RESULTS

To demonstrate the energy efficiency of wireless networkswith selfish energy allocation and selfish partner selection,this section provides a numerical example for a wirelessnetwork with non-fading path-loss channels using orthogo-nal amplify-and-forward (OAF) cooperative relaying. In eachtransmission session,K source andK destination nodesare randomly placed on a disk of radiusR = 10 meters(with uniform distribution). The squared channel magnitudebetween each node pairi and j is then calculated asHij =(

(xi − xj)2 + (yi − yj)

2)−γ/2

where γ = 4 is the path-loss parameter and(xi, yi) is the cartesian coordinate pair ofnodei. The relative energy efficiency of several schemes, aver-aged over the random node positions, are compared in Figure 4with respect to a system with centrally controlled (CC) energyallocations and CC pairing assignments. In Figure 4, a relativeenergy efficiency of one corresponds to the minimum totalnetwork transmission energy (obtained through CC energyallocations and CC pairing assignments) and a relative energyefficiency of zero corresponds to direct transmission.

The results in Figure 4 show that a system with purelyselfish energy allocations (NBS/NE) and endogenously formednode pairings (SR) can achieve a relative energy efficiency ofapproximately half of that of a system with centrally controlledenergy allocations and pairings for values ofK ≥ 10. Sincea stable roommate node pairing solution does not alwaysexist, we provide “optimistic” and “pessimistic” bounds ontherelative energy efficiency of NBS/NE energy allocations with

SR pairings by using the centrally controlled pairing assign-ment and direct transmission, respectively, when a SR pairingsolution is not found. Note that the results corresponding toCC energy allocations are not stable with respect to unilateraldeviations in the sense of an NE. This is because some nodesmay receive negative payoffs under CC energy allocation andthese nodes would rationally choose defection. Also note thatall of the results corresponding to NBS/NE energy allocationsare stable with respect to unilateral deviations in the sensethat no single node can improve its payoff by defecting. TheNBS/NE results with SR pairings are stable with respect tobilateral (pairwise) deviations when the SR pairing solutionexists, whereas CC and random pairings are not, in general,stable with respect to bilateral deviations.

This numerical example confirms that, in the absence ofcentral control in a wireless ad hoc network, sources canendogenously form cooperative pairs and selfishly allocatetheir transmission/relaying energies to improve the overallnetwork energy efficiency with respect to direct transmission.The results also suggest that SR pairings are significantly moreefficient than random pairings and only suffer a small loss withrespect to CC pairings.

2 4 6 8 10 12 14 160

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

number of nodes in transmission session (K)

rela

tive

ener

gy e

ffici

ency

CC energy allocation with random pairingsNBS/NE energy allocation with CC pairingsNBS/NE energy allocation with optimistic SR pairingsNBS/NE energy allocation with pessimistic SR pairingsNBS/NE energy allocation with random pairings

Fig. 4. Relative energy efficiency of a wireless network using OAF cooper-ative relaying with centrally controlled (CC) or selfish (NBS/NE) energy al-location and with random, centrally controlled (CC), or optimistic/pessimisticstable roommates (SR) partner assignments.

VII. C ONCLUSION

This paper employs both non-cooperative and coopera-tive game theoretic tools to analyze the energy efficiencyof wireless ad hoc networks with selfish energy allocationand endogenous partner selection. The novelty of this studyis that cooperation is established without the added com-plexity of extrinsic incentive mechanisms, altruistic nodes,and/or community enforcement. We first described a two-player repeated relaying game and developed the necessaryand sufficient conditions under which “natural” cooperation ispossible. These conditions were derived for both fading and

non-fading channels. By incorporating axiomatic bargainingmodels, we then showed how to calculate a unique Pareto-efficient cooperative energy allocation that can be locallycomputed and agreed upon by selfish nodes without centralauthority. We then extended the two-player model to networkswith K > 2 players and proposed a technique to endogenouslyform cooperative partnerships without central control throughthe stable roommate algorithm. Finally, we provided numericalresults based on OAF relaying and quantified the energyefficiency of an ad hoc wireless network with selfish sourceswith respect to a network with centrally optimized energyallocations and pairings.

REFERENCES

[1] Q. Zhao and L. Tong, “Energy efficiency of large-scale wireless net-works: Pro-active vs. reactive networking,”IEEE Journal on SelectedAreas of Communications, vol. 23, pp. 1100–1112, May 2005.

[2] S. Marti, T. J. Giuli, K. Lai, and M. Baker, “Mitigating routingmisbehavior in mobile ad hoc networks,” inin Proc. ACM/IEEE An-nual International Conference on Mobile Computing and Networking(Mobicom), pp. 255–265, 2000.

[3] J. Yang, A. G. Klein, and D. Brown III, “Cooperation without extrinsicincentive mechanisms in wireless networks with selfish nodes,” Acceptedto appear in IEEE Signal Processing Magazine, 2009.

[4] J.-P. Hubaux, T. Gross, J.-Y. Le Boudec, and M. Vetterli,“Towardsself-organized mobile ad hoc networks: the terminodes project,” IEEEComm. Mag., vol. 39, pp. 118–124, Jan. 2001.

[5] L. Blazevic, L. Buttyan, S. Capkun, S. Giordano, J.-P. Hubaux, andJ.-Y. L. Boudec, “Self organization in mobile ad hoc networks: theapproach of terminodes,”IEEE Communications Magazine, vol. 39,pp. 166–174, June 2001.

[6] M. Jakobsson, J.-P. Hubaux, and L. Buttyan, “A micro-payment schemeencouraging collaboration in multi-hop cellular networks,” in FinancialCrypto 2003, 2003.

[7] L. Lai and H. El Gamal, “On cooperation in energy efficientwirelessnetworks: The role of altruistic nodes,”IEEE Transactions on WirelessCommunications, vol. 7, pp. 1868–1878, May 2008.

[8] M. Felegyhazi, J.-P. Hubaux, and L. Buttyan, “Nash equilibria of packetforwarding strategies in wireless ad hoc networks,” inIEEE Trans.Mobile Computing, Vol. 5, May. 2006, pp. 1044–1049, May 2006.

[9] J. Yang and D.R. Brown III, “Energy efficient relaying games in cooper-ative wireless transmission systems,”Signals, Systems and Computers,2007. ACSSC 2007. Conference Record of the Forty-First AsilomarConference on, pp. 835–839, Nov. 2007.

[10] J. N. Laneman, D. N. C. Tse, and G. W. Wornell, “Cooperative diversityin wireless networks: Efficient protocols and outage behavior,” IEEETransactions on Information Theory, vol. 50, no. 12, pp. 3062–3080,2004.

[11] D. Gale and L. Shapley, “College admissions and the stability ofmarriage,” The American Mathematical Monthly, vol. 69, pp. 9–15,January 1962.

[12] D. Gusfield and R. W. Irving,The Stable Marriage Problem: Structureand Algorithms. Cambridge, Massachusettes: The MIT Press, 1989.

[13] J. Yang, D. Gunduz, D.R. Brown III, and E. Erkip, “Resource allocationfor cooperative relaying,” inProceedings of the Conference of Informa-tion Sciences and Systems (CISS 2008), (Princeton, NJ), Mar 2008.

[14] K. Binmore,Fun and Games: A Text on Game Theory. Lexington, MA:D.C. Heath, 1992.

[15] R. Axelrod, The Evolution of Cooperation. Basic Books, 1984.[16] E. Kalai and M. Smorodinsky, “Other solutions to nash’sbargaining

problem,” Econometrica, vol. 43, no. 3, pp. 513–518, 1975.[17] X. Cao, “Preference functions and bargaining solutions,” in IEEE

Conference on Decision and Control, pp. 164–171, 1982.[18] J. Yang and D.R. Brown III, “The effect of channel state information on

optimum energy allocation and energy efficiency of cooperative wirelesstransmission systems,” inProceedings of the Conference of InformationSciences and Systems (CISS 2006), (Princeton, NJ), pp. 1044–1049, Mar2006.

[19] M. Jackson and A. Wolinsky, “A strategic model of socialand economicnetworks,”Journal of Economic Theory, vol. 71, pp. 44–74, June 1996.