Profit Maximization via Energy Consumption Reduction in Cooperative Resource Management Systems

Profit Maximization via Energy Consumption Reduction in Cooperative Resource Management Systems

Matteo SerenoDipartimento di InformaticaUniversità di Torino, Italy

Joint work with C. Anglanoand M. GuazzoneUniversità del Piemonte Orientale, Alessandria, Italy

Outline• First (and motivating) example:

cooperation among mobile network providers• Cooperative game theory approach• Basic notions of cooperative game

theory

• A more genaral approach for mobile networks

• Another application of the framework: ccoperation among cloud providers

• Discussion

oCellular networks represent a promising area to develop energy efficient techniques Enormous diffusion of cellular

accesses In these networks a significant

fraction of power consumption is in charge to the provider (e.g., the impact of the reduction of this consumption is an evident result)

The energy consumed by the access networks is largely wasted (e.g., the networks are dimensioned on peak hour traffic)

Cooperation among Mobile Network Providers

o In general the same area is normally covered by several competing providers

oWhen the traffic is low, the networks become over-dimensioned and one can serve the entire area

oThe other access networks are switched off and all the customers are served by the network that remains active

Cooperation among Mobile Network Providers

In the literature there are many studies of this approach (for instance Ajmone, Meo et al)Most of the previous proposals focus the attention on the evaluation of the energy savingsIn 2011 I shifted the attention on different issues

The cooperation among the providers (called players in terms of G.T.)Cooperation that must considered a paradigm to improve the performance of wireless networks and/or the energy efficiency

Cooperation among Mobile Network Providers

I addressed questions such asWhy the providers should cooperate ?What are the cooperative structures ? Efficiency and stability of the cooperation(s)Allocation of the benefits in appropriate manner among the cooperative players

• Game theory framework to describe the interaction among providers

• Cooperative Game Theory to deal with the formation of coalitions

• Games with transferable utility, where the utility (e.g., monetary value) can be divided in any manner among the coalition members

• Criteria to allocate of the benefits/profits among the coalition members

Game Theory Approach

Cooperative Games:Basic Idea

Cooperative game theory provides analytical tools to study the behavior of rational players when they cooperate

Canonical Coalition GamesThe grand coalition of all players is an optimal structure and is of major importance.Key question: “How to stabilize the grand coalition”

Coalition Formation GamesThe network structure that forms depends on gains and costs from cooperation.Key question: “How to form an appropriate coalitional structure and how to study its properties”

Coalition Graph GamesPlayers’ interactions are governed by a communication graph structure.Key question: “How to stabilize the grand coalition or form a network structure taking into account the communication graph”

We consider a given area (a cell in terms of mobile networks) covered by n providers

Network providers = Players N is the set of players/providers (with |N|=n)ui is the number of users serverd by the i-th network providerv({i}) = profit of the i-th provider when there are ui users in its cell (value in terms of G.T.)

Game Theory Approach: Problem formulation

User subscription fees Energy + other costs

v({i}) = Moneyi(with ui users) – Costi(with ui users)

M. Sereno, “Cooperative game theory framework for energy efficient policies in wireless networks”, ACM e-Energy Conference, Madrid 2012.

Game Theory Approach: Coalition Games

Formation of cooperative groups (coalitions)

A coalition S (with S N) is an agreement among the players in S

In a coalition S one access network is active while the other |S| - 1 are switched off

S

Game Theory Approach: Value of a coalition

v({i}) = value of i and v({j}) = value of jv({i}{j}) = (Moneyi(ui) + Moneyj(uj)) - Costi(ui+uj)

The utility that i and j receive from the division of v({i}{j}) is the player’s payoff xi and xj

The coalition value v quatifies the worth of a coalition (v: 2N ℝ)A coaltion game can be defined as a pair (N,v)

sum of subscriptions fees in the two networks (i, and j)

costs in charge to network i (e.g., we assume that i remains active and j is switched off) to serve the custormers ui and uj

N = coalition among all the playersv(N) = value of Nxi = payoff that player i receives (i.e., xi is a fraction of v(N))

A payoff vector x is efficient iff = v(N)

It is individual rational if xi ≥ v({i}) (i.e., every player receives a payoff that is greater that the one it gets when acting alone)

Game Theory Approach: Properties of a coalition

StabilityAssume that x is the payoff vectorA coalition N is unstable if there is a subset of players S⊂N

if v(S) >

S

The players have incentive for coalition S

The Core: the most renowned conceptFor a TU game, the core is a set of payoff allocations (x1, . . ., xN) satisfying two conditions

The cores of cooperative games can be emptyIn this case the grand coalition cannot be stabilizedSome players or groups of players are better off when acting alone that when cooperating (with other players of the grand coalition)

We can check whether the core is non-empty (e.g, with superadditivity)

Game Theory Approach: Properties of a coalition

Cost Allocation or Surplus Sharing many proposals (in the literature)Comparison among three allocation strategies:

Equal ShareShapley valueNucleolus

Properties:EfficiencySymmetryDummy playerAdditivityMonotonicityConsistency/Stability

Allocation Strategies: from v(N) to x

Efficiency: a payoff vector x is an efficient cost allocation method if

Monotonicity: if some player’s contribution to all coalitions to which he belongs increases, then that player’s allocation should not decrease. Similarly, if costs increase (and the value thereby decrease) no one should receive more that they had before.

Consistency/Stability: an allocation of v(N) should be viewed as fair by all possible subgroups of N.No subset of players should find incentive to change the allocation.

=v(N)

• There is no method suitable for all the cases• The axioms can help to narrow down the set of possible allocation methods

• For our system the game is not a one-shot affair, but is reassessed periodically

• The Shapley value does not ensure the consistency (i.e., it may yield unstability)

–But there are cases where we can prove that the Shapley value is in the core (so it is stable)

• The nucleolus does not ensure monotonicity

• For our problem the Shapley value is better than Nucleolus

What is the best allocation strategy ?

The players join a coalition in some random order. The marginal contribution that a player makes to the overall value of the coalition can depend on at which point in coalition formation process that the player joinsThe Shapley value is defined as the average marginal contribution of a player, taken over all possible ways to form the grand coalition N

Marginal contribution: m(i, S) = v(S) - v(S – {i})

The Shapley value of player i

The Shapley Value

𝜑𝑖 (v )= ∑𝑆  𝑁𝑖∈𝑆

|S− {i }|!∨N−S∨!n !

m(i, S)

• The framework is helpful for the design of cooperative strategies

• PoC for the application of Cooperative Game Theory to Energy Optimization of Cellular Networks

Pros/Cons of the approach

Use of the coalition formation games when the grand coalition is not feasible

Can we extend the applicability of the

framework ?There are time intervals where the load of the networks is such that only one of them can serve all the users

• After the 2012 that other similar studies appeared (different settings)• In 2014, using the Coalition Formation Games, we propose a more

realistic and flexible model– More realistic Base Station power consumption model with load independent

and load dependent power consumption terms– User QoS requirements in terms of bandhwidth requirements – Penalties incurred by the network operator if a user receives downlink rate

lower than its QoS requirement– Costs to set up (and manage) a coalition among network providers– Use of Coalition Formation Games + Optimization to accounts for a wide set

of network scenarios

The problem becomes a profit maximization for the network providers with respect to their energy

consumptions

Extensions

C. Anglano, M. Guazzone, M. Sereno “Maximizing profit in green cellular networks through collaborative games”, Computer Networks Journal, Dec 2014.

•Cooperation gains are limited by a cost•The grand coalition is NOT guaranteed to form•Cluster the network into partitions

• New issues: network topology, coalition formation process, environmental changes, etc

•Key Questions– How can the players form coalitions?– What is the network structure that will form?– How can the players adapt to environmental changes such

as mobility, the deployment of new users, or others?– Can we say anything on the stability of the network

structure?

Coalition Formation Games: Main Properties

To cooperate a set of network operators must form a coalition: they must agree to share their own base stations and users among themThere can be several different coalitions that can be formed

Different compositions, different profits

To join a coalition a network operator must find it profitable

Better than other possible choices: join another coalition (or working alone)

Coalition Formation Games for cellular network operators

The profit must not be not ephemeral and a network operator seeks properties such asStability: none of the coalition participants finds that it is more profitable to leave the coalition

Lack of stability causes monetary losses

Fairness: when joining a coalition a network operator expects that the resulting profits are fairly divided among the participants

N = set of players/network_operatorsSN = coalitionAn agreement among the network operators in S

At any given time the set of players is partitioned into a coalition partition ={S1, S2, …, Sl} where for each k SkN, for jk SjSk=, and =N

Coalition Formation Games for cellular network operators

Agreement: the network operators of the coalition S decide to serve all the custormers of the coalition minimizing the costs resulting from serving all these users using all the resources provided by the coalition

Difference with the previous formulation

The coalition value v(S)

v(S) = R(S) – Q(S) – K(S)

Coalition Formation Games for cellular network operators

Sum of revenue rates of the individual users for each of the network operator of the coalition

Coalition load cost rate: computed minimizing the costs resulting from serving all the users of the coalition using all the coalition resources

Coalition formation cost rate: the term takes into account the cost incurred by players to establish and maintain the coalition(e.g., costs for system reconfiguration to enable user migration and handover)

Players (network operators) have preferences over coalitions Class of Coalition Formation Games where the worth of the coalition only depends on the players of that coalition (the other players in other coalitions do not affect this value)

Hedonic Coalition GamesEach player i can compare all the possible coalitions it may joinPreference relation i: For any i N and S1, S2 N

S1 i S2

player i prefers being in coalition S1 rather than in S2 (or at least i prefers equally both coalitions)i is a reflexive, complete and transitive binary relation over all the set of coalitions that player i can formWe assume that

S1 i S2 ui(S1)≥ ui(S2)

Coalition Formation Process

={S1, S2, …, Sl} is a coalition partition on Ni is a playerS(i) is the coalition which includes i

i decides to leave its current coalition S(i) to join another coalition Sk (with Sk) iff

Ski S(i)that is, if its utility (payoff) in the new coalition exceeds the one it is getting in its current coaltion

{S(i), Sk} → {S(i)\{i}, Sk{i}}

Coalition formation Games: The Algorithm

The shift rule → provides a mechanism through which any player can leave its current coalition and joinThis rule can be seen as a selfish decision

AlgorithmEach player searches asynchronously with respect to the other players the set of possible coalitions it may joinand evaluates whether it is preferable to remain in its current coalition or join another coalitionThe algorithm starts with 0={{1}, {2}, …, {n}}, i.e., initially there are no coalitions

Coalition formation Games: The Algorithm

The coalition formation is a dynamic process where each player evaluates the convenience of being in different coalitionsProblem: the player may re-evaluate the same coalitions⇒ endless coalition formation processSolution: to avoid generating twice the same candidate coalition the player stores the identity of the coalitions that have been already evaluated (a pruning strategy defined in W. Saad et al “Hedonic coalition formation for distributed task allocation among wireless agents”, IEEE Trans. on Mob. Comm., 2011)

Definition of function ui(S)

= 𝜑𝑖 (S ) ,   if   S∉h ( i )

Shapley value for player i in coalition S

The history set where player i stores the identity of the coalitions that have already evaluated

Existence of stable partitionsAssumption: the players are myopicStability concepts for hedonic games (Bogomolnaia and Jackson, 2002)• individual deviations • coalitional deviations

In our studies we only account for individual deviations: we exlude collusions among the players

Hedonic games: stability

they do not take into account how their decisions to form a coalition will affect in the future the decisions of other playersA stability definition of this type is the Nash stability

A partition is Nash stable if no player can benefit by moving from his coalition to another (possibly empty) coalition

A partition is core stable if there is no subset of players that would like to separate and form their own coalition

We derived a distributed hedonic coalition formation algorithm that converges to a final Nash stable partition

Time discretizationIn each time step we compute the coalition valuesv(S) = R(S) – Q(S) – K(S) for the term Q(S) we develop an optimization modelFor the coalition S we define a Integer Linear Program modeling the problem of allocating a set of users onto a set of BSs

The term K(S) represents the costs to set up and maintain the coalition S

Computation of the costs: optimization model

• Simulations of a several (realistic) scenarios– System configurations

with 5 network operators (5 Base stations) placed in 1km x 1km square area

– BSs parameters (e.g., bandwidth, power consumpution, and so on) derived from the literature

– Load of the BSs derived from real data traffic (avaliable in the literature)

– Time duration: 1 week

Evaluation

• Metric: net profit– Ratio between the profit

obtained by player with the algorithm and the profit it gets when acting alone

– Computed for each discrete step

• Large set of experiments: – heterogeneous/homogeneos

scenarios, – network operators with

identical/different load profiles,

– …

In the experiments we performed we observed net profit increments that vary from 70% to 150% (for each network operator)

Summary of the evaluationEnergy costs greatly influence the achieved profitsNetwork operators with heavier load can offload their users to other operators thus limiting their QoS penalty (higher profit)Network operators with lighter load can host users of other operators (higher profit due to a better use of their network)The discretization step has a significant impact

Pros/Cons and (possible) Developments • A viable cooperation

scheme among a group of network operators to reduce the costs (i.e., energy costs)

• The proposed solution could be “easily” implemented in real scenarios

A distributed approach that does not need (strong) synchronization among the network operators

• The agreement among different (competing) network operators is not so “easy”Economic convenienceIncentive policies

• The solution of the optimization could be computationally expensive (in case of a large number of BSs)

• Implementation of multi-cells scenario (e.g, development of credit-based approaches among the network operators)

• Fast approximation for the optimization model• Scenarios with owners and tenants

A Cloud Provider (CP) aims at. . .Maximize Net Profit = (Revenues - Costs)

(subject to SLAs)

Another application of the framework: cooperation among cloud providers

A significant fraction of the operational costs for a datacenter derives from power consumption (e.g., from 20% to 40%)Cloud Coalition: Members agree to mutually share their own resources to run the end-user applications of the other members

Benefits:• Lending underutilized/unused resource capacity to

amortize electricity costs• Borrowing resource capacity to . . .

– Serve more customers– Exploitation of (geographically) different energy prices – Reduce electricity costs

Basic Idea

Problems: How to form coalitions?Many different alternatives

Goal:Coalitions that areStableFair

Contributions: Application of the Cooperative Game Theory Framework to this problem

Hedonic Games for Cloud Coalitions

Problem & Goal

When alone,Each Cloud Provider aims at maximizing its own net profit

In a coalition S,The Cloud Provider pays for the energy consumed to serve the hosted user processes (e.g., VMs) and receives the profits for its service

The net profit can be expressed as

The idea

ProfitR(S)=

Sum of revenues of the individual users for each of the CP of the coalition

Coalition cost: computed minimizing the costs resulting from serving all the users of the coalition using all the resources of all CPs

Coalition formation cost: the term takes into account the cost incurred by the CPs to establish and maintain the coalition

Each Cloud Provider aims at mazimize its share of ProfitR(S)Hedonic Game-Based Framework (with Transferable Utility) to model cooperation between CPsEach CP

Pays for the energy consumed by each hosted VM, butReceives a reward for doing so

The Hedonic Coalition Formation GameTo model coalition formationEach CP assigns preferences over coalitions

Ingredients

Ingredients (cont’d)Players set N All CPsPlayers Coalition S ⊆N Coalition of CPsCoalition Structure ={S1, S2, …, Sl} Partition of NCoalition Value v(S) Coalition net

profit ProfitR(S)Profit Allocation xi(S) Shapley Value

Hedonic Coalition Formation Game (N, v, i)Player i assigns preferences i over coalitionsFor the player i SaiSb iff

Sa has not been evaluated yet (use of the history), andxi(Sa)≥xi(Sb)

The Hedonic Shift Rule →i

•Player i leaves its current coalition to either– enter another more preferred one, or– stay alone

Enabling of →i

• Given and iN, “→i” is enabled for i iff Sk such thata) i∉Sk

b) has not already visited Sk, andc) i strictly prefers {Sk, i} over the current coalition

Hedonic shift rule algorithm

The algorithm is similar to the previous one– Convergence to a final f

– Proof of the Nash stability– Asynchronous and distributed

Two metrics– Net Profit Increment– Energy Saving

Evaluation by simulation (400 random regerated scenarios)Coalition vs no-CoalitionSynthetic summary of the results (in %)

•Energy Saving min 21.6 max 33.6•Profit Increment min 10.5 max 20.1

Hedonic shift rule algorithm (cont’d)

Conclusions and Future developments• The framework is enough flexible and could include other features• The optimization problem requires additional work (e.g., to obtain

fast approximate solutions)

Other problems stem: …. . . …

A related problem: Coalitions of Data centers geographically distributed

Coalition(s) of DCs to minimize energy costs

Different (and variable) energy costs in different

regions

Change in workloads (of the power networks)

Coalitions (and in general geographical load balancing) may create a vicious cycle among electricity demand, cost and price!!!

Time to finishThank you for the attention

?

I am here at INRIA until the end of March 2016matteo.sereno@di.unito.it matteo.sereno@inria.fr