Hybrid Overlay/Underlay Cognitive Femtocell Networks…ncel.ie.cuhk.edu.hk/sites/default/files/07042288.pdf · IEEETRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL.14,NO.6,JUNE 2015 3259

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 14, NO. 6, JUNE 2015 3259

Hybrid Overlay/Underlay Cognitive FemtocellNetworks: A Game Theoretic Approach

Bojiang Ma, Student Member, IEEE, Man Hon Cheung, Vincent W. S. Wong, Senior Member, IEEE, andJianwei Huang, Senior Member, IEEE

Abstract—Femtocell networks have the potential to satisfy theincreasing demand of mobile data usage. The recently proposedconcept of cognitive femtocell network provides an effective wayto further improve the spectrum spatial and frequency reuse. Inthis paper, we study the subchannel allocation problem for orthog-onal frequency division multiple access (OFDMA)-based hybridoverlay/underlay cognitive femtocell networks. While most of theprevious related studies did not fully exploit the potential of spatialand frequency reuse of the network, we propose a hybrid overlayand underlay spectrum access mechanism to further improvethe performance of cognitive femtocell networks. We formulatethe subchannel allocation problem as a coalition formation gameamong femtocell users under the hybrid access scheme, and ana-lyze the stability of the coalition structure. We propose an efficientalgorithm based on the solution concept of recursive core, andachieve a stable and efficient allocation. Simulation results showthat the proposed algorithm achieves an improvement in aggregatenetwork throughput up to 72% comparing to the overlay onlyscheme, 35% comparing to the underlay only scheme, and 18%comparing to a recently proposed coalition formation algorithmin the literature.

Index Terms—Cognitive femtocell networks, hybrid overlay/underlay cognitive radio, coalitional game theory.

I. INTRODUCTION

A femtocell access point (FAP) is a small low-power cellu-lar base station, deployed either by the users or service

providers to improve the coverage and capacity of cellularsystems. There have been increasing interests in femtocelldeployment in wireless cellular networks. Subscribed femtocellcustomers can autonomously deploy FAPs to create cellularhotspots [1]. Due to the autonomous deployment of FAPs,interference management becomes a challenging issue [2]. Onepromising solution to address the interference issue is to useorthogonal frequency division multiple access (OFDMA) forfemtocells [3], [4]. In OFDMA systems, the available spectrum

Manuscript received March 1, 2014; revised October 8, 2014; acceptedJanuary 8, 2015. Date of publication February 13, 2015; date of current versionJune 6, 2015. This work was supported by the Natural Sciences and EngineeringResearch Council (NSERC) of Canada and the General Research Fund (ProjectNumber CUHK 412713) established under the University Grant Committee ofthe Hong Kong Special Administrative Region, China. The associate editorcoordinating the review of this paper and approving it for publication wasC.-B. Chae.

B. Ma and V. W. S. Wong are with the Department of Electrical andComputer Engineering, The University of British Columbia, Vancouver, BCV6T 1Z4, Canada (e-mail: [email protected]; [email protected]).

M. H. Cheung and J. Huang are with the Department of InformationEngineering, The Chinese University of Hong Kong, Shatin, Hong Kong(e-mail: [email protected]; [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TWC.2015.2403363

is divided into orthogonal subchannels. The system allocatesdifferent users with different groups of orthogonal subchannels.However, in an area with lots of femtocells, it is unlikely thatall users can be allocated different orthogonal channels at thesame time, since the number of channels may be far less thanthe number of users. Furthermore, to utilize the limited licensedspectrum more efficiently, the service providers would considerfrequency reuse in the network planning. The frequency reuserequires adaptive scheduling algorithms to mitigate the co-channel interferences among users using the same channel, andcognitive radio (CR) can be a promising technology to realizesuch flexible interference management [5].

Spectrum overlay and spectrum underlay are two approachescommonly considered in cognitive radio networks (CRNs).In an overlay cognitive system, secondary users (SUs) areonly allowed to use unoccupied spectrum holes (channels).Spectrum underlay allows SUs to simultaneously operate infrequency bands where primary users (PUs) are active, butunder strict SU transmission power constraints [6]. The hybridoverlay/underlay spectrum access combines the benefits of bothoverlay and underlay methods, creating a new approach tofurther exploit both the unused spectrum regions and the under-utilized spectrum areas. Hybrid overlay/underlay CR spectrumaccess is flexible to deal with different femtocell densities inboth urban areas and rural areas [7]. When the spectrum isless crowded such as in rural areas, SUs are more likely totransmit on idle subchannels. In case of dense deployment offemtocells such as in urban areas, where not all users can findunused spectrum bands, the hybrid access scheme helps furtherimprove the spectrum efficiency and system throughput, whileguaranteeing the quality of service (QoS) of PUs.

Several recent studies have considered femtocell networkswith CR capabilities. Meerja et al. in [8] proposed a sens-ing scheme for the overlay cognitive femtocell networks, anddetermined the throughput by using a Markov chain model.Lien et al. in [9] proposed a CR resource management scheme,which guarantees the QoS in terms of delay for femtocell net-works. Attar et al. in [10] studied the interference managementschemes with cognitive macrocell and femtocell base stations,and proposed two game theoretic mechanisms to mitigate theinterference. Urgaonkar et al. in [11] investigated the oppor-tunistic cooperation between femtocell users and macrocellusers, and designed a control algorithm for cooperation underboth the relay model and the interference model. Adhikary et al.in [12] discussed the impact of open and closed access in-terference cancelations in cognitive femtocells, and proposeda downlink interference alignment scheme. Al-Rubaye et al.

1536-1276 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

3260 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 14, NO. 6, JUNE 2015

Fig. 1. Hybrid overlay/underlay cognitive spectrum access system. (a) OFDMA, (b) cognitive overlay, (c) cognitive underlay, and (d) hybrid cognitive overlay/underlay.

in [13] proposed a priority queuing strategy for data packetsdelivery in indoor cognitive femtocell systems. From a gametheoretic point of view, Gharehshiran et al. in [14] considereda collaborative subchannel allocation for cognitive femtocellnetworks using coalitional game theory. In [15], Pantisano et al.used coalition formation game to study the cooperations be-tween macrocell and femtocell in the uplink spectrum sharing.In [16], Guruacharya et al. applied the concept of networkmultiple-input-multiple-output (MIMO) to small cell networksand proposed a scheme based on coalition formation game toperform the cluster-wise joint beamforming.

In the analysis of the cooperation between femtocell and mac-rocell with CR capabilities, femtocell users are usually regardedas unlicensed SUs of the CRNs and access the spectrum with theoverlay scheme [8], [10], [11], [14]. However, such treatmentmay be oversimplifying. In traditional CRNs, SUs are assumedto be unlicensed low priority users. However, in cognitivefemtocell networks, the femtocell users are also regular sub-scribers of the same cellular provider which serves the macrousers, hence they should have a higher priority than the SUs intraditional CRNs. In the closely related work [14], Ghareshiranet al. focused on the behavior of the users with a cognitiveability, but did not address the spectrum sharing issue for PUsand SUs of the cognitive femtocell networks. They formulatedthe subchannel allocation problem as a cooperative game incharacteristic form, but did not consider the interference.

To further improve the overall system performance, besidesdesigning spectrum access schemes for PUs and SUs, it is alsoimportant to encourage the cooperation among SUs, which mayalso generate interference among themselves. Coalitional gametheoretical approaches allow SUs to form coalitions, which canmitigate the interference cooperatively and improve the spec-trum efficiency. In this paper, we consider the femtocell usersoperating under a hybrid CR access scheme, which is moreadaptive to the density change of the femtocell deployment, andcan improve the spectrum efficiency substantially verified byour extensive simulations. Moreover, the femtocell users havea higher priority comparing to those in the previous studies [8],[10], [11], [14]. Specifically, we apply the cooperative gametheory [17], [18] to study the subchannel allocation problemin such a hybrid CR based femtocell network. To achieve thecooperation among SUs, femtocell users can form coalitionsand coordinate their transmissions using the femtocell gateway.

We formulate the problem as a coalition formation game inpartition form [18] with negative externalities (due to theinterference), and propose a modified recursive core algorithmto find an efficient stable solution of the game. The maincontributions of this paper are as follows:

• Hybrid Overlay/Underlay Femtocell and New SolutionConcept: We design a hybrid overlay/underlay CR schemefor femtocell networks. We formulate the subchannel al-location problem as a coalition formation game with neg-ative externalities, to effectively characterize and mitigatethe interference. We analyze the stability and efficiency forthe network game, and characterize a stable and efficientsolution of the coalition formation game lying in therecursive core of the game.

• New Recursive Algorithm: Based on the solution conceptof the recursive core [19], we propose a modified recursivecore (MRC) algorithm to achieve an efficient and stablecoalition formation for the hybrid CR scheme.

• Superior Performance: Simulation results show that ourproposed MRC algorithm achieves a higher aggregatethroughput for femtocells than the overlay only scheme,underlay only scheme, and a hybrid overlay/underlayscheme without MRC. It also outperforms another recentlyproposed coalition formation algorithm [14].

The rest of the paper is organized as follows. In Section II, wepresent the system model. We formulate the coalition formationproblem in Section III, and propose the algorithm based on therecursive core in Section IV. Simulation results are presented inSection V, followed by concluding remarks in Section VI.

II. SYSTEM MODEL

We consider an OFDMA cognitive femtocell network withsome FAPs and femtocell user equipment (FUEs). The fem-tocell network is within the range of a macrocell base station(MBS). The macrocell user equipment (MUEs) are regardedas PUs and the FUEs are regarded as SUs. Different from atraditional CRN, the FUEs have a higher priority than that in acognitive femtocell network. In Fig. 1, we show the four spec-trum sharing schemes for (a) OFDMA, (b) cognitive overlay,(c) cognitive underlay, and (d) hybrid cognitive overlay/underlay, respectively. In Fig. 1(a), all users are allocated or-thogonal subchannels in different time slots using OFDMA and

MA et al.: HYBRID OVERLAY/UNDERLAY COGNITIVE FEMTOCELL NETWORKS: A GAME THEORETIC APPROACH 3261

Fig. 2. Hybrid overlay/underlay cognitive femtocell networks.

no cognitive users are considered. Fig. 1(b)–(d) consider the CRsystems. In Fig. 1(b), SUs detect and use idle spectrum holesto transmit under the cognitive overlay scheme. In Fig. 1(c),SUs can use the same subchannel with PUs without performingany detection under the cognitive underlay scheme, as long asthe signal to interference plus noise ratio (SINR) requirementsof PUs are satisfied. Fig. 1(d) represents our proposed hybridoverlay/underlay spectrum sharing scheme, where we allowthe SUs to adapt the ways of accessing the licensed spectrumaccording to the status of the PUs. If the PU is detected tobe idle at the selected subchannel, then the SU uses the idlesubchannel with the overlay scheme. If no idle subchannel isavailable, the SU uses an occupied subchannel when the SINRrequirements of PUs are satisfied with the underlay scheme.

Users in both the macrocell and femtocells utilize resourcesbelonging to the same service provider. An example of hybridoverlay/underlay cognitive femtocell network system is shownin Fig. 2. In the network, each FAP is connected to the corenetwork (CN) through an Internet protocol (IP) backhaul andfemtocell gateway (FGW) [1]. The FGW serves as the coor-dinator of FAPs for a local area, such as FAPs in the sameapartments building or nearby houses. For a sparse femtocelldeployment, such as femtocells in the detached houses in therural area, the spectrum resources are sufficient for both MUEsand FUEs to occupy completely orthogonal subchannels. Insuch a sparse scenario, the cognitive overlay is preferred tomaximize the performance of each user. On the other hand,femtocell deployment can be highly dense in apartment oroffice buildings. In such a dense scenario, the number ofavailable subchannels may not be sufficient for all users tooccupy different orthogonal subchannels. Multiple users mayhave to share a common subchannel, under the condition thatthe interference level for each MUE is acceptable. Therefore,the hybrid overlay/underlay access method becomes the mostflexible option in a high density deployment case.

Let us consider a set of FAPs F in the coverage of a macro-cell, where an FAP is indexed by i. The set of FUEs associatedwith FAP i is Ki, and the size of Ki is usually between 1 and4 [20]. The operator has a total available spectrum of W Hz.We consider a set of licensed subchannels M (with the size|M| = W

B ), where each subchannel has a bandwidth of B Hzand consists of several consecutive subcarriers.

We assume that the number of subchannels is not sufficientfor all MUEs and FUEs to have separated subchannels. In ashared spectrum OFDMA system, signals from a macrocell andfemtocells are likely to transmit simultaneously in the samesubchannels. Therefore, co-channel interference (CCI) mayoccur. The FAP has the cognitive ability to detect whether a co-channel user (macrocell user or femtocell user) is nearby, and isresponsible of allocating subchannels to its users. When an FAPis about to allocate subchannels, it will first sense the spectrumenvironment to determine the available spectrum bands. Thesensing coordination among multiple FAPs can be implementedby exchanging neighboring femtocell information throughthe FGW. We assume that each FAP is equipped with twotransceivers. One of them is used for spectrum sensing, whilethe other one is used for both intra-femtocell and inter-femtocellcommunications on the selected channels. Thus, FAP can per-form spectrum sensing and data transmission simultaneously.

Once a FAP has sensed the spectrum, it will send the informa-tion regarding the used and unused spectrum list to the databaseat the FGW. FAPs from other femtocells can access these infor-mation from the database, and share the spectrum informationwith neighboring FAPs. We assume that the backhaul has abun-dant bandwidth for the transfer of data and control packets.1

1We consider a typical femtocell scenario with less than four FUEs in anFAP [20], and each FAP only serves subscribed users (i.e., closed access). Theoperator can check backhaul requirement at the installation of an FAP to makesure the backhaul bandwidth is enough for the designed uses [21].


Fig. 3. Interference scenario for downlink communications in femtocellnetworks.

When an FAP detects the return of a MUE to a previouslyavailable subchannel, it will coordinate with other FAPs basedon the proposed cooperative game and adjust the subchannelallocation to its FUEs.

A. Channel Model

We focus on the downlink communications from the FAPsto the FUEs and from the MBS to the MUEs.2 Each FUEwill be allocated one subchannel by its corresponding FAP. Wewill use the co-channel SINR in evaluating the average linkquality and thus the data rate of each user. With concurrenttransmissions of multiple users (can be both FUE and MUE)on the same subchannel, the transmission rate of each user isconstrained by the aggregate interference from other users. Theinterference scenario is shown in Fig. 3, where the solid-linearrow represents the signal of downlink transmission and thedash-line arrow represents the interference signal. Note thatonly FUEs and MUEs on the same subchannel will interfereeach other.

We use P(m)ik to represent the transmission power of FAP i

to FUE k on subchannel m ∈ M, the corresponding downlinkreceived SINR is

SINR(m)ik =

P(m)ik

∣∣∣h(m)ik

∣∣∣N

(m)0 + I

(m)ik

, (1)

where h(m)ik is average channel gain of the link between FAP

i and FUE k in channel m, and incorporates the impact ofboth path loss and large-scale fading. Specifically, we haveh(m)ik = G

(m)ik d−γ

ik , where G(m)ik is the fading gain between FAP

i and FUE k on subchannel m, d−γik is the distance between

FAP i and FUE k, γ is the pathloss exponent, N (m)0 is the

background noise power of subchannel m, and I(m)ik is the

aggregate interference power received by FUE k on subchannelm with FAP i as the source node. For the link between FAP iand FUE k, we assume that FUE k can achieve a transmission

2The analysis can be generalized to other scenarios with minor modifications,such as the changes in the parameters characterizing the positions of theinformation source, interference sources, and the transmission power. Forexample, in the scenario that femtocell and macrocell use separate frequencybands, our hybrid overlay/underlay formulation is still applicable by treatingthe transmitting FAPs as PUs.

rate R(m)ik on subchannel m according to the Shannon capacity

formula, i.e.,

R(m)ik = ηB log2

(1 + SINR(m)

ik

), (2)

where η ∈ [0, 1] is a coefficient describing the efficiency of thetransceiver design. In the hybrid overlay/underlay CR accessscheme, FUEs attempt to find unoccupied subchannels first. Ifno spectrum hole exists, the FUEs will share a subchannel withMUEs subject to interference constraints. We denote b as theMBS. For the hybrid overlay/underlay access scheme in (1),we have

I(m)ik =

∑j∈F\{i}

P(m)jk

∣∣∣h(m)jk

∣∣∣+ P(m)bk

∣∣∣h(m)bk

∣∣∣ , (3)

where the first summation represents the interference fromother FAPs in the same subchannel m, and the second termrepresents the interference from the MBS transmitting in thesame subchannel m.

Using equations (1), (2), and (3), the data rate Rik of FUE kwith FAP i as the source node can be written as

Rik =∑m∈M

R(m)ik . (4)

Note that the transmission rate of FUE k with FAP i asthe source node is the summation of data rates on all usedsubschannels. For a subchannel m that is not allocated to aFUE k by FAP i, the transmission power P

(m)ik and the data

rate R(m)ik are both zero. With the OFDMA approach, an FAP

can communicate with all its FUEs simultaneously. Therefore,we define the data rate of FAP i as the aggregate data rate of allFUEs associated with FAP i, which can be represented as

Ri=∑k∈Ki

Rik=∑k∈Ki

∑m∈M

ηBlog2

⎛⎝1+

P(m)ik

∣∣∣h(m)ik

∣∣∣N

(m)0 +I

(m)ik

⎞⎠. (5)

When OFDMA is used in the femtocell networks, differentFUEs associated with the same FAP will be allocated orthogo-nal subchannels and do not interfere with each other. However,there still exist interference from FUEs associated with otherFAPs as well as MUEs. One way to reduce such interferenceis to encourage mobile devices to form coalitions, such thatdevices in the same coalition can coordinate and reduce inter-ferences, thus improve the performance of the network.

In the next section, we will describe the challenging problemof coalition formation problem, and explore coalition formationgame to analyze such a problem.

III. PROBLEM FORMULATION

In this section, we formulate the subchannel allocation prob-lem in hybrid overlay/underlay cognitive femtocell networksas a coalition formation game. Our goal is to form a stablecoalition structure to maximize the aggregate throughput offemtocell users.


A. Subchannel Allocation Under the HybridOverlay/Underlay Access Scheme

As the spectrum resources available in all cells belong to thesame service provider, our target is to perform a centralizedsubchannel allocation to efficiently utilize the limited licensedspectrum. The key idea is to enable FAPs to cooperate witheach other, to minimize mutual interferences and increase theaggregate throughput. To achieve this, we will formulate thecoalition formation problem, and find the stable and efficientpartition that maximizes the aggregate throughput for all fem-tocell users.

In CRNs, the roles of PUs and SUs are often predetermined.For the ease of exposition, we consider MUEs as PUs of thenetwork, and the subchannel allocation for MUEs is performedprior to the coalition formation of FAPs. However, it should benoted that FUEs have a higher priority than that assumed in [8],[10], [11], [14], since here the femtocell equipment can transmitwith the maximum power allowed in the same subchannels asthe macrocell equipment. We then need to establish a subchan-nel allocation rule for the cooperative FAPs within the samecoalition, so that they can avoid strong mutual interference.Let us denote MS as the set of available subchannels for thecoalition S . For FAP i ∈ S , we denote MFAP

i as the set ofsubchannels allocated to FAP i. When coalitions are formed,to let each FUE have at least one subchannel, we let FAP i,which is in coalition S , allocate one subchannel to each of its|Ki| FUEs. To reduce mutual interference within each coalition,the principle of subchannel allocation is to let each FAP inthe same coalition have as many orthogonal subchannels aspossible. Following the principle, we design the subchannelallocation as the following two steps. In step 1, MFAP

i iscomputed as follows:

MFAPi =

⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩

{(i− 1)|Ki|+ 1 mod|MS|} ,...

{(i− 1)|Ki|+ l mod|MS|} ,...

{(i− 1)|Ki|+ |Ki| mod|MS|}

⎫⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎭

={MFAP

i1 , . . . ,MFAPil , . . . ,MFAP

i|Ki|

}, (6)

where i ∈ S . In step 2, to avoid allocating subchannel {0} toan FAP, we check and replace element {0} with {|MS|} inMFAP

i as follows: For 1 ≤ l ≤ |Ki|, if MFAPi

⋂{0} �= ∅ and

MFAPil = {0}, we set MFAP

il = {|MS|}.As an example, consider a coalition S with size |S| = 3,

the number of FUEs in each coalition member FAPs 1, 2,3: {|K1|,K2|,K3|} = {2, 3, 4}, the available number of sub-channels for the coalition |MS| = 6, the subchannel allocationdecision is MFAP

1 = {{1}, {2}}, MFAP2 = {{4}, {5}, {6}},

MFAP3 = {{3}, {4}, {5}, {6}}. From the example, we note

that different FAPs might have overlapping subchannel alloca-tions, since

∑i∈S |Ki| can be larger than |MS|. This is allowed

in the hybrid overlay/underlay cognitive systems, and does notconflict with the goal that the aggregate throughput of SUs ismaximized while the SINR levels for PUs are guaranteed.

B. Coalition Formation Game in Partition Form

After defining the subchannel allocation rules, in this subsec-tion, we introduce the basic elements of games and formulateour coalition formation game as a game in partition form. Ingeneral, a coalition formation game involves a set of players,who attempt to form coalitions, to improve their performanceor utilities. To specify a coalition formation game, we needto specify a set of players and a value which quantifies thebenefits of the game. The definition of the value determines theform (e.g., characteristic form, partition form) and type [e.g.,transferable utility (TU), non-transferable utility (NTU)] of thegame [18].

Next, we introduce the elements of the coalition formationgame.

Players: The set of FAPs F = {1, . . . , F}.Coalition: The set of players S ⊆ F that coordinate their

transmissions and subchannel allocations form a coalition S .Partition: Let A(Q) be the set of partition or coalition

structure of set Q. For a partition π ∈ A(Q), we have ∪S∈πS =Q and S ∩ S = ∅, ∀S, S ∈ π with S �= S . As an example, forQ = {1, 2, 3}, π ∈ A(Q) can be one of the following fivecoalition structures: {{1},{2},{3}}, {{1},{2,3}}, {{2}, {1, 3}},{{3}, {1, 2}}, and {{1, 2, 3}}.

Strategy: Each player chooses to join a proper coalitionwhich results in the largest value of the game.

From the elements of the game above, we note that FUEsare not players. They accept the subchannel allocations fromtheir corresponding FAPs. However, FUEs can be allocatedboth occupied and unoccupied subchannels. Given a partitionπ and FAP i ∈ S , we can compute the interference for FUE kin FAP i from other equipment transmitting in subchannel m as,

I(m)ik (S, π) =

∑j∈F\{i}

P(m)jk (S, π)

∣∣∣h(m)jk

∣∣∣+ P(m)bk

∣∣∣h(m)bk

∣∣∣ , (7)

where P(m)jk (S, π) is the transmission power of FAP j to FUE

k on subchannel m under partition π. Note that P (m)jk (S, π) is a

function of (S, π), since the subchannel allocation is based onthe coalition and partition as mentioned. The first summationrepresents the aggregate interference from all other FAPs(j ∈ F \ {i}) in the same subchannel m, and the second termrepresents the interference from the MBS transmitting in thesame subchannel m. Equation (7) shows that the interferencereceived by an FUE depends not only on which coalition theFUE has joined, but also on how other FAPs form coalitions.Given the interference on any subchannel of any FUE in FAPi in coalition S under the partition π as shown in (7), theaggregate data rate of the users in FAP i is

Ri(S, π)=∑k∈Ki

∑m∈MFAP

i

ηB log2

⎛⎝1+

P(m)ik (S, π)

∣∣∣h(m)ik

∣∣∣N

(m)0 +I

(m)ik (S, π)

⎞⎠.

(8)

Due to the fact that the players are the subscribed users ofthe wireless service provider, the payoffs of the players are notdetermined solely by the data rate that they can achieve. Thepayoff will be reduced if a player acts selfishly and has negativeimpact on the network, such as generating severe interference to


other users. Meanwhile, the wireless service providers are theoperators of the networks. They can control the payoffs of theusers from two aspects. They can physically provide differentservices to cooperative and non-cooperative users, and finan-cially give rewards and punishment. The users are encouragedto act cooperatively and improve the overall performance ofthe network. With the considerations mentioned above and tobe simple, for the cooperative coalition formation game, wedefine the payoff of player i ∈ S , which is the average data rateallocated to player i given the partition of the network is π, i.e.,

xi(S, π) =∑

i∈F Ri(S, π)|F| . (9)

We note that for any coalition S ⊆ F , the payoff of everyplayer in the coalition depends on the overall partition π of thenetwork, i.e., on the players in S as well as on the players in F \S . This is referred to the coalition formation game in partitionform [18].

C. Modified Recursive Core for Coalition Formation GameWith Negative Externalities

Negative externality is often used to characterize the indirectcost imposed by one user on other users when sharing somecommon resources. In our work, the negative externality corre-sponds to the interferences generated by a user to another user.A coalitional game is defined as superadditive if the value ofa union of any two disjoint coalitions is no less than the sumof the values of individual coalition. In superadditive games,any two coalitions will not be worse off by merging into alarger coalition. Therefore, the grand coalition, which containsall players as a coalition, will form. Due to the externality, theproposed game is not superadditive, hence the grand coalitionis not optimal [18]. As an example, let us consider a 2-playernetwork. If there is only one available channel, the formationof grand coalition indicates that both players prefer to jointhe same coalition and jointly allocate the resources. This maynot be true when two players are very far from each other, asin that case each user can fully utilize the channel with littleor no mutual interferences to each other. Therefore, coalitionformation game in partition form with externalities allows adifferent solution concept than the grand coalition.

To solve the above coalition formation game in partition formwith negative externality, we propose the solution concept ofmodified recursive core. The recursive core [19] is an importantsolution concept for coalition formation games in partition formconsidering externality.3 For any partition π, that

⋃S∈π S = F ,

the value of the game equals the summation of the payoff forall players’ payoffs as follows [15]

v(π) =∑i∈F

xi(S, π) =∑S∈π

∑i∈S

xi(S, π). (10)

Subsequently, we use the recursive core as the solutionconcept for the game (F , v). The recursive core can be regardedas a generalization of the concept of core for games in charac-teristic form, to games in partition form and with externalities.

3Different from the core or Shapley value, recursive core allows modeling ofexternalities for games in partition form.

If the recursive core concept is applied to a coalition formationgame in characteristic form, it degenerates to the concept of thecore [22].

Recursive core is defined recursively using the residual core,which is the core of a residual game. Therefore, before intro-ducing the concept of recursive core, we need to introduce theconcept of a residual game and residual core [19].

Definition 1: Let (F , v) be a coalition formation game. Aresidual game (R, v) is a coalition formation game in partitionform defined on a set of players R, after the partitions forplayers in F \ R have been decided. The players that in F \ Rare called deviators, while the players in R are called residualplayers.

Consider the coalition formation game (F , v), and let Sbe a coalition of deviators. Let R = F \ S denote the set ofresidual players. The residual game (R, v) is defined as a gamein partition form based on the set of players R. As discussedin [19], once the residual game is defined, it can be solved asan independent game in partition form, regardless the fact thatit is a residual game. In fact, a game in partition form can beregarded as a group of residual games, and each one of theresidual games can be solved independently. The solution ofany residual game is defined as the residual core as follows [19].

Definition 2: The residual core of a residual game (R, v) isa set of possible partitions of R that can be formed.

Note that if we have F instead of R in Definition 2, then thecorresponding residual core becomes the recursive core. Givenany coalition formation game (F , v), residual games are gameswith fewer players than the original game and are therefore lesscomplicated. Since a residual core for a residual game of alarger game is also the recursive core for the game of all theresidual players, the criterion that a partition can be formedis the same as defined in recursive core. Given the coalitionformation game (F , v), the outcome as a pair which consiststhe payoff vector and the partition, the recursive core can befound by playing residual games recursively, which is definedas follows [19].

Definition 3: The recursive core C(F , v) of a coalition for-mation game (F , v) is inductively defined as follows:

1) One-player Game. The recursive core of a coalition for-mation game with only player i is the only outcome andpartition C({i}, v) = ((v({i})), {i}).

2) Inductive Assumption. Suppose that the residual coreC(R, v) for all games with at most |F| − 1 play-ers has been defined. Let Ω(R, v) denote a set ofall possible outcomes of game (R, v). The inductiveassumption A(R, v) about the game (R, v) is A(R, v) ={C(R, v), if C(R, v) = ∅,Ω(R, v), otherwise.

3) Dominance. Define x as the payoff vector of the playersand πF as the partition of the user set F . The outcome(x, πF ) is dominated via the coalition S forming parti-tion if for at least one outcome (yF\S , πF\S) ∈ A(F \S, v), there exists an outcome ((yS ,yF\S), πS ∪ πF\S) ∈Ω(F , v) such that (yS ,yF\S) x.

4) Recursive Core. The recursive core of a game of |F|players is the set of undominated outcomes denoted byC(F , v).


The following statement better explains the concept of dom-inance in step 3). Given a current partition πF and the corre-sponding payoff vector x, a deviation of the members of S fromπF to πS ∪ πF\S results an outcome ((yS ,yF\S), πS ∪ πF\S),where (yS ,yF\S) x. This outcome is more rewarding for theplayers of S , and thus (x, πF ) is dominated via the coalition S .

According to the above definition of recursive core, we canobtain the recursive core recursively. We first start with a one-player game. The only outcome is defined as the recursive coreof the one-player game. We then consider a game with moreplayers. Proceeding recursively by considering the inductive as-sumption, we obtain the undominated outcomes as the residualcore of each game with at most |F| − 1 players. We obtain therecursive core C(F , v) for the game (F , v) from the residualcore of the game with |F| − 1 players.

To illustrate the operation of finding the recursive core,we give a three-player game example as follows. In a three-player game ({1, 2, 3}, v) with players 1, 2, and 3, we canfind the recursive core as follows. First, the initial partitionand the recursive core is defined as C({1}, v). Second, wehave A({1}, v) = C({1}, v), since the recursive core of theone player game is C({1}, v). We consider the dominanceand select the best result for the game of two players toobtain the recursive core for the 3-player game. The 2-playergame can be constructed from 1-player game by includ-ing one more player. We define the set of partitions fora 2-player game as {π21, π22}, where π21 = {{1, 2}} andπ22 = {{1}, {2}}. Third, we obtain two outcomes for the2-player game {(xπ21

, {{1, 2}})} and {(xπ22, {{1}, {2}})},

where xπ21and xπ22

are the payoff vectors of the two play-ers. If xπ22

xπ21, then A({1, 2}, v) = {(xπ21

, {1, 2})}. Ifxπ21

xπ22, then A({1, 2}, v) = {(xπ22

, {{1}, {2}})}. Oth-erwise, A({1, 2}, v) = Ω({1, 2}, v) which includes both out-comes for the 2-player game. Now, we are ready to obtain therecursive core of the 3-player game, by using the residual coreof 2-player game and checking the possible dominant outcomesincluding the third player. The third player can either join anycoalition of the residual core for the 2-player game, or stay asa singleton coalition. For example, if we have A({1, 2}, v) ={(xπ21

, {1, 2})}, the recursive core C({1, 2, 3}, v) can include{(xπ31

, {{1, 2, 3}})}, {(xπ32, {{1, 2}, {3}})}, or both, based

on the dominant relationship of the vectors xπ31and xπ32

,where π31 = {{1, 2, 3}} and π32 = {{1, 2}, {3}}. Note thatbased on the different residual cores for the residual games, anypossible outcome of the original game can be in the recursivecore. However, with the process of searching residual coreand using residual core to reach the final recursive core, wesolve the game with a lower complexity comparing with theexhaustive search method. In addition, since every partitionsolely determines the payoffs for all the players, the recursivecore can be regarded as a set of partitions that allows theplayers to organize in a way that achieve the Pareto optimalpayoff vectors.

The idea of the original recursive core is to find a set ofundominated outcomes of the (F , v). However, as the objectiveis to improve the aggregate performance for the network insteadof each individual player, for the game of our network, we onlyneed one undominated outcome which results in the largest sum

of the payoffs of all players. This reduces the computationalcomplexity significantly. To do so, instead of searching for theresidual core (i.e., a set of undominated outcomes for residualgames), we search for one undominated outcome that resultsin the largest value of of each residual game and keep theoutcome as the modified residual core of each residual game.By recursively finding the modified residual core of a largerresidual game, we obtain the modified recursive core of theoriginal game in the end.

Before introducing the specific process of obtaining modifiedrecursive core, we need to define a preference relation for eachFAP. The definition is as follows.

Definition 4: For any FAP j ∈ F , we define a preference re-lation �j as a complete, reflexive and transitive binary relationover the set of all coalitions that FAP j can possibly form.

Note that according to the definition of binary relation, abinary relation Z on a set T is: (a) complete if ∀t, t′ ∈ T :tZt′ or t′Zt; (b) reflexive if ∀t ∈ T : tZt; (b) transitive if∀t, t′, t′′ ∈ T : [tZt′ and t′Zt′′] ⇒ tZt′′. Since we let the FAPsautonomously form the coalitions, the above definition is usedto compare the preference of FAP j over different coalitionstructures (i.e., partitions). For FAP j ∈ F , given two partitionsπ1 ⊆ A(F) and π2 ⊆ A(F), π1 �j π2 indicates that FAP jprefers joining a coalition to form partition π1 over joininganother coalition to form partition π2, or at least, FAP j has nopreference. Moreover, j is defined as a strict transitive binaryrelation, and π1 j π2 indicates that FAP j strictly prefers toform partition π1 than partition π2. We define the operation todecide the preference as follows:

π1 j π2 ⇔ v(π1) > v(π2), (11)

where π1 ⊆ A(F) and π2 ⊆ A(F) are any two partitions thatcontain FAP j, and the vj is as in (10).

According to the definitions of the preference relation in(11), FAP j ∈ F prefers joining a new coalition to form a newpartition that FAP j has never been a member of, if and onlyif such a decision results in a larger value of the game v. Itis important to note that the preference is also rational that theplayers prefer the strategy which maximizes their payoffs, sincethe payoffs of players are proportional to the value of the gameaccording to (9) and (10).

Besides a larger value of the game for the new partition, wealso need the SINR for all MUEs to be greater than a threshold.The SINR of MUE n on subchannel m can be computed asfollows:

SINR(m)n =

P(m)bn

∣∣∣h(m)bn

∣∣∣N

(m)0 +

∑i∈F P

(m)in (S, π)

∣∣∣h(m)in

∣∣∣ , (12)

where P (m)bn is the transmission power of the MBS on subchan-

nel m, |h(m)bn | is average channel gain of the link between the

MBS and MUE n, P (m)in (S, π) is the transmission power of

FAP i on subchannel m in coalition S under partition π, |h(m)in |

is average channel gain of the link between FAP i and MUE n.We define the switch rule for coalition formation consideringthe SINR constraint of MUEs as follows.


Definition 5: (Switch Rule) Given a partition π, and coali-tions Si,Sl ∈ π, FAP j ∈ Si is willing to leave its currentcoalition Si and join another coalition Sl with i �= l, if and onlyif Sl

⋃{j} j Si and SINR(m)

n > θ, n ∈ N ,m ∈ M, where θis the threshold according to the SINR requirements of MUEs.

The switch rule indicates that an FAP is willing to leave itscurrent coalition and join another coalition, if and only if thenew coalition is strictly preferred over the current coalition.According to the switch rule, all the FAPs can make decisionsautonomously to form coalitions in the system.

In our game, we define the modified recursive core (MRC)by modifying the third and fourth steps in the original recursivecore in Definition 3. In the third step (i.e., dominance), insteadof defining the undominated outcome according to the payoffvectors of the players, we use the value of game in (10) inthe MRC. Specifically, we let each player switch coalitionsaccording to the switch rule and autonomously join the coalitionthat results in larger value of the game v, until the coalitionstructure converges to the one that results in the largest value ofthe game (and maximum payoff), and no player wishes to leavethe current coalition. The outcome of MRC, which results in thelargest value of the game, is one of the undominated outcomesin the original recursive core definition. In the fourth step (i.e.,core generation), the MRC is defined as the outcome with theoptimal value of the game rather than the set of undominatedoutcomes. Consequently, the achieved MRC lies in the originalrecursive core.

D. Stability Analysis of the Modified Recursive Core

To analyze the stability of partition achieved by our method,we next define the stability of coalition structure [23], [24].As in [23], Apt et al. proposed the concept of the defectionfunction D, which is a function that associates with everynetwork partition. By defining Dp as a defection functionthat allows formation of all partitions, one can decide whichcoalition to join given the partition of π.4

Definition 6: A partition π = {S1,S2, . . . ,SP } is Dp-stableif for all partitions π′ �= π, v(π) ≥ v(π′).

In other words, a coalition partition π is Dp-stable if noplayer has an incentive to move from its current coalition toanother coalition. Therefore, no player can result in a largervalue of the game by acting according to the switch rule whenthe current partition is Dp-stable.

Theorem 1: Using the modified recursive core, starting fromany initial partition, all the FUEs will always converge to a finalpartition π∗, which is Dp-stable.

Proof: Suppose the partition π∗ is not Dp-stable. In otherwords, we can find a player j ∈ F that can act according to theswitch rule to achieve a larger value of game. However, due tothe finite number of partitions of a set, the assumption contra-dicts to the second step of finding the modified recursive corewhich no player can leave partition π∗ to obtain a larger valueof game. Therefore, the partition π∗ is a Dp-stable partition. �

4The p represents the defection function is about the formation of allpartitions, which is the function we need in our definition.

IV. ALGORITHM DESIGN BASED ON

THE MODIFIED RECURSIVE CORE

In this section, we propose a modified recursive core (MRC)algorithm as shown in Algorithm 1 to solve the subchannelallocation problem for the hybrid overlay/underlay cognitivefemtocell network using coalition formation game describedin Section IV. It allows each FAP to decide which coalitionit should join to maximize the value of the game (i.e., theaggregate throughput of all SUs in the network).

Algorithm 1 Modified Recursive Core (MRC) Algorithmfor Hybrid Overlay/Underlay Cognitive Femtocell CoalitionFormation Game

1: Initialize Pi, ∀i ∈ F , hi,j , ∀j, i ∈ F , j �= i, N0, θ2: Set π(0)∗ := {{1}, . . . , {|F|}}3: Set {{MMUE

1 }, . . . , {MMUE|N | }} := {{1}, . . . , {|N |}}

4: Set {{M(0)FAP1 }, . . . , {M(0)FAP

|F| }} :=

{{{1}, . . . , {1}}, . . . , {{1}, . . . , {1}}}5: Set k := 1, V := {V1, . . . , V|F|} := {1, . . . , |F|}6: while V1 �= V|F| do7: for i ∈ F do8: for S ∈ {π(k−1)∗ \ {i}} ∪ ∅ do9: Set π(k) := {π(k−1)∗ \ S,S ∪ {i}}10: Update M(k)FAP

i according to (6)11: Compute SINR(m)

n , ∀m∈MMUEn , n∈N using

(12)12: If SINR(m)

n > θ, ∀m ∈ MMUEn , n ∈ N

13: Determine v(π(k)) using (10)14: end if15: If v(π(k)) > v(π(k−1))16: Set π(k)∗ := π(k)

17: Set M(k)FAP ∗i := M(k)FAP

i

18: Set v(π(k))∗:= v(π(k))

19: end if20: If v(π(k)) ≤ v(π(k−1))21: Set π(k)∗ := π(k−1)

22: Set M(k)FAP ∗i := M(k−1)FAP

i

23: Set v(π(k))∗:= v(π(k−1))

24: end if25: end for26: Set Vi := v(π(k))

∗

27: end for28: Set k := k + 129: end30: Set π∗ := π(k−1)∗, MFAP ∗

i := M(k−1)FAP ∗i , ∀i ∈ F

31: Output: (Modified Recursive Core for the game)Output the stable core of game (F , v) consisting of both thefinal partition π∗ and subchannel allocation decision MFAP ∗

i .

We start with the modified recursive core searching process,by considering the initial partition of the game with all FAPsas singleton coalition (line 2). Let MMUE

n be the set of sub-channels allocated to MUE n. The initial subchannel allocationallocates each MUE an unoccupied suchannel (line 3), andallocates all FUEs the same subchannel initially (line 4). We


use vector V to record the value of the game for comparison ofdifferent iterations, and initialize the value of V (line 5). Next,we let the players autonomously join potential coalition whichresults in a larger value of the game one by one in each iteration.In the iteration k, different possible coalition structures π(k)

are considered by each FAP. Player i either joins a coalitionS ∈ π(k−1)∗ formed in the previous iteration, or forms a newcoalition (when S = ∅) in lines 8 and 9. As an example, inthe first iteration where k = 1, player 1 can either stay as asingleton coalition {1} (i.e., the partition π(1) = π(0)) or joinone of the coalitions {{2}}, . . . , {{|F|}}. Corresponding toeach different partition, the subchannels MFAP

i are allocatedaccording to (6) in line 10. After verifying that the SINR levelfor all MUEs satisfies the threshold θ (line 12), we evaluatethe value of the game using (10) (line 13). Next, we recordthe partition π(k) that results in the largest value of the gamev(π(k)) so far (line 16 and 21). We also keep a record of the bestvalue of the game to verify the convergence (line 26). Underthe condition that the value of game remains the same for thefirst and |F|-th iteration (line 6), all the |F| players are satisfiedwith the formed coalitions. Therefore we obtain the MRC andthe corresponding subchannel allocation.

We now discuss the computational complexity and otherpractical implementation details of our algorithm. Regardingthe computational complexity of solving cooperative games,computing the core is NP-complete in general [25]. Computingthe Shapley value is sharp-P-complete, which means that it isas hard as any counting problem in NP [26]. The computationalcost in finding the modified core [14] using exhaustive searchis O(M2KN !), where M , K and N are the number of basestations, number of active users, and number of subchannelsrespectively. In our algorithm, the computational cost for eachFAP to find the optimal coalition is O(|F|2), where |F| is thenumber of the FAPs. Regarding the frequency of updating thecoalition structure, since the FAPs are stationary once installed,it is not necessary to update the coalition structure frequently.Regarding the information required by the algorithm, the ap-proximated average channel gain can be obtained according tothe locations of users, which can be calculated by triangulationmethods with network listening and announced to all FAPs bythe service provider. The transmission power levels for MBSand FAP are fixed and known by all devises. The noise powerin the coverage range of an FAP can be measured locallyat each FAP.

V. PERFORMANCE EVALUATION

In this section, we evaluate the performance of the proposedMRC algorithm through numerical simulations. We assume that|F| FAPs are randomly placed in a square region of l × l m2.The coverage range of each FAP is a disk with radius r.We assume that the FUEs can sense the number of availablesubchannels. Each FAP is responsible for the downlink com-munications of all its FUEs concurrently. We will vary thenumber of FAPs, MUEs, and subchannels in the network toevaluate the performance of the proposed algorithm. Unlessstated otherwise, we adopt the simulation parameters from atypical femtocell setting as summarized in Table I.

TABLE ISIMULATION PARAMETERS

Fig. 4. Snapshot of coalition formation in a cognitive femtocell network for|F| = 12, |N | = 8.

Using MATLAB as our simulation tool, we repeat the ex-periment 1000 times using Monte Carlo simulations for eachnetwork setting, and calculate the average value of the perfor-mance metrics over different network topologies.

We first present a sample coalition formation for a cognitivefemtocell network deployed within a 200× 200 m2 squareregion with twelve FAPs and eight MUEs, and each FAP hastwo FUEs. Fig. 4 shows the coalition formation result. The bluediamonds and red circles represent the FAPs and FUEs in thefemtocell network, and the green squares represent the MUEs,respectively. FAPs 1 and 4 form a coalition, FAPs 2, 7 and 11form a coalition, FAPs 6, 10 and 12 form a coalition, FAPs 3,5, 8, and 9 form singleton coalitions, respectively.

To evaluate the performance of the proposed hybrid spectrumaccess scheme, we implement the MRC algorithm for thehybrid spectrum access scheme. We evaluate the aggregatethroughput of all femtocell users under different MUE densi-ties [27], and different FUE densities by varying the numberof MUEs |N | and FAPs |F| in the system. For the overlayspectrum access scheme, the subchannels are only allocatedwhen there are vacant subchannels in the system. Otherwise, theFUEs will keep waiting until a vacant subchannel appears. Forthe underlay spectrum access scheme, each FUE is randomly al-located one subchannel under the condition that the SINR of theMUE in that subchannel is greater than the threshold θ. For thehybrid overlay/underlay access scheme without MRC, we first


Fig. 5. Aggregate throughput of FAPs versus number of FAPs |F| when|N | = 5, |M| = 9.

allocate vacant subchannels to the FUEs. If all the vacant sub-channels are occupied, we then randomly allocate the occupiedsubchannels to FUEs, while satisfying the SINR requirementsof the MUEs in those subchannels. For our proposed hybridspectrum access scheme with MRC, when there are vacant sub-channels, each FUE is allocated a vacant subchannel. When novacant subchannel exists, the FAPs form coalitions and allocatesubchannels with coordination according to Algorithm 1.

In Fig. 5, we plot the aggregate throughput of FAPs versusthe number of FAPs in the network. As shown in Fig. 5,the aggregate throughput obtained by our hybrid access MRCalgorithm outperforms the hybrid scheme without MRC, over-lay, and underlay spectrum access schemes. The aggregatethroughput of overlay scheme remains flat, since the numberof unoccupied subchannels for overlay FAPs remains fixedregardless of the number of FAPs. However, the hybrid MRCalgorithm continues to improve the aggregate throughput andoutperforms the hybrid scheme without MRC and the underlayscheme. This is because the MRC increases the aggregatethroughput by allocating subchannels in an optimal mannerthrough coalition formation, whereas the hybrid scheme with-out MRC and underlay scheme cannot successfully mitigate theinterference among femtocells, especially when the number ofFAPs is large. When there are 26 FAPs in the network, the MRCscheme outperforms the hybrid without MRC, underlay, andoverlay schemes by 15%, 16%, and 72%, respectively.

In Fig. 6, we show the aggregate throughput of FAPs ver-sus the total number of available subchannels, while fixingthe number of FAPs in the network at ten. For the overlayscheme, the aggregate throughput remains zero when all sub-channels are occupied by MUEs, and increases linearly withrespect to the number of available subchannels. For the hybridscheme without MRC, when there are no vacant subchannels, itachieves almost the same performance as the underlay scheme.It outperforms the underlay scheme when the number of vacantsubchannel increases. This is due to the higher chance for thehybrid schemes to allocate a vacant subchannel to an FUE.However, as the hybrid scheme without MRC cannot mitigate

Fig. 6. Aggregate throughput of FAPs versus number of subchannels |M|when |N | = 3, |F| = 10.

Fig. 7. Individual throughput of FUE versus the total number of switcheswhen |N | = 2, |M| = 4.

the interference among femtocells, we can significantly im-prove the performance by allowing FAPs to form coalitionsaccording to the MRC algorithm. When ten subchannels areavailable, the hybrid scheme with MRC outperforms the hybridscheme without MRC, underlay, and overlay schemes by 19%,35% and 62%, respectively.

In Fig. 7, we show the throughput of an individual user versusthe total number of switches (i.e., the number of switch rule ex-ecuted) under a particular network topology. With two MUEs,four FUEs, and four available subchannels, the throughput ofeach user converges after six switches.

In Fig. 8, we show the average number of iterations versusthe number of FAPs, in a network of six subchannels and fouractive MUEs. The average number of iteration increases as thenumber of FAPs increases. With twenty FAPs in the network,the average number of iterations is less than 5.5. This indicatesthat, on average, each player switches less than 5.5 times to


Fig. 8. Average number of iterations to reach the optimal stable partitionversus the number of FAPs |F| when |N | = 4, |M| = 6.

Fig. 9. Average coalition size of MRC algorithm versus the number of FAPs|F| when |M| = 6.

join the optimal coalition and the network achieves the stablepartition.

In Fig. 9, we plot the average coalition size versus the numberof FAPs, when six subchannels are available and two to fouractive MUEs exist. The average size of each coalition increasesas the number of FAPs increases. This is because more FAPsresult in a smaller distance between FAPs on average anda higher possibility of subchannel reuse. The size of eachcoalition decreases when the number of MUE decreases. Itindicates that when more resources become available, FAPsintend to be less cooperative.

In Fig. 10, we compare the performance of our coalitionformation algorithm with the distributed coalition formationalgorithm (DCFA) in [14]. We use the same network configu-ration to compare the performance of coalition formation algo-rithm in a fair manner. We plot the aggregate throughput versusthe number of FAPs in the network for both MRC algorithm

Fig. 10. Aggregate throughput of MRC algorithm and DCFA algorithm [14]versus the number of FAPs |F|.

and DCFA algorithms. Under different numbers of subchannelsand different FAP densities, our MRC algorithm outperformsthe DCFA algorithm substantially. With DCFA algorithm, onlywhen the interference is negligible, the achieved modified coreand the corresponding coalition structure will result in anefficient and non-blocking payoff vector of players. When theinterference is not negligible, such as in the densely populatedhybrid overlay/underlay systems in this paper, the coalitionstructure and subchannel allocation obtained by DCFA maynot be the optimal result. The impact of the interference cangreatly degrade the performance using DCFA. Our MRC al-gorithm takes the negative externality due to interference intoconsideration when forming coalitions, and results in bettersolutions comparing to DCFA algorithm. With 23 FAPs, twoMUEs and six subchannels in the network, our MRC algorithmoutperforms the DCFA algorithm by 18%.

VI. CONCLUSION

In this paper, we investigated the hybrid overlay/underlayspectrum sharing mechanism for cognitive femtocell networks.Specifically, we studied the subchannel allocation problem forthe hybrid overlay/underlay cognitive femtocell networks. Weformulated the problem as a coalition formation game with neg-ative externalities, which models the interactions and coopera-tions among the communication links between the FAPs and theFUEs. We proposed the MRC algorithm based on the recursivecore in coalitional game theory. Simulation results showed thata substantial improvement of the aggregate network throughputis achieved over the overlay scheme, the underlay scheme, ahybrid overlay/underlay scheme without MRC, and a recentlyproposed coalition formation algorithm. In the future work, wewill study a generalized system with multiple MBSs. We willconsider the case where MUEs and FUEs in cognitive femtocellnetworks have exact the same priority. We will also investi-gate the local search method and a quasi-distributed algorithmfor the FAPs.


REFERENCES

[1] V. Chandrasekhar, J. G. Andrews, and A. Gatherer, “Femtocell networks:A survey,” IEEE Commun. Mag., vol. 46, no. 9, pp. 59–67, Sep. 2008.

[2] V. Chandrasekhar and J. G. Andrews, “Uplink capacity and interfer-ence avoidance for two-tier femtocell networks,” IEEE Trans. WirelessCommun., vol. 8, no. 7, pp. 3498–3509, Jul. 2009.

[3] D. López-Pérez, A. Valcarce, G. De La Roche, and J. Zhang, “OFDMAfemtocells: A roadmap on interference avoidance,” IEEE Commun. Mag.,vol. 47, no. 9, pp. 41–48, Sep. 2009.

[4] Y. Sun, R. Jover, and X. Wang, “Uplink interference mitigation forOFDMA femtocell networks,” IEEE Trans. Wireless Commun., vol. 11,no. 2, pp. 614–625, Feb. 2012.

[5] S. Haykin, “Cognitive radio: Brain-empowered wireless communica-tions,” IEEE J. Sel. Areas Commun., vol. 23, no. 2, pp. 201–220,Feb. 2005.

[6] Q. Zhao and B. Sadler, “A survey of dynamic spectrum access,” IEEESignal Process. Mag., vol. 24, no. 3, pp. 79–89, May 2007.

[7] J. Zou, H. Xiong, D. Wang, and C. Chen, “Optimal power allocation forhybrid overlay/underlay spectrum sharing in multiband cognitive radionetworks,” IEEE Trans. Veh. Technol., vol. 62, no. 4, pp. 1827–1837,May 2013.

[8] K. A. Meerja, P.-H. Ho, and B. Wu, “A novel approach for co-channel interference mitigation in femtocell networks,” in Proc. IEEEGLOBECOM, Houston, TX, USA, Dec. 2011, pp. 1–6.

[9] S. Lien, C. Tseng, K. Chen, and C. Su, “Cognitive radio resource man-agement for QoS guarantees in autonomous femtocell networks,” in Proc.IEEE ICC, Cape Town, South Africa, May 2010, pp. 1–6.

[10] A. Attar, V. Krishnamurthy, and O. N. Gharehshiran, “Interference man-agement using cognitive base-stations for UMTS LTE,” IEEE Commun.Mag., vol. 49, no. 8, pp. 152–159, Aug. 2011.

[11] R. Urgaonkar and M. J. Neely, “Opportunistic cooperation in cognitivefemtocell networks,” IEEE J. Sel. Areas Commun., vol. 30, no. 3, pp. 607–616, Apr. 2012.

[12] A. Adhikary, V. Ntranos, and G. Caire, “Cognitive femtocells: Breakingthe spatial reuse barrier of cellular systems,” in Proc. IEEE ITA Workshop,San Diego, CA, USA, Feb. 2011, pp. 1–10.

[13] S. Al-Rubaye, A. Al-Dulaimi, and J. Cosmas, “Cognitive femtocell,”IEEE Veh. Technol. Mag., vol. 6, no. 1, pp. 44–51, Mar. 2011.

[14] O. N. Gharehshiran, A. Attar, and V. Krishnamurthy, “Collaborative sub-channel allocation in cognitive LTE femto-cells: A cooperative game-theoretic approach,” IEEE Trans. Commun., vol. 61, no. 1, pp. 325–334,Jan. 2013.

[15] F. Pantisano, M. Bennis, W. Saad, and M. Debbah, “Spectrum leasing asan incentive towards uplink macrocell and femtocell cooperation,” IEEEJ. Sel. Areas Commun., vol. 30, no. 3, pp. 617–630, Apr. 2012.

[16] S. Guruacharya, D. Niyato, M. Bennis, and D. I. Kim, “Dynamic coali-tion formation for network MIMO in small cell networks,” IEEE Trans.Wireless Commun., vol. 12, no. 10, pp. 5360–5372, Oct. 2013.

[17] M. J. Osborne and A. Rubinstein, A Course in Game Theory.Cambridge, MA, USA: MIT Press, 1994.

[18] W. Saad, Z. Han, M. Debbah, A. Hjorungnes, and T. Basar, “Coali-tional game theory for communication networks: A tutorial,” IEEE SignalProcess. Mag., vol. 26, no. 5, pp. 77–97, Sep. 2009.

[19] L. Á. Kóczy, “A recursive core for partition function form games,” TheoryDecision, vol. 63, no. 1, pp. 41–51, Aug. 2007.

[20] J. Andrews, H. Claussen, M. Dohler, S. Rangan, and M. Reed, “Femto-cells: Past, present, and future,” IEEE J. Sel. Areas Commun., vol. 30,no. 3, pp. 497–508, Apr. 2012.

[21] P. Xia, V. Chandrasekhar, and J. G. Andrews, “Open vs. closed accessfemtocells in the uplink,” IEEE Trans. Wireless Commun., vol. 9, no. 12,pp. 3798–3809, Dec. 2010.

[22] D. Ray, A Game-Theoretic Perspective on Coalition Formation.New York, NY, USA: Oxford Univ. Press, 2008.

[23] K. R. Apt and A. Witzel, “A generic approach to coalition formation,” Int.Game Theory Rev., vol. 11, no. 3, pp. 347–367, Sep. 2009.

[24] W. Saad, Z. Han, T. Basar, M. Debbah, and A. Hjorungnes, “A coalitionformation game in partition form for peer-to-peer file sharing networks,”in Proc. IEEE GLOBECOM, Miami, FL, USA, Dec. 2010, pp. 1–5.

[25] V. Conitzer and T. Sandholm, “Complexity of constructing solutions in thecore based on synergies among coalitions,” Artif. Intell., vol. 170, no. 6/7,pp. 607–619, May 2006.

[26] X. Deng and C. H. Papadimitriou, “On the complexity of coopera-tive solution concepts,” Math. Oper. Res., vol. 19, no. 2, pp. 257–266,May 1994.

[27] D. Ngo, L. Le, and T. Le-Ngoc, “Distributed Pareto-optimal power controlfor utility maximization in femtocell networks,” IEEE Trans. WirelessCommun., vol. 11, no. 10, pp. 3434–3446, Oct. 2012.

Bojiang Ma (S’13) received the B.Eng. degreefrom Southeast University, Nanjing, JiangSu, China,in 2009, and M.A.Sc. degree from University ofVictoria, Victoria, BC, Canada, in 2011. He is cur-rently working towards the Ph.D. degree at the Uni-versity of British Columbia (UBC), Vancouver, BC,Canada. His research interests include interferencemanagement and resource allocation for heteroge-neous wireless networks using optimization theoryand game theory, with current focus on small cellnetworks, home wireless networks, and mm-wave

communication systems.

Man Hon Cheung received the B.Eng. and M.Phil.degrees in information engineering from the ChineseUniversity of Hong Kong (CUHK), in 2005 and 2007,respectively, and the Ph.D. degree in electrical andcomputer engineering from the University of BritishColumbia (UBC), in 2012. Currently, he is a Post-doctoral Fellow in the Department of InformationEngineering, CUHK. He received the IEEE StudentTravel Grant for attending IEEE ICC 2009. He wasawarded the Graduate Student International ResearchMobility Award by UBC and the Global Scholarship

Programme for Research Excellence by CUHK. He serves as a Technical Pro-gram Committee member for IEEE ICC, Globecom, and WCNC. His researchinterests include the design and analysis of wireless network protocols usingoptimization theory, game theory, and dynamic programming, with current fo-cus on mobile data offloading, mobile crowd sensing, and network economics.

Vincent W. S. Wong (SM’07) received the B.Sc.degree from the University of Manitoba, Winnipeg,MB, Canada, in 1994, the M.A.Sc. degree from theUniversity of Waterloo, Waterloo, ON, Canada, in1996, and the Ph.D. degree from the University ofBritish Columbia (UBC), Vancouver, BC, Canada,in 2000. From 2000 to 2001, he worked as a Sys-tems Engineer at PMC-Sierra Inc. He joined theDepartment of Electrical and Computer Engineeringat UBC in 2002 and is currently a Professor. Hisresearch areas include protocol design, optimization,

and resource management of communication networks, with applications towireless networks, smart grid, and the Internet. He is an Associate Editor of theIEEE TRANSACTIONS ON COMMUNICATIONS. He has served on the editorialboards of IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY and Journalof Communications and Networks. He has served as a Technical ProgramCo-chair of IEEE SmartGridComm’14, as well as a Symposium Co-chairof IEEE SmartGridComm’13 and IEEE Globecom’13. He is the Chair of theIEEE Communications Society Emerging Technical Sub-Committee on SmartGrid Communications and IEEE Vancouver Joint Communications Chapter.

Jianwei Huang (S’01–M’06–SM’11) received thePh.D. degree from Northwestern University in 2005.He is an Associate Professor and Director of the Net-work Communications and Economics Lab (ncel.ie.cuhk.edu.hk), the Department of Information Engi-neering, Chinese University of Hong Kong. He isthe co-recipient of 7 Best Paper Awards, includingIEEE Marconi Prize Paper Award in Wireless Com-munications in 2011. He has co-authored four books:Wireless Network Pricing, Monotonic Optimizationin Communication and Networking Systems, Cogni-

tive Mobile Virtual Network Operator Games, Social Cognitive Radio Net-works, as well as four “ESI Highly Cited Papers.” He has served as an Editorof IEEE TRANSACTIONS ON COGNITIVE COMMUNICATIONS AND NET-WORKING, IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS—COGNITIVE RADIO SERIES, and IEEE TRANSACTIONS ON WIRELESSCOMMUNICATIONS. He has served as a Guest Editor of IEEE TRANSACTIONSON SMART GRID, IEEE JOURNAL ON SELECTED AREAS IN COMMUNICA-TIONS, IEEE Communications Magazine, and IEEE Network. He has servedas Associate Editor-in-Chief of IEEE Communications Society TechnologyNews, Chair of IEEE Communications Society Multimedia CommunicationsTechnical Committee, and Vice Chair of IEEE Communications Society Cog-nitive Network Technical Committee. He is a Senior Member and a Dis-tinguished Lecturer of IEEE Communications Society.

Documents

Hybrid Overlay/Underlay Cognitive Femtocell Networks…ncel.ie.cuhk.edu.hk/sites/default/files/07042288.pdf · IEEETRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL.14,NO.6,JUNE 2015 3259