16
1 P2P Content Distribution to Mobile Bluetooth Users Uichin Lee, Member, IEEE, Sewook Jung, Dae-Ki Cho, Alexander Chang, Junho Choi, and Mario Gerla, Fellow, IEEE Abstract—Using handheld devices such as cellular phones and smart phones for personal entertainment has become com- monplace in today’s lifestyle. Virtually all of these devices are equipped with Bluetooth technology that can be used to distribute entertainment contents such as music and movie clips. Mobile users can download content from the opportunistically available infrastructure (e.g., Digital Billboards) as well as direct Peer-to- Peer (P2P) collaboration which significantly increases content availability/coverage. P2P content distribution protocol design is heavily influenced by the characteristics of Bluetooth, which is the main departure from Internet-based content distribution. However, little has been done to understand the performance of overall Bluetooth operations, ranging from peer discovery to data downloading, in dynamic environments with mobility, interference, and different Bluetooth versions/chipsets. In this paper, we first perform extensive experiments to measure the performance of Bluetooth in dynamic environments. We find that Bluetooth-based content distribution faces several challenges such as time/energy consuming resource discovery and limited bandwidth even with the enhanced features of the latest Bluetooth version. We then propose strategies that can effectively improve the performance of resource discovery and downloading phases. Our simulation and experimental results document the improve- ments obtained with our proposed techniques. Index Terms—Content distribution, Bluetooth, Measurement I. I NTRODUCTION Small, portable devices equipped with one or more wireless technologies (e.g., WiFi, Bluetooth, ZigBee, etc.) are becom- ing increasingly popular, ranging from smart phones (e.g., iPhone, Blackberry) to digital music players (e.g., Microsoft Zune, iPod touch). Beyond the means of wireless synchro- nization as cable replacement, such radio technologies pave the way of novel applications to mobile users, thus delivering new opportunities to all facets of the industry. In particular, this paper deals with one fast growing application, namely, proximity advertising and marketing using Bluetooth-enabled digital billboards [1]. Handset maker Nokia and music label Manuscript received April 19, 2005; revised January 11, 2007. This research is supported through participation in the International Technology Alliance sponsored by the U.S. Army Research Laboratory and the U.K. Ministry of Defense under Agreement Number W911NF-06-3-0001, and; by ARMY MURI under funding W911NF0510246. Copyright (c) 2009 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to [email protected]. U. Lee is with Bell Labs, Alcatel-Lucent, 791 Holmdel-Keyport Road, Holmdel, NJ 07733 USA (e-mail: [email protected]). S. Jung is with Broadcom Cooperation, 16340 West Bernardo Dr San Diego, CA 92127 USA (e-mail: [email protected]). A. Chang is with Yahoo!, Inc., 3333 Empire Ave, Burbank, CA 91504 USA (e-mail: [email protected]). J. Choi is with Heavy Iron Studios, 6601 Center Drive West, Suite 400 Los Angeles, CA 90045 USA (e-mail: [email protected]). D.-K. Cho. and M. Gerla are with the Department of Computer Science, University of California at Los Angeles, Los Angeles, CA 90095 USA (e- mail: [email protected]; [email protected]). EMI provide a service that lets coffee shop customers listen to music via Bluetooth [2]. Similarly, WideRay has placed Bluetooth-enabled kiosks in selected music stores, video game stores, and theaters across the country to send messages asking if users are interested in getting more information about various items the store is selling, e.g., music, ring tones, video etc. More interestingly, CBS makes clips from its prime-time program line-up available for free via CBS Outdoor billboards located in NYC’s Grand Central Station to allow smart phone or PDA users to watch clips [3]. So far, mobile users have been tapping this abundance of data directly from the opportunistically available digital billboards. Given the fact that such billboards are located at popular places (e.g., subway stations, department stores, tour sites, and stadiums), and more users simultaneously download from the network while on the move, it is natural to expect that several of them may be interested in downloading the same content. Thus, it makes sense to explore the extension of opportunistic networking to include direct Peer-to-Peer (P2P) exchanges. There are obvious advantages in this P2P and infrastructure downloading synergy. P2P technology enables us to overcome the short contact duration with a BT-AP due to the short communication range (10m) and mobility, obviating the needs of installing BT-APs every 10m. Thus, our goal is to devise an efficient P2P content distribution protocol in this mobile environment with opportunistic infrastructure supports. A good model is offered by Internet-based P2P indexing (re- source discovery) and content distribution protocols. A logical overlay network (e.g., Chord [4], Gnutella [5]) is built on top of the Internet to provide efficient ways of resource discovery, even with the dynamics of user participation (called churning). In addition, BitTorrent-like P2P file swarming is commonly used for content distribution where content is divided into a number of small pieces and users cooperatively share whatever pieces they have [6]. Then the question is whether we can adapt these techniques to mobile environments. The overlay approach may not be feasible in mobile environments because we cannot assume Internet-like connectivity [7], [8]. While proposals for DHT indexes in mobile networks exist (e.g., VRR [9], MADPastry [10]), it is neither practical to maintain an overlay network to distant mobile users nor feasible to guarantee the cooperativeness of the intermediate users if they are not interested in the content [11]. The data access patterns become opportunistic since data can be exchanged whenever there is a user of the same interests or the infrastructure in the neighborhood. BitTorrent-like P2P file swarming should be used to exploit the opportunistic connectivity efficiently. Thus, our focus is on opportunistic P2P file swarming in mobile environments. P2P protocol design in mobile environments is dependent on

P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

1

P2P Content Distribution to Mobile Bluetooth UsersUichin Lee,Member, IEEE,Sewook Jung, Dae-Ki Cho, Alexander Chang, Junho Choi, and

Mario Gerla,Fellow, IEEE

Abstract—Using handheld devices such as cellular phonesand smart phones for personal entertainment has become com-monplace in today’s lifestyle. Virtually all of these devices areequipped with Bluetooth technology that can be used to distributeentertainment contents such as music and movie clips. Mobileusers can download content from the opportunistically availableinfrastructure (e.g., Digital Billboards) as well as direct Peer-to-Peer (P2P) collaboration which significantly increases contentavailability/coverage. P2P content distribution protocol designis heavily influenced by the characteristics of Bluetooth, whichis the main departure from Internet-based content distribution.However, little has been done to understand the performanceof overall Bluetooth operations, ranging from peer discoveryto data downloading, in dynamic environments with mobility,interference, and different Bluetooth versions/chipsets. In thispaper, we first perform extensive experiments to measure theperformance of Bluetooth in dynamic environments. We findthat Bluetooth-based content distribution faces several challengessuch as time/energy consuming resource discovery and limitedbandwidth even with the enhanced features of the latest Bluetoothversion. We then propose strategies that can effectively improvethe performance of resource discovery and downloading phases.Our simulation and experimental results document the improve-ments obtained with our proposed techniques.

Index Terms—Content distribution, Bluetooth, Measurement

I. I NTRODUCTION

Small, portable devices equipped with one or more wirelesstechnologies (e.g., WiFi, Bluetooth, ZigBee, etc.) are becom-ing increasingly popular, ranging from smart phones (e.g.,iPhone, Blackberry) to digital music players (e.g., MicrosoftZune, iPod touch). Beyond the means of wireless synchro-nization as cable replacement, such radio technologies pavethe way of novel applications to mobile users, thus deliveringnew opportunities to all facets of the industry. In particular,this paper deals with one fast growing application, namely,proximity advertising and marketing using Bluetooth-enableddigital billboards [1]. Handset maker Nokia and music label

Manuscript received April 19, 2005; revised January 11, 2007. This researchis supported through participation in the International Technology Alliancesponsored by the U.S. Army Research Laboratory and the U.K. Ministryof Defense under Agreement Number W911NF-06-3-0001, and; by ARMYMURI under funding W911NF0510246.

Copyright (c) 2009 IEEE. Personal use of this material is permitted.However, permission to use this material for any other purposes must beobtained from the IEEE by sending a request to [email protected].

U. Lee is with Bell Labs, Alcatel-Lucent, 791 Holmdel-Keyport Road,Holmdel, NJ 07733 USA (e-mail: [email protected]).

S. Jung is with Broadcom Cooperation, 16340 West Bernardo DrSan Diego,CA 92127 USA (e-mail: [email protected]).

A. Chang is with Yahoo!, Inc., 3333 Empire Ave, Burbank, CA 91504 USA(e-mail: [email protected]).

J. Choi is with Heavy Iron Studios, 6601 Center Drive West, Suite 400 LosAngeles, CA 90045 USA (e-mail: [email protected]).

D.-K. Cho. and M. Gerla are with the Department of Computer Science,University of California at Los Angeles, Los Angeles, CA 90095 USA (e-mail: [email protected]; [email protected]).

EMI provide a service that lets coffee shop customers listento music via Bluetooth [2]. Similarly, WideRay has placedBluetooth-enabled kiosks in selected music stores, video gamestores, and theaters across the country to send messages askingif users are interested in getting more information aboutvarious items the store is selling, e.g., music, ring tones,videoetc. More interestingly, CBS makes clips from its prime-timeprogram line-up available for free via CBS Outdoor billboardslocated in NYC’s Grand Central Station to allow smart phoneor PDA users to watch clips [3].

So far, mobile users have been tapping this abundanceof data directly from the opportunistically available digitalbillboards. Given the fact that such billboards are locatedatpopular places (e.g., subway stations, department stores,toursites, and stadiums), and more users simultaneously downloadfrom the network while on the move, it is natural to expectthat several of them may be interested in downloading thesame content. Thus, it makes sense to explore the extension ofopportunisticnetworking to include direct Peer-to-Peer (P2P)exchanges. There are obvious advantages in this P2P andinfrastructure downloading synergy. P2P technology enablesus to overcome the short contact duration with a BT-AP due tothe short communication range (10m) and mobility, obviatingthe needs of installing BT-APs every 10m. Thus, our goal isto devise an efficient P2P content distribution protocol in thismobile environment with opportunistic infrastructure supports.

A good model is offered by Internet-based P2P indexing (re-source discovery) and content distribution protocols. A logicaloverlay network (e.g., Chord [4], Gnutella [5]) is built on topof the Internet to provide efficient ways of resource discovery,even with the dynamics of user participation (called churning).In addition, BitTorrent-like P2P file swarming is commonlyused for content distribution where content is divided intoanumber of small pieces and users cooperatively share whateverpieces they have [6]. Then the question is whether we canadapt these techniques to mobile environments. The overlayapproach may not be feasible in mobile environments becausewe cannot assumeInternet-like connectivity[7], [8]. Whileproposals for DHT indexes in mobile networks exist (e.g.,VRR [9], MADPastry [10]), it is neither practical to maintainan overlay network to distant mobile users nor feasible toguarantee thecooperativenessof the intermediate users if theyare not interested in the content [11]. The data access patternsbecome opportunistic since data can be exchanged wheneverthere is a user of the same interests or the infrastructure inthe neighborhood. BitTorrent-like P2P file swarming shouldbeused to exploit the opportunistic connectivity efficiently. Thus,our focus is onopportunistic P2P file swarmingin mobileenvironments.

P2P protocol design in mobile environments is dependent on

Page 2: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

2

underlying radio technology, which is the main departure fromthat in the Internet. P2P file swarming is generally dividedinto two phases, namely resource discovery (in the immediateneighborhood) and downloading. The protocol design of eachcomponent must carefully consider the characteristics of Blue-tooth to obtain better performance. However, little efforts havebeen made to understand the performance of overall Blue-tooth operations, from peer discovery to data downloading,indynamic environments with mobility and interference (WiFi,obstacles). In addition, the performance variation among dif-ferent Bluetooth versions was not examined, although differentBluetooth versions are currently populated in the market due tothe slow adaptation of Bluetooth standards as shown in Figure1. Measurement studies thus far reported only the performanceof specific operations in static scenarios with a certain Blue-tooth version [12], [13], [14], [15]. For instance, Kasten etal. [13] reported the issues of the power consumption andpeer discovery latency in a customized Bluetooth-based sensornode; and Leopold showed peer discovery and throughputmeasurement results of a Bluetooth v1.1 device [14]. As aresult, the impacts of the Bluetooth characteristics on protocoldesign and evaluation were not addressed in recent contentdistribution systems [16], [17], [18], [19], [20].

In this paper, we perform extensive experiments to accu-rately characterize the Bluetooth performance in a dynamicmobile environment. We emulate a mobile user with anAmigobot, a programmable robot that can travel at a speedup to 2.2m/s [21]. Our measurements include peer discovery,connection setup, and download bandwidth under variousconditions such as mobility, interference, different Bluetoothversions. To the best of our knowledge, this is the firstexperiment with Bluetooth in an externally controlled mobileenvironment. From our experimental results, we find thatBluetooth-based content distribution faces several challenges.

• Resource discovery in Bluetooth is time/energy con-suming even with the latest Bluetooth version 2.0. InBluetooth, one must first discover its neighbors (peerdiscovery) and then create a connection to each neighborsequentially to resolve one’s query (content discovery).

• Mobile Bluetooth users experience significant throughputdrop. In particular, the measured throughput varies widelyover distance and is much lower than the simulationresults. We discover that this is mainly due to the auto ratecontrol algorithm in Bluetooth called Channel QualityDriven Data Rate (CQDDR) implementation in Bluetoothdevices. The incorrect choice of a packet type results ina significant performance loss as noted in [22].

• We find that Bluetooth devices are not optimized for“mobile” environments. The advance features such astransmission power control that adjusts power to saveenergy, and adaptive frequency hopping that prevents theuse of interfered channels, do not work well, mainlybecause of their long response time as oppose to a shortcontact period in mobile scenarios.

• We report that the particular type of Bluetooth ver-sions/chipset has a great impact on peer discovery, con-nection setup, and data throughput.

0

0.1

0.2

0.3

0.4

0.5

0.6

1.1 1.2 2.0 1.1 1.2 2.0

2006/11 2007/11

Fraction of devices

Cordless

Cellular

Palm

SmartPhone

Laptop

Fig. 1. Bluetooth version distribution: The statistics were gathered in theUCLA campus on Nov. 29/30, 2006 and Nov. 29, 2007. Bluetooth versioninformation was collected from those devices that allowed making connections(75% of 402 devices in 2006 and 60% of 351 devices in 2007).

Given this, we improve Bluetooth-based contention distri-bution and make the following contributions.

• We investigate the performance improvement strategiesof resource discovery/downloading phases. First, we pro-pose solutions that can effectively reduce the contentdiscovery latency; thus, for given contact time, a node canspend more time for data transfer. Second, we consideran energy efficient peer discovery scheme that savesenergy with minimal performance penalty. Third, wepropose a method for adaptive Bluetooth packet selectionso as to fully utilize the channel. Finally, we evaluatecoding techniques such as rateless/network coding thatcan potentially increase the availability of data.

• These schemes are validated with a combination of simu-lations and testbed experiments. Our results show that (1)the cooperative resource discovery method significantlyreduces the latency; (2) adaptive inquiry mode where theinquiry mode is initiated by other peers conserves energywith minimal performance degradation; (3) a receiverfeedback scheme for packet size selection maximizes thechannel utilization; and (4) network/rateless codes yieldup to 15% improvement in the tested scenarios.

Note that our measurement results can also be applicable tooptimizing networking protocols. For instance, Delay TolerantNetwork (DTN) routing [23], [24] which aims at providingreliable data delivery even with connectivity disruptions(dueto mobility and sparse node density) can maximize data trans-fer per contact, thereby increasing the end-to-end throughput.The rest of this paper is organized as follows. Section II-Bintroduces the basics of Bluetooth. Section III describes theexperimental setup and presents our measurement results. Sec-tion IV describes various schemes to enhance the performanceof a file swarming protocol in Bluetooth. The following sectionevaluates the proposed schemes via extensive simulations.Related work is reviewed in Section VIII. Finally, Section IXconcludes the paper.

II. BACKGROUND

In this section, we overview Bluetooth-based P2P contentdistribution and illustrate detailed protocol operationsof Blue-tooth to understand their impacts on content distribution.

Page 3: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

3

A. Bluetooth-based P2P Content Distribution

In our scenario, a static Bluetooth-enabled digital billboardlocated at a popular place (e.g., subway stations, departmentstores, tour sites, and stadiums) is a publisher of content [1],[2], [3]. It initially pushes content meta-data to mobile users,which includes a unique channel ID (e.g., SHA-1 hash of thecontent title), content version, timestamp, short description,producer, media type, etc. For a given content channel, differ-ent versions can be distinguished using the version field in themeta-data. Based on the content channel description, mobileusers can selectively subscribe various content channels,e.g.,headline news video clip, tourist information, etc.

Mobile users who have subscribed the same content channelproactively query other peers (content pulling). After findingpeers of interest, they share content by using BitTorrent-likefile swarming. The data source (i.e., static billboard) divides afile into n fixed size pieces. Peers can download pieces fromstatic billboard and other peers. Each node has a bitmap of theavailable pieces for efficient piece reconciliation. Whenever aconnection is available, peers first exchange their bitmapstofind out missing pieces through simple bit operations. The sizeof a typical bitmap is as small as tens of bytes even if thereare more than 100 blocks. To be more precise, givenn pieces,the size of a bitmap table is⌈n/8⌉ bytes. For example, thesize of a bitmap table for 100 blocks only takes 13 bytes.

B. Bluetooth Overivew

Bluetooth is defined as layered protocol architecture. Radiospecifies details of the physical layer of Bluetooth. Basebandconcerns with peer discovery, connection setup, addressing,packet format, power control, and timing etc. Link ManagerProtocol (LMP) is responsible for link setup between Blue-tooth devices and link management. This includes the controland negotiation of baseband packet size and transmissionpower. Logical link control and adaptation protocol (L2CAP)adapts upper-layer protocols (e.g., segmentation and reassem-bly, protocol multiplexing, flow control per L2CAP channel,error control and retransmissions, etc.) to the Baseband layervia Host Control Interface (HCI). In this section, we reviewRadio and Baseband of Bluetooth. Readers can find the detailsof the Bluetooth radio system in other papers [25], [26]

In Bluetooth, the total bandwidth is divided into 79 channels(each 1 MHz). Frequency hopping (FH) occurs by jumpingfrom one physical channel to another in a pseudorandomsequence. The hop rate is 1600 hops per second, so thateach physical channel is occupied for 625µs. This intervalis referred to as a slot and is numbered sequentially. Thebasic unit of networking in Bluetooth is apiconet, consistingof a master and from one to seven active slave devices. Themaster determines a hopping sequence based on its BluetoothID (referred as an FH channel) that shall be used by all deviceson this piconet. The FH channel is shared between a masterand slaves using Time Division Duplex (TDD); i.e., dataare transmitted in one direction at a time with transmissionalternating both directions. Multiple slaves share the piconetmedium using Time Division Multiple Access (TDMA). ThisFH channel is a form of Code Division Multiple Access

Inquiry Scan

256 A-trains 256 B-trains

Inquiry Window (Tw_inq)

Inquiry Response Packet

Inquiry Packet

Random Back-off

Interval

Inquiry Scan

Window (Tw_inq_scan)

Interval (Tinq_scan)

Inquiry Packet

Master

Time

Slave

Fig. 2. Bluetooth inquiry procedure example: The size of the inquiry windowis 5.12s (=4×1.28s). The master node sends inquiry packets during the inquirywindow. The slave node periodically listens to a specific channel to wait foran inquiry packet (inquiry scan). After receiving an inquiry packet, the slaveperforms a random back-off and then sends an inquiry response.

(CDMA), and thus, several piconets can coexist with minimalinterference. Two piconets will occasionally use the samephysical channel during the same time slot, causing a collisionand data loss, but this happens rarely, and it is recoveredby forward error correction (FEC) and error detection/ARQtechniques. Multiple piconets in the same area are referredtoas ascatternetand different piconets can be interconnected.However, scatternet connection support is defined as optionalin all the Bluetooth specifications; thus, many Bluetooth chipsdo not support scatternet [19].

Physical link and packet types: The Bluetooth link sup-ports synchronous services such as voice traffic via the syn-chronous connection oriented (SCO) link, and asynchronousservices such as bursty data traffic via the asynchronousconnectionless (ACL) link. Since we focus on the data traffic,only the ACL link is illustrated in the following. Basebanddefines an ACL link to provide packet switched connectionbetween master and active slaves (one ACL link per slave).The master interleaves traffic between multiple active slaves(up to 7 slaves).

Achievable data rates on the ACL link vary depending onthe number of slots per packet (1, 3, and 5 slots) and on theforward error correction (FEC) strategy. The more the numberof slots, the larger is the payload size. FEC packets formatsare DM1 (17B), DM3 (121B), and DM5 (224B), and the non-error coded formats are DH1 (27B), DH3 (183B), and DH5(339B). For instance, DH5 can achieve the maximum rateof 732.2Kbps and 433.9Kbps for asymmetric and symmetriccommunications respectively. The maximum asymmetric andsymmetric data rates are 732.2 Kbps and 433.9Kbps with DH5respectively. As of Bluetooth v2.0 Enhanced Data Rate (EDR),Phase Shift Keying (PSK) modulation has been added. PSKincreases coding bits per symbol to 2 and 3 bits, thus intro-ducing EDR packet typeswithout FEC as 2-DH1/2-DH3/2-DH5 (packet size = 2×DHx, maximum asymmetric andsymmetric rate = 1448.5Kbps and 869.7Kbps, respectively)and 3-DH1/3-DH3/3-DH5 (packet size = 3×DHx, maximumasymmetric and symmetric rate = 2178.1Kbps and 1306.9Kbpsrespectively).

Peer Discovery: The first step in establishing a piconet is toidentify devices in range that wish to participate in the piconet.As shown in Figure 2, the inquiry procedure begins when thepotential master transmits an ID packet withInquiry Hopping

Page 4: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

4

Sequence(Tx slot) and waits for the response packets from theslaves with theInquiry Response Hopping Sequence(Rx slot).The Inquiry Hopping Sequence consists of 32 unique wake-up frequencies distributed equally over the 79 frequenciesandis divided into two distinct sequences calledA- andB-train.Each train contains 16 physical channels and must be repeatedat least 256 times (i.e., 2.56s=256×10ms) before switching toanother train. During the inquiry window (Tw inq), the masterrepeats the following: send two inquiry packets in each Txslot, and then listen to response packets from other devicesinthe following Rx slot. The potential slaves periodically (i.e.,Tinq scan) enter the Inquiry scan state and listen to a specificchannel to search for inquiry packets for inquiry scan window(Tw inq scan). When a node receives an inquiry packet, itenters the Inquiry Response state and returns an FHS packet(i.e., inquiry response) after backing off a random number ofslots to avoid collision. A FHS packet contains the deviceaddress, page scan repetition mode (PSRM) and clock offset(CO), required by the master to initiate a connection. As ofBluetooth v1.2 theinterlaced inquiry scanis introduced. In theoriginal scheme, because the master repeats a specific packettrain many times, a slave node could miss the whole packettrains if it listens to a channel that belongs to the other packettrain (e.g., inquiry A train/scan B train, or inquiry B train/scanA train). To prevent such pathological situations, the interlacedscan allows a slave node to scan two trains (i.e., A and B)in a row. Thus, the probability of missing an inquiry packetbecomes negligible [27].

Connection setup: Once the master has found deviceswithin its range, it can establish a connection to a device(i.e., paging). For a device to be paged, the master uses theslave device address to calculate the page hopping sequence.To synchronize the phase in the sequence, the master usesthe estimate of the slave’s clock from the inquiry response,ifavailable. To synchronize the phase, the master node performsthe similar procedure as the inquiry procedure, but using thepage hopping sequence. The potential slaves periodically (i.e.,Tpage scan) enter the Page scan state and stay there for pagescan window (Tw page scan) to search for paging packets. Theslave page scan repetition mode (i.e., scan frequency) can beeither R1 (Tpage scan ≤ 1.28s) orR2 (Tpage scan ≤ 2.56s).The slave’s setting (i.e.,R1 or R2) is informed to the mastervia the PSRM field of the inquiry response packet. Since thepaging interval should be greater than the page scan interval,the master sets the number of packet train repetition (Npage)based on the slave’s configuration, namelyNpage ≥ 128 forR1 andNpage ≥ 256 for R2. The master then starts sendingthe packet trainNpage times. After receiving the pagingpacket, the slave immediately returns the response packet(without back-off). Finally, the master then responds withitsFHS packet (master response), and the slave sends back anacknowledgement. As a result, a connection is established andboth master and slave begin to use the connection hoppingsequence defined in the master’s FHS packet.

Power class: A Bluetooth device is classified into threepower classes: Class 1, Class 2, and Class 3 (100m, 10m, and1m radio range respectively). Portable devices (PDA, SmartPhones, etc.) typically use Class 2 interfaces, and commercial

Bluetooth-based content distribution systems such as Blue-Casting [28] and BlueBlitz [29] use Class 1 interfaces. Theoutput powers of Class 1, Class 2, and Class 3 must be in rangeof [1mW (0dBm), 100mW (20dBm)], [0.25mW (-6dBm),2.5mW (4dBm)], and [1mW (0dBm), N/A] respectively. Thenominal output power is only defined in Class 2 as 1mW(0dBm). It is mandatory for Class 1 devices, but optionalfor Class 2 devices to implement power control. Based onReceived Signal Strength Indication (RSSI), devices exchangemessages (via LMP) to adjust output power.

III. M EASUREMENTSTUDY OF BLUETOOTH-BASED

CONTENT DISTRIBUTION

In this section, we evaluate the Bluetooth-based contentsharing system through measurement. We are mainly inter-ested in measuringpeer discovery/connection setup latencyanddata rate of a mobile user. Peer discovery latency denotesthe time to discover a random node. Connection latency isthe time to make a connection to a discovered peer. Theefficiency of the connection is measured through the downloadthroughput. These performance metrics may depend on variousfactors; namely, a class of Bluetooth devices (i.e., Class 1or2) and versions (i.e., Bluetooth 1.1, 1.2, or 2.0 EDR), speedof users, distance between users, and WiFi interference. Wemeasure the impacts of these variables using the metrics ofinterests. In this section, after describing the experimentalsetup we first show the results of peer discovery/connectionlatency. We then present the downloading throughput of amobile user followed by the impact of WiFi interference.Finally, the downloading performance in a real environmentis presented.

A. Measurement Setup

Measurement hardware: Evaluating mobile users is chal-lenging since it is nontrivial to control the speed of users.Thus, the key ingredient is the ability to control the speed ofmobile users so that we can repeat the experiment multipletimes in order to accurately measure the performance. To thisend, we use Amigobot, a programmable robot that can travelat a speed up to 2.2m/s. Amigobot can be controlled wirelesslyin a remote machine using a 900MHz radio modem pair calledAmigoWireFree. A mobile user is emulated by an Amigobotcarrying a laptop on its top. In the experiment, the speeds ofthe Amigobot are set to 0.5, 1.0, and 1.5m/s (slow, moderate,and fast walking speeds respectively [30], [31]) and our speedmeasurement results confirm that it can achieve the specifiedsettings of interest. Note that Amigobots are used to measuredata rates under various scenarios (Section III-C and SectionIII-D); for peer discovery/connection setup (Section III-B), weuse static scenarios. The laptop used is Dell Latitude D610with a Pentium M 770 processor (2.0Ghz) and 512MB RAM.

The following Bluetooth dongles are used for experiments.In the case of Class 2 devices, we use Belkin F8T003v (CSRchipset, Bluetooth v1.1), Bluetake BT009Si (Silicon Wave,Bluetooth v1.2), and Belkin F8T013V (Broadcom chipset,Bluetooth v2.0 EDR). Since most commercial Bluetooth-basedcontent distribution systems such as BlueCasting [28] and

Page 5: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

5

PeerDiscovery

hci_inquiry()

ConnectionSetup

connect()

failuresuccess

SendDatasend()

time outsuccess

ConnectionSetup

listen()

ReceiveDatarecv()

Master Slave

Fig. 3. Measurement software operations

BlueBlitz [29] use the Class 1 interfaces, we also experimentwith a Class 1 device, Belkin F8T012V (Broadcom chipset,Bluetooth v2.0 EDR) in order to see the potential benefits ofits high transmission power for data transfer to Class 2 devices.Note that a long range transfer is only feasible between Class1 devices because Bluetooth requires a bidirectional link forhandshaking.

Measurement Environment: Unless otherwise mentioned,the experiments were carried out in the second level of under-ground parking lot to exclude external factors, such as WiFiinterference and obstacles (i.e., human). There was no physicalinterference of human/vehicles because the experiments wereperformed late at night.

Measurement software: To measure the metrics of inter-ests, we develop a measurement tool using BlueZ v3.7 [32].BlueZ is one of the most popular Bluetooth host protocol stackimplementations and is included in the official Linux kernel.BlueZ implements core protocols (e.g., L2CAP, RFCOMM,etc.) and provides the BSD socket interfaces. The softwareis used for peer discovery, connection, and data transfer.On one side, the measurement software operates as masternode, and on the other side, it operates as slave node. Peerdiscovery is carried out usinghci-inquiry function by themaster node. The duration of inquiry (inquiry length) isone of the parameters ofhci-inquiry and its unit is 1.28s.1

The actual data transfer is realized through L2CAP sockets.L2CAP is a connection-oriented protocol that sends individualdatagram of fixed maximum length. The default transportpolicy is to retransmit until success or total connection failure,thus providing areliable point-to-point connection. L2CAPuses Protocol Service Multiplex (PSM) ports to differentiatemultiple applications on the same host. The slave node bindsa PSM port and waits for a connection. The master nodecan initiate a connection to the slave node by calling L2CAPconnectsocket function. An ACL connection is used for datatransfer, but this connection initiation routine (i.e., paging) ishidden beneath the L2CAP software layer in BlueZ kernelmodule. Note thatconnectrequires the timeout value, but thisis nothing to do with the actual paging interval. As described inSection II-B, the paging interval is determined by PSRM valuein the inquiry response. Our tested devices all use PSRM modeR2 in which the page scan interval is less than 2.56s. In thiscase, the specification recommends thatNpage, the number

1The inquiry window size isinquiry length× 1.28s.

of paging train repetitions, should be greater than 256, i.e.,> 2.56s. During the period, if the paging node receives apage response, it immediately returns and starts exchangingadditional packets to setup an L2CAP connection. Therefore,given thatNpage = 256 is used, we can assume that the overallconnection takes roughly less than 3s. In our experiment, thisvalue is used as a connection timeout value. If the connectiontimes out, the programretries until it succeeds.

The overall procedure can be summarized as follows (seeFigure 3). The master node first callshci-inquiry to discoverpeers. The number of inquiry attempts is logged to measurethe discovery latency. Upon finding a node, it tries to makean L2CAP socket connection to the peer. The latency ofL2CAP connectfunction is logged to measure the connectionlatency. As described above, by setting connection timeoutas3s each connection attempt takes less than 3s. The numberof connection attempts is also logged. Dummy data with achunk (packet) of 2300B is continuously transmitted untilthe connection is lost. Hereafter, we refer a chunk of 2300Bas a packet. For every packet transfer, the master logs thetransmission power level (hci-read-transmit-power-level). Theslave node sets on inquiry and page scan, and listens to aspecific port in order to accept an L2CAP connection. Oncethe connection is created, the slave starts receiving packets andlogs link quality (hci-read-link-quality) and Receive SignalStrength Indication (RSSI) (hci-read-rssi) information. Forevery 500ms period, it logs the total amount of received dataand the data throughput. In addition, the slave node logs HCIevent messages usinghcidump, a packet snooping program forBluetooth. As we will see later,hcidumpat the receiver sideallows us to infer the current packet type for data transfer.This information is managed by Link Management Protocol(LMP) in Bluetooth and is not revealed to the upper layer.

B. Peer Discovery/Connection Setup Latency

Peer discovery and connection latencies are the key com-ponents of Bluetooth-based content distribution. Given limitedcontact duration, the above overheads determine the usefultime for actual data transfer. In this section, we investigatevarious parameters impacting the performance of these met-rics; namely, inquiry length, distance between two peers, andBluetooth versions. First, we show the impact of inquiry lengthin the range of[1, 6] with different Bluetooth versions. Recallthat the unit of the inquiry window is 1.28s; e.g., inquirylength 1 is equal to 1.28s. If the inquiry fails, then the masternode tries again. We measured the average number of suchtrials to discover a peer. Upon discovery we measured theconnection setup latency. To exactly measure the impact ofdistance, we usestatic scenarios where the distance betweentwo peers was set to 0m to 20m with a gap of 5m. Bluetoothv1.1, v1.2, and v2.0 EDR were used. For each configuration,we ran experiments 50 times to get the average value.

Figure 4 shows the average number of trials for variousBluetooth versions. Bluetooth v2.0 devices can be successfullydiscovered with inquiry length 1, and v1.2 devices take lessthan 2. This is mainly due to the use of interlaced scanoperations introduced as of Bluetooth v1.2. In the interlaced

Page 6: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

6

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1 2 3 4 5 6

Avg. number of trials

Inquiry window size

1.1 to 1.11.1 to 1.21.1 to 2.0

(a) Bluetooth v1.1

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1 2 3 4 5 6

Avg. number of trials

Inquiry window size

1.2 to 1.11.2 to 1.21.2 to 2.0

(b) Bluetooth v1.2

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1 2 3 4 5 6

Avg. number of trials

Inquiry window size

2.0 to 1.12.0 to 1.22.0 to 2.0

(c) Bluetooth v2.0

Fig. 4. Peer discovery as a function of inquiry length with various Bluetooth versions

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

0 5 10 15 20

Avg. number of trials

Distance (m)

2.0 to 1.12.0 to 2.01.1 to 2.01.1 to 1.1

Fig. 5. Avg. number of trials as a function ofdistance

0

0.5

1

1.5

2

2.5

3

0 5 10 15 20

Avg. connection latency (s)

Distance (m)

2.0 to 1.12.0 to 2.01.1 to 2.01.1 to 1.1

Fig. 6. Average connection latency as a functionof distance

1

2

3

4

5

6

7

8

1.1 1.2 2.0

Total latency (s)

Bluetooth version of the inquiring node

To 1.1To 1.2To 2.0

Fig. 7. Total latency: discovery latency + connec-tion setup latency

inquiry scan, scanning one inquiry packet train is immediatelyfollowed by scanning the other train. The probability ofmissing an inquiry packet train is thus negligible [27], butit is still possible for the following reason. The specificationrecommends random back-off before a node returns an inquiryresponse, mainly to reduce the chances of collision whenmultiple devices have opened scan windows and an inquiringdevice begins transmitting inquiry packets. If the scanninginterval is larger than 1.28s, the range of [0,1023] is usedfor back-off; otherwise, the range of [0,127] is used. Sincethe average back-off interval is one half of the maximuminterval, the probability of failure with inquiry length 1 isgiven as P [failure|Tw inq = 1, Tmax bf = 1024] = 512 ×

0.625ms/1.28s = 0.249 and P [failure|Tw inq = 1, Tmax bf =

128] = 64 × 0.625ms/1.28s = 0.03. From the figures, we seethat the implementation may vary by vendors: Bluetooth v1.2may useTmax bf = 1024 as the maximum back-off windowsize, and Bluetooth v2.0 does not implement any random back-off.

Unlike other Bluetooth versions, Bluetooth v1.1 requiresthat inquiry length be at least 3. In our experiments, we triedsmaller inquiry lengths (namely 1 and 2), but discovery wasnot successful in most of the trials. Theoretically speaking,since inquiry length 1 and 2 show about 0.36 and 0.48 ofdiscovery probabilities according to [27], the discovery processcould be modeled using geometric distribution assuming eachtrial is independent. On average, by repeating a trial 3 timeswith inquiry length 1, we should be able to discover a peer.However, according to our finding this is not true with smallinquiry length. In practice, we found that for Bluetooth v1.1we need at least inquiry length 3. Bluetooth v1.1 scans onlya single train, and missing a train incurs 2.56s (i.e., inquiry

length 2) of penalty. We suspect that our tested Bluetoothdevice may always start with the same train, thus requiring atleast inquiry length 3. The question is then whether we shouldrepeat an inquiry procedure with small inquiry length, say 3or try with bigger inquiry length. For the sake of analysis,we assume that each trial is independent one another if theinquiry length is greater or equal to 3. Then, from Figure 4, wecan calculate the average latency for each Bluetooth version;i.e., the average number of trials is multiplied by the inquirylength. Let us find the optimal inquiry length to discover eachBluetooth version; i.e., by comparing the average latency ofthe plots with “1.1/1.2/2.0 to v1.1” in Figure 4. For instance,for “v1.1 to v1.1” the latency with inquiry length 3 and 4is 4.83s and 5.47s respectively.2 Thus, we conclude that theoptimal inquiry length to discover Bluetooth v1.1 and v1.2 orhigher is given as 3 and 1 respectively.

We then measure the impact of distance on discoverylatency. Figure 5 shows the average number of trials as afunction of distance. To discover Bluetooth v1.1 and v2.0devices, we use inquiry length of 3 and 1 respectively. Thedistance is not a critical factor for device discovery. It confirmsthe fact that the inquiry procedure is robust, because an inquirypacket is repeatedly sent and a small size inquiry packet haslow packet error rate.

Finally, we present the results of connection latency. Again,connection latency is the time for a node to connect to adiscovered peer. Figure 6 shows the results as a functionof distance and Bluetooth versions. Surprisingly, the latency

2The average number of trials for inquiry length 3 and 4 is 1.07 and 1.25,respectively. Since unit inquiry length takes 1.28s, the average latency withinquiry length 3 and 4 is 1.07×1.28×4=4.83s and 1.26×1.28×3=5.47s,respectively.

Page 7: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

7

between Bluetooth v2.0 devices is twice larger than thatbetween Bluetooth v1.1. The latency for a Bluetooth v2.0device to connect to a Bluetooth v1.1 device takes 5 timeslonger than that for the other direction. Similar to inquiry, theconnection latency is not sensitive to distance. Figure 7 showsthe total latency for discovery and connection. In our scenariosunder consideration, it takes much longer to discover and makea connection to a Bluetooth v1.1 device.

C. Download Throughput of Mobile Users

We measured the downloading throughput of a mobile useras follows. A static node as a master transferred data to amobile user, initially located at the same place. As describedbefore, the mobile user was emulated using an Amigobot. Weprogrammed the robot to travel up to 25m at the speed of 0.5,1.0, and 1.5m/s to mimic slow, moderate, and fast walkingspeeds respectively. The impact of various Bluetooth versionswas explored by testing different Bluetooth versions at themobile user side. We used a Class 1 Bluetooth v2.0 deviceto investigate the benefits of high transmission power at thesender side. We ran each configuration 5 times to calculate theaverage.

Figure 8 shows the average of the total amount of datareceived while an Amigobot travels 25 meters. It shows thata user moving at a moderate speed (i.e., 1m/s) can downloadseveral mega bytes. As speed increases, the download datasize decreases. The decrement is nonlinear since for givenfixed distance the traveling time is inversely proportionaltospeed (i.e., 0.5m/s: 50s, 1m/s: 25s, 1.5m/s: 16.6s) – the contactduration is critical in the mobile environment. Figure 9 showthe data rate as a function of distance with different speedsrespectively. The results show that the overall trend of data rateis independent of speed and is a function of distance. Due tothe lack of space, we do not present the results of Bluetoothv2.0 Class 1 results because the overall results are quite similarto Class 1 results, except that it shows higher RSSI on average.High transmission power (100mW vs. 1mW) improves theperformance less than 5%, which validates the importance ofa symmetriclink in Bluetooth – Bluetooth protocol operationsare bi-directional (e.g., peer discovery, data transfer, etc.).However, later we will show that a Class 1 device may bringimprovement in presence of WiFi interference, at the cost ofincreased interference to WiFi.

Figure 10 shows that the RSSI value gradually decreaseswith distance. On the other hand, the data rate sharply dropsafter passing by around the 5 meter mark. This sharp dropis counter intuitive because bit error rate rather graduallydecreases with distance [33]. We find that this is mainly dueto Channel Quality Driven Data Rate (CQDDR), the auto ratecontrol protocol in Bluetooth. The specification states thatquality measurement of a link at the receiver side can beused to dynamically control the packet type transmitted fromthe remote device to improve throughput (closely related toFEC and ARQ). However, the incorrect choice of a packettype may result in a significant performance loss as notedby Chen et al. [22]. Therefore, the CQDDR policy plays akey role in determining the throughput, especially in mobile

environments. The results of Bluetooth v1.1 and v1.2 inFigure 8 show that different policies may bring considerablethroughput difference.

Let us further investigate the behavior of CQDDR in ourtested Bluetooth devices. CQDDR is implemented in the LMP.The receiver side LMP can ask the sender side LMP totransmit packets using a preferred packet type. Although thereis no HCI function that can access current packet type beingused, the current packet type can beinferred at the receiverside as follows. L2CAP layer segments an L2CAP packetby ACL MTU size, which can be read from the Bluetoothdevice. For example, given 1017B ofACL MTU (the defaultvalue of Broadcom chipset), 2300B of an L2CAP packet (+4bytes for L2CAP header) is segmented into two 1017B andone 270B packets. Each segmented packet is attached with a4byte ACL header (i.e., 1021B and 274B packets). A seriesof L2CAP segment packets are then sent to the lower layer.Bluetooth baseband processes those packets: if the size of anincoming packet is larger than that of the current packet type, anode fragments the incoming packet. For instance, in the aboveexample if the current baseband packet type is 3-DH5, whichcan load up to 1021B, the segmented packet with size 1021B issent directly using a 3-DH5 packet. If the current packet type is2-DH5 (max 367B), the baseband layer segments the payload,i.e., by generating two 367B packets and one 283B packet.The L2CAP layer of the receiver will reassemble the payload.Thus, by reading HCI event messages at the L2CAP layer,we can determine the current packet type. In our experiment,we usehcidump to log events. Due to the lack of space, weonly present our main findings. First, the tested Bluetooth v2.0devices start with 3-DH5 and then changes to 2-DH5 and 2-DH3 as link quality degrades. Second, the tested Bluetoothv1.1 devices use DH5 by default and as the distance increases(i.e., after passing on average 10m), it changes the packet typeto DM5. Third, the tested Bluetooth v1.2 devices continue touse DH5 regardless of the distance.

D. Impact of WiFi Interference

Both Bluetooth and IEEE 802.11 WiFi (i.e., 802.11 b/g)operate in the same frequency spectrum, i.e., the 2.4GHzISM band. Thus, it is expected that the performance of thesedevices is adversely affected in the presence of one other. Inview of this, the Coexistence Task Group 2 (TG2) of IEEE802.15 has included an adaptive frequency hopping (AFH)mechanism. AFH considers the channel condition and changesthe hopping frequency dynamically, thus enabling coexistencewith other devices in the 2.4GHz ISM band. AFH consists oftwo steps: channel classification and adaptive control protocol.Channel classification keeps the list of “good” and “bad”channels based on the channel quality. This information isthen exchanged through an adaptive control protocol. Giventhis, AFH kernel chooses a set of hop frequencies to use byavoiding as many bad channels as possible. Note that if thenumber of “good” channels is less than 15 (i.e., the minimumnumber of channels that FCC requires Bluetooth to hop over),some bad channels would still be used.

We evaluate the throughput of mobile Bluetooth users inpresence of WiFi interference. The system configuration is

Page 8: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

8

500

1000

1500

2000

2500

3000

3500

4000

4500

0.5 1 1.5

Total amount of

data downloaded (KB)

Speed (m/s)

2.0 C12.0 C21.2 C11.2 C21.1 C11.1 C2

Fig. 8. Downloaded data size

0

200

400

600

800

1000

1200

1400

1600

0 5 10 15 20 25

Data rate (Kbps)

Distance (m)

0.5m/s1.0m/s1.5m/s

Fig. 9. Date rate of a 2.0 user (C2)

-12

-10

-8

-6

-4

-2

0

0 5 10 15 20 25

RSSI

Distance (m)

0.5m/s1.0m/s1.5m/s

Fig. 10. RSSI of a 2.0 user (C2)

30m

7m

1.5m

WiFi

Am

igo

bo

t

dir

ec

tio

n

N WC

NWS

N BS NBC

BT

Fig. 11. WiFi interference test setup

presented in Figure 11. Note that there was no other WiFiinterference in our experiments. Recall that we performed theexperiments on the second level of an underground parkinglot to exclude external factors. We use a Linksys Wireless-Grouter which is configured with channel 8. NodeNWC withOrinoco Gold 802.11b PCMCIA card is configured as a WiFiclient located 30 meters apart from the router. Both WirelessRouter andNWS are directly attached to the hub. WiFiinterference is induced usingiperf, a bandwidth measuringtool [34], by generating 4Mbps of UDP traffic fromNWC toNWS . For each experiment, the bandwidth ofNWS is loggedto show the impact of Bluetooth on WiFi. NodeNBS is located7 meters away from theNWC and sends dummy data usinga Bluetooth connection to the mobile nodeNBC .

Figure 12 shows the results of total amount of downloadeddata. Compared to Figure 8, WiFi interference incurs through-put loss up to 30% in the scenarios under consideration. AFHis supposed to effectively circumvent this situation in bothBluetooth v1.2 and v2.0, but the loss is prevalent in both cases.To see this, we checked the AFH channel map viahci-read-afh-map. It returns 79 1-bit fields that represent the state ofthe hop sequence specified by the most recent AFH messageexchange by LMP. If channeln is used, the corresponding bitis 1; otherwise it is 0. Our tested devices used all availablechannels even with WiFi interference. Interestingly, whenwetested them in a static environment, it took about 30-60seconds to detect interference. Although the choice of channelestimation method is vendor specific, it is generally believedthat some common methods like packet error rate (PER) or biterror rate (BER) are used. However, these methods generallyare on a channel-by-channel basis, and thus require a longerresponse time. Moreover, the interference learning is onpersessionbasis in our tested devices. A session must last at leastthat period of time to detect it, and the learned informationispurged after a session is closed. Thus, AFH is not effective in

mobile environments due to its long response time as opposeto short contact duration.

From the figure, we also see that unlike Figure 8, a Class 1device brings more than 20% of throughput gain for Bluetoothv2.0 but the performance of other versions is merely improved.We plot the rate over distance for Bluetooth v2.0 Class 1 andClass 2 with WiFi in Figure 13 and Figure 14. The figuresshow that the impact of WiFi on a Class 2 device is visibleenough to observe a drastic throughput decrement when it isnearNWC . On the other hand, Class 1 is fairly resilient toWiFi interference. With a Class 2 sender, WiFi throughputdrops to around 3.2Mbps, whereas with a Class 1 sender,it drops to around 2.9Mbps (additional 10% drop). Figure14 shows that the data rate drops at the very beginning dueto its high transmission power unlike Figure 13. The hightransmission power has a longer interference range. Thus, itis detrimental to the performance of WiFi users.

Note that LMP also uses a power control scheme: if thereceived signal strength differs too much from the preferredvalue, then it may request an increase or a decrease of the TXpower. Most products implement power control to save energyalthough it is not mandatory to implement the function. Thecurrent transmission power can be read byhci-read-transmit-power-level. Interestingly, we were not able to observe anypower control working during the experiments. We examinedtested devices to see whether power control is working or not:we located two laptops close by and initiated data transfer.After passing more than 30 seconds the transmission powerlevel gradually decreased. Note that the RSSI values were quitestable.

E. Summary

The experimental results can be summarized as follows.• Inquiry and connection latencies vary widely among

different versions and chipmakers. Distance is not criticalfor peer discovery and connection latencies. The optimalinquiry lengths for v1.1 and v1.2 or above are given as3 and 1 respectively.

• The data throughput of a mobile user is mainly deter-mined by CQDDR. Transferring data with high power(using a Class 1 device) does not significantly improvethe performance (less than 5% most of the cases). Foran efficient connection management, the data rate/RSSIdynamics can be used to estimate the distance betweenpeers.

Page 9: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

9

0

500

1000

1500

2000

2500

3000

3500

4000

4500

0.5 1 1.5

Total amount of

data downloaded (KB)

Speed (m/s)

2.0 C12.0 C21.2 C11.2 C21.1 C11.1 C2

Fig. 12. Downloaded data size with interference

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0 5 10 15 20 25 2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

Bluetooth Data Rate (Mbps)

WiFi Data Rate (Mbps)

Distance (m)

Class 2 BluetoothWiFi

Fig. 13. Date rate with interference (C2)

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0 5 10 15 20 25 2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

Bluetooth Data Rate (Mbps)

WiFi Data Rate (Mbps)

Distance (m)

Class 1 Bluetooth WiFi

Fig. 14. Date rate with interference (C1)

• Both WiFi and Bluetooth affect the throughput of eachother. Bluetooth v2.0 EDR is more vulnerable to WiFiinterference (35% throughput loss for some case) than theother versions. Bluetooth Class 1 improves the through-put, but it adversely affects the WiFi throughput (addi-tional 10% drop). A long response time as oppose toa short contact duration inmobile environment makespower control/AFH not work well in our tested devices.

IV. PERFORMANCEENHANCEMENT FEATURES

We have characterized the system by extensively evaluatingdiscovery/connection latency and download throughput. Givena dynamic environment with mobility, our results confirmthat resource discovery is time/energy consuming, and amobile user may experience a considerable throughput drop.In this section, we propose strategies that can improve theperformance of file swarming operations (i.e., resource discov-ery/downloading). For efficient resource discovery, we presenta cooperative resource discovery method to minimize resourcediscovery latency, and an energy efficient peer discoveryprotocol that saves energy with minimal performance penalty.To improve the downloading phase, we propose an adaptivepacket size selection protocol that maximizes the utilizationof the physical channel, and evaluate coding techniques suchas rateless/network coding that can potentially increase theavailability of data.

A. Cooperative Resource Discovery

The standard Bluetooth stack includes a service discoveryprotocol (SDP), proposed by the Bluetooth Special InterestGroup (SIG). SDP provides a simple discovery mechanismbased on requesting service classes or service attributes (i.e.,file ID) successively from all devices in range. Since Bluetoothsupports broadcast within a piconet, a node must form apiconet; i.e., the node as master should connect to each nodeasslave sequentially. It is in fact equivalent for a node to connectto each node and send a message one by one. As a result, thenumber of connection attempts per node will increase, andthe chance of a useful contact will decrease because eachconnection attempt will take at least several seconds as shownin the previous section.

Peer discovery in Bluetooth can be used for broadcastingwithout connection setup. Recall that a master node broadcastsan inquiry packet and waits for inquiry response packets fromthe slave nodes. Sedov et al. showed that the 24-bit Class of

Device (CoD) field of an inquiry response (FHS packet) canbe used to expedite service discovery [35]. The CoD field hasa variable format indicated using the format type field withinthe CoD. Bluetooth SIG assigned format type “00” to defineservice category/device class (e.g., computer/laptop). In ourprotocol, we use format type “11” which is reserved for futureuse. A node can recognize that other nodes are running ourapplication by checking this format type.

In our scenario, content has a uniquely identifiable ID thatis generated by applying a collision resistant one-way hashfunction, e.g., SHA1. Since the length of ID is longer than22bits (after excluding 2 bit format type), we take the lower22 bits of ID. Each node sets the CoD field with the file thatit is interested in downloading. Note that the CoD field canbe set by an application using thehci-write-class-of-devicecommand. After an inquiry, a master node can find neighbornodes that are currently downloading files of interest. It thenconnects to one of those nodes to exchange pieces. Before thefile exchange, these nodes can exchange a list of peers thatthey have collected thus far to expedite file ID dissemination.Each entry includes a timestamp to denote when a node wasdiscovered. We can set a time threshold to flush obsolete peersin the list, thus saving bandwidth.

In February 2007, the Bluetooth v2.1 standard draft wasreleased. The major enhancement is the Extended Inquiry Re-sponse (EIR) mode. The potential slave node can immediatelysend an EIR packet (max 240B) after returning the inquiryresponse. Thus, the EIR packet can contain more information,e.g., file bitmap (of pieces that a node has), etc.

B. Energy Efficient Peer Discovery

Power consumption in a mobile device is a critical issue,and it is preferable to design an energy efficient protocol,yet preserving the performance. Meier et al. [36] showed thestatistics of power consumption per unit time in each Bluetoothoperation, i.e.,CIdle = 20mA, CInquiry = 38mA, CScan =49mA, CData = 35mA. Although the scan operation is moreexpensive than an inquiry operation, the inquiry has to berepeated for a longer period of time. The total amount ofconsumed energy by inquiry is muchhigher than inquiry scan.

Bluetooth-based content sharing requires nodes to period-ically alternate their role as master and slave so that theycan discover each other (i.e., P2P mode). Let us see howmuch energy a node spends in an hour to perform theP2P mode. Assume that the time spent for inquiry is the

Page 10: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

10

HCI data

HCI data

Bluetooth

Embeded S/W

HCI data

HCI data

HCI data

LMP

L2CAP

Start

Cont

Start

Cont

HCI data

HCI data

HCI data Cont

Cont

ACL Packet

HCI

HCI

USB

ACL M TU

CH FlgHCI-HDR

Len

CH FlgHCI-HDR

Len

CH FlgHCI-HDR

Len

CH FlgHCI-HDR

Len

CH FlgHCI-HDR

Len

CH FlgHCI-HDR

Len

CH FlgHCI-HDR

Len CH FlgHCI-HDR

Len

BlueZ Linux

Kernel Module

L2CAP PayloadLenL2CAP-HDR

CID

Fig. 15. Bluetooth stack overview: An L2CAP packet is fragmented intomultiple HCI ACL packets of size ACL MTU. These packets are thentransferred to LMP which is implemented in the Bluetooth chipset via HCI.The packet is sent without further fragmentation if the packet size is less thanor equal to the size of the current HCI ACL packet; otherwise it is fragmentedinto multiple packets. The shaded part of an ACL packet is wasted due toincorrect ACL MTU size.

same as that for inquiry scan (i.e., 30 minutes). Thus, itspends38mA × 30m × 1h

60m= 19mAh for the inquiry and

⌊30m × 60s1m

× 1

1.28s⌋ × 49mA × 11.25ms × 10−3s + [30m −

⌊30m × 60s1m

1

1.28s⌋×11.25ms×10−3s]×20mA = 36458.7mAs ≈

10.13mAh for the inquiry scan. During the idle period ofinquiry scan (99.2%), most devices can change their state tothe “standby” state (which consumes 2mAs), instead of the“idle” state [37]. Therefore, the actual energy consumptionof the inquiry scan is approximately 1.21mAh – that is whymost devices (e.g., cell phones) typically stay in the inquiryscan mode. The cost of inquiry is significant, consideringthe fact that recent cell phones such as LG chocolate andMotorola MOTOKRZR K1m are equipped with less than900mAh battery.

We propose a simple solution to reduce the frequencyof inquiry by using the concept of sociological orbits [38].Sociological orbits are probabilistic mobility models wherenodes move between a set of hubs, e.g., subway stations orbus stops. In Bluetooth-based content distribution, such hubsbecome information exchange bazaar [39] such that peoplewith the shared interests can cooperatively share the content.Since it is less likely that information exchange happens intransit to hubs, we propose aadaptive inquiry mode: a nodecontinues to stay in theinquiry scanstate unless some othernodes in theinquiry mode wake it up by creating an actualconnection. To be precise, in the very beginning nobody is inthe inquiry mode. Upon passing by an AP, the node will beactivated. Any node in the inquiry mode can then wake upothers. If a node fails to create a connection or to find anynodes over a certain threshold, it then goes into the inquiryscan state to save energy. Instead of switching back to the scanstate abruptly, the inquiry interval can be dynamically adjustedusing the contact frequency (or recent activity) as proposed byDrula et al. [40].

C. Adaptive ACL MTU Size Selection

Let us illustrate the journey of application data in Bluetooth.Application data sent to the L2CAP layer are loaded into anL2CAP packet. The packet contains an L2CAP packet headerwith length and channel ID (CID).3 The L2CAP packet is

3CID is used to identify a logical channel end point of a device.

then loaded into an HCI ACL packet. If it is larger thanACL MTU , it is fragmented into multiple HCI ACL packetsof size MTU. Note that theACL MTU value is a fixed valueread from the chipset when the BlueZ kernel module is loaded.An HCI ACL packet is then sent to LMP in the Bluetoothchipset. If the current ACL packet type cannot accommodatean incoming HCI ACL packet, it will be further fragmentedinto multiple packets of size of the current ACL packet type.The overall procedure is shown in Figure 15. The problemhappens when theACL MTU is not multiple of the currentACL packet size. We cannot fully utilize the channel; e.g.,in the figure, the shaded part of an ACL packet is wasted.Thus, we have to set theACL MTU value based on thecurrent packet type of LMP. However, Bluetooth does notreveal any information of the current ACL packet type (i.e.,any information of CQDDR in LMP) as shown in SectionIII-C. To illustrate this, consider the following example.An ap-plication sends a series of 1013B packets. The L2CAP packetsize is 1017B (1013B + 4B L2CAP header). Assuming thatACL MTU is 1017B (a default value of Broadcom chipset),there is no L2CAP level fragmentation. Given that LMP startswith the packet type of 3-DH5 (max 1021B), an L2CAP packetis fully loaded into a 3-DH5 packet (1017B+4B ACL header).Now assume that the packet type has changed to 2-DH5 (max679B). A 1017B L2CAP packet cannot be fitted into a 2-DH5packet. At the baseband layer, the packet is fragmented andloaded into two 2-DH5 packets, i.e., 679B+338B. Since thesecond packet cannot be fully utilized, this results in 24%throughput loss, i.e., 341B out of 1358B (=2×679B).

Although there is no HCI function available to read thecurrent ACL packet type, in Section III, we show that the“receiver” can know the sender’s packet type by readingHCI event messages at the L2CAP layer just ashcidump.As distance increases, the packet type gradually changes tosmaller size due to CQDDR (e.g., 3-DH5 to 2-DH5). Thus,we propose that the receiver notifies the packet type change tothe sender so that it can adjust the L2CAP payload size. Thefeedback overhead is minimal because the response packet issmall.

D. Resource Availability Improvement

It is likely that there are more users with partial downloads(due to short contact duration, churning) than ones with acomplete file in the network. Upon contacting these users thepartial downloads tend to have overlapping information. Asauser collects more and more pieces, the problem of overlap-ping is exacerbated (known as a coupon collection problem).It is known that coding techniques such as rateless/networkcoding increase the availability of resources in the network,thus mitigating the coupon collection problem [41], [42], [43].We consider the following coding techniques:

Rateless Code: In rateless codes (e.g., Online Code, RaptorCode, etc.), a file withn blocks is used to generate an infiniteseries of encoded blocks at the APs [41]. Each block isattached with its ID and is disseminated through the lossy“contact”-based Bluetooth network. A user upon collection(1 + ǫ)n blocks can decode the blocks with high probabilitywhereǫ is a system parameter.

Page 11: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

11

Network Code: Network coding involves a coding operationin any intermediate nodes in the network. At the AP, a streamsof encoded blocks are generated through random linear com-bination of the original blocks; i.e.,c =

∑n

k=1ekpk whereek

is drawn randomly over a finite fieldF and pk denotes thek-th original block. The corresponding code vector is the setof coefficients used for coding, i.e.,e = (e1, · · · , en)T . TheAP distributes the coded block with its encoding vector, i.e.,c =

[

ec

]

. Similarly, an intermediate node withm coded blocks(denotedck) in its local memory can generate a re-encodedblock on-the-fly, i.e.,

∑m

k=1ekck whereek is drawn randomly

over a finite fieldF. The code vector of the re-encoded blockis used to find a “helpful” node that has at least one linearlyindependent coded block. If helpful, this re-encoded blockis transferred. After collectingn linearly independent codedblocks, a node can decode the blocks. Readers can find thedetails in the related papers [42], [43].

V. SIMULATION

In this section, we simulate a Bluetooth-based contentswarming algorithm to evaluate the proposed features:i)cooperative resource discovery;ii) data encoding (options: noencoding; network coding; end-to-end rateless coding, and;iii) normal and adaptive inquiry mode. Since coding strategyand inquiry mode are “orthogonal” options, we will test bothoptions jointly in the same experiment. Finally, we show theperformance of the system in presence of different Bluetoothversions.

A. Simulation Setup

We implement the proposed algorithms in the UCBT ns-2 based Bluetooth simulator, a publicly available open sourceBluetooth simulator.4 UCBT implements the majority of Blue-tooth protocols, such as baseband, LMP, L2CAP, and BNEP.

Mobility : We assume Bluetooth devices (AP’s and mobiles)are placed in a long corridor with fixed boundary (for example,an airport alley, or a shopping mall). Users are moving withwaypoint motion from East to West or from West to East.Waypoints are randomly chosen in the initial stage. To mimicrealistic pedestrian motion, speeds are selected based on thenormal distribution with the mean speed (0.8, 1.0, 1.2, or1.4m/s) and the variance (mean speed/3). There is no pauseat the end of each waypoint. Mean speeds are selected basedon the mobility model as reported in [30]. To add a randomfactor, direction is changed periodically with an offset intherange [-10, 10] degrees with respect to the original direction.When a node reaches the North or South boundary, it isreflected back in to the working area. When a node reachesEast or West bound, it moves out of the area. A new nodeis then generated at the other boundary with same speed anddirection as the eliminated node.

Metrics: We measure the progress using download percent-age of all the nodes that have passed the simulated area duringthe simulation. By dividing the summation of the downloadedblocks by the nodes, we can calculate the expected number

4https://www.ececs.uc.edu/cdmc/ucbt

of downloaded blocks. As time tends to infinity, by the stronglaw of large number, this is equal to the ensemble averageof the number of blocks that a random node can downloadwhile passing by the simulated region. The percentage iscalculated by dividing the average by the total number ofblocks. For instance, given that 100 10KB blocks, 50% meansthat a random user can download 500KB while passing by asimulated region.

Inquiry methods: Every Bluetooth node has two stages:Inquiry and Connection stages. In the inquiry stage, nodesperform peer discovery by alternating inquiry and inquiry scanbased on the selected inquiry mode. The Connection stagebegins after a connection is established. If there is no usefuldata to transfer in both directions, the connection is terminated.It is maintained until all useful data are transferred in bothdirections. After this, they enter the Inquiry stage.

• Normal Inquiry ModeEvery Bluetooth node does inquiryand inquiry scan alternately. This alternately changinginquiry and inquiry scan during inquiry stage maintainedthroughout simulation time.

• Adaptive Inquiry ModeAll nodes except Access Point(AP) initially do inquiry scan only, to save energy. A nodereactively starts peer discovery after the first connection.If there is no incoming connection for 10s, it switchesback to the inquiry scan only mode.

Data transfer methods: There are two types of nodes inour system: APs and mobile nodes. APs have the completefile. They are static and are randomly distributed in the area.Mobile nodes initially have no files to exchange. They canreceive data blocks from APs or other mobile peers. They arealso randomly distributed in the moving area and they movebased on selected speed.

• Access Point (AP) only Data TransferThis option corre-sponds to downloading from AP only (ie, no P2P contentsharing via Bluetooth). The AP option is tested as areference, to show the relative improvement introducedby P2P sharing.

• Non-coded Data TransferAfter the connection is madebetween two nodes, block bitmaps are exchanged to showavailability of blocks in each node [19]. Each node thuscan make a “helpful” data list, i.e., a list of blocks thatcan help the peer. If the helpful data list is non empty, oneblock is randomly selected from the list and transferredto the peer.

• Network Coding Data TransferFor AP-to-Peer connec-tions, an AP distributed coded blocks by encoding blocksvia network coding. Similarly, for P2P connections eachnode generates a coded block by linearly combining allthe blocks. Note that the code vector is first transferredto the other party to check the helpfulness; thus, actualdata transfer happens only if it is helpful. The28 Galoisfield is used for network coding.

• Rateless Coding Data TransferIn an AP, data is encodedusing Online Code [41], cached in background and thentransmitted. Each encoded block has its unique ID. Aftera connection with the peer is made, the peers exchangedata block ID lists (instead of data block bitmaps as innormal data transfer). The data block list contains all

Page 12: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

12

0

10

20

30

40

50

60

0 120 240 360 480 600 720 840 960

Dow

nloa

d P

erce

ntag

e (%

)

Time (s)

APP2P(Non, Normal)P2P(RC, Normal)P2P(NC, Normal)

(a) Normal

0

10

20

30

40

50

60

0 120 240 360 480 600 720 840 960

Dow

nloa

d P

erce

ntag

e (%

)

Time (s)

APP2P(Non, Adaptive)P2P(RC, Adaptive)P2P(NC, Adaptive)

(b) Adaptive

Fig. 16. Download percentage vs. time

0

10

20

30

40

50

60

0.8 1 1.2 1.4

Ave

rage

Dow

nloa

d P

erce

ntag

e (%

)

Average Speed (m/s)

APP2P(Non, Normal)P2P(RC, Normal)P2P(NC, Normal)

(a) Normal

0

10

20

30

40

50

60

0.8 1 1.2 1.4

Ave

rage

Dow

nloa

d P

erce

ntag

e (%

)Average Speed (m/s)

APP2P(Non, Adaptive)P2P(RC, Adaptive)P2P(NC, Adaptive)

(b) Adaptive

Fig. 17. Download percentage vs. speed

block IDs that a certain node has. After exchanging datablock list, each node constructs the helpful data list forthe peer. If the list is non empty, one block is randomlyselected and transferred to the peer.

Parameter Summary: The area is a corridor with 100mlength and 5m width where we deploy 50 nodes. We set thenumber of APs to 1 to keep the system simple. Mean speedsare selected between 0.8 and 1.4 m/s to simulate pedestrianmobility. Unless otherwise mentioned, we use the mobility of1m/s and the DH5 packet by default. The AP distributes a4.8MB file. The size of a block is 24KB (total 200 blocks).Rateless and network codes use the same block size for coding.Note that due to the space limitation, we do not presentthe impact of the block size, but the overall performance isconsistent with the default configuration.

B. Simulation Results

Download Progress: Figure 16 shows average downloadpercentage at 10 second intervals. For Figure 16(a), normalinquiry mode is used and for Figure 16(b), adaptive inquirymode is used. In this experiment, as mentioned earlier, weevaluate both inquiry modes and coding strategies. Startingwith normal vs. adaptive inquiries, network coding and ratelesscoding data transfer, normal and adaptive inquiry mode showsalmost the same performance. But, non-coded data transferwith adaptive inquiry mode shows better performance thanthat with normal inquiry mode. When adaptive inquiry modeis used, an AP node transfers data and also wakes up mobilenodes. Other nodes in the adaptive inquiry mode do the same.In this case, those inquiring nodes always have data and there-fore, this decreases unhelpful connections. As a result, over-head for data transfer is reduced; thus, download percentageincreases. For coded data transfers, their data blending feature

already helps to increase download percentage, therefore itmakes the adaptive inquiry mode less effective.

Moving to the data transfer options, P2P transfers haveconsistently twice the download percentage as AP transfer.As for the P2P coding options, rateless coding and networkcoding methods show better performance than the non-codingmethod after the download percentage reaches approximately30%. These coding methods reduce overlapping of data blocksbetween two connected nodes; thus, they increase downloadpercentage. Rateless coding encodes data only in AP or atnodes that are acting as AP’s after recovering the entirefile; whereas network coding encodes blocks in all nodesafter receiving data blocks from others and therefore data aremore well blended. Thisblending feature of network codingincreases the download percentage much more than ratelesscoding.

Figure 17 shows average download percentage during 1000seconds. For Figure 17(a), the normal inquiry mode is used,and for Figure 17(b), the adaptive inquiry mode is used. Asspeed increases, download percentage decreases for all cases.In fact, as speed increases, nodes move out of communi-cation range faster and therefore packet drops happen morefrequently. Also, lifetime of nodes decreases. This results inmore frequent regenerations of nodes. Note that when a nodeis regenerated, all received data are deleted. This decreasesaverage download percentage.

All P2P data transfers show twice download percentagethan AP data transfer. Among P2P data transfers, downloadpercentages can be ranked in the following order; network cod-ing, rateless coding, and non-coding because of overlappingof blocks and data blending feature that mentioned previoussection. Adaptive inquiry mode has almost the same downloadpercentage as normal inquiry mode.

Page 13: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

13

0

20

40

60

80

100

120

140

0.8 1 1.2 1.4

Num

ber

of In

quiry

Average Speed (m/s)

APP2P(Normal)

P2P(Adaptive)

Fig. 18. Number of inquiry vs. speed

0

20

40

60

80

100

0 120 240 360 480 600 720 840 960

Dow

nloa

d P

erce

ntag

e (%

)

Time (s)

0%25%50%75%

100%

Fig. 19. Download percentage vs. the faction ofBluetooth v2.0 nodes

0.5

0.6

0.7

0.8

0.9

1

2 3 4 5

Fraction of useless

connection attempts

Number of files

Fig. 20. Percentage of useless connection attempts

Inquiry Frequency : Figure 18 shows the number of in-quiries used during 1000 seconds. In AP mode, only APperforms inquiry, so the number of inquiries is calculated fromAP node. In P2P mode, this number is calculated by averagingtotal number of inquiries from all nodes. The adaptive inquirymode has 60% of inquiries compared to normal inquiry mode.Inquiry consumes more energy than inquiry scan. So, by usingadaptive inquiry mode, nodes can save energy. Our resultsshow that normal and adaptive inquiry modes show almost thesame connection times. By using the adaptive inquiry mode,nodes can save energy, while having same discovery efficiencyand data download percentage.

Presence of Different Bluetooth Versions: We show theimpact of different Bluetooth versions in presence. In thesimulation, we use Bluetooth v2.0 users as well as Bluetoothv1.2 users. Bluetooth v2.0 uses the 3-DH5 packet by defaultand Bluetooth v1.2 uses the DH5 packet by default. Weincrease the fraction of Bluetooth v2.0 users with a gap of25%. Figure 19 shows the results in presence of differentBluetooth versions. The graph clearly shows that the presenceof different Bluetooth versions has a critical impact on thesystem performance.

Cooperative resource discovery: Thus far we consider ascenario where the AP distributes a single file such that everynode in the network is interested in downloading the file. Inreality, multiple files are shared at the same time. A nodeshould first find who are currently sharing the file of interestby connecting its neighbors one by one. As the number offiles increases, there will be more useless connection attempts,thus decreasing the useful contact period. The problem willnot happen with cooperative resource discovery, because eachnode learns this information from inquiry or its neighbors.To illustrate the benefits of cooperative resource discovery,we increase the number of files from 2 to 5 and dividethe users into 2 to 5 groups of (approximately) equal sizeaccordingly. For instance, given two files shared, we divide49users into two groups: one with 25 and the other with 24. Wesimulate the scenario of 50 nodes with 1m/s average speed.The percentage of useless connection attempts is plotted inFigure 20. The figure clearly shows that useless connectionattempts clearly increases as the number of files increases,which can be prevented with cooperative resource discovery.When a single file is shared, there are about 7 connections pernode, but the number of attempts increases as the number of

files increases.

VI. EXPERIMENTS

In this section, we present the evaluation of adaptive ACLMTU selection and then deliver our preliminary field testresults.

Adaptive ACL MTU size selection: We modify the BlueZstack in Linux Kernel v2.6.15.1. During the L2CAP con-nection setup (l2cap-do-connect), mtu field of the L2CAPconnection data structure (l2cap-conn) is initialized withACL MTU . This value is used to fragment the L2CAPpacket inl2cap-do-send()(defined inl2cap.c). Thus, wheneverreceiving a feedback a node updatesmtu field (via a newlydefined socket option). We use the same setting as in SectionIII-C. We compare the performance of adaptive ACL MTUselection using Bluetooth v2.0 Class 2 interfaces. The speedof an Amigobot is set to 0.5m/s. Figure 21 shows the results.It shows that the receiver feedback improves the overallthroughput more than 20%.

Testbed experiment results: We implement our protocolusing Python.5 We use PyBluez, a Python library to accessBlueZ functions in Linux. The software can be configured aseither BT-AP or peer. A BT-AP performs inquiry continuallyto discover mobile users. Upon discovering a node, it startssending missing pieces to a mobile user. In contrast, mobileusers alternate between inquiry and inquiry scan periodicallyto discover one another. If multiple peers are detected, a nodeselects the node with highest RSSI value. Recall that RSSI canbe a good measure of distance estimation as shown in SectionIII-C.

We used Kensington 33348 Bluetooth dongles (v2.0 EDR,Class 2, and Broadcom chipset) for both mobile users and aBT-AP. As in Figure 22, four people with a laptop walkedaround the tested area at a nominal walking speed (1-1.5m/s).We performed our experiment in the third floor of BoelterHall at UCLA. The BT-AP distributed a 5MB file. We usedthe piece size of 8KB (total 640 pieces). We measured thedownload progress over time. Figure 23 shows the progress ofmobile users.6 As time passes, we notice that nodes have morechances to collect pieces using P2P connections. It shows that

5The software is available at http://netlab.cs.ucla.edu/cgi-bin/usemod10/wiki.cgi?Bluetooth

6For the sake of illustration, we plot a mark in every 10 pieces.

Page 14: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

14

0

200

400

600

800

1000

1200

1400

1600

1800

0 5 10 15 20 25

Data rate (Kbps)

Distance (m)

FeedbackDefault

Fig. 21. Performance comparison with receiverfeedback

50m

15m

BT-AP

#1

#2

#3

#4

Moving Direction

Fig. 22. Experiment scenario (BoelterHall, UCLA)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 100 200 300 400 500 600 700

Download Porgress (%)

Time (s)

User 1User 2User 3User 4

Fig. 23. Download progress of mobile users

a user can download around 1.6MB from the AP. From thelog, more than 56% of the blocks were downloaded via P2P,i.e., 1471 out of 2560 (640×4) pieces. The figure shows that asusers collect more pieces, there will be more opportunitiesofP2P download, i.e., most of the short downloading percentageincrements in the figure. The packet type change pattern wasconsistent with the previous experiment results: 3-DH3→ 2-DH5 → 2-DH3.

Our preliminary test shows that there are still rooms forimprovements. First, a connection must be closed as early aspossible when a user is out of range. We observed that a useroften failed to utilize a contact period because of this. A gooddesign choice would be installing multiple interfaces in the BT-AP. Second, peer selection must be carefully done to preventthe case of making unnecessary connection attempts to a nodemoving away.

VII. D ISCUSSION

Potential benefits of Bluetooth over other wireless tech-niques: P2P wireless networking with handheld devices canbe realized mainly with Bluetooth, WiFi and cellular. Inthis paper, we claim that Bluetooth is a viable solution.First, Bluetooth is energy-efficient; the low-power idle modeof Bluetooth is about 40 times more energy efficient thanthat of WiFi [44]. Cellular radio, say 2.5G and 3G, isextremely power hungry compared to Bluetooth and WiFi(e.g., up to 2000mW transmission power); moreover, it iscostly and has bandwidth limitations and latency problems.Second, Bluetooth efficiently utilizes the spectrum becauseit uses channel hopping over the 2.4GHz ISM band that isshared with license-free data communications (e.g., 802.11b/g, ZigBee, etc.) [45], [46], [47]. WiFi has the disadvan-tage of providing less efficient spectrum reuse in crowdedenvironments, say shopping malls; and chaotic deploymentsof WiFi access points (i.e., unplanned and unmanaged) makethe situation worse [48]. ZigBee is reported to be vulnerableto WiFi interference [49].7 Finally, Bluetooth is ubiquitous

7Crossbow reported that packet loss caused by WiFi interference wasrelatively low (<15%) in their tested scenario [50], but recent measurementresults show that a ZigBee network may experience much higher packet lossrate [51], [49], [52]. Thus, several interference mitigation strategies have beenproposed [49], [52]. Musaloiu-E et al. [49] proposed a method of estimatingthe degree of WiFi interference to dynamically switch frequencies wheninterference is detected. Ho et al. [52] proposed to use dualradio interfaces(ZigBee and WiFi) such that before ZigBee data transfer, a WiFi RTS/CTSis sent first to keep interfering WiFi devices quiescent.

in portable devices. With continuing proliferation, Bluetooth-enabled devices are forecast to increase by over 40% thisyear, to around 800 million units worldwide [53]. While weadvocate Bluetooth for P2P networking, we recognize howeverthe fact that other technologies arecomplementaryand willin fact all be usedopportunisticallyby the mobile users indifferent applications.

Energy efficiency and multi-radio scenarios: Bluetooth 2.0and 802.11g can achieve the throughput of about 2.1Mbps and31Mbps respectively [54]. Assuming that transmission poweris 81mW and 890mW for Bluetooth and WiFi [44], energy-per-kilo-bit (the ratio of transmission power to throughput)is given as 0.0385mJ for Bluetooth and 0.0287mJ for WiFi.Although WiFi achieves better energy efficiency for datatransfer, it has the disadvantage of providing less efficientspectrum reuse in crowded environments. WiFi shares themedium using CSMA and per node throughput is inverselyproportional to the number of contenders, assuming that theyfairly share the bandwidth. Thus, energy-per-kilo-bit is roughly0.0287×kmJ wherek is the number of contenders. In contrast,Bluetooth can fully sustain the maximum bandwidth up to 20different piconets [55]. As the number of contenders increases,WiFi could show worse energy-per-kilo-bit performance thanBluetooth.

Due to the short radio range, mobile Bluetooth users mayfail to find other users of interest, suffering from intermittentconnectivity or prolonged disconnections. As a complementsolution, WiFi can also be used to provide a better coverage.Knowing that keeping WiFi always on quickly drains a battery,Pering et al. [44] proposed a set of policies where Bluetoothisused by default, and WiFi is used on-demand based on end-to-end application requirements, especially when requiring higherbandwidth.8 In practice, this may not always be true due to lessefficient spectrum reuse of WiFi in crowded environments (interms of number of users and unmanaged deployment of WiFiAPs). Nonetheless, using WiFi on-demand can help mitigate acoverage problem of Bluetooth-based content distribution. Forinstance, when mobile users detect that the content availabilityfrom neighboring nodes is below a certain threshold, theycooperatively turn on WiFi and start exchanging data overWiFi. Also, nodes switch off WiFi whenever the availabilitycriterion is met and then exchange data using Bluetooth. It

8They showed that the low-power idle mode of WiFi (Cisco PCM-350) andBluetooth (BlueCore3) consumes 390mW and 5.8mW, respectively

Page 15: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

15

would be part of our future work to evaluate the tradeoffsbetween energy consumption and download delay in variousswitching policies.

VIII. R ELATED WORK

Bluetooth-based content distribution uses Bluetooth AccessPoints (BT-AP) to distribute content to the mobile users. Itcan be classified based on cooperativeness of users. Non-cooperative content distribution is based on a client servermodel; BT-APs are the servers, and clients download filesonly from the APs. BlueCasting [28] and BlueBlitz MagicBeamer [29] are a widely accepted proximity marketing sys-tem such that they can identify Bluetooth users and delivertailored messages. LeBrun et al. [16] proposed BlueSpot,a public-transit based content distribution system accessiblesuch that riders can opportunistically access data during theirtravel time via Bluetooth. In contrast, cooperative contentdistribution is based on a P2P model such that users can sharethe downloaded files from APs. Farkas et al. [17] proposed apush-based P2P content distribution model: A file was dividedinto blocks, and blocks were multicasted through Bluetoothpiconets. Given that nodes were synchronized and static, aring overlay was formed to schedule parallel piconets toefficiently distribute blocks. Unfortunately, the scheme did notconsider network dynamics such as mobility, churning, lackof synchrony, and intermittent connectivity. Jung et al. [19]implemented cooperative file swarming by optimizing the peerdiscovery in Bluetooth to better utilize the short contact dura-tion among mobile users. In the Haggle Project [56], Leguay etal. [18] proposed various content distribution strategiesto dis-tribute asmall sizefile to a group of users in a large scale urbannetwork. Users can cooperatively relay a file to other interestedusers, and a set of mobile bridges (regardless of interests)canbe used to further expedite delivery. However, none of theabove works attempts to consider the actual characteristics ofP2P Bluetooth communications in realistic environments (withmobility, heterogeneous Bluetooth versions, WiFi interference,etc.), even though that has a significant impact on the protocoldesign and evaluation. In this paper, we first evaluate Bluetoothcommunications in such an environment. The evaluation re-sults are then used to examine various aspects of a system, e.g.,resource discovery, limited bandwidth, resource availability,and heterogeneous Bluetooth versions/chipsets. Based on this,we propose techniques for efficient file swarming.

In spite of its popularity, the performance measurements ofBluetooth devices have not been thoroughly explored. Kastenet al. [13] evaluated their customized Bluetooth-based sensordevice as a part of the Smart-its project and reported theissues of the power consumption and the inquiry latency.Similarly, Siegemund et al. [15] measured the performancethe inquiry procedure and then proposed a cooperative peerdiscover protocol. The measurement study by Leopold [14] isclose to our work. The inquiry procedure and the throughputwere evaluated using Bluetooth v1.1 devices. The authorfound that the inquiry was not sensitive to distance and thelatency distribution was centered on the train length. Themeasured throughput varied widely over distance and was

much lower than their simulation results, which the authorwas not able to explain. Our work differs in thati) we explorethe interoperability issues of various Bluetooth versions(v1.1,v1.2, and v2.0 EDR),ii) we measure the throughput of a“mobile” user, iii) we find the reason behind the throughputvariation over distance,iv) we explore the impact of interfer-ences (e.g., WiFi/Obstacles). Note that our inquiry results areconsistent with [14] over all Bluetooth versions. Beaufouretal. [12] showed that the connection latency follows a long-taildistribution, i.e., quite a few take longer than 3 seconds. Incontrast, we find that the connection setup rarely takes morethan 3 seconds.

WiFi and Bluetooth coexistence issues are well documentedin the literature [46], [57], [47]. Shoemake [46] performedcoexistence testing and showed that interference results inconsiderable throughput loss. Punnoose et al. [57] identifiedthat effective bandwidth degrades more rapidly than the packetloss rate owing to deferred transmissions caused by carriersensing in WiFi. In contrast, Bluetooth performance startstodegrade rapidly when the interfering WiFi signal is comparableto the desired signal level since Bluetooth does not use“carrier sensing.” Shuaib et al. [47] evaluated the coexistenceof 802.11g with Bluetooth. They showed that the shorter thedistance between Bluetooth node and WiFi node, the less is theimpact on the throughput, mainly due to the power control inBluetooth. Mander et al. [58] evaluated the adaptive frequencyhopping (AFH). They showed that the AFH improves thethroughput more than 30%. Unlike [14], they claimed thataverage throughput does not vary significantly with distance.In this paper, we evaluate the impact of WiFi interferenceon “mobile” Bluetooth users. Contrary to the previous re-sults [47], [58], we find that in the mobile environmenti)power control/AFH may not work well potentially due todynamics of RSSI, andii ) the average throughput is dependenton the distance due to CQDDR.

IX. CONCLUSION

In this paper, we studied the practical implementation ofP2P content distribution to Bluetooth users. We measured theperformance of Bluetooth operations such as peer discovery,connection setup, and data throughput in dynamic environ-ments with mobility, interference, and different Bluetoothversions. We showed that resource discovery is time/energyconsuming even with the latest Bluetooth version; mobileBluetooth users experience considerable throughput drop notonly from the imperfect auto rate selection implementationin Bluetooth, but also from external interferences (obstacles,WiFi); and the advance features (power control and adaptivefrequency hopping) did not work well. We then introducedstrategies that can improve the performance of resource dis-covery and downloading phases, and validated the proposedschemes via simulations and experiments. Our results showedthat (1) the cooperative resource discovery effectively reducedthe latency with minimal overhead; (2) the adaptive inquirymode significantly reduced the frequency of inquiry; (3) theadaptive ACL MTU size selection scheme achieved a 20%throughput improvement; and (4) coding-based file swarmingimproved the system performance by 15%.

Page 16: P2P Content Distribution to Mobile Bluetooth Usersmslab.kaist.ac.kr/wikipages/files/bluetooth_cdn_tvt... · 2010. 1. 13. · In this section, we overview Bluetooth-based P2P content

16

REFERENCES

[1] A. Nandan, S. Tewari, S. Das, G. Pau, M. Gerla, and L. Kleinrock,“AdTorrent: Delivering Location Cognizant Advertisementsto Car Net-works,” in IFIP WONS, Les Menuires, France, Jan. 2006.

[2] “BBC NEWS: Music trial taps into Bluetooth,” http://news.bbc.co.uk/1/hi/technology/4392534.stm.

[3] “CBS Goes Bluetooth To Promote Fall TV Line-Up,” http://www.telecomweb.com/tnd/18847.html.

[4] I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan,“Chord: A Scalable Peer-to-peer Lookup Service for Internet Applica-tions,” in SIGCOMM’01, San Diego, CA, Aug. 2001.

[5] “Gnutella,” http://en.wikipedia.org/wiki/Gnutella.[6] B. Cohen, “Incentives Build Robustness in BitTorrent,”in P2PECON’03,

Berkeley, CA, Jun. 2003.[7] M. Conti, E. Gregori, and G. Turi, “A Cross-layer Optimization of

Gnutella for Mobile Ad Hoc Networks,” inMobiHoc’05, Urbana-Champaign, IL, Sep. 2005.

[8] U. Lee, J.-S. Park, J. Yeh, G. Pau, and M. Gerla, “CodeTorrent: ContentDistribution using Network Coding in VANETs,” inMobiShare’06, LosAngeles, CA, Sep. 2006.

[9] M. Caesar, M. Castro1, E. B. Nightingale, G. O’Shea, and A. Row-stron, “Virtual Ring Routing: Network routing inspired by DHTs,” inSIGCOMM’06, Pisa, Italy, Sep. 2006.

[10] T. Zahn and J. Schiller, “Performance Evaluation of a DHT-basedApproach to Resource Discovery in Mobile Ad Hoc Networks,” inWONS’07, Obergurgl, Austria, Jan. 2007.

[11] T. Qiu and I. Nikolaidis, “On the Performance and Policies of Mo-bile Peer-to-Peer Network Protocols,” inCNSR’04, Fredericton, N.B.,Canada, May 2004.

[12] A. Beaufour, M. Leopold, and P. Bonne, “The Bluetooth Radio System,”in WSNA’02, Atlanta, Georgia, Sep. 2002.

[13] O. Kasten and M. Langheinrich, “First Experiences withBluetooth in theSmart-Its Distributed Sensor Network,” inPACT’01, Barcelona, Spain,Oct. 2001.

[14] M. Leopold, “Evaluation of Bluetooth Communication: Simulation andExperiments,” Dept. of Computer Science, Univ. of Copenhagen, Tech.Rep., Spring 2002.

[15] F. Siegemund and M. Rohs, “Rendezvous Layer Protocols for Bluetooth-Enabled Smart Devices,”Personal and Ubiquitous Computing Journal,vol. 7, no. 2, pp. 91–101, Jul. 2003.

[16] J. LeBrun and C.-N. Chuah, “Feasibility Study of Bluetooth-BasedContent Distribution Stations on Public Transit Systems,” in ACMMobiShare, Los Angeles, CA, Sep. 2006.

[17] P. S. Lorant Farkas, Balazs Bakos, “A Practical Approach to Multicas-ting in Bluetooth Piconets,” inWCNC’06, Las Vegas, USA, Apr. 2006.

[18] J. Leguay, A. Lindgren, J. Scott, T. Friedman, and J. Crowcroft, “Op-portunistic Content Distribution in an Urban Setting,” inCHANTS’06,Pisa, Italy, Sep. 2006.

[19] S. Jung, U. Lee, A. Chang, D. Cho, and M. Gerla, “BlueTorrent:Cooperative Content Sharing for Bluetooth Users,” inPerCom’07, WhitePlains, NY, Mar. 2007.

[20] L. McNamara, C. Mascolo, and L. Capra, “Media Sharing basedon Colocation Prediction in Urban Transport,” inMobiCom08, SanFrancisco, CA, Sep. 2008.

[21] “AmigoBot,” http://www.activrobots.com.[22] L.-J. Chen, R. Kapoor, M. Y. Sanadidi, and M. Gerla, “Enhancing

Bluetooth TCP Throughput via Link Layer Packet Adaptation,” inICC’02, Anchorange, AK, May 2002.

[23] K. Fall, “A Delay Tolerant Networking Architecture forChallengedInternets,” inSIGCOMM’03, Karlsruhe, Germany, Aug. 2003.

[24] U. Lee, S. Y. Oh, K.-W. Lee, and M. Gerla, “RelayCast: ScalableMulticast Routing in Delay Tolerant Networks,” inICNP’08, Orlando,FL, Oct. 2008.

[25] J. C. Haartsen, “The Bluetooth Radio System,”IEEE Personal Commu-nications, vol. 7, no. 1, pp. 28–36, Jul. 2000.

[26] “Bluetooth SIG,” Bluetooth Specification v2.0, 2004.[27] B. S. Peterson, R. O. Baldwin, and J. P. Kharoufeh, “Bluetooth Inquiry

Time Characterization and Selection,”IEEE TMC, vol. 5, no. 9, pp.1173–1187, Sep. 2006.

[28] “BlueCasting,” http://www.bluecasting.com.[29] “BlueBlitz,” http://www.blueblitz.com.[30] “New York City Pedestrian Level of Service Study - PhaseI, 2006.”[31] K. Teknomo, “Microscopic Pedestrian Flow Characteristics: Develop-

ment of an Image Processing Data Collection and Simulation Model,”Ph.D. dissertation, Tohoku University Japan, Sendai, 2002.

[32] “BlueZ Bluetooth protocol stack for Linux,” http://www.bluez.org.

[33] A. Madhavapeddy and A. Tse, “A Study of Bluetooth Propagation UsingAccurate Indoor Location Mapping,” inUbiComp’05, Tokyo, Japan, Sep.2005.

[34] “Iperf,” http://dast.nlanr.net/Projects/Iperf.[35] I. Sedov, S. Preuss, C. Cap, M. Haase, and D. Timmermann, “Time and

energy efficient service discovery in Bluetooth,” inVTC’03, Jeju, Korea,Apr. 2003.

[36] L. Meier, P. Ferrari, and L. Thiele, “Energy-efficient bluetooth net-works,” in Technical Report 204, Computer Engineering and NetworksLaboratory (TIK), ETH Zurich, Jan. 2005.

[37] J.-C. Cano, J.-M. Cano, E. Gonzalez, C. Calafate, and P.Manzoni,“Power Characterization of a Bluetooth-based Wireless Node for Ubiq-uitous Computing,” inICWMC’06, Bucharest, Romania, July 2006.

[38] J. Ghosh, S. Yoon, H. Q. Ngo, and C. Qiao, “Sociological Orbit for Ef-ficient Routing in Intermittently Connected Mobile Ad Hoc Networks,”in Technical Report TR-2005-19, University at Buffalo, Apr. 2005.

[39] M. Motani, V. Srinvasan, and P. S. Nuggehalli, “PeopleNet: EngineeringA Wireless Virtual Social Network,” inMobiCom’05, Cologne, Ger-many, Sep. 2005.

[40] C. Drula, C. Amza, F. Rousseau, , and A. Duda, “Adaptive Energy Con-serving Algorithms for Neighbor Discovery in OpportunisticBluetoothNetworks,” IEEE JSAC, vol. 25, no. 1, pp. 96–107, Jan. 2007.

[41] P. Maymounkov and D. Mazieres, “Rateless Codes and Big Downloads,”in IPTPS’03, Berkeley, CA, Feb. 2003.

[42] P. Chou, Y. Wu, and K. Jain, “Practical Network Coding,”in Allerton’03,Allerton, Oct. 2003.

[43] C. Gkantsidis and P. Rodriguez, “Network Coding for Large ScaleContent Distribution,” inINFOCOM’05, Miami, FL, USA, Mar. 2005.

[44] T. Pering, Y. Agarwal, R. Gupta, and R. Want, “CoolSpots: Reducing thePower Consumption of Wireless Mobile Devices with Multiple RadioInterfaces,” inMobiSys’06, Uppsala, Sweden, Jun. 2006.

[45] R. Gummadi, D. Wetherall, B. Greenstein, and S. Seshan, “Understand-ing and Mitigating the Impact of RF Interference on 802.11 Networks,”in SIGCOMM’07, Kyoto, Japan, Aug. 2007.

[46] M. B. Shoemake, “Wi-Fi (IEEE 802.11b) and Bluetooth coexistenceissues and solutions for the 2.4 GHz ISM Band,” Texas Instruments,Tech. Rep., Feb. 2001.

[47] K. Shuaib, M. Boulmalf, F. Sallabi, and A. Lakas, “Performanceanalysis: Co-existence of IEEE 802.11g with Bluetooth,” inWOCN’05,Dubai, United Arab Emirates UAE, Mar. 2005.

[48] A. Akella, G. Judd, S. Seshan, and P. Steenkiste, “Self Management inChaotic Wireless Deployments,” inMobiCom’05, Cologne, Germany,Sep. 2005.

[49] R. Musaloiu-E. and A. Terzis, “Minimising the Effect of WiFi interfer-ence in 802.15.4 Wireless Sensor Networks,”Int. J. Sensor Networks,vol. 3, no. 1, pp. 43–54, 2008.

[50] “Avoiding RF Interference between WiFi and Zigbee,” http://www.xbow.com/Support/Supportpdf files/ZigBeeandWiFiInterference.pdf.

[51] BGR, “WLAN Interference with IEEE 802.15.4,” Zensys Inc., Tech.Rep., March 2007.

[52] J. Hou, B. Chang, D.-K. Cho, and M. Gerla, “Minimizing 802.11Interference on Zigbee Medical Sensors,” inBodyNets’09, Los Angeles,CA, Apr. 2009.

[53] “Bluetooth-enabled Equipment Shipments to Hit 800 million this Year,”http://www.mobiletechnews.com/info/2007/10/14/130333.html.

[54] E. Ferro and F. Potorti, “Bluetooth and Wi-Fi Wireless Protocols: aSurvey and a Comparison,”IEEE Wireless Communications, vol. 12,no. 1, pp. 12–26, 2005.

[55] S. Souissi and E. F. Meihofer, “Performance Evaluation of a BluetoothNetwork in the Presence of Adjacent and Co-channel Interference,”in IEEE Emerging Technologies Symposium’00, Richardson, TX, Apr.2000.

[56] “Haggle Project,” http://www.haggleproject.org.[57] R. J. Punnoose, R. S. Tseng, and D. D. Stancil, “Experimental Results

for Interference between Bluetooth and IEEE 802.11b DSSS Systems,”in VTC’01, Rhodes, Greece, May 2001.

[58] S. Mander, D. Reading-Picopoulos, and C. Todd, “Evaluating theAdaptive Frequency Hopping Mechanism to Enable Bluetooth –WLANCoexistence,” inLCS’03, London, UK, Sep. 2003.