0018-9162/02/$17.00 © 2002 IEEE56 Computer

Rover: Scalable Location-AwareComputing

Consider a group touring the museums inWashington, D.C. The group arrives at aregistration point, where each personreceives a handheld device with audio,video, and wireless communication capa-

bilities—an off-the-shelf PDA available in the mar-ket today. A wireless-based system tracks the loca-tion of these devices and presents relevantinformation about displayed objects as the usermoves through the museum. Users can query theirdevices for maps and optimal routes to objects ofinterest. They can also use the devices to reserveand purchase tickets to museum events later in theday. The group leader can send messages to coor-dinate group activities.

The part of this system that automatically tai-lors information and services to a mobile user’slocation is the basis for location-aware comput-ing. This computing paradigm augments the moretraditional dimensions of system awareness, such astime-, user-, and device-awareness. All the tech-nology components to realize location-aware com-puting are available in the marketplace today.What has hindered the widespread deployment oflocation-based systems is the lack of an integra-tion architecture that scales with user populations.

We developed Rover technology to meet thisneed.1 We have completed the initial implementa-tion of a system based on it and have validated itsunderlying software architecture, which achievessystem scalability through fine-resolution, applica-tion-specific resource scheduling at the servers andnetwork.

ROVER ARCHITECTURERover technology tracks the location of system

users and dynamically configures application-levelinformation to different link-layer technologies andclient-device capabilities. A Rover system representsa single domain of administrative control, managedand moderated by a Rover controller. Figure 1shows a large application domain partitioned intomultiple administrative domains, each with its ownRover system—much like the Internet’s DomainName System.2

End users interact with the system through Roverclient devices—typically wireless handheld unitswith varying capabilities for processing, memoryand storage, graphics and display, and networkinterfaces. Rover maintains a profile for each device,identifying its capabilities and configuring contentaccordingly. Rover also maintains end-user profiles,defining specific user interests and serving contenttailored to them.

A wireless access infrastructure provides connec-tivity to the Rover clients. In the current implemen-tation, we have defined a technique to determinelocation based on certain properties of the wirelessaccess infrastructure. Although Rover can leveragesuch properties of specific air interfaces,1 its loca-tion management technique is not tied to a particu-lar wireless technology. Moreover, different wirelessinterfaces can coexist in a single Rover system or indifferent domains of a multi-Rover system. Softwareradio technology3 offers a way to integrate the dif-ferent interfaces into a single device. This wouldallow the device to easily roam between different

Rover technology adds a user’s location to other dimensions of systemawareness, such as time, user preferences, and client device capabilities.The software architecture of Rover systems is designed to scale to largeuser populations.

SumanBanerjeeSulabh AgarwalKevin KamelAndrzejKochutChristopherKommareddyTamerNadeemPankajThakkarBao TrinhAdel YoussefMoustafaYoussefRonald L.LarsenA. UdayaShankarAshokAgrawalaUniversity of Maryland, College Park

C O V E R F E A T U R E

Rover systems, each with different wireless accesstechnologies.

A server system implements and manages Rover’send-user services. The server system consists of fivecomponents:

• The Rover controller is the system’s “brain.”It manages the different services that Roverclients request, scheduling and filtering thecontent according to the current location andthe user and device profiles.

• The location server is a dedicated unit thatmanages the client device location serviceswithin the Rover system. Alternatively, appli-cations can use an externally available loca-tion service, such as the Global PositioningSystem (GPS).4

• The streaming-media unit manages audio andvideo content streamed to clients. Many oftoday’s off-the-shelf streaming-media units canbe integrated with the Rover system.

• The database stores all content delivered to theRover clients. It also serves as the stable storefor the user and client states that the Rovercontroller manages.

• The logger interacts with all the Rover servercomponents and receives log messages fromtheir instrumentation modules.

The server system exports a set of well-definedinterfaces through which it interacts with the het-erogeneous world of users and devices. Third-partydevelopers can use these interfaces to develop appli-cations that interact with the system.

For multi-Rover systems, we define protocolsthat allow interaction between the domains. This

enables users registered in one domain to roam intoother domains and still receive services from the sys-tem.

ROVER SERVICESRover offers two kinds of services to its users.

We refer to them as basic data services and trans-actional services.

• Basic data services use text, graphics, audio,and video formats. Users can subscribedynamically to specific data componentsthrough the device user interface. Rover filtersthe available media formats according to thedevice’s capabilities. The basic data serviceinvolves primarily one-way interaction.

• Transactional services have commit semanticsthat require coordinating state between theclients and Rover servers. E-commerce inter-actions are examples of this service class.

Location is an important attribute of all objectsin Rover. Several techniques exist for estimating anobject’s location, including the GPS and radio-fre-quency techniques based on signal strength or sig-nal propagation delays. The choice of techniquesignificantly affects the granularity and accuracyof the location information. Rover therefore uses atuple of value, error, and time stamp to identify anobject’s location.1 The value is an estimate of theobject’s location (either absolute or relative to somewell-known location). The error identifies theuncertainty in the estimate. The time stamp iden-tifies when the estimate was made.

The accuracy required of location informationdepends on the context of its use. For example, an

October 2002 57

Database

Streaming-media unit

Localnetwork

Internet/Public Switched

Telephone Network

Location server

Rover controller

Base stations

Client devices

Rover system

Firewall

Rover system

Intercontroller protocolsRover system

Logger

Figure 1. Roverphysical architec-ture in the contextof a multi-Roversystem. The Rovercontroller managessystem services,including communi-cation betweenindividual Roversystems.

58 Computer

accuracy of 4 meters is adequate to providewalking directions from the user’s currentlocation to another location about 500 metersaway. However, it is inadequate for locatinga particular painting on a museum walldirectly in front of the user. Obviously, theaccuracy of location information improvessignificantly with direct user input. For exam-ple, the user can directly input a location ona map displayed on his or her device.

Rover includes support for operations to fil-ter, zoom, and translate map display. Filter opera-tions are keyed to a set of attributes that identifycertain properties of map objects. Rover generatesthe filters based on the user’s context. The filters selectand map the appropriate object subset for display.For example, one user may be interested only in therestaurants in a specific area, while another wants toview only museum and exhibition locations.

A displayed map’s zoom level identifies its granu-larity. The user’s context, consisting of a location andprofile, determines its default setting. For example,inside a museum, the map shows a detailed museumlayout. When the user steps outside, the displayzooms out to an area map with points of interest inthe geographic vicinity. Additionally, the user canchoose to alter the major zoom level through explicitinput. In either case, appropriate filters are used toselectively display different objects on a map at anyzoom level. Rover automatically translates the mapdisplayed on the client device as the user moves to anew region.

SYSTEM SCALABILITYTwo potential bottlenecks can hinder the system’s

scalability. One is the server system, which musthandle a large number of client requests with tightreal-time constraints. The other is the wirelessaccess points, which have limited bandwidth.

To handle the large volume of real-time requeststhat users generated, we developed the actionmodel, a fine-grained, real-time, application-spe-cific architecture that allows Rover systems to scaleto large user populations.

To make its implementation more efficient, wedivided the Rover server components into two classesbased on the user request volumes they handle:

• primary servers directly communicate with theclients and therefore directly handle large vol-umes of user requests. They include the Rovercontroller, location server, and streamingmedia unit; and

• secondary servers, which communicate only

with primary servers to provide back-end sys-tem capabilities, include the Rover databaseand logger.

Only the primary servers need to implement theAction model.

Action modelThe Action model avoids the overhead of thread

context switches and allows more efficient sched-uling of execution tasks. The Rover controllerimplements this architecture.

In the action model, scheduling occurs in“atomic” units called actions. An action is a smallpiece of code that has no intervening I/O opera-tions: Once an action begins execution, anotheraction cannot preempt it. Consequently, given aspecific server platform, it is easy to accuratelybound an action’s execution time.

A server operation is a transaction, either client- oradministrator-initiated, that interacts with the Rovercontroller: Examples in the museum scenario wouldinclude registerDevice, getRoute, and locateUser. Aserver operation consists of a sequence—or moreprecisely, a partial order—of actions interleaved withasynchronous I/O events. Each server operation hasone action for each kind of I/O event response.

A server operation at any given time has zero ormore actions eligible for execution and is in one ofthree states:

• ready-to-run—at least one of the server oper-ation’s actions is eligible for execution but noneis executing;

• running—one action is executing (in a multi-processor setup, several of the operation’sactions can execute simultaneously); or

• blocked—the server operation is waiting foran asynchronous I/O response, and no actionsare eligible to be executed.

An action controller uses administrator-definedpolicies to decide the execution order of the eligibleactions. The scheduling policy can be simple andstatic, such as priorities assigned to server opera-tions. It also can be time based, such as earliest-deadline-first or a function of real-time costs. In anycase, the controller picks an eligible action and exe-cutes it to completion, then repeats the process—waiting only if there are no eligible actions,presumably because all server operations are wait-ing for I/O completions.

A simple action API defines the management andexecution of actions:

A task-schedulingarchitecture

handles the largevolume of real-time

requests that users generate.

• init (action id, function ptr)—initializes a new action (identified by actionid) for a server operation. Function ptridentifies the function (or piece of code) asso-ciated with the action.

• run (action id, function parame-ters, deadline, deadline failedhandler ptr)—marks the action as eligibleto run. Function parameters are theparameters used in executing this instance ofthe action. Deadline is optional and indi-cates the time (relative to the current time) bywhich the action should execute; the deadlineis soft—that is, violating it leads to somepenalty but not system failure. If the actioncontroller cannot execute the action within thedeadline, it will execute the function indicatedby deadline failed handler ptr. Thisparameter can be null, indicating that no com-pensatory steps are needed.

• cancel (action id, cancel handlerptr)—cancels a ready-to-run action, providedit is not executing. Cancel handler ptrindicates a cleanup function; it can be null.

Actions versus threadsThere are several ways to implement the Rover

controller using a thread model. For example, eachserver operation could have a separate thread, oreach user could have a separate thread handlingall its operations. Both of these approaches implya large number of simultaneously active threads aswe scale to large user populations, resulting inlarge overheads for thread switching.

A more sensible approach is to create a small setof “operator” threads that execute all operations—for example, one thread for all registerDevice oper-ations, one for all locateUser operations, and so on.

This approach reduces the thread-switchingoverhead, but there are drawbacks. For one, thethreads package restricts the ability to optimizescheduling, especially in time-based scheduling.More importantly, each operator thread executes

its set of operations in sequence, which severelylimits the ability to optimally schedule the eligibleactions within an operation and across operations.Of course, each thread could keep track of all itseligible actions and do scheduling at the actionlevel, but this essentially re-creates the action modelwithin each thread.

We compared the performance of action-basedversus more traditional thread-based systems intwo kinds of server operations, shown in Figure 2:

• Scenario A—a computation-intensive scenariowith 10,000 processor-bound server opera-tions, in which each server operation has threecompute blocks, interleaved with two file-write operations. In each of these server oper-ations, the second and third I/O computeblocks need not await completion of the priorfile I/O write operation.

• Scenario B—an I/O-intensive scenario with100 I/O-bound server operations, in whicheach server operation has three computeblocks, interleaved with two network I/Ooperations. In each of these server operations,the second and third compute blocks can startonly after the prior network I/O operation fin-ishes. We use UDP to implement network I/Ointeraction. Since our focus is on the compar-ison of action-based versus thread-based sys-tems, we avoid issues of packet loss andretransmissions by considering only thoseexperiments in which no UDP packets werelost in the network.

We used two execution platforms: M1 and M2.M1 runs Linux on a 600-MHz Intel Pentium IIIprocessor with 96 Mbytes of RAM. M2 runs Solarison a Sun Ultra 5 with a 333-MHz Sparc processorand 128 Mbytes of RAM. For the thread-basedimplementation, we used the LinuxThreads libraryfor the M1 platform and the Pthreads library forthe M2 platform; both are implementations of thePosix 1003.1c threads package.

October 2002 59

Compute block A…

Compute block A…

Compute block A…

Log todisk

Database

ServerOperation A{

File I/O

File I/O

write(…);

write(…);

}

Compute block B

Compute block B

Compute block B

ServerOperation B{

Network I/O

Network I/O

}

recvfrom(…);sendto(…);

recvfrom(…);sendto(…);

Figure 2. Serveroperations used inthe experimentalevaluation of theaction model. Sce-nario A (left) inter-leaves computationwith file-write oper-ations. Scenario B(right) interleavescomputation withnetwork I/O interac-tions.

60 Computer

The total execution time for the three computeblocks in each server operation A was 0.1518 msfor M1 and 0.9069 ms for M2. The ping networklatency for the network I/O in server operation Bvaried between 30 and 35 ms.

In the action-based scenarios, we implementedeach compute block as a separate action. In thethread-based scenarios, we experimented with dif-ferent numbers of threads, executing each threadan equal number of server operations for perfectload balancing between the different threads.

Table 1 shows the overheads obtained in eachcase, where overhead is the total execution timeminus the fixed, identical, and unavoidable com-putation and communication costs for the two sce-narios. The results represent the mean executionoverheads of 10,000 server operations in scenario Aand 100 server operations in scenario B, which wererequired to obtain low variance. The computation-intensive server operations show little performancegain in trying to overlap computation with file I/Ocommunication—not enough to justify the over-head of a multithreaded implementation. Amongthe thread-based implementations, a thread-basedsystem with a single thread performs best.

For the I/O-intensive server operations, a multi-threaded implementation is useful because com-

putation and communication can overlap. Conse-quently, the best performance for the thread-basedsystem occurs with the maximum number ofthreads—specifically, one thread for each serveroperation.

However, as Table 1 shows, the action-basedimplementation in both scenarios has about anorder of magnitude lower overhead compared withthe best thread-based implementation.

COMMUNICATION INTERFACESFor the wireless interface to client devices, we

considered two link-layer technologies: IEEE802.11 and Bluetooth. Bluetooth is power efficientand therefore better at conserving client batterypower. According to current standards, Bluetoothcan provide bandwidths up to 2 Mbps. In contrast,IEEE 802.11 is less power efficient but widelydeployed and currently provides bandwidths up to11 Mbps. In areas where these high-bandwidthalternatives are not available, Rover client deviceswill use the lower bandwidth interfaces that cellu-lar wireless technologies provide.

Figure 3 shows how Rover’s controller interactswith other parts of the system and with the exter-nal world. The controller uses the location inter-face to query the location service about the

Client devices

Rover controller

Logger

User infobase

Actions

Location service

Transportinterface

Locationinterface

Actionscheduler and

controller

User/deviceprofile

management

Instrumentation

Contentmanagement

Securitypolicies

Back-endinterface

Administratorinterface

Contentinterface

Billing/e-commerce

servicesAdministrator Content

provider

Serverassistants’interface

Content infobase

Figure 3. Rover’slogical architecture.The Rover controllerinteracts with theexternal worldthrough interfacesfor location, trans-port, administration,back-end services,content, andsecondary serverassistants.

Table 1. Comparisons of overheads for action-based and thread-based systems (in milliseconds).

Figure 2 Machine Action- Thread-based (threads used)scenario specifications based 1 5 10 50 100

A M1: Pentium/Linux 24.27 299.36 299.93 300.46 304.50 310.31 A M2: Sparc/Solaris 62.82 1,000.90 1,012.54 1,041.60 1,012.83 1,031.25 B M1 controller, 11.61 3,711.94 1,302.20 1,011.49 893.10 728.30

M2 database

positions of client devices and the transport inter-face to identify data formats and interaction pro-tocols for communicating with the clients. It usesthe server assistants’ interface to interact with sec-ondary servers like the database and the streaming-media unit and the back-end interface to interactwith external services, such as credit card autho-rization for e-commerce purchases. Third-partyproviders typically offer these external services.

System administrators can use the admin inter-face to oversee the Rover system, including moni-toring the Rover controller, querying client devices,updating security policies, issuing system-specificcommands, and so on.

The content interface lets content providersupdate the information and services that the Rovercontroller serves to client devices. Having a separatecontent interface decouples the data from the con-trol path.

INITIAL IMPLEMENTATIONWe have successfully built Rover prototype sys-

tems and tested them in both indoor and outdoorenvironments at the University of Maryland, CollegePark. A preliminary Windows-based test imple-mentation ran Windows 2000 for the controller andWindows CE for the client devices. However, thecurrent implementation runs under Linux.

We implemented the Rover controller on an IntelPentium machine running Red Hat Linux 7.1 andthe clients on Compaq iPAQ model H3650 PocketPCs running Familiar’s Linux distributions for PDAs(http://familiar.handhelds.org). Wireless access isover IEEE 802.11 wireless LANs. Each CompaqiPAQ includes a wireless card that attaches to thedevice through an expansion sleeve.

We have experimented with a set of eight clientdevices and have tested various functionalities ofthe system.

For our outdoor experiments, we interfaced aGarmin e-Trex (http://www.garmin.com/products/etrex/) GPS device to the Compaq iPAQs andobtained device location accuracy between 3 and 4meters. Figure 4 shows the iPAQ Rover client’sdefault display, which marks different user locationsas dots on an area map.

We implemented the indoor Rover system in a26.6 × 70-meter area on the fourth floor of theComputer Science Department building. In thisimplementation, the location service uses signal-strength measurements from different base stations.About 12 base stations are distributed all over thebuilding, and the client device can typically receivebeacons from five or six of them. We get an accu-

racy of about 2 meters in this environment, usingvery simple signal-strength estimation techniques.

In both these environments, we implemented thebasic Rover system functionality, which included

• user activation and deactivation procedures;• device registration and deregistration proce-

dures;• periodic broadcast of events of interest from

the Rover controller to the users in specificlocations;

• unicasts from the controller according to user-specified time, location, or context-dependentconditions;

• both simple-text-messaging and voice-chatinteraction between users; and

• an administrator’s console, allowing a globalview of all system users and their locations.

The administrator can directly interact with allusers or a specific subset based on location or otheruser attributes. Users have the option of makingtheir location visible to other users.

R over is currently available as a deployable sys-tem using specified technologies, both indoorsand outdoors. Ultimately, our goal is to pro-

vide a completely integrated system that uses dif-ferent technologies and allows a seamless experienceof location-aware computing to clients as they movethrough the system. With this in mind, we have var-ious short- and long-term projects:

• Experiment with a wider range of clientdevices, both those with limited text andgraphics display capabilities and those that can

October 2002 61

Figure 4. Roverclient defaultdisplay for CompaqiPAQ marks userlocations as dots onan area map.

62 Computer

support richer functionality, such as location-aware streaming video services.

• Integrate more wireless air interfaces with theRover system. Bluetooth is a logical next tech-nology to experiment with. In the longer term,we expect to work with cellular providers todefine and implement mechanisms that will letRover clients interact over the cellular inter-face.

• Implement more location-determination tech-niques. We are experimenting with new mech-anisms for better location estimation, includinga signal propagation delay-based technique,called PinPoint Technology, developed at theUniversity of Maryland.

• Implement the multi-Rover system.• Deploy Rover campus-wide at the University

of Maryland, College Park. Initially, we expectto deploy independent Rover systems to serveclients of specific departments. These systemswill subsequently interact using the inter-Rovercontroller protocols of a multi-Rover system.We will colocate the Rover controllers with theWeb servers and integrate the content man-agement for both systems.

We believe that Rover technology will greatlyenhance the user experience in many places, includ-ing museums, amusement and theme parks, shop-ping malls, game fields, offices, and businesscenters. We designed the system specifically to scaleto large user populations and expect its benefits toincrease with them. �

References1. S. Banerjee et al., Rover Technology: Enabling Scal-

able Location-Aware Computing, tech. reports UMI-ACS-TR 2001-89 and CS-TR 4312, Dept. ComputerScience, Univ. of Maryland, College Park, Md., Dec.2001.

2. P. Mockapetris, “Domain Names: Implementationand Specification,” Internet Engineering Task Force,RFC 1035 (Internet standard), Nov. 1987;http://www.ietf.org/rfc/rfc1035.txt.

3. J. Mitola, “The Software Radio Architecture,” IEEEComm., vol. 5, May 1995, pp. 26-38.

4. B. Hofmann-Wellenhof, H. Lichtenegger, and J.Collins, GPS: Theory and Practice, Springer-Verlag,Wien, N.Y., 1997.

Suman Banerjee is a PhD candidate in the Depart-ment of Computer Science at the University ofMaryland, College Park. He received an MS in

computer science from the University of Maryland,and a BTech in computer science and engineeringfrom the India Institute of Technology in Kanpur.He is a student member of the IEEE. Contact himat suman@cs.umd.edu.

Sulabh Agarwal is a software developer with EpicSystems Corp., Madison, Wisconsin. He receivedan MS in computer science from the University ofMaryland, College Park. He received a BTech fromthe India Institute of Technology, Delhi.

Kevin Kamel is a faculty research assistant at theUniversity of Maryland Institute for AdvancedComputer Studies, College Park. He received a BSin computer engineering from the University ofMaryland, College Park. Contact him at kamelkev@umiacs.umd.edu.

Andrzej Kochut is a PhD student in computer sci-ence at the University of Maryland, College Park.He received an MS in computer science from theUniversity of Warsaw, Poland. Contact him atkochut@cs.umd.edu.

Christopher Kommareddy is a PhD student in theDepartment of Electrical and Computer Engineer-ing, University of Maryland, College Park, wherehe received an MS in electrical and computer engi-neering. Contact him at kcr@cs.umd.edu.

Tamer Nadeem is a PhD student in computer sci-ence at the University of Maryland, College Park.He received an MS in computer science from theUniversity of Maryland, and an MS and a BS incomputer science from Alexandria University,Egypt. He is a student member of the IEEE and theACM. Contact him at nadeem@cs.umd.edu.

Pankaj Thakkar is a software engineer withAskJeeves. He received an MS in computer sciencefrom the University of Maryland, College Park, anda BTech in computer science and engineering fromthe India Institute of Technology, Delhi.

Bao Trinh is a faculty research assistant in theDepartment of Computer Science at the Universityof Maryland, College Park, where he received anMS in computer science. Contact him at bao@mindlab.umd.edu.

Adel Youssef is a PhD student in computer scienceat the University of Maryland, College Park. Hereceived an MS in computer science from the Uni-

versity of Maryland, College Park, and an MS andBS in computer science from Alexandria Univer-sity, Egypt. Contact him at adel@cs.umd.edu.

Moustafa Youssef is a PhD candidate in computerscience at the University of Maryland, CollegePark. He received an MS in computer science fromthe University of Maryland, and an MS and a BS incomputer science from Alexandria University,Egypt. He is a student member of the IEEE, theIEEE Computer Society, and the IEEE Communi-cation Society. Contact him at moustafa@cs.umd.edu.

Ronald L. Larsen is dean of the School of Infor-mation Sciences at the University of Pittsburgh. Hereceived a PhD in computer science from the Uni-versity of Maryland. He is a member of the IEEE,the IEEE Computer Society, and the ACM.

A. Udaya Shankar is a professor in the computerscience department and the Institute for AdvancedComputer Studies at the University of Maryland,College Park. He received a PhD in electrical engi-neering from the University of Texas at Austin.Contact him at shankar@cs.umd.edu.

Ashok Agrawala is a professor in the Departmentof Computer Science at the University of Marylandand holds joint positions with the University ofMaryland Institute for Advanced Computer Stud-ies (UMIACS) and the Department of ElectricalEngineering. He received an MS and a PhD inapplied mathematics from Harvard University andan ME and a BE in electrical engineering from theIndian Institute of Sciences, Bangalore. He is a Fel-low of the IEEE and a member of the ACM, AAAS,and Sigma Xi. Contact him at agrawala@cs.umd.edu.

October 2002 63