SmartSantander: IoT experimentation over a smart city testbed

Computer Networks 61 (2014) 217–238

Contents lists available at ScienceDirect

Computer Networks

journal homepage: www.elsevier .com/ locate/comnet

SmartSantander: IoT experimentation over a smart city testbed

1389-1286/$ - see front matter � 2013 Elsevier B.V. All rights reserved.http://dx.doi.org/10.1016/j.bjp.2013.12.020

⇑ Corresponding author. Address: Universidad de Cantabria, Laborato-rios de I+D de Telecomunicacion, 39005 Santander, Spain. Tel.: +34942200914; fax: +34 942201488.

E-mail addresses: [email protected] (L. Sanchez), [email protected] (L. Muñoz), [email protected] (J.A. Galache),[email protected] (P. Sotres), [email protected] (J.R. Santana),[email protected] (V. Gutierrez), [email protected](R. Ramdhany), [email protected] (A. Gluhak), [email protected] (S. Krco), [email protected] (E. Theodoridis), [email protected] (D. Pfisterer).

Luis Sanchez a,⇑, Luis Muñoz a, Jose Antonio Galache a, Pablo Sotres a, Juan R. Santana a,Veronica Gutierrez a, Rajiv Ramdhany b, Alex Gluhak c, Srdjan Krco d, Evangelos Theodoridis e,Dennis Pfisterer f

a University of Cantabria, Plaza de la Ciencia s/n, Santander 39005, Spainb Lancaster University, South Drive, Lancaster LA1 4WA, United Kingdomc University of Surrey, Faculty of Engineering & Physical Sciences, Guildford GU2 7XH, United Kingdomd Ericsson Serbia, Milentija Popovica 5a/V, Belgrade 11070, Serbiae Research Academic Computer Technology Institute, Patras 26500, Greecef University of Lübeck, Ratzeburger Allee 160, Lúbeck 23562, Germany

a r t i c l e i n f o a b s t r a c t

Article history:Received 1 July 2012Received in revised form 5 August 2013Accepted 6 December 2013Available online 27 December 2013

Keywords:Internet of ThingsExperimentationResearchSmart cityTestbed

This paper describes the deployment and experimentation architecture of the Internet ofThings experimentation facility being deployed at Santander city. The facility is imple-mented within the SmartSantander project, one of the projects of the Future InternetResearch and Experimentation initiative of the European Commission and represents aunique in the world city-scale experimental research facility. Additionally, this facility sup-ports typical applications and services of a smart city. Tangible results are expected toinfluence the definition and specification of Future Internet architecture design from view-points of Internet of Things and Internet of Services. The facility comprises a large numberof Internet of Things devices deployed in several urban scenarios which will be federatedinto a single testbed. In this paper the deployment being carried out at the main location,namely Santander city, is described. Besides presenting the current deployment, in thisarticle the main insights in terms of the architectural design of a large-scale IoT testbedare presented as well. Furthermore, solutions adopted for implementation of the differentcomponents addressing the required testbed functionalities are also sketched out. The IoTexperimentation facility described in this paper is conceived to provide a suitable platformfor large scale experimentation and evaluation of IoT concepts under real-life conditions.

� 2013 Elsevier B.V. All rights reserved.

1. Introduction

The Internet of Things (IoT) has recently risen in prom-inence due to significant advances in enabling device-

technologies, such as Radio Frequency Identification (RFID)tags and readers, Near Field Communication (NFC) devices,and embedded sensor/actuator nodes. With this emer-gence of interconnected devices and services, the IoT hasbeen touted to become the next major extension to thecurrent fixed and mobile networking infrastructures. Re-cent predictions [1] foresee that IoT will form an essentialpart of the Future Internet (FI), as its connected deviceswill outnumber the computers and mobile devices utilisedby human users by orders of magnitude. If such a scenariounfolds, it is not hard to conclude that the design of the FIand its architecture will be strongly influenced by therequirements of the IoT.

http://crossmark.crossref.org/dialog/?doi=10.1016/j.bjp.2013.12.020&domain=pdf

http://dx.doi.org/10.1016/j.bjp.2013.12.020

mailto:[email protected]














http://dx.doi.org/10.1016/j.bjp.2013.12.020

http://www.sciencedirect.com/science/journal/13891286

http://www.elsevier.com/locate/comnet

218 L. Sanchez et al. / Computer Networks 61 (2014) 217–238

However, the IoT has many facets and exceeds thescope of currently-available deployments mainly due totwo issues. Firstly, current IoT-like deployments are essen-tially closed and vertically-integrated solutions tailored tospecific application domains. Secondly, new technologiesand solution-optimisations are constrained in terms ofapplicability to the context under which they have beentested. For example, the research on one of the predomi-nant areas of IoT, namely Wireless Sensor Networks(WSN), the experimentally-driven one in particular, hasprimarily focused on advances within WSN islands, provid-ing optimised solutions for the resource-constrained de-vices of which they are composed.

Realising the vision of the IoT, therefore, requires anagreed architectural reference model, based on open proto-col solutions and key enabling services that enable interop-erability of deployed IoT resources across differentapplication domains and contribute to horizontal re-useof the deployed infrastructure [2,3]. Additionally, a majorgoal of IoT research is to integrate WSN into a globallyinterconnected infrastructure, moving from the currentlyexisting Intra-net to a real Inter-net of Things [4].

Based on this precept, the SmartSantander project [5]mainly targets the creation of a European experimentaltest facility for the research and experimentation of archi-tectures, key enabling technologies, services and applica-tions for the IoT in the context of a smart city. Thisfacility aims to leverage key IoT-enabling technologiesand to provide the research community with a unique-in-the-world platform for large scale IoT experimentationand evaluation under real-world operational conditions.Setting an experimental facility into a city context has spe-cial significance for IoT research for three main reasons: (1)the pervasiveness of IoT-based technologies that form partof the Smart City infrastructure fabric and the realism ofexperimentation achieved through their use; (2) the infra-structural scale and heterogeneity (devices, protocols andservices), and the population of users that are key enablersfor a broad range of experimentation; (3) the diversity ofproblems and application domains in dense techno-socialeco-systems such as Smart Cities that provide invaluablesources of challenging functional and non-functionalrequirements. As their infrastructure exhibit these proper-ties Smart Cities provide excellent environments and are,indeed, catalysts for, IoT research.

Four contributions are presented in this paper. Firstly,this paper describes the architectural reference model foropen real-world IoT experimentation facilities defined inthe SmartSantander project. More specifically, it highlightsthe key challenges addressed in establishing an urban city-scale IoT experimentation facility and illustrates the plat-form usage though a representative set of implementeduse cases. Secondly, as the deployment of large-scale dis-tributed multi-purpose multi-stakeholder IoTinfrastruc-ture is complexity-fraught and not risk-averse (often acompromise over platform capabilities, overall usefulnessand cost), we regard the experience gained from our phys-ical deployment process as another valuable contribution.In this respect, the paper provides detailed insight on theactual physical deployment of a large-scale heterogeneousIoT infrastructure over the city of Santander. The third

contribution consists on presenting the solutions adoptedfor making the facility usable for the experimenters. TheIoT experimentation support framework relies on the inte-gration of existing components from SENSEI [6], WISEBED[7] and Telefonica Ubiquitous Sensor Networks (USN) Plat-form [8]. However, due to SmartSantander’s uniquerequirements we have implemented additional mecha-nisms to address support for large-scale, horizontality, het-erogeneity, mobility testing, as well as security, privacy,and trust. Finally, describing the different supported exper-imentation capabilities of the deployed facility is the lastcontribution presented in the paper.

The paper is organised as follows. In Section 2 relatedwork and facilities for experimental IoT research are pre-sented. Section 3 describes the SmartSantander platform’shigh-level architecture emphasising the main require-ments and testbed singularities that have been consideredfor the realisation of the experimental facility. Section 4provides insights on the deployed IoT infrastructure atthe city of Santander. The mechanisms that have beenimplemented for the testbed management in terms of re-source discovery and testbed monitoring are described inSection 5. Section 6 presents the solutions developed forsupporting the experimentation on top of the SmartSant-ander infrastructure. Finally, Section 7 concludes the paperpresenting some of the work to be accomplished in thenear future.

2. Related work

Despite significant technological advances, difficultiesassociated with the evaluation of IoT solutions under real-istic conditions in real world experimental deploymentsstill hamper their maturation and significant roll out. Theuse of experimental facilities is considered a key enablerto facilitate the design and evaluation of novel IoT systemsthat work more reliably under realistic operational condi-tions and for their evaluation. A plethora of testbeds haveemerged in the past decade. Many of these are lab-basedtestbeds which suffer from various shortcomings such asrealism of experimentation environment, limitations ofscale and mobility testing support, heterogeneity of under-lying experimentation substrate or the lack end userinvolvement in IoT experimentation. The reader is referredto [9] for a more detailed survey and analysis of these test-beds. Our work aims to overcome several of these short-comings and provide a facility for experimentation withIoT deployments in urban environments and SmartCityservices and applications that can be enabled on top ofthese.

Existing efforts that most closely match our target envi-ronment are smart city deployments such as Oulu SmartCity (outdoor sensor nodes) [10] or CitySense (embeddedPCs with WiFi interfaces deployed on lamp posts) [11].Although they offer IoT devices for service enablement(Oulu Smart City) or experiments (Citysense), they do notadequately provide provisions addressing experimentationrequirements such as IoT device heterogeneity, support ofrealistic mobility scenarios and lack adequate scale neces-sary for carrying out large experiments or user trials.

L. Sanchez et al. / Computer Networks 61 (2014) 217–238 219

Furthermore they are not designed with the intent to serveboth as service provisioning and experimentationinfrastructures.

Some of these aforementioned requirements are par-tially tackled in lab based testbeds. For example the Kan-seiGeni [12], SensLab [13] and iLab.t [14] testbedsprovide adequate heterogeneity by offering different moteplatforms at the IoT tier and GW tier (KanseiGeni) devicesfor experimentation. However, the target deploymentenvironment differs from urban outdoor environments,so do the underlying tools or mechanisms that have beendesigned to manage these. Although the scale of these test-beds is significant for indoor testbeds, a city deploymentcan easily exceed these numbers by an order of magnitude.

Similarly WISEBED [15] offers large IoT device hetero-geneity by providing support for testbed federation. In factour framework builds upon WISEBED and its underlyingcapabilities and extends these for the use in a larger scaleout-door environment. For example our work adds supportfor wireless reprogramming of experimentation nodes, im-proved usability for experimenters for selection adequateexperimentation resources and increased robustness andlower configuration overhead for management of testbednodes.

Unlike the other testbeds, our testbed provides accessto mobile experimentation nodes that are embedded inreal urban infrastructures, e.g. buses or public service vehi-cles, in order to allow more realistic mobility experiments.Furthermore our testbed has the ability to involve real cit-izens into the experimentation life cycle.

3. IoT testbed requirements and architecture

This section, first, elaborates on the requirements forproviding a rich IoT experimentation environment andaddressing many open research challenges in the area ofIoT testbeds. Based on these requirements, it provides anoverview of the architecture of the SmartSantander testbedand the features of the platform.

3.1. Design considerations

As reported in previous work [9], existing network test-bed facilities have several limitations that make them failto provide adequate support for the emerging require-ments of experimental IoT research. The SmartSantanderfacility offers a variety of properties and features to over-come many of these shortcomings and integrates theminto a holistic experimentation environment. In the follow-ing we highlight the key requirements along multipledimensions and provide considerations on how the Smart-Santander facility addresses them.

3.1.1. Experimentation realismLive testbeds provide a degree of experimentation real-

ism that even the most detailed simulation cannot achieve[16]. We argue that, for IoT-technology experimentation,even lab-based testbeds do not suffice to evaluate researchprototypes under realistic conditions and to facilitate theirtransfer into real world deployments. IoT technologies are

heavily dependent on ambient environmental conditionsin which they are deployed, including the service logic ofthe diverse IoT applications. Smart cities in particular arean important emerging domain for the IoT in which a mul-titude of application areas intersect and therefore repre-sent a realistic/fertile experimentation medium for IoTtechnologies. To this end, the SmartSantander facility con-sists of an urban deployment within the city of Santanderand other partner sites. This enables more realistic experi-mentation and faster maturation of IoT solutions for themass market.

3.1.2. ScaleReal-world experimentation in a target deployment

environment also requires experimentation at adequatescale. While smaller-scale testbeds with populations oftens up to hundreds of nodes were sufficient for mostWSN experiments, many IoT experiments demand an or-der of magnitude larger scale. In order to facilitate experi-mentation at scale SmartSantander offers access tothousands of IoT experimentation nodes, which can be uti-lised for advanced experimentation scenarios.

3.1.3. HeterogeneityFuture Internets of Things will consist of a wide variety

of devices integrated with other FI infrastructure and ser-vice provisioning platforms. For reasons of applicability,it is expected that the development and evaluation of pro-tocols and other IoT technologies be undertaken underconditions that is representative of the degree of heteroge-neity inherent in the Internet of Things. In this respect, theSmartSantander provides a multi-tier architecture thatencompasses the most relevant device tiers of IoT systems.The IoT device tier, in particular, offers a diverse set of het-erogeneous IoT nodes (sensors, actuators, QR and NFC tagsand mobile-phone-based sensing-platforms) connected viadifferent network technologies, with different mobility(fixed or mobile), and with different sensing/actuationmodalities.

3.1.4. MobilityThe IoT is composed of fixed and mobile devices which

can also interact with each other in real life scenarios.While some indoor testbeds offer robot-controlled mobil-ity, it is often difficult to reproduce real life mobility pat-terns in such testbeds. SmartSantander thereforeprovides support for realistic mobility by deploying a partof the infrastructure on moving real world entities, such asbuses, public service vehicles or taxis. Furthermore themobility of users is opportunistically leveraged by allowingthe smartphone of a citizen to report information capturedin a participatory manner [17].

3.1.5. User support and end user involvementUnlike many IoT and FI testbeds that are geared to-

wards supporting the experimental researcher as its maintarget user, the SmartSantander facility has taken a broad-er approach. The deployment of such a facility in the heartof a city and the considerable costs involved motivate theexploitation of the facility beyond the experimental re-search community. The facility has therefore been


conceived not only to act as a testbed for research with IoTtechnologies but for the development and evaluation of IoTenabled Smart City services and applications targetingdevelopers of commercial Smart City services and applica-tions. Furthermore, SmartSantander also targets end usersby providing IoT enabled services to the citizens of Santan-der and to other beneficiaries at the different testbed sites.The involvement of concrete end users adds anotherdimension to the evaluation capabilities of the platformby allowing not only the assessment of technical perfor-mance of IoT solutions, but also their user adoption and so-cial impact.

3.1.6. ReliabilityHaving in mind the purpose of the infrastructure, in

particular that it is intended to be used for service provi-sion, reliability of the complete system represents animportant requirement to ensure smooth and uninter-rupted operation.

3.2. Architecture overview

The SmartSantander platform follows a three-tieredarchitecture consisting of an IoT device tier, an IoT gateway(GW) tier and server tier. Fig. 1 illustrates the three tiersrepresenting different classes of devices and services thatcomprise the SmartSantander infrastructure.

The IoT node tier provides the necessary experimenta-tion substrate consisting of IoT devices. These devices aretypically resource-constrained (in terms of power, memoryand energy availability) and export sensing or actuatingcapabilities. This tier accounts for the majority of the de-vices utilised in the testbed. Due to their outdoor deploy-ment these devices are subject to harsh environmentalconditions (physical damage, weather influences, powersupply). To ensure their reliable operation, a number ofmeasures have been undertaken. These devices are de-ployed at hard-to-reach locations to minimise damagefrom vandalism. For dependability, dual power supplies(electric distribution network combined with batteries)and dual communication interfaces are installed. For reli-ability, multiple communication paths to a gateway are en-abled for sensor reading collection and for maintenance(e.g. over-the-air firmware and application updates) and

Fig. 1. Logical separation of 3-tier node architecture into a testbed

a set of management procedures is implemented to ensurerapid detection of malfunctioning nodes.

The IoT gateway node tier links the IoT devices at theedges of the network to a core network infrastructure.The nodes of the GW tier are also part of the programmableexperimentation substrate, in order to allow experimenta-tion for different inter-working and integration solutionsof IoT devices with the network elements of a current orFI. The GW tier devices are typically more powerful thanIoT nodes but at the same can still be based on embeddeddevice architectures – and are thus more resource-con-strained than devices of the server tier.

The server tier provides more powerful server deviceswhich are directly connected to the core network infra-structure. The servers can be used to host IoT data reposi-tories and application servers that can be configured torealise a variety of different IoT services and applicationsor to investigate approaches for real world data miningand knowledge engineering. The server tier benefits fromvirtualisation in a cloud infrastructure, ensuring high reli-ability and availability of all components and services.

The proposed architecture is agnostic to the communi-cation technologies between the different elements at thedifferent tiers. In this sense, realisations of the architecturecan be carried out using different communication technol-ogies between servers, GW nodes and IoT nodes. The com-munication solutions adopted for the Santander testbed,for instance, are described in Section 4.2.

A key design consideration is to minimise the requiredhuman intervention to make both use and managementof such large scale infrastructure tractable. Thus, the archi-tecture has been separated into a Testbed observation andmanagement plane and an IoT experimentation plane.

The Testbed observation and management plane com-prises all the functionalities of the testbed dealing with dy-namic management, plug-and-play configuration andautomated fault management of the SmartSantanderframework. A testbed user will invoke the APIs offeredthrough the IoT experimentation plane in order to configure,run and control its experiments. Most experiments willutilise the nodes of the IoT node tier; however some end-to-end experiments or holistic IoT solution evaluations willrequire also the involvement of the gateway and server tierin the IoT experimentation plane.

observation and management and an experimentation plane.


It should be noted that this separation is logical anddoes not automatically imply that functionalities on differ-ent planes are hosted on different network nodes. In somecases, functions of both planes can be part of the same de-vice while in other ones also a physical separation may ex-ist. Physical separation has the advantage that experimentsare not influenced by testbed observation and managementplane functions, which may impair performance resultson resource-constrained devices. However this comes atthe cost of additional hardware.

In order to realise this architecture we propose a refer-ence model for IoT experimentation testbeds that encom-passes both testbed observation/management, and IoTexperimentation planes. We contend that such facilities re-quires, as illustrated in Fig. 2, the provision of testbed fea-tures by four main sub-systems: (1) Authentication,Authorisation and Accounting (AAA), (2) testbed manage-ment, (3) experimental support, and (4) application sup-port. In our reference model, each subsystem comprisesseveral functional blocks that implement the functionalityexpected from the subsystem. Subsystems may spanacross the three node-tiers requiring different componentsor logic to be deployed at each tier. Subsystems export anumber of interfaces. Interfaces in our referencemodel architecture are notional entities that expose the

Fig. 2. Reference model architecture

functionality of the different subsystems through a collec-tion of APIs. In concrete instantiations of the referencemodel, these interfaces may be realised through technolo-gies such as Web Services, RESTful APIs, messaging proto-cols or event handling, to name but a few.

The AAA subsystem controls the access to the testbedby authenticating users, authorising the invocation of par-ticular testbed services based on user privileges and mon-itoring the level of platform-use by users. Its services areexposed via the Access Control Interface (ACI).

The Testbed Management Subsystem encapsulates thefunctionalities concerning the automatic management ofthe facility. Through the exported Management SupportInterface (MSI), it provides access to functions such as re-source discovery, dynamic resource registration, resourceor software component reconfiguration, and testbed mon-itoring and fault management. The MSI interface is usedprincipally by the testbed administrator to ensure theoperation of the facility.

The Experimentation Support Subsystem (ESS) embod-ies the experimentation plane functionality of the testbedby providing functions for testbed resource selection, spec-ification of experiments including resource configurations,reservation of testbed resources, scheduling of experi-ments as well as deployment and execution control of

of the SmartSantander facility.


experiments and data collection/analysis. Essentially, itprovides operations to assist the user during the entireexperimentation life-cycle. The ESS’s functionality is ex-posed through the experimental support interface (ESI)which is mainly used by scientific researchers; it also pos-sible to access the service functions of the ExperimentationSupport Subsystem (ESS) through this interface.

The Application Support Subsystem (ASS) offers via itsApplication Support Interface (ASI) a wide range of datamanagement functions that can operate on information re-trieved from the devices at the IoT node tier. For instance,it enables service applications to discover and select sensordata streams, issue commands to actuators, subscribe tosensor data events and access recorded sensor data forthe purpose of data mining. Not only Smart City serviceprovision will be supported through this interface but alsoexperiments at service level that mainly needs access tothe data collected within the infrastructure.

The three main functional features that have to be sup-ported in SmartSantander, i.e. experiment support, plat-form management and service provision, necessitatefunctionality mapping and simultaneous deployment onthe three architecture tiers. These three aspects have tocoexist at each tier in such a way that all of them are sup-ported but do not affect each other significantly. The onlyexception to this full coexistence is found at IoT node tierwhere some of the devices pose limits to the experimenta-tion that can be carried out over them.

Testbed-management data and sensor-observations arestored in data repositories which can be accessed by soft-ware components from each subsystem. The repositoriestypically capture representations of the information mod-els defined in our reference model. More specifically, wehave defined information models for the specification ofexperiments, the description of testbed resources, thespecification of observations and sensor readings and thespecification of logical node topologies for experimenta-tion. These information models are not presented here, asthey are beyond the scope of this paper.

Our reference model reflects the experimentation/ser-vice-provisioning duality that we believe is crucial to theoverall usefulness of the platform and for the definition ofexploitation models to ensure its sustainability. As a con-crete realisation of this reference model, the SmartSantan-der platform supports both experiment-execution andservice applications (smart city services) concurrently with-in its infrastructure. Our solution for the coexistence ofexperiments and smart city services relies on a combinationof dedicated nodes and of sensor-data sharing. Sets of dedi-cated nodes for experimentation and for smart city applica-tions are required to maintain Service-Level Agreements(SLAs) brokered with the different stakeholders of the plat-form. Sharing of data streams from virtually every IoT nodein the platform enables a multitude of experiments andsmart city services to coexist as sensor-data consumers.

4. Santander testbed deployment

The objectives of SmartSantander’s deployed IoT infra-structure are two-fold as well as concurrent. As a testbed,

it enables experimental assessment of cutting-edge scien-tific research. However, as mentioned in Section 3, thistestbed goes beyond the experimental validation of novelIoT technologies. It also aims at supporting the assessmentof the socio-economical acceptance of new IoT solutionsand the quantification of service usability and performancewith end users in the loop. For instance, it simultaneouslysupports the trial and subsequent provisioning of smartcity services. To attract the widest interest and demon-strate the usefulness of the SmartSantander platform, thedeployment of the IoT experimentation infrastructure hasbeen undertaken to realise the most interesting and im-pact-generation use cases. In this respect, application areashave been selected based on their high potential impact onthe citizens, thus enabling the execution of extensiveexperiments to obtain insights into the uptake of IoT-basedservices deployed in a live environment. Also taken intoconsideration in the selection of application use cases arethe diversity, dynamics and scale of the IoT environment.All these aspects increase the potential of the testbed forthe evaluation of advanced protocol solutions.

4.1. Use cases and scenarios

This section outlines some of the selected use cases andscenarios that underpinned the Santander testbeddeployment.

4.1.1. Environmental monitoringThe current solutions for environment monitoring in

urban settings usually rely on a small number of measure-ments stations placed at fixed locations. Although theaccuracy of the measurement equipment in these units ishigh, their cost effectively excludes large-scale deploymentto obtain measurements at finer granularity.

With the introduction of IoT technology, it is now pos-sible to deploy a large number of low cost sensors for afraction of cost of the current technology [18,19]. TheseIoT sensors do not provide the same degree of accuracybut using a large number of measurement points andintelligent processing of the measurements it is possibleto obtain sufficiently accurate measurements. In the envi-ronmental monitoring use case, readings gathered fromfixed and mobile sensors are used as the initial indicatorof the severity of the environment pollution (air quality,noise levels and luminosity levels) covering large areas.In case where conditions are observed, special alarms aregenerated by the system. If these observations last for longperiods of time in some specific geographical region, thenmore accurate environment monitoring equipment isdeployed. Moreover, in order to comply fully with environ-mental-monitoring legislation, devices offering a high-degree of accuracy are deployed temporarily at theidentified pollution hotspots, thereby resulting in the cov-erage of a broad area at the fraction of the cost.

4.1.2. Outdoor parking management and driver guidanceThe outdoor parking management use case implies the

development and deployment of a parking space manage-ment service in the city of Santander. Essentially, thissmart-city service enables monitoring the occupancy of


outdoor parking spaces on the streets of the Santander citycentre for parking-bay usage and accounting. To imple-ment this service, ferromagnetic wireless sensors are bur-ied under the asphalt at each bay. Peer equipment suchas repeaters are deployed in an area to guarantee connec-tivity with the Internet such that parking occupancy infor-mation can be disseminated instantaneously to drivers, therelevant traffic control management organisations in thecity or local authorities. Further, sensor data from parkingbays is aggregated and used to feed parking status infor-mation to display panels located at street intersections.These data streams can be subscribed to by mobile phoneapplications providing, for example, navigation help to freeparking spaces. Similarly, historical parking occupancydata can be analysed by municipal authorities to deter-mine the level of parking provisioning in the city.

4.1.3. Parks and gardens precision irrigationThe Precision Irrigation use case is aimed at augmenting

the automated irrigation systems currently deployed alongparks and gardens to evaluate plants’ requirements inwater and provide for more precise on-demand irrigation.Automatic irrigation systems in use in city parks and gar-dens are schedule-based i.e. run preconfigured programsbased on timetables irrespective of weather conditions orthe water requirements of the vegetation at particularareas. Different species of shrubbery and trees have vary-ing requirements in terms of water consumption, whichis also influenced by other factors such as soil humidity.The development of WSN precision irrigation and parkmonitoring applications makes it easier to increase effi-ciency and cut down costs. IoT devices spread around thepark and gardens enable agricultural data such as air tem-perature and humidity, soil temperature and moisture, leafwetness and rainfall to be collected. The real-time informa-tion from the sites provides a solid base for park techni-cians to adjust strategies at any time. Instead of takingdecisions based on some uncertain average condition,which may not be even close to reality, or having to bephysically present on-site constantly, a precision park irri-gation approach recognises differences and automatesmanagement actions accordingly.

4.1.4. Augmented realityThe augmented reality use case aims at augmenting the

city scape or locations in the city with IoT endpoints toprovide context-sensitive information and services at theselocations. This initially involves augmenting Points OfInterest (POI) in the city, for example touristic sites, shopsand public spaces with NFC tags. These tags are used to ex-pose services or information relevant to the location/con-text to site-visitors. As an example of this service usage,the site-visitor’s mobile-phone display can be overlaidwith relevant services or tourist-targeted information,depending on their location or direction of vision. For in-stance, the augmented reality use case provides touristswith a ‘‘stroll in the city’’ experience by supplying themwith location-sensitive information such as description ofmonuments in their preferred language. Whilst NFC tagshave been deployed at the various POIs in the city, weare currently envisaging a commercial exploitation of this

platform capability that involves augmenting shops withNFC tags as a means for advertising sales opportunities tocustomers. This will provide shops with new opportunitiesto build and strengthen customer relationships [20].

There are several potential windfall applications thatcan exploit the data collected from this platform capability.Location information and visitor frequency can be used togauge the popularity of sites and to adjust visitor-manage-ment strategies accordingly. Tags can be coupled withmore advanced services such as ‘‘feedback’’ from the citi-zens to the city council.

4.1.5. Participatory sensingIn this scenario, mobile phones are used as sensors,

feeding sensed physical data such as GPS coordinates,direction (compass) and environmental data such as noiseor temperature to the SmartSantander platform. Users canalso subscribe to services such as ‘‘the pace of the city’’,where they can get alerts for specific types of events cur-rently occurring in the city. Users also can report the occur-rence of such events, which will subsequently bepropagated to other users that are subscribed to therespective type of events.

4.2. Deployed infrastructure

The deployment of IoT devices to compose the Smart-Santander infrastructure has been motivated both byrequirements for ‘in-situ’ experimentation and by theaforementioned smart city services. This section thereforeprovides details in terms of hardware specifications anddeployment locations of the SmartSantander’s IoT devices.The deployment of the IoT devices in a natural setting pre-sented unprecedented challenges; this report also de-scribes the problems encountered and the solutionsformulated to resolve them.

4.2.1. Deployment locationsThe IoT experimentation facility deployed in Santander

has been settled on a cyclic approach with two of theplanned phases already undertaken.

The objective of the first cycle of deployment was tocreate a meshed WSN on fixed locations that would serveas a testing environment for the experimental validationof advanced WSN-related mechanisms. The deploymentalso influenced by the city of Santander smart-city servicerequirements and strategy, focused on three geographicalareas of significance to the smart-city services. To achievethe maximum possible impact to the citizens, the deploy-ment process intentionally accomplishes a concentrationof IoT devices in the city centre (a 1 km2 area). This areahas the highest IoT node density in Santander and frequentusage provides insights into the acceptance of IoT-basedservices running in live environments.

Fig. 3 shows an excerpt view of the Santander city cen-tre deployment. The different icons represent the deployednodes (i.e. Carbon Monoxide – CO –, light intensity, noise,temperature, and car presence detection sensors). Follow-ing the architecture described in Section 3, the deploymentincludes clusters of wireless sensors and gateway devicesacting as cluster heads.


Once the areas for the deployment were decided, thenext step in the deployment process was to specify whereto physically install the devices. In this sense, the key fac-tor influencing the decision was ensuring a viable powersupply to all the devices. Although, WSNs are typically con-sidered autonomous in terms of power needs, this assump-tion does not reconcile with the envisaged high-frequencymulti-user usage model of our platform. Energy autonomyis achieved through the use of long-lasting batteries andmost importantly, energy efficient mechanisms. However,testbed experimentation requires frequent node-softwareupdates, which impose a stiffer power consumption pen-alty on IoT nodes than can be realistically met by batteriesalone. To this end, WSN-experimentation testbeds such as[21,15] or [22] rely on permanent power supplies for theirnodes or exhibit a reduced node lifetime.

A hybrid solution to IoT node power requirements wasadopted to minimise the infrastructure’s energy consump-tion signature on the power grid, but ensure the survivabil-ity of its experimentation nodes. To fulfil the need forproximity to a power source, sensor devices were attachedto public lampposts (as illustrated by the picture in Fig. 4).The sensor devices are also endowed with rechargeablebatteries and a charging circuit. Thus, daylight operationof the nodes (lampposts turned off) draws power fromthe batteries which are charged at night when the lamp-posts are turned on. Nightly operation of the nodes relieson the power from the lamppost. This solution guaranteespower supply even under energy-hungry experimentationscenarios. Corresponding electrical adaptation and protec-tions (transformer, fuse and differential protection) wereadded in order to obey municipal regulation.

Although this solution was feasible for sensor nodessupporting the environmental monitoring service, proxim-ity to permanent power supplies for parking sensor nodesis impossible due their deployment location (buried underthe asphalt, see Fig. 4). Thus, due to their exclusive relianceof batteries, power consumption on these nodes is keptminimal using energy efficient mechanisms similar tothose presented in commercial products like [23–25]. Thisguarantees a device lifetime of over 3 years. Experimenta-tion over these nodes is restricted only to accessing car-presence detection information.

Fig. 3. Santander city centre d

Gateway devices have other deployment peculiarities inthat they require a constant power supply and connectivityto the Internet. The solution was to install most of thesedevices at municipality premises located along the areato be covered. These premises are connected through a fi-bre-optic ring which allows GWs to be connected to ahigh-capacity backbone network. Where no such munici-pality premises were available, access to the Internet isachieved through WAN connectivity via a 3G telecoms net-work interface.

The first cycle of IoT deployment yielded 740 points ofpresence in the city. Each point of presence is equippedwith several sensors making a total of more than 50 noisesensors, 600 temperature sensors, 500 light intensity sen-sors and 30 CO sensors. Additionally, 390 nodes with carpresence detection modules have been installed in parkingbays and 23 GWs have been installed to ensure connectiv-ity between the IoT node tier and the server tier.

In the second cycle, three additional fixed-node clusterstotalling approximately 50 IoT nodes were added to theinfrastructure. These clusters support the smart irrigationuse case and offer sensing capabilities via 45 temperatureand relative humidity sensors, 25 soil moisture and soiltemperature sensors, 4 weather stations with solar radia-tion, atmospheric pressure, anemometer and rainfall sen-sors, and 2 water flow sensors. The second cycle alsoimproved node heterogeneity with the deployment of150 mobile devices on top of public transport buses,municipality fleet vehicles and taxis. These nodes provideuseful mobility patterns for experimentation as well assupport environmental monitoring service. Besides the en-hanced experimentation possibilities, we envisage thesenodes to serve multiple application domains such as smartpublic transportation management and traffic conditionsassessment. Further, to support experimentation basedon alternative technologies and facets of the IoT paradigm,2000 Quick Response (QR) and NFC tags (cf. Fig. 5) havebeen deployed over the city (at touristic POIs, bus stopsand municipality’s premises). These collectively supportthe operation of the augmented reality smart-city service.

Finally, citizens’ smartphones are also part of the test-bed. A Participatory Sensing mobile app has been devel-oped within the SmartSantander project to enable these

eployment excerpt view.

Fig. 4. (a) Wireless sensor nodes attached to lamppost and (b) wireless sensor nodes buried under the asphalt.

Fig. 5. (a) Detail of sensor nodes installed on public bus and (b) QR/NFC tag attached to bus stop.


devices to send sensed physical measurements as well asmobile phone users’ observations (text, images and video).

4.2.2. Hardware deployedOur deployment topology organises each cluster of sen-

sor nodes around a Gateway (GW) device which providesmanagement operations for that node-cluster and connec-tivity to the server tier. Where nodes are out of radio rangeof the Gateway device, we employ repeater nodes to en-sure connectivity.

Gateway devices are intended to perform data packetrouting functions so that sensor observations are trans-ported from the sensor devices to the server tier as wellas executing several experimentation and testbed manage-ment functions. Thus Gateway devices must be amply pro-visioned in terms of memory/processor capacity and offercommunication interfaces towards both the WSN andexternal networks. To fulfil these requirements, embeddedPCs based on the ALIX board have been used as Gatewaydevices. They have increased capacity in terms of processor(500 MHz) and memory (256 MB RAM and up to 32 GB fordata storage). They are configured to each include twoXbee-Pro [26] radio modules for communicating with theWSN, as well as WiFi, 3G, Bluetooth and Ethernet inter-faces so that they can be connected to the rest of theSmartSantander infrastructure (i.e. SmartSantander back-end and other Gateways). The GW devices run Linux OSand are encased in a housing that is IP67 compliant resis-tant to vandalism. The small size ensures ease of

installation at appropriate ground clearance without beingtoo conspicuous.

Sensor nodes installed on lampposts are based on theATmega1281 microcontroller and are endowed with 8 KBSRAM, 4 KB EEPROM, 128 KB FLASH and an extra storingSD memory with 2 GB capacity. For Input/Ouput, they have7 analogue and 8 digital interfaces available for externalsensor connection, as well as 1 PWM, 2UART, 1 I2C and 1USB interfaces. Depending on the device, the correspond-ing sensing probes are connected to a sensor board placedon top of the main board. This enabled the deployment ofIoT nodes with diverse sensing capabilities, each with aconfiguration designed to support particular experimentor smart-city service classes. The sensing capabilities ofour IoT devices include: air quality (temperature and COsensors), noise (noise sensor), temperature (temperaturesensor), luminosity (light and temperature sensors), irriga-tion monitoring sensor (temperature, relative humidity,soil moisture and soil temperature sensors) and environ-mental station (temperature, relative humidity, solar radi-ation, atmospheric pressure, anemometer and rainfallsensors).

The most noteworthy characteristic of these devices re-lated to their wireless communication interfaces is thatthey are provisioned to provide two separate communica-tion channels: one for the experimentation plane and onefor the management/service plane. This is a departure fromcontemporary WSN testbeds [12,15] or [21], which havetraditionally relied on wired connections (e.g. USB) for sup-porting testbed and experiment management mechanisms.


In the traditional approach, only actual experiments usethe nodes’ wireless interface. For instance, during a routingalgorithm experiment route discovery and maintenancemessages are exchanged via the wireless interface, butnode reprogramming and events reporting are donethrough the wired interface. However, a wired backhaulfor our sensor nodes is impractical in the SmartSantanderplatform, given the geographical distribution of sensornodes. Making the service/management plane share thesame communication channels as the experimentationplane introduces contention on the radio module and thepossibility of interference between the different sourcesdata traffic thus creating non-deterministic behaviourwhich is undesirable for repeatable experiments.

The solution adopted at the IoT nodes level is, as shownin Fig. 6, based on the inclusion of two XBee-Pro radiomodules (operating at 2.4 GHz frequency) on each lamp-post sensor device. One of the modules implements nativeIEEE 802.15.4 protocol, whilst the other runs IEEE 802.15.4protocol modified with the proprietary routing protocol,called Digimesh [27].

The two interfaces allow the creation of two physicallyindependent wireless networks. The network based on thenative IEEE 802.15.4 interfaces is fully devoted to experi-mentation. Researchers deploying their experiments onthese sensor nodes will be allowed to freely use the IEEE802.15.4 interface for communicating with other sensornodes within the scope of their experiment. On the otherhand, the network based on the Digimesh interfaces isused for carrying, to and from the cluster gateway, serviceprovision related information as well as for supporting allthe testbed and experiment management mechanisms.The reason for using a low data rate interface for this sec-ond network is mainly for guaranteeing low power con-sumption on the nodes.

IoT nodes that are installed on vehicles are alsoequipped with a native IEEE 802.15.4 interface that canbe freely used within the scope of an experiment to com-municate not only among other devices deployed on vehi-cles, but also with devices installed on lampposts.However, since vehicles are moving all over the city, thebackhaul network for service provision and managementmechanisms handling is based on General Packet RadioService (GPRS). Power consumption is not that critical forthese nodes as they are powered through the vehicle bat-teries which represent a large energy supply for this kindof device. These devices are equipped with sensors for

Fig. 6. IoT node deployed in SmartSantander.

detecting air pollutants such as nitrogen dioxide (NO2),CO, ozone (O3) as well as detection of particles in suspen-sion, temperature and air humidity. Most importantly,they are also equipped with GPS so that all their observa-tions come geo-localised and they also report speed andcourse of the vehicle.

Finally, for the participatory sensing use case, citizenssmartphones are used as yet another IoT device. In thissense, by means of an App developed within the Smart-Santander project, device sensing capabilities (i.e. GPS,acceleration, microphone, etc.) are exploited. However,what is more interesting is that through the same App,users are able to report events happening on the city (e.g.hole in the street, malfunctioning street light, full wastebasket, unattended taxi stop, etc.) participating in observ-ing the city conditions.

4.2.3. Testbed inter-tier connectivityInter-tier connectivity in the SmartSantander testbed

(the Santander WSN) is arranged through different com-munication technologies. This section describes the net-work topology of the facility. As illustrated by Fig. 7,fixed IoT nodes are organised into clusters that form amesh network of nodes providing both single-hop connec-tivity (via the native 802.15.4 interface) and as well asmulti-hop data transfer to the gateway and server tier(via the Digimesh-enabled radio interface). All the devicesin a cluster form part of the same mesh network and mayserve the experimentation plane or service plane or both.IoT nodes that are physically close but belong to differentcluster groups are not part of the same mesh networkand therefore cannot relay each other’s sensor observa-tions towards the servers. All sensor observations, testbedmanagement and experiment management traffic have tobe forwarded through the cluster head, i.e. the gatewaynode. Multiple egress points for multi-home cluster groupshave not been considered.

However, this hierarchical topology of cluster groups isnot imposed on the selection of nodes for experimentation.

Fig. 7. Testbed physical network diagram.


As experiment traffic is transmitted via the native IEEE802.15.4 interface, experimenters are given free rein torealise the topology they desire. The testbed does not im-pose any restriction on the use of the second radio inter-face. The only condition that has to be fulfilled for two ofthese IoT nodes to be able to communicate is the existenceof a radio link between them. In essence, all the devices de-ployed are part of the same physical network as long as it ispossible to find a set of IEEE 802.15.4 links connecting, on amulti-hop manner, any pair of the deployed nodes. Thisfact is presented when in Fig. 7 native IEEE 802.15.4 linksare set between IoT nodes in different clusters.

Connectivity for the IoT nodes deployed on vehicles dif-fers from the case of static ones. These devices are not partof any cluster but they use a GPRS connection to directlyreport the observations captured by their sensors and tosupport testbed and experimentation management proce-dures. However, the native IEEE 802.15.4 is capable ofinteracting with the fixed devices. Hence, these nodescan also be part of the abovementioned experimentationnetwork as long as the vehicle on which they are mountedcomes close to any of the fixed ones.

GWs are the cluster heads for the fixed IoT nodes.Depending on where the GW is deployed, several possibil-ities for connecting them to the Internet, thus to the Plat-form Servers, arise. Whenever it has been possible, GWshave been deployed at one of the City Council or Universitypremises. This kind of location allows direct access to awired Intranet. If it is not possible to find such location,GPRS connection is used to connect the GW to the corenetwork.

Platform Servers are directly connected to the core net-work using the network of the University of Cantabria.

5. Large-scale IoT testbed management

Beside the deployment, management of the testbed isan extremely challenging task. Developing a dependablelarge-scale IoT platform necessitates robust techniquesfor realising out-of-band management and control planes.

Over time, there are dynamic variations to networkcontext and to application requirements. Node member-ship of the network changes as new nodes are added, fail(due to power outage or hardware failure) or are discon-nected (due to transient connectivity in the case of mobilenodes). Individually, each IoT node may transition througha number of possible states during the operation of thetestbed; the responsiveness of a node to issued commandsdepends on its current state. Further, supporting multipleapplication domains introduces dynamic variations in thespatial and temporal characteristics of sensor data basedon the new requirements of developed applications andservices. Last but not least, most of the devices deployedmay concurrently run experiment code from researcherswhile providing sensor readings for the service provision.

With the scale and variety of testbed managementevents to track, one cannot assume human interventionalone is sufficient to provide timely response to eventsand remediation to faults; a certain degree of automationis required, keeping the human in the loop only for

decision-making and policy-specification. This sectiontherefore covers features for the dynamic management ofthe SmartSantander testbed. Initially, the three main pro-cesses that are carried out for the testbed managementare presented. Next, the components realising them areintroduced. Moreover, the resource discovery mechanismsof the SmartSantander platform, outlining the informationmodels used for resource description and the registrationprocess for new nodes, are presented. Finally, the monitor-ing feature of the testbed is described.

5.1. Testbed management procedures

Management processes are performed dynamically bythe Management and Fault-Monitoring Subsystem,namely: resource discovery, resource monitoring and test-bed reconfiguration.

The resource discovery process involves detecting newIoT resources in the testbed, registering them for use andrecording the resource descriptions using standard models.Only having all the resources appropriately describedusing these information models to uniformly describe theattributes, capabilities and roles of the devices, experi-menters or application developers will be able to selectthe testbed resources that best fit their needs.

The resource monitoring process concerns the depend-ability of the testbed platform (i.e. its robustness with re-spect to software component or equipment failure). IoTdevices can run out of battery power, be subjected to hard-ware failure, accidental damage or vandalism whilst theyrun experiments and generate experiment traces or sensordata streams. Each experimentation node can be reserved,flashed with an experimenter’s code, reset or enter an ‘idle’state of service-observations reporting. Ensuring the cor-rect execution of the IoT testbed’s services in the face ofsuch dynamicity and ensuring the testbed’s resilience tofailures, therefore, requires continuous monitoring of thestate of its IoT resources.

On the detection of hardware failures, fault-remedia-tion strategies require that the testbed is reconfigured toomit the faulty nodes from future experimentation or ser-vice-provisioning. Reconfiguration for testbed manage-ment is not confined to only executing fault-remediationstrategies. As dynamic variation in the platform executioncontext occurs, reconfiguration of the platform’s compo-nents is required to deliver optimal performance at alltimes. The reconfiguration strategy usually involves chang-ing control parameters to optimise the operation of run-ning components and communication protocols.Parameter-based reconfiguration is also required whenapplication requirements change; for example the tempo-ral granularity of sensor data can be dynamically adjustedto suit the requirements of service applications.

5.2. Components for dynamic testbed management

A number of components have been implemented toprovide mechanisms for resource discovery, resource mon-itoring and testbed reconfiguration. As illustrated by Fig. 8,these components are deployed at different tiers of theplatform.


At the portal server level, the following components areresponsible for providing functionality for testbedmanagement.

� Resource Directory (RD). It supports the resource dis-covery process by enabling the storage and lookupof resource descriptions for IoT nodes. It exports aRepresentational State Transfer (REST) interface forquerying and retrieval of IoT resources based on auser’s set of criteria (e.g. sensed phenomena, sensorlocality, etc.).� IoTResourceManager. It handles the registration of new

IoT nodes in the platform and updates the status ofIoT nodes based on the reception of status reports frommonitoring components. It also issues reconfigurationcommands to the Experiment Support Subsystem andthe Application Support Subsystem based on IoT nodefailure detection.� TRConfigurator. This component configures and controls

the execution of the Testbed Runtime [7] (TR) withinthe Experiment Support Subsystem. It specifies the setof nodes available for experimentation to the TR in

Fig. 8. Components for te

the form of a WiseML specification. It reacts to reconfig-uration commands issued by the IoT Resource Managerto change the set of nodes available for reservation.� USNConfigurator. This component (re)configures the

USN [8] to provide services for the deployment of appli-cations based on the set of IoT nodes (sensors and actu-ators) reserved for the purpose of service-provision. Itadapts the resource description of IoT nodes to the Sen-sorML format used in the USN for the purpose of node-registration. It reacts to reconfiguration commandsfrom the IoT Resource Manager to register/unregisterIoT nodes in/from the USN.� PSensRegistrationManager. This component performs

participatory sensing resource discovery by triggeringthe registration of IoT nodes such as smartphones andtablets. It is also responsible for monitoring the statusof these nodes and forwarding status reports to theIoT Resource Manager.

At the gateway and IoT node tier, the following compo-nents encapsulate functionality for resource discovery, re-source monitoring and reconfiguration.

stbed management.

1 The protobuf messages are included as payload into ActiveMQ Bytes-Message messages. To enable event demultiplexing and handler invocation,each event-carrying BytesMessage includes an identifier as a MessageProperty for specifying the event type.


� NodeManager. In terms of resource discovery, this com-ponent detects new nodes and triggers the registrationwith the RD. It also monitors the status of all nodesassociated with its host GW. As periodic service mes-sage frames are routed from these nodes to the gate-way, the NodeManager component intercepts themessage frames to either detect new or dead nodes.NodeManager also maintains GW status using periodicbeacon messages.� Node Application Programming Interface (API). A set of

core function implementations from the Node API areincluded in every software image flashed onto the sen-sor nodes. They export management functions that canbe invoked through command packets by the Node-Manager to facilitate resource discovery and monitor-ing. For discovery, device-specific parameters such asthe radio chip’s MAC address are queried from eachnode by the NodeManager using the getPropertyValue()operation. For monitoring the integrity of each node,the isAlive() operation allows the NodeManager to ver-ify the live state of each node. Status parameter-valuessuch as a node’s CPU load, memory utilisation and bat-tery-level can also be directly queried through the get-PropertyValue() operation.

All interactions between the management componentsin the Portal Server tier and GW tier occur through thepropagation of events. To this end, as can be seen inFig. 8, the management plane provides dedicated event-channels for IoT resource registration, resource monitoringand testbed reconfiguration within a distributed event bus.The event bus is realised through a component, called theEvent Broker (not shown in Fig. 8), which embodies a gen-eric communication substrate for disseminating manage-ment events. The Event Broker forms a distributed ‘EventBus’ to which all testbed management components areconnected. It implements a topic-based publish-subscribeevent model wherein events are disseminated to subscrib-ers based upon their type. The event bindings between themanagement components are then asynchronous, distrib-uted and multi-party.

� Asynchronous: Event publishers do not block while pro-ducing events and subscribers are notified asynchro-nously when an event is received; this is an excellentfit for with unreliable, resource constrained WSNs.� Distributed: Local or remote bindings are semantically

identical, allowing components to be easily bound tolocal or remote event sources.� Multi-party: the event bindings allow multiple consum-

ers to be bound to the same publisher; this allows forrich interactions between components. For instance, itsuffices for a management console to subscribe to thethree event channels to receive information aboutongoing resource registration, monitoring and reconfig-uration on the platform.

The interface to the event bus is simple and lightweight.The Event Broker defines two publish-subscribe topics foreach management channel, one for request events and an-other for the reply events. Request-reply protocols are used

for each management task to instil robustness in the faceof Wide Area Network (WAN) connections to remote IoTnodes.

The Event Broker uses the ActiveMQ message brokersystem [28] to implement the management event topicsand event delivery functionality. To ensure reliable opera-tion, features such as durable subscriptions and persis-tence of event topics are used from ActiveMQ. As such,events are cached for components holding durable sub-scriptions, should they fail or be reloaded. Event typesare implemented using Google Protocol Buffers1 [29]; thisenables the event typing system to be extensible and lan-guage-independent. The event system must first be extensi-ble, as to enable the addition of new event types since newcomponent-interactions can be introduced to support newplatform-features. In this respect, the addition of new eventtypes and their corresponding parser/builder functionalityshould be as seamless as possible. Secondly, as the platformcomponents are developed using various technologies, bind-ings for the event types to different programming languagesare desirable.

Table 1 summarizes the development stage and the tar-get goal of all these components.

5.3. Resource discovery

The discovery of resources is an essential feature of anIoT platform as it serves to support selection of resourcesmatching a user’s set of criteria (e.g. sensed phenomena,sensor locality or measurement frequency).

In addition to the heterogeneity in terms of their hard-ware characteristics and context attributes that IoT nodesexhibit, they also differ by their intended roles within theplatform. As shown by the taxonomy of SmartSantanderdevices illustrated by Fig. 9, IoT nodes assume one of thefollowing roles:

� Infrastructural Nodes: These are essentially the portalservers, gateway computers and repeater nodes thatform part of the backbone network in the IoT facility,serving to run services of the testbed to support exper-imentation, service-provision and testbed management.Although these are mainly infrastructural nodes i.e.part, they can participate in experimentation by hostingexperiment software through the use of application-sandboxing entities such as application servers, OSGIcontainers or virtual machines.� Experimentation IoT Nodes: These are IoT nodes

deployed to support experimentation. They are man-aged by the TR, which provides services for experiment-ers for node reservation and Over-The-Air Programming(OTAP) support for software deployment. They need notbe exclusive to experimentation; the ubiquity of sens-ing for service applications often require that thesenodes whilst generating experiment traces concurrentlyfeed sensor data readings to the USN platform.

Table 1Implementation status of components for testbed management.

Component name Development stage Target goal

Resource directory Implemented and integrated Thousands of resource descriptions storedEvent broker Integrated Scalable distribution of asynchronous events. Hundreds of events per minuteIoTResourceManager Implemented and integrated Handling of registration and monitoring events. Hundreds of events per minuteTRConfigurator Implemented and integrated Handling of registration and monitoring events. Hundreds of events per minuteUSNConfigurator Implemented and integrated Handling of registration and monitoring events. Hundreds of events per minutePSensRegistrationManager Implemented and integrated Dynamic registration of participatory sensing related resourcesNodeManager Implemented and integrated Dynamic registration and monitoring of sensor devicesNode API Implemented and integrated Expose sensor devices management functionalities

Fig. 9. Taxonomy of IoT resources in the SmartSantander platform.


� Service-Only IoT Nodes: This class of nodes is reservedexclusively for the provision of services. They only deli-ver the observations they generate to the USN entity.These nodes cannot be reprogrammed or queried byexperimenters.� Participatory Sensing Nodes: These are handheld devices

(for example, mobile phones and tablets) running par-ticular Participatory Sensing applications for eventtracking or collection of sensed data. They feed observa-tions to SmartSantander’s USN entity to support citizen-targeted services.

Each class of IoT resources is described by a specific setof attributes that list hardware characteristics of the devicesuch as node type, sensing capabilities, mobility model toname a few or time-varying context parameters such asposition and device state. Resource discovery entails thatthese resources are searchable in terms of these attributes.The resource discovery process for the SmartSantandertherefore, encompasses two main activities:

1. Resource Management: the specification of an IoTResource Description Model that allows the diversityof IoT resources to be described in a consistent and uni-form manner. This subsumes the storage of resourcedescriptions with search capabilities to facilitatelookup.

2. Resource Registration: the generation of resourcedescriptions for new IoT resources and maintenancebased on the resources’ dynamic state.

The two resource discovery activities are described inmore details in the following sub-sections.

5.3.1. Resource managementIn significantly large testbeds like SmartSantander,

monitoring of nodes available in the system is one of themost important requirements for its efficient functioning.The concept of RD is often used [30,31] in this kind ofenvironments.


RD is an entity that stores descriptions of resourcesavailable in a system at a given time. It provides two mainfunctions:

� Resource registration and storage of their descriptionsin the RD.� Discover of resources by searching through the stored

resource descriptions.

Hence, the RD represents one of the important buildingblocks of the SmartSantander platform.

The initial version of SmartSantander was based on theimplementation done in the FP7 SENSEI project [32]. Sev-eral extensions and modifications have been implementedon top of it to suit the new SmartSantander requirements.Further to this, the underlying Structured Query Language(SQL) database (DB) that stored resource descriptions hasbeen replaced with a MongoDB to ensure better perfor-mance and more flexible handling of various resourcedescription documents [33].

The RD provides two main interfaces using REST-basedweb services for interaction with the users and resources:

� Resource Publication Interface (RPI) allows resources toregister with the RD by submitting their descriptions tothe appropriate Uniform Resource Identifier (URI) of theRD. This is implemented using the POST (registration ofa new resource), PUT (update of an existing resourcedescription) and DELETE (deleting a resource descrip-tion from the RD) methods of HyperText Transfer Proto-col (HTTP). The resource description is submitted as aparameter of the mentioned methods.� Resource Lookup Interface (RLI) allows resource users

(applications and various platform components) tosearch for resources with required characteristics. Thisis implemented using the GET method of HTTP withappropriate set of key-value pairs as parameters of aquery. RD identifies resources with the matching char-acteristics and responds with the list of resourcedescriptions. The resource descriptions contain not onlya description of the resource, but also an URI that usershould use to interact with the resource. Users can per-form once-off lookups or can subscribe to RD asking tobe informed whenever the query is satisfied. In otherwords, the users get informed whenever a resourcewith specified characteristics becomes available orceases to be available.

Attending to the taxonomy presented in Fig. 9, re-sources are described through Extensible Markup Lan-guage (XML) documents. Each resource descriptioncaptures the main characteristics of the sensors and datathey produce (type of sensors, accuracy, manufacturer,range, location, etc.) as well as the type and characteristicsof the IoT nodes (testbed server, gateway, experimentationnode, service only node, participatory sensing node, con-nection address and type, etc.).

5.3.2. Dynamic resource registrationThe dynamic registration of IoT resources involves

event interactions between the NodeManager, the

PSensRegistrationManager and the IoTResourceManagercomponents. The registration of experimentation and ser-vice nodes is triggered by the NodeManager componentresiding at each GW. The NodeManager component pro-duces registration request events i.e. the event set {NO-DE_REG_REQUEST,GW_REG_REQUEST}, and subscribes tothe corresponding registration acknowledgement eventsi.e. the event set {NODE_REG_REPLY,GW_REG_REPLY}.

On intercepting periodic frames emitted by new nodes,the NodeManager formulates a registration request event(the NODE_REG_REQUEST event) and uses the interfaceoperations of the event bus to dispatch the event on theRegistration Channel (step 1, Fig. 10). This registrationevent request contains the attributes required to create avalid Resource Description for the IoT node. As illustratedby the sequence of event dispatches in Fig. 10, theNODE_REG_REQUEST event is received (step 2) and pro-cessed by the IoTResourceManager component to publisha Resource Description in the RD through the latter’s RPIinterface (step 3). This is a simplification, prior to publish-ing the Resource Description for the new IoT node, the IoT-ResourceManager performs checks to see if this node hasnot been seen in the platform before and verifies that theGW node is not disabled. The registrations of GWs andtheir associated IoT nodes are maintained as soft statethrough the use of timers for resource invalidation anddeletion. GW nodes are responsible for sending invalida-tion requests for IoT sensor nodes that are no longer withinits reach. They also send HELLO message-events to the IoT-ResourceManager periodically to indicate their operationalstatus. After a number of missed HELLO message-events,GWs and their associated IoT nodes are first disabled (afteran invalidation timeout) in the RD and subsequently de-leted, should they fail to reappear after a deletion timeout.

After the Resource Description publication, the IoTRe-sourceManager sends reconfiguration commands throughthe Reconfiguration Channel to the TRConfigurator andUSNConfigurator components. For example, it dispatchesthe ADD_SENSOR_REQ events to the TRConfigurator com-ponent (step 4) to add an IoT sensor node for experimenta-tion. The TRConfigurator receives these reconfigurationevents (step 5), having subscribed to them and proceedsto generate the new configuration (containing the newIoT resource) for the TR. Next, it uses the configurationinterface of the TR to install the new configuration (step6) and upon a successful response, it sends back to theIoTResourceManager a reply event (ADD_SENSOR_REP,ADD_GW_REP or ADD_PS_REP) containing the result ofthe reconfiguration execution (step 7 and step 8).

If the IoT node is destined for supporting city-services,the IoTResourceManager sends the ADD_SERVICE_REQreconfiguration request event to the USNConfigurator(step 9). This reconfiguration request is received by theUSNConfigurator (step 10) which effectuates a secondaryregistration on the USN sub-system by issuing a registra-tion message in SensorML [34], the information modelused by the USN for the description of resources. Uponthe completion of this task, a reply event is sent back tothe IoTResourceManager component (step 11) indicatingthe outcome of the reconfiguration request. It is only afterthe completion of the registration and reconfiguration

Fig. 10. Event interactions for experiment IoT node registration.


tasks that the IoTResourceManager publishes a reply eventto inform the NodeManager component of the outcome ofits registration request (step 12 and step 13). The flexibil-ity of the event-based bindings used in the design is suchthat, unsuccessful registrations of IoT resources are auto-matically picked up by management consoles listeningfor the relevant events, namely, NODE_REG_REPLY,GW_REG_REPLY and PS_REG_REPLY.

5.4. Testbed monitoring

Due to uncontrollable factors (e.g. weather) far awayfrom the safety of lab, testbed monitoring is very crucialfor proper operation, maintenance, etc. As described inSection 5.1, the components for dynamic testbed manage-ment also support monitoring of resource availability andstatus, at three different levels of the architecture: PortalServer, GWs and IoT Nodes. As depicted in Fig. 10, testbedmonitoring is possible utilising the Monitoring Channelwhich is established in parallel with Registration andReconfiguration Channels. This setting permits extremelydynamic behaviour as it realise simultaneous resource

registration and monitoring and appropriate testbedreconfiguration according to observations made by theother two channels.

Similarly to the Experiment IoT Node registration thereare event interactions for monitoring of resources. The keycomponent for these interactions is NodeManager. There isone instance of NodeManager running at each GW nodeand each NodeManager instance has the responsibility tonotify the main system for the status of the correspondingGW and the attached to it IoT nodes. Periodically, Node-Manager notifies with and HELLO event (on behalf of itsGW) signalling the IoT Resource Manager that the GW isup and running. In the case that IoT Resource Managerdoes not receives a HELLO event for an already registeredGW for a certain period of time (configured as parameter)then assumes that this GW is out of order and properly up-dates RD and reconfigures TR. Upon a fresh HELLO eventfrom this GW, it is restored as active and components arereconfigured.

Furthermore, NodeManager is responsible for updatingIoT Node status of nodes attached to its GW. Node Managereither by observing passing messages from the GW or by


explicitly diffusing special wireless commands to IoTNodes in the range of the GW, can extract knowledge aboutthe status of IoT Nodes and information about them likebattery level, free memory etc. Then with NODE_STA-

TUS_REQUEST events Node Manager informs IoT ResourceManager about the status of the IoT Nodes. When an IoTnode is not detected through passing messages or doesnot reacts to the special wireless commands then is consid-ered as out of order.

ESS should be always aware for all of these changes ofresource node status (GW and IoT), in order to properly re-serve nodes for an experiment, execute an experiment, etc.As mentioned in Section 5.1, the key component for keep-ing up to date the ESS is TRConfigurator. TRconfigurator isthe responsible for generating and maintaining the appro-priate configuration state for the ESS components by trans-forming resource descriptions, included within the eventsexchanged through the Reconfiguration Channel, in thevarious formats (i.e. WiseML, Resource Description Frame-work – RDF –) that are used by the aforementionedcomponents.

6. Testbed usage: IoT experimentation

While testbed management is the most critical part fortestbed administrators, the main aim of SmartSantandertestbed is to be open and ready to be used by experiment-ers. In this sense, it is important to highlight that theexperimental facility is not only heterogeneous from thepoint of view of the infrastructure that forms the testbedbut also when looking at the kind of experimentation thatis supported. In the following sections the two basic exper-imentation approaches that are supported are described.

6.1. Service level experimentation and smart city serviceprovision

Deployment at a city scale enables direct interactionwith a large base of end-users. Interaction with real end-users allows not only assessment of technologies but alsoassessment of services. As the environment in which thetestbed is deployed is a Smart City scenario, SmartSantan-der aims also at experimentation at service level.

We refer to service level experimentation when exper-imenters make use of any of the information gathered bythe deployed infrastructure in order to build a smart cityapplication or service. The target of these applications orservices is, in general, to improve the efficiency of the cityand facilitating a more sustainable development of the cityand its citizens.

The platform enables, through the ASI, access to anypiece of information gathered by any of the deployed sen-sors generally following a publish/subscribe approach. TheASI also enables access to historic records of sensorsmatching a particular search criterion. Mostly, the querieswill be related to the sensing capabilities and location ofthe IoT nodes.

Although the main aim of the deployed infrastructure isto enable experimenters to test their developments, thedeployed infrastructure is already being used to support

actual smart city services. These services are already beingassessed by their corresponding end-users (e.g. municipal-ity technicians, citizens, etc.) in several on-going trials.

Regarding the precision irrigation use case, 48 IoT nodesequipped with agricultural sensors measuring parameterslike air temperature and humidity, soil temperature andmoisture, atmospheric pressure, solar radiation, windspeed/direction and rainfall have been deployed.

SmartSantander developed and integrated a precisionirrigation service that estimates plants’ requirements inwater in the different subareas of the deployment. Thereal-time information from the field enables park techni-cians to adjust irrigation strategies at any given time. In-stead of taking decisions based on uncertain averageconditions, which may not be even close to reality, or hav-ing to be constantly physically present on-site, a precisionpark irrigation approach recognises differences andaccordingly automates management actions. For this rea-son a smartphone application, developed for the Androidplatform, complements the main web application provid-ing easy access to the measured parameters inside the parkareas.

The Key Performance Indicators (KPI) evaluated duringthe trial mainly targeted the assessment of how realisticand accurate is the presentation of the park/garden’s statusand the assessment of how much the use of the IoT-sup-ported irrigation service facilitates savings in certain re-sources like water and labour. Municipality technicianshave helped in the assessment as they were given accessto the implemented services and were asked to comparetheir assessment from in-field visits with the informationavailable from the IoT-supported irrigation service. Thefeedback received from them was that the accuracy of theirrigation status reported through the implemented serviceswas high enough to rely on it for taking the decision whetherto water the park or not. Fig. 11(a) shows a heat-map derivedfrom real-time measurements collected by sensors alreadydeployed in Las Llamas Park. Similar maps and reports havebeen really valuable for the park managers in having a quick,remote and sufficiently accurate assessment of park status.

The service also provides other details for technicians tobe able to make a more in-depth evaluation. For example,Fig. 11(b) and (c) respectively show the soil moisture andthe rainfall observed by sensors deployed in the Las LlamasPark during February 2013. The low values for the soilmoisture tension, which indicates that the terrain satu-rated of water, shown in Fig. 11(b) fits with the rainyweather during the first two weeks of the month exposedin Fig. 11(c). However, after these first two rainy weeksthere is a one-week dry period, where soil moisture in-creases up to 28 centibars. Nevertheless, this is not consid-ered enough to start the irrigation system in the park.Hence, during this month, the use of IoT technology al-lowed parks and gardens managers to avoid visits to thepark in order to do in-field inspection of the different areasand take the decision whether to irrigate or not.

6.2. IoT device level experimentation life cycle

In addition to service level experimentation, IoTdevice level experimentation is also supported by the

Fig. 11. (a) Soil moisture tension heatmap in Las Llamas park. (b) Soil moisture tension during February 2013 in Las Llamas park. (c) Rainfall (mm) duringFebruary 2013 in Las Llamas park.


SmartSantander testbed. Among the several differences interms of the requirements imposed by these two kinds ofexperiments, one has to be highlighted. While service levelexperimentation does not generally need to modify thebehaviour of the IoT node but just need to access the infor-mation it gathers, scientific experimentation typicallyneeds to have complete control over the IoT device andmost of the times the experiments comprises flashing theIoT node with a binary image integrating the technology/protocol/mechanism that is to be evaluated.

In this sense, a scientific experiment lifecycle has beendefined for SmartSantander testbed and correspondingmechanisms have been implemented in order to addresseach of the different phases defined.

During specification phase, mainly dealing with the re-source selection, the user is assisted with an exploration ofavailable testbed resources and their static and dynamicproperties and topological interdependencies. The user isable to formulate queries for specific resource propertiesin order to satisfy the requirements for a particular exper-imentation scenario which are matched against the testbedresources descriptions in order to provide the user with aselection of testbed resources fulfilling the desired proper-ties. Furthermore, during setup phase, dealing with actualreservation and scheduling, ESS assures that experimentsdo not collide in time. Finally, on the execution phaseexperimenter is empowered with experiment executioncontrol, experiment monitoring, data collection andlogging.

6.2.1. Resource requirementsThe SmartSantander platform aims at supporting exe-

cution of various experiments on a large scale. Each exper-iment involves a number of various IoT nodes, dependingon the type of experiment. It is possible and preferable tohave multiple experiments running at the same time, usingdifferent nodes at various or even same locations. Withthousands of available nodes and multiple simultaneousrequests for execution of experiments with differingrequirements and involving a large number of nodes, it isnecessary to provide tools and procedures for automaticassignment and scheduling of available IoT nodes to eachexperiment taking into consideration the capabilities ofeach node and requirements of experiments.

RD is used for this purpose. It contains semanticdescriptions of all available resources, including informa-tion about the capabilities of the IoT nodes that are impor-tant for experimenters. It is envisaged that in a similarmanner, semantic descriptions of experiments will bestored in a RD (the same or a separate instance). Thesedescriptions will contain information about the nature ofthe experiment, the information needed, capabilities ofthe IoT nodes and supported protocols, duration; how anexperiment is influencing the environment (for exampleone experiment might influence the outcome of anotherdue to the activities undertaken – obstructing radio trans-missions, making some information unavailable or chan-ged), etc.

The platform will be then in position to reason over theprovided semantic descriptions, matching not just anexperiment with appropriate resources, but also makingsure that the new experiment will not interfere with otherexperiments scheduled at the same time. This mechanismwill greatly improve the efficiency of the allocation of re-sources to the experiments and will ensure proper condi-tion for all simultaneous experiments as well as theservices running on the platform.

6.2.2. Resource reservation and provisioningESS implementation architecture [35] was designed

with generality in mind and at the architecture’s core aset of standardized web service APIs allows a technology-agnostic standardized way for users to access a testbed’sresources. The so-called (TR) is the reference implementa-tion of the APIs for testbed management and experimentexecution defined in the WISEBED project. It creates anoverlay network for easy node addressing and message ex-change independent from the actual underlying networkconnections.

One of these APIs is the Reservation System (RS) API,which allows users to reserve a set of resources (i.e., IoTdevices) for experimentation. This API allows experiment-ers to select a subset of resources, uniquely identified by aUniform Resource Name (URN), based for example on de-vice type, attached sensors, mobility support, etc. As ithas been introduced in the previous section, resource man-agement solutions that have been put forward in Smart-Santander testbed aims at improving the efficiency of the


allocation of resources to the experiments and will ensureproper condition for all simultaneous experiments as wellas the services running on the platform.

As can be seen in Fig. 12, after successful authentica-tion, experimenters can reserve, by sending the set of URNsto the RS web service, the devices that best fit theirrequirements for a certain period of time. The RS thenchecks authorization and reserves the devices if they areavailable for the desired time period. As return value theuser receives a secret reservation key which he uses to ac-cess his experiment through the WSN API. This secret keyis called the reservation key in the following. As a resultof this invocation, users obtain a so-called secret reserva-tion key which is used later on to identify the user as theowner of this reservation.

6.2.3. Experiment controlWith a valid reservation key a user is able to interact

with the testbed via WSN web service API. Interactionmeans either to control an experiment (i.e., to reprogramor to reset devices), or to interact with a running experi-ment (e.g., to send command or to receive benchmarkingresults). For both types of interactions users have full con-trol over the complete experimental setup.

In order to interact with an experiment, users send theirsecret reservation key to the testbed via a web service calland if the experiment has started, they get a private webservice endpoint URL to interact with the reserved re-sources. This Uniform Resource Locator (URL) points toan instance of the WSN API. The details of the full API inter-action are depicted in Fig. 12. For the experimenter, thesesteps are automated and a number of clients to these APIsare available ranging from a command-line client to web-based interface (cf. Fig. 13), which both support scriptable

Fig. 12. Interaction of the different WISEBED APIs to obt

experiments (e.g., for automatic execution of repeatedexperiments, etc.). To receive output generated by devicesas well as status updates about the experiment, the userprovides the testbed with a URL where he exposes theso-called controller API. The methods of the experimentercontroller API are called by the WSN API implementationto send experiment output or asynchronous status updateson ongoing operations, like (re-)programming nodes, tothe owner of the experiment.

This enables the experimenter to not only be able tocontrol the experiment behaviour but also to transparentlyget the traces of the experiment so that they can be ana-lysed on a real-time basis or stored for offline assessmentand post-processing.

The overlay network created by the TR performs mes-sage forwarding and offers communication primitives thatare used for the control and management of experiments.This overlay network handles the messages exchanged be-tween the experimenter controller and the IoT Nodes en-abling a virtual point-to-point connection between theexperimenter host and each of the reserved nodes. Objec-tive is to have one virtual connection per IoT Node in thetestbed, accessed through an exclusive connector, enablingnot only data collection or experiment logging but alsoexperiment control operations to be triggered on a perIoT Node basis. This provides a great degree of flexibilitywithin an experiment.

While achieving this might be straightforward for thosetestbeds relying on a wired connection to each of the IoTNodes deployed, in the SmartSantander fully wireless con-text described in Section 4, exclusive virtual connectionwith each of the testbed IoT Nodes required appropriatemultiplexing and demultiplexing of the communication(both uplink and downlink) since all the virtual connec-

ain a private testbed instance for experimentation.

Fig. 13. Web-based interface for experimentation.


tions with the IoT Nodes are supported over one singlephysical connection (i.e. the wireless link between theGW and the IoT Nodes of his cluster). SmartSantandertestbed implements at GW level the modules that generatethe handlers for each of the IoT Nodes that are managed bythat gateway. The TR will be able to interact with the IoTNodes through these handlers as if they were point-to-point physical communication ports towards the under-lying IoT Nodes.

6.2.4. IoT nodes remote reprogrammingAmong the different experiment control functionalities,

there is one that required particular attention as it is thebasis for a testbed to be considered an experimentationfacility. The ability to re-program the IoT Nodes at any timeduring the experiment is critical and has been carefully ad-dressed within the SmartSantander testbed.

It comes without saying that direct reprogramming ofthe IoT Nodes deployed in SmartSantander testbed is nota possibility. Further to this, it has been already statedthat no wired infrastructure is available to support fastand resilient flashing of nodes. Thus, remote reprogram-ming of IoT Nodes is being carried out through OTAPmechanisms implemented as part of the ESS. Bearing inmind that clusters of IoT Nodes deployed embraces mul-tihop wireless networks configurations, it is more preciseto speak about Multihop Over The Air Programming(MOTAP).

Several MOTAP mechanisms are available in the litera-ture [36]. However, none of them had an available imple-mentation for the devices actually deployed. Thus, inorder to make the deployed platform as dynamic andreconfigurable as possible, a reliable MOTAP protocol hasbeen implemented for flashing nodes over the air eitherin unicast, multicast or broadcast fashions, as many timesas needed. As it can be seen in Fig. 13, flashing and reset-ting operations are available to the experimenter.

7. Conclusions

As has been indicated, Internet of Things is foreseen to bean essential part of the FI. In this paper the key features andproperties that are supported by SmartSantander testbedhave been described. The experimental research facility pre-sented in this paper aims at supporting the testing of pro-posed protocols, services and configurations in a realisticsetting at an appropriate scale. Shortcomings of the existingtestbeds in terms of scale, heterogeneity, mobility and moreimportantly realism of experimentation environment andend-user involvement are overcome by the holistic experi-mentation environment deployed in SmartSantander.

This paper presents the testbed architecture as well asthe main deployment issues and experiences. In this sense,specific mechanisms have been integrated in order to guar-antee that the testbed is ready to provide to the experi-menters all the potential that such a large scale testbedhas. Particular attention has been put on the testbed man-agement as it is of utmost importance to keep track of allthe testbed resources thus guaranteeing the dependabilityof the facility. Moreover, the solutions developed are ontheir own a significant contribution that addresses thechallenging tasks that are raised by the scale and varietyof testbed management events to track.

The facility will be further improved and enhanced byfederating it with additional sites providing access to aneven larger number and more varied types of IoT devices.

Acknowledgements

This work is funded by research project SmartSantan-der, under FP7-ICT-2009-5 of the 7th Framework Pro-gramme of the European Community. Authors would liketo acknowledge the collaboration with the rest of partnerswithin the consortium leading to the results presented inthis paper.


References

[1] M. Botterman, Internet of things: an early reality of the futureinternet, in: Workshop Report prepared for European Commission,Information Society and Media Directorate General, NetworkedEnterprise & RFID Unit (D4), Prague, May 2009.

[2] G. Santucci, Internet of the future and internet of things: what is atstake and how are we getting prepared for them?, eMatchconference, Oslo, September 2009.

[3] Real World Internet Position Paper <http://rwi.future-internet.eu/images/c/c3/Real_World_Internet_Position_Paper_vFINAL.pdf>.

[4] M. Zorzi et al., From today’s INTRAnet of things to a future INTERnetof things: a wireless and mobility-related view, IEEE Wirel. Commun.17 (6) (2010) 44–51.

[5] FP7-ICT-2009-5-257992, Project SmartSantander <http://www.smartsantander.eu>.

[6] FP7-ICT-2007.1.1-215923, Project SENSEI <http://www.sensei-project.eu>.

[7] FP7-ICT-2007.1.6-224460, Project WISEBED <http://www.wisebed.eu/>.

[8] J. Bernat, et al., Ubiquitous sensor networks in IMS: an ambientintelligence Telco platform, in: Proceedings of ICT Mobile Summit,2008.

[9] A. Gluhak et al., A survey on facilities for experimental internet ofthings research, IEEE Commun. Magaz. 49 (11) (2011) 58–67.

[10] T. Ojala, Open urban testbed for ubiquitous computing, in:Proceedings of Conference on Communications and MobileComputing, Shenzhen, April 2010.

[11] R.N. Murty, et al., CitySense: an urbanscale wireless sensor networkand testbed, in: Proceedings of Conference on Technologies forHomeland Security, Waltham, May 2008.

[12] M. Sridharan, et al., Kanseigenie: software infrastructure forresource management and programmability of wireless sensornetwork fabrics, in: Proceedings of Next Generation InternetArchitectures and Protocols, 2010.;G. Werner-Allen, et al., Motelab: a wireless sensor network testbed,in: Proceedings of Symposium on Information Processing in SensorNetworks, Los Angeles, April 2005.

[13] SensLab, Very Large Scale Open Wireless Sensor Network Testbed,2010 <http://www.senslab.info/>.

[14] S. Bouckaert, et al., The w-iLab.t testbed, in: Proceedings ofConference on Testbeds and Research Infrastructures for theDevelopment of Networks & Communities, Berlin, May 2010.

[15] G. Coulson et al., Flexible experimentation in wireless sensornetworks, Commun. ACM 55 (1) (2012) 82–90.

[16] E. Nordstrom, P. Gunningberg, C. Rohner, O. Wibling, Evaluatingwireless multi-hop networks using a combination of simulation,emulation, and real world experiments, in: Proceedings of the 1stInternational Workshop on System Evaluation for Mobile Platforms(MobiEval ‘07), ACM, New York, NY, USA, pp. 29–34.

[17] J. Burke, et al., Participatory sensing, in: Proceedings of Conferenceon Embedded Networked Sensor Systems, Boulder, October 2006,Computer Science Department, University of Virginia, ‘‘VineLabWireless Testbed’’, 2009 <http://www.cs.virginia.edu/_whitehouse/research/testbed>.

[18] G. Barrenetxea, et al., SensorScope: out-of-the-box environmentalmonitoring, in: Proceedings of Conference on InformationProcessing in Sensor, Networks, 2008, 22–24 April 2008, pp. 332–343.

[19] K. Martinez, J.K. Hart, R. Ong, Environmental sensor networks, IEEEComp. 37 (8) (2004) 50–56.

[20] L. Frayer, High-Tech Sensors Help Old Port City Leap Into SmartFuture <http://www.npr.org/blogs/parallels/2013/06/04/188370672/Sensors-Transform-Old-Spanish-Port-Into-New-Smart-City>.

[21] V. Handziski, et al., TWIST: a scalable and reconfigurable testbed forwireless indoor experiments with sensor networks, in: Proceedingsof Workshop on Multi-Hop Ad Hoc Networks: From Theory toReality, 2006.

[22] P. Dutta, et al., Trio: enabling sustainable and scalable outdoorwireless sensor network deployments, in: Proceedings of Conferenceon Information Processing in Sensor Networks, 2006.

[23] http://www.nedapavi.com/products/sensit/sensit.html.[24] http://www.tst-sistemas.es/en/solutions/parking/.[25] http://www.streetline.com/parksight/.[26] Point-to-Multipoint RF Modules <http://www.digi.com/products/

wireless-wired-embedded-solutions/zigbee-rf-modules/point-multipoint-rfmodules/>.

[27] Digimesh Routing Protocol <http://www.digi.com/technology/digimesh/>.

[28] Apache ActiveMQ <http://activemq.apache.org/>.[29] Google Protocol Buffers <https://developers.google.com/protocol-

buffers/docs/overview>.[30] Z. Shelby, S. Krco, CoRE Resource Directory, Internet Draft, May 2012

<http://datatracker.ietf.org/doc/draft-shelby-core-resource-directory/>.

[31] ETSI TS 102 690, Machine to Machine Communications (M2M);M2M Functional Architecture <http://webapp.etsi.org/WorkProgram/Report_WorkItem.asp?WKI_ID=30459>.

[32] V. Tsiatsis, et al., The SENSEI Real World Internet Architecture,Towards the Future Internet, IOS Press, 2010, pp. 247–256.

[33] S. Jokic, et al., Evaluation of a document oriented resource directoryperformance, in: Proceedings of the Telecommunication Forum2011, Belgrade, November 2011.

[34] Sensor Model Language (SensorML) <http://www.ogcnetwork.net/SensorML>.

[35] G. Coulson et al., Flexible experimentation in wireless sensornetworks, Magaz. Commun. ACM 55 (1) (2012) 82–90.

[36] S. Brown, C.J. Sreenan, Updating Software in Wireless SensorNetworks: A Survey, Technical Report UCC-CS2006-13-07, 2006.

Dr. Luis Sánchez received both the Telecom-munications Engineering and Ph.D. degree bythe University of Cantabria, Spain, in 2002 and2009 respectively. He is assistant professor atthe Dept. of Communications Engineering atthe University of Cantabria. He is active onmeshed networking on heterogeneous wire-less scenarios and optimization of networkperformance through cognitive networkingtechniques.

Prof. Luis Muñoz received both the Tele-communications Engineering degree andPh.D. from the Polytechnical University ofCataluña (UPC), Spain, in 1990 and 1995,respectively. He is head of the Network Plan-ning and Mobile Communications Laboratorybelonging to the Communications Engineer-ing Department (DICOM) at the University ofCantabria, Spain. His research focuses onadvanced data transmission techniques, het-erogeneous wireless multihop networks andapplied mathematical methods for telecom-

munications. He has participated in several National and Europeanresearch projects belonging to the 4th, 5th, 6th and 7th Framework Pro-gram in which he is technical manager of SmartSantander. He has pub-
lished over 150 journal and conference papers. He serves as editor ofseveral journals. In parallel to this activity, he serves as consultant for theSpanish Government as well as for different companies in Europe andUSA.
Jose Antonio Galache received Telecommu-nications Engineering degree by the Univer-sity of Cantabria, Spain, in 2004. He iscurrently active on different European pro-jects framed under the smart city paradigm,such as SmartSantander, as well as othersfostering collaboration between Europe andJapan, such as ClouT project, sharing experi-ences within smart cities framework. Hisresearch interests are on WSNs, wirelessrouting protocols and middleware platformsfor managing heterogeneous networks, as

well as M2M communications within the frame of a smart city. He iscurrently working in the realization of his Ph.D.

http://refhub.elsevier.com/S1389-1286(13)00433-7/h0020











r Networks 61 (2014) 217–238

Pablo Sotres works as research fellow in the
Network Planning and Mobile Communica-tions Laboratory, which belongs to theCommunications Engineering department atthe University of Cantabria, Spain. Hereceived Telecommunications Engineeringdegree by the University of Cantabria in2008. He is currently involved on differentEuropean projects framed under the smartcity paradigm, such as SmartSantander; andrelated to inter-testbed federation, such asFed4FIRE.
238 L. Sanchez et al. / Compute

Juan Ramón Santana is a Telecommunica-tion Engineer graduated in 2010 in theUniversity of Cantabria, who is working asresearch fellow in the Network Planning andMobile Communications Laboratory, a tele-communication research group from thesame university. Prior to his current occu-pation, he did an internship in the Univer-sity of Strathclyde (Glasgow) working on IoTsolutions for six months. Some of the pro-jects in which he is currently involved areSmartSantander and EAR-IT, European col-

laborative projects under the Seventh Framework Programme, relatedto the Smart City paradigm. He has multiples areas of interestincluding WSN (Wireless Sensor Networks), M2M communications and
Mobile phone application research.
Verónica Gutierrez received Telecommuni-cations Engineering degree by the Universityof Cantabria, Spain, in 2000. She is currently asenior researcher at the Network Planningand Mobile Communications Laboratory atthe University of Cantabria. She is active ondifferent European projects framed under thesmart city paradigm, such as SmartSantander,as well as others leveraging the Living Labconcept, sharing experiences within smartcities framework. Her research interests areon participatory sensing, augmented reality

and middleware platforms for IoT-based smart city services development.She is currently working in the realization of her Ph.D.

Rajiv Ramdhany is a senior researcher withthe Next Generation Distributed SystemsGroup at Lancaster University, where heworks on Adaptive Systems Software for AdHoc Networks. He has worked on improvingthe adaptability of operating systems featuressuch as protocol stacks and concurrency sup-port, particular in embedded systems andother resource-constrained environments. Healso has a background in middleware proto-cols. His research interests include the Inter-net of Things for smarter cities, multi-user

exploitation of commodity Wireless Sensor Networks and communica-tion protocols for ad hoc networks. Rajiv received his Ph.D. in 2011 for hiswork on ‘‘Supporting the dynamic deployment of ad hoc routing proto-
cols’’ and an M.Sc. in Distributed Systems in 2003 at Lancaster University.In between, Rajiv has worked as a lecturer in Computer Science at theUniversity of Mauritius.
Dr. Alexander Gluhak is a senior researchfellow at the Centre for Communication Sys-tems Research (CCSR) at the University of Sur-rey, UK, where he is coordinating Internet ofThings related research activities. He is alsocurrently technical manager of the ICT-FP7SENSEI project. He completed a Dipl.-Ing. (FH)degree from the University of Applied Sciencesin Offenburg, Germany in 2002 and a Ph.D.degree at the University of Surrey in 2006. Afterhis Ph.D. graduation he has held researchpositions with CSSR and later the Ericsson Ire-

land Research Centre. He was involved in the UK Virtual Centre of Excel-lence on Mobile and Personal Communications and contributed actively toseveral large European research projects, such as e-SENSE and SENSEI. His
research interests are next generation network architectures – in particularthe integration of the Internet of Things, service-oriented sensor networksand scalable context information infrastructures for next generation net-works. He is currently care taker of the Real World Internet cross domaintopic of the Future Internet Assembly (FIA) and serves as member on the FIAorganisational committee.
Dr. Srdjan Krco is with Ericsson since 2000where he held a number of positions and hasworked in and managed a number of productdevelopment and research projects. Since2008, Srdjan leads a research and innovationteam focusing on M2M, Internet of Things andtheir application in various domains includingthe smart cities. He has participated in severalFP7 projects (PROSENSE, SENSEI, SmartSant-ander, IOT-i, IoT6) and is active in the FutureInternet Assembly and IoT Forum. In 2007 hereceived the Innovation engineer of the Year

Award in Ireland from the Institute of Engineers of Ireland. Srdjan holds apart time assistant professor position at the University of Belgrade.

Dr. Evangelos Theodoridis (http://www.evangelostheodoridis.org) graduatedComputer Engineering and InformaticsDepartment, University of Patras, Greece.After that received a Master’s Degree in ‘‘Sci-ence and Technology of Computers’’ andreceived a Ph.D. in Computer Science from theDepartment of Computer Engineering &Informatics of the University of Patras, Greece.His research interests are focused in DataStructures, Algorithms, Information Retrieval,Databases, Data Mining, Web Technologies

and Internet of Things. Right now he is a researcher at Computer Tech-nology Institute and adjunct faculty of Computer Engineering and Infor-matics Department, University of Patras, Greece.

Dennis Pfisterer is an assistant professor atthe University of Lübeck, Germany. Heworked as a research assistant at the Euro-pean Media Laboratory in Heidelberg in thearea of resource adaptive systems. In 2003, hejoined Prof. Fischer’s group at the Braun-schweig Institute of Technology, and followedhim in 2005 to Lübeck to continue hisresearch on sensor networks at the Institute ofTelematics. He finished his doctorate in 2007and his habilitation in 2012. His currentresearch interests include distributed sys-

tems, the Internet of Things (IoT), wireless sensor networks, web services,service-oriented architectures, peer-to-peer networks, semantic webtechnologies, and network security.

Documents

SmartSantander: IoT experimentation over a smart city testbed