WINLAB

Capturing an Experiment: A Wireless Testbed Perspective

Ivan SeskarSeskar (at) winlab (dot) rutgers (dot) edu

Rutgers, The State University of New Jerseywww.winlab.rutgers.edu

WINLAB

Experiment Support Architecture

System

Control

MeasureDe

plo

y &

Co

nfig

ure

Experimenter

Results

Definition

WINLAB

ORBIT Interface

Orbit Nodes

Chasis Manager (CM)

Resource Controller

Orbit Measurement Lib

Collection Servicee

Collection Server

berkeley DB

Experiment Management Service

TFTP Server

Frisbee Server

APT Server

Experiment Controller

SQL DB

Collections Manager

Chassis Manager Controller

-End1

*-End2

*

Orbit Services

WINLAB

Orbit Measurement Library (OML)

User application

OML interface to user application

OML transport layer

OML data filter, id = xx

OML data filter, id = yy

OML data filter, id = zzpl

ugg

able

filte

rs, c

hose

n

by th

e ex

perim

ente

r

XDR Encoded data over multicast channel.

OML transport layerBerkeley DB

OML XDR decoder

SQL DB

OML SQL module

Orbit node Collection server

WINLAB

Defining an Experiment

Experiment

Static

- Mapping nodes to prototype

- Binding free properties

Dynamic

P1

Pi

Topology Application

Properties

Measurements

Test code

Prototype

App 1

Sys

Mapping Strategy

WINLAB

Experiment Definition

Experiment.name = "tutorial-1"Experiment.project = "Orbit::Tutorial"Experiment.startMode = Experiment::REBOOT

## Define nodes used in experiment#node([2, 3], :sender) {|n| n.prototype("http://apps.orbit-lab.org/sender", { :if => Node::W0_IF, :packetSize =>128 Experiment.parameter("packetSize"), :rate => Experiment.parameter("rate") })}node([3, 4]) {|n| n.prototype("http://apps.orbit-lab.org/receiver", {:if => Node::W0_IF})}

## Configure environment#node("/*/*").net.w0 {|n| n.essid = "HelloWorld" n.mode = "ad-hoc" n.channel = 1 n.xmitPower = 1 n.bitrate = "11Mbps" # Force single rate n.ip = "%10.0.%x.%y"}

# Start experimentwhenReady {|e, n| p.packetSize = 1024 p.rate = 250 e.sleep 5.0 e.done}

OMF Experiment Description Language, OEDL

WINLAB

ORBIT Grid

WINLAB8

Hardware Differences in commercial hardware Difference in tolerances and ageing for various low cost

components

Issues Affecting Archiving/Repeatability

Software Software or firmware bugs that may manifest as

inconsistent experimental results

Environment Uncontrolled interference over time and space Co-located infrastructure, opening and closing of doors,

movement of people etc.

WINLAB9

Device Calibration Setup

WINLAB10

Transmitter calibration Reciver calibration

• Calibrate transmit and receive side of the wireless cards

• Record corrections to be applied for each card based on the discrepancies observed

Calibration: Device Characterization

WINLAB11

Calibration: Temporal repeatabilityThe Experiment

WINLAB12

Calibration: Spatial repeatability

Blue sender sends to blue receiver

Yellow sender sends to yellow receiver

Experiment repeated on a different set of nodes with same topology

Spatial throughput variationsw.r.t mean for experimentduration averaged overdifferent experimental runs

WINLAB13

Calibration• Node calibration procedure necessary to

address hardware dependencies• Temporal variation less than device variability• Periodic re-calibration necessary• Need to automate the on-site calibration

procedure so that it can “monitor” the testbed “drift”

• Bounds on acceptable performance – Baseline standard

WINLAB

Archiving ORBIT ExperimentsExisting Support: • Image• Experiment script and/or commands on the console• State of services and environmental traces• Measurements (if using OML)• Device calibration info

What is missing• Full metadata • State of devices• Full RF environment (not necessarily critical for repeatability)

WINLAB

Wild Environment: WiMAX Basestation

Roof Antenna

• Wide-area packet based communication system (4G)• Operational with an educational license

Basestation (IDU) Unit

RF (ODU)Amplifier

WINLAB

MeasuredRSSI

Wild Environment: Coverage

WINLAB

Wild Environment: Mobility

Trajectory

RSSI

WINLAB

Instead of Conclusion

• Repeatability/replayability incentive

• Standardized anomaly monitoring• Storing metadata in archives• Baseline performance tests and

calibration