Embed Size (px)
Citation preview
Ex
pe
rim
en
tall
y D
riv
en
Re
se
arc
h
T-1
10.6
130
Sy
stem
s E
ng
inee
rin
g i
n D
ata
C
om
mu
nic
ati
on
s S
oft
wa
re
Ma
tti
Sie
kk
inen
30
.10
.20
12
Ou
tlin
e
• E
xperim
enta
lly d
riven r
esearc
h
–
What
and w
hy?
–
Ho
w t
o d
o it?
–
Wh
at
are
th
e c
ha
llen
ge
s?
• T
estb
eds a
nd tools
–
Cu
rre
ntly a
va
ilab
le
–
Co
min
g in
th
e f
utu
re
• C
onclu
sio
ns
Wh
at
is e
xp
eri
me
nta
lly
dri
ve
n r
es
ea
rch
?
• S
tudy h
ow
solu
tions (
pro
tocol, a
lgorith
m)
work
in
opera
tional setu
p
• G
oals
:
–
Un
de
rsta
nd
ho
w in
div
idu
al so
lutio
ns b
eh
ave
in
re
al w
orld
–
Un
de
rsta
nd
wh
at
are
in
tera
ctio
ns b
etw
ee
n in
div
idu
al so
lutio
ns
–
Imp
rove
th
e b
eh
avio
r th
rou
gh
re
de
sig
n o
r n
ove
l so
lutio
ns
• H
ow
?
–
De
sig
n,
imp
lem
en
t, d
ep
loy,
run
, a
nd
me
asu
re
Wh
at
is e
xp
eri
me
nta
tio
n?
• Q
: H
ow
is e
xperim
enta
tion d
efined?
• A
: It isn’t!
Wh
at’
s t
he
dif
fere
nc
e?
• T
esting
• E
xperim
enta
tion
• B
enchm
ark
ing
• T
rial
• P
ilot
• F
ollo
win
g s
lides a
re a
n a
ttem
pt to
define term
inolo
gy b
y
“Experim
enta
lly D
riven R
esearc
h”
FIR
Ew
ork
s (
EU
-IC
T)
white p
aper
Ex
pe
rim
en
tati
on
vs
. T
es
tin
g
• T
esting is w
ell
defined, e.g
. by E
TS
I –
Co
nfo
rma
nce
te
stin
g,
Inte
rop
era
bili
ty t
estin
g
• E
xperim
enta
tion (
in o
ur
dis
cip
line)
is n
ot w
ell
defined
–
Th
e o
rde
rly o
r m
eth
od
ica
l o
bse
rva
tio
n o
f th
e v
aria
tio
n o
f fa
cts
re
su
ltin
g
fro
m a
rtific
ial stim
uli
in a
re
pro
du
cib
le e
nviro
nm
en
t th
at
co
nfirm
s a
h
yp
oth
esis
(ve
rifica
tio
n)
or
reje
cts
it
(fa
lsific
atio
n)
• In
short
:
–
Mo
dify s
tim
uli !
ob
se
rve
im
pa
ct
–
Hyp
oth
esis
is v
alid
if w
e c
an
sh
ow
th
at
an
exp
erim
en
t ca
n b
e
rep
rod
uce
d
“Exp
erim
en
tally
Drive
n R
ese
arc
h”
FIR
Ew
ork
s w
hite
pa
pe
r
Ex
pe
rim
en
tati
on
vs
. T
es
tin
g (
co
nt.
)
• K
now
ledge a
bout th
e r
ight stim
uli
and o
bserv
ation p
oin
ts o
f a
syste
m?
–
Te
sti
ng
: kn
ow
n lis
t o
f stim
uli
an
d o
bse
rva
tio
n p
oin
ts f
or
ch
eckin
g
co
rre
ctn
ess
–
Ex
pe
rim
en
tati
on
: se
arc
h f
or
the
rig
ht
stim
uli
an
d o
bse
rva
tio
n p
oin
ts
for
asse
ssm
en
t
• Level of m
atu
rity
of th
e k
now
ledge a
bout th
e b
ehavio
r of a
syste
m?
–
Dra
w a
lin
e b
etw
ee
n r
es
ea
rch
(e
xp
erim
en
tatio
n)
an
d d
evelo
pm
en
t (t
estin
g)
“Exp
erim
en
tally
Drive
n R
ese
arc
h”
FIR
Ew
ork
s w
hite
pa
pe
r
Ex
pe
rim
en
tati
on
vs
. B
en
ch
ma
rkin
g
• B
en
ch
mark
ing
–
Co
ntr
olle
d (
oft
en
op
tim
al) c
on
ditio
ns
–
Me
asu
re f
un
ctio
na
l a
nd
/or
pe
rfo
rma
nce
me
tric
s
–
Co
mp
are
re
su
lts t
o a
n e
xis
tin
g s
pe
cific
atio
n
• E
.g.
we
ll-a
cce
pte
d in
du
str
y s
tan
da
rd
“Exp
erim
en
tally
Drive
n R
ese
arc
h”
FIR
Ew
ork
s w
hite
pa
pe
r
En
su
rin
g q
ua
lity
in
ex
pe
rim
en
tati
on
• S
om
e im
port
ant pro
pert
ies o
f experim
enta
tion
–
Ve
rifia
bili
ty
• W
e c
an
fin
d a
fo
rma
l m
od
el th
at
is e
qu
iva
len
t to
exp
erim
en
tal m
od
el
(pro
du
ce
s s
am
e r
esu
lts t
ha
n t
he
exp
erim
en
ts)
–
Re
liab
ility
• P
rob
ab
ility
th
at
syste
m w
ill p
erf
orm
its
in
ten
de
d f
un
ctio
n d
urin
g a
sp
ecifie
d
pe
rio
d o
f tim
e u
nd
er
sta
ted
co
nd
itio
ns
–
Re
pe
ata
bili
ty
• W
e g
et
alw
ays t
he
sa
me
re
su
lt w
ith
sa
me
pa
ram
ete
rs a
nd
in
pu
t va
ria
ble
s
–
Re
pro
du
cib
ility
• W
e g
et
sa
me
re
su
lts w
ith
sa
me
exp
erim
en
tatio
n s
etu
p o
n a
diffe
ren
t e
xp
erim
en
tal syste
m
“Exp
erim
en
tally
Drive
n R
ese
arc
h”
FIR
Ew
ork
s w
hite
pa
pe
r
Sc
ale
an
d c
os
t o
f e
xp
eri
me
nta
tio
n
• Larg
e s
cale
?
–
In L
SI/
VL
SI
circu
it d
esig
n it
wa
s la
rge
-sca
le 1
00
-50
00
circu
it e
lem
en
ts
ve
ry-la
rge
-sca
le (
5k-5
0k),
su
pe
r-la
rge
-sca
le (
50
k-1
00
k)
ultra
-la
rge
-sca
le (
>1
00
k) !
• C
ost
facto
r as a
measure
–
Co
st
is a
fu
nctio
n o
f co
mp
lexity,
dim
en
sio
n a
nd
en
viro
nm
en
tal
co
nd
itio
ns
–
La
rge
-sc
ale
: co
st
exce
ed
s t
he
co
st
of
a la
bo
rato
ry e
nviro
nm
en
t b
y
on
e o
r m
ore
le
ve
ls o
f m
ag
nitu
de
“Exp
erim
en
tally
Drive
n R
ese
arc
h”
FIR
Ew
ork
s w
hite
pa
pe
r
Wh
y is
ex
pe
rim
en
tally
dri
ve
n r
es
ea
rch
necessary
?
• E
valu
ate
solu
tions e
ven if!
–
an
aly
tica
l m
od
elin
g o
f th
eir b
eh
avio
r is
difficu
lt
–
sim
ula
tio
n is u
nfe
asib
le (
do
es n
ot
sca
le e
no
ug
h)
–
su
ita
ble
sim
ula
tio
n t
oo
ls d
o n
ot
exis
t
• S
ee n
on-t
rivia
l dependencie
s b
etw
een p
ara
mete
rs
–
Sim
ula
tio
ns a
re o
nly
as g
oo
d a
s t
he
mo
de
ls t
he
y r
ely
on
• P
rovid
e input fo
r sim
ula
tors
–
Th
ink a
bo
ut
wh
ere
wo
rklo
ad
s a
nd
sim
ula
tio
n m
od
els
co
me
fro
m
–
E.g
. to
po
log
ies f
or
sim
ula
tin
g r
ou
tin
g p
roto
co
ls
Ou
tlin
e
• E
xperim
enta
lly d
riven r
esearc
h
–
What
and w
hy?
–
Ho
w t
o d
o it?
–
Wh
at
are
th
e c
ha
llen
ge
s?
• T
estb
eds a
nd tools
–
Cu
rre
ntly a
va
ilab
le
–
Co
min
g in
th
e f
utu
re
• C
onclu
sio
ns
Typ
ical p
rocess
• (R
e)D
esig
n &
im
ple
ment solu
tion a
nd d
eplo
y it
–
E.g
. o
ve
rla
y r
ou
tin
g p
roto
co
l
• E
xperim
enta
tion a
nd c
olle
ction o
f m
easure
ments
–
Tra
ffic
tra
ce
s,
ap
plic
atio
n lo
gs,
etc
.
–
Inp
ut
for
da
ta a
na
lysis
• In
fere
nce / A
naly
sis
–
Ca
n b
e a
lso
in
teg
rate
d in
to m
ea
su
rem
en
t
–
Le
arn
ho
w s
olu
tio
n b
eh
ave
s a
nd
(o
ptio
na
lly)
go
ba
ck t
o
be
gin
nin
g
13
Ex
pe
rim
en
t &
me
as
ure
Da
ta
an
aly
sis
(Re
)De
sig
n
& i
mp
lem
en
t
Ex
pe
rim
en
tati
on
ap
pro
ac
he
s
• E
mula
tions
• C
losed p
roprieta
ry testb
eds
–
Usu
ally
sm
all
or
me
diu
m s
ize
• O
pen t
estb
eds
–
Sm
all
to la
rge
sca
le
• T
he Inte
rnet
–
Hu
ge
sca
le
Em
ula
tio
ns
• In
our
dis
cip
line: S
om
eth
ing in b
etw
een s
imula
tions a
nd r
eal
testb
eds
• P
art
of th
e s
yste
m is r
eal
–
Re
pro
du
ce
s e
xa
ctly t
he
orig
ina
l kin
d o
f b
eh
avio
r –
Wh
ere
as s
imu
late
d s
yste
m b
eh
ave
s a
cco
rdin
g t
o a
mo
de
l
• T
ypic
al case:
–
Re
al d
evic
es (
or
virtu
al m
ach
ine
s)
run
nin
g a
re
al a
pp
lica
tio
n
• A
s o
pp
ose
d t
o a
pp
lica
tio
n w
ork
loa
d f
or
sim
ula
tor
–
Sim
ula
ted
ne
two
rk
• N
S-2
/3
• W
hat’s the p
oin
t?
–
Ca
n m
ea
su
re b
eh
avio
r o
f re
al d
evic
es a
nd
ap
plic
atio
ns o
ve
r co
ntr
olle
d
ne
two
rk e
nviro
nm
en
t –
Eva
lua
te h
ow
ne
two
rk p
ara
me
ters
im
pa
ct
de
vic
e/a
pp
lica
tio
n b
eh
avio
r
Em
ula
tio
n e
xa
mp
le
sin
gle
co
mp
ute
r
Clo
se
d t
es
tbe
ds
• Y
ou c
an b
uild
your
ow
n testb
ed
–
In y
ou
r o
wn
la
b
–
No
t o
pe
n t
o o
the
rs
–
Oft
en
sm
all
sca
le
• A
dvanta
ges
–
“Ea
sy”
to c
on
tro
l –
(Sh
ou
ld)
kn
ow
exa
ctly w
ha
t is
go
ing
on
–
“Ea
sy”
to c
olle
ct
me
asu
rem
en
ts
• P
roble
ms
–
Ca
n b
e s
ub
sta
ntia
l a
mo
un
t o
f w
ork
•
Eve
n if
sm
all
sca
le
–
Re
pro
du
cib
ility
of
resu
lts c
an
be
qu
estio
na
ble
•
Ca
n y
ou
co
ntr
ol a
ll th
e r
ela
ted
pa
ram
ete
rs?
Ex
am
ple
of
clo
se
d t
es
tbe
d
• V
ery
sm
all
multi-hop s
tream
ing s
etu
ps
• C
oncre
te w
alls
for
limitin
g inte
rfere
nce
Op
en
te
stb
ed
s
• T
estb
eds that are
built
for
use b
y o
thers
as w
ell
• S
mall
to larg
e s
cale
–
Sm
all/
me
diu
m:
sin
gle
la
b s
etu
p
–
La
rge
: co
op
era
tive
te
stb
ed/p
latf
orm
• T
here
are
severa
l availa
ble
for
researc
h p
urp
oses
–
Pla
ne
tla
b,
em
ula
b, !
• T
he Inte
rnet is
an e
xtr
em
e c
ase
–
Op
en
to
eve
ryo
ne
–
So
, ca
n b
e v
iew
ed
as o
ne
ve
ry la
rge
op
en
te
stb
ed
–
Th
e m
ost
rea
listic c
ase
fo
r so
lutio
ns t
arg
ete
d f
or
Inte
rne
t d
ep
loym
en
t
Ou
tlin
e
• E
xperim
enta
lly d
riven r
esearc
h
–
What
and w
hy?
–
Ho
w t
o d
o it?
–
Wh
at
are
th
e c
ha
llen
ge
s?
• T
estb
eds a
nd tools
–
Cu
rre
ntly a
va
ilab
le
–
Co
min
g in
th
e f
utu
re
• C
onclu
sio
ns
Wh
y is
ex
pe
rim
en
tati
on
ch
alle
ng
ing
?
• U
sually
cannot experim
ent w
ith o
pera
tional netw
ork
s
–
No
arc
hite
ctu
ral su
pp
ort
–
Ho
w t
o d
ep
loy s
olu
tio
ns?
–
Ho
w t
o d
o m
ea
su
rem
en
ts?
• C
ost
–
Ne
ed
sp
ecia
lize
d s
olu
tio
ns
–
Ne
ed
la
rge
sca
le d
ep
loym
en
ts
• W
e o
fte
n w
an
t In
tern
et
like
re
alis
m f
rom
eva
lua
tio
n
–
Bo
th c
osts
, la
bo
r a
nd
eq
uip
me
nt
• A
vaila
bili
ty o
f m
eans to d
o it
–
too
ls,
pla
tfo
rms,
ne
two
rks,
etc
.
–
co
st
issu
e
Sim
ula
tio
n !
Em
ula
tio
n !
Ex
peri
me
nta
tio
n
• C
ost vs. re
alis
m
Ou
tlin
e
• E
xperim
enta
lly d
riven r
esearc
h
–
What
and w
hy?
–
Ho
w t
o d
o it?
–
Wh
at
are
th
e c
ha
llen
ge
s?
• T
estb
eds a
nd tools
–
Cu
rre
ntly a
va
ilab
le
–
Co
min
g in
th
e f
utu
re
• C
onclu
sio
ns
Wh
at
is a
va
ila
ble
?
No
w
• T
estb
eds
–
Pla
ne
tLa
b
–
Em
ula
b
–
OR
BIT
–
NIT
OS
–
CityS
en
se
–
WIS
EB
ED
• A
ltern
ative a
ppro
aches
–
OpenF
low
–
Clic
k
–
Ne
tFP
GA
In t
he f
utu
re
• T
estb
eds
–
GE
NI
–
Sm
art
Sa
nta
nd
er
– !
• F
edera
ted testb
eds
Pla
ne
tLa
b
• G
lobal dis
trib
ute
d s
yste
m infr
astr
uctu
re
–
pla
tfo
rm f
or
lon
g r
un
nin
g s
erv
ice
s
–
testb
ed f
or
ne
two
rk e
xp
erim
en
ts
Pla
ne
tla
b:
fac
ts a
nd
fig
ure
s
• Launched in M
arc
h 2
002
• 1011 n
odes a
round the w
orld
–
35
co
un
trie
s
–
54
5 s
ite
s (
un
ive
rsitie
s,
rese
arc
h la
bs)
–
11
43
no
de
s
• A
colle
ction o
f m
achin
es d
istr
ibute
d a
round the g
lobe
–
Mo
st
of
the
ma
ch
ine
s a
re h
oste
d b
y r
ese
arc
h in
stitu
tio
ns
• A
ll m
achin
es a
re c
onnecte
d to the Inte
rnet
• A
ll m
achin
es a
re a
dm
inis
tere
d b
y a
syste
m c
alle
d
MyP
LC
–
Th
e s
oft
wa
re is s
up
po
rte
d o
n s
eve
ral fla
vo
rs o
f L
inu
x
Pla
ne
tLa
b a
rch
ite
ctu
re
• S
ite:
physic
al lo
cation o
f a P
lanetL
ab n
ode
–
E.g
. F
rau
nh
ofe
r In
stitu
te o
r U
CL
• N
od
e:
serv
er
that ru
ns c
om
ponents
of P
lanetL
ab s
erv
ices
• S
lice:
set of allo
cate
d r
esourc
es d
istr
ibute
d a
cro
ss P
lanetL
ab
–
UN
IX s
he
ll a
cce
ss t
o p
riva
te v
irtu
al se
rve
rs o
n a
se
lecte
d s
et
of
Pla
ne
tLa
b n
od
es
–
Use
r a
ssig
ns a
se
t o
f P
lan
etL
ab n
od
es t
o a
slic
e
• V
irtu
al se
rve
rs f
or
tha
t slic
e a
re c
rea
ted
on
ea
ch
of
the
assig
ne
d n
od
es
• S
liver:
slic
e r
unnin
g o
n a
specific
node
–
Yo
u c
an
use
ssh t
o lo
gin
to
a s
live
r o
n a
sp
ecific
no
de
• F
air s
hare
allo
cation o
f re
sourc
es p
er
sliv
er
–
CP
U a
nd
lin
k b
an
dw
idth
Pla
ne
tLa
b a
rch
ite
ctu
re (
co
nt.
)
• N
odes
Pla
ne
tLa
b a
rch
ite
ctu
re (
co
nt.
)
• S
lice 1
Pla
ne
tLa
b a
rch
ite
ctu
re (
co
nt.
)
• S
lice 2
Pla
ne
tLa
b a
rch
ite
ctu
re (
co
nt.
)
• S
lices 1
&2
Us
ing
Pla
ne
tLa
b
• C
entr
al W
ebsite that m
anages
–
All
acco
un
ts
–
All
no
de
s
–
All
reso
urc
es
• R
egis
tering w
ith o
ne o
f 3 P
LC
s (
your
Pla
netL
ab C
entr
al)
–
PL
US
A (
pla
ne
t-la
b.c
om
)
–
PL
Eu
rop
e (
pla
ne
t-la
b.e
u)
–
PL
Ja
pa
n (
pla
ne
t-la
b.jp)
• D
iffe
rent kin
ds o
f m
em
bers
hip
s a
vaila
ble
dependin
g o
n
kin
d o
f in
stitu
tion
–
HII
T a
nd
Aa
lto
(C
om
ne
t@E
LE
C)
are
me
mb
ers
Pla
ne
tLa
b:
pro
s a
nd
co
ns
• P
ros:
–
Acce
ss t
o la
rge
sca
le d
istr
ibu
ted
re
so
urc
es
–
Ru
n e
xp
erim
en
ts w
ith
co
mp
lete
co
ntr
ol o
ve
r e
ach
no
de
•
Re
str
icte
d r
oo
t a
cce
ss
–
Co
nn
ectivity o
ve
r re
al In
tern
et
pa
ths
–
Sca
le f
rom
on
e t
o f
ew
to
ma
ny n
od
es
–
Mo
nito
r C
PU
an
d n
etw
ork
tra
ffic
–
De
plo
y lo
ng
-ru
nn
ing
exp
erim
en
tal se
rvic
es
• C
ons:
–
Sh
are
d r
eso
urc
es
• C
row
de
d m
ach
ine
s c
an
ca
use
bia
se
d e
xp
erim
en
ts
–
Sta
bili
ty
• N
od
es g
o d
ow
n
• M
an
ag
em
en
t o
fte
n n
ot
hig
h p
rio
rity
Em
ula
b
• N
etw
ork
te
stb
ed
–
Pro
vid
es r
em
ote
acce
ss t
o
cu
sto
m e
mu
late
d n
etw
ork
s
• D
evelo
ped a
t U
niv
ers
ity o
f U
tah a
round the y
ear
2000
–
Cu
rre
ntly m
an
y d
ep
loym
en
ts
aro
un
d t
he
wo
rld
exis
t
–
Priva
te,
op
en
, sp
ecific
p
urp
ose
(e
.g.
tea
ch
ing
, re
se
arc
h,
se
cu
rity
re
se
arc
h)
!
• W
ho c
an u
se it?
–
Op
en
de
plo
ym
en
ts c
an
be
u
se
d b
y e
xte
rna
l re
se
arc
he
rs
34
Em
ula
b C
ha
rac
teri
sti
cs
• P
hysic
al vie
wpoin
t: L
arg
e s
witched L
AN
with c
ontr
ol softw
are
•
How
it w
ork
s
–
Cre
ate
s c
usto
m n
etw
ork
to
po
log
ies s
pe
cifie
d b
y u
se
rs in
NS
–
So
ftw
are
ma
na
ge
s P
C c
luste
r, s
witch
ing
fa
bric
• U
ser
can
–
Re
pla
ce
no
de
OS
so
ftw
are
–
Co
nfig
ure
lin
k t
op
olo
gy
• U
se
s V
irtu
al L
AN
s t
o im
ple
me
nt
arb
itra
ry a
nd
iso
late
d t
op
olo
gie
s
–
Co
ntr
ol “p
hysic
al” lin
k c
ha
racte
ristics
• S
ha
pe
la
ten
cy/b
an
dw
idth
/dro
ps/e
rro
rs
• D
on
e v
ia in
vis
ibly
in
terp
ose
d “
sh
ap
ing
no
de
s”
• A
lso h
as a
set of w
irele
ss 8
02.1
1 c
apable
nodes
–
Bo
th in
fra
str
uctu
re a
nd
ad
-ho
c m
od
e s
up
po
rte
d
E
mu
lab
: E
xa
mp
le T
op
olo
gy
Co
nfi
gu
rati
on
36
set ns [new
Sim
ula
tor]
sourc
e tb
_com
pat.tc
l
set node
A [$ns n
ode]
set nodeB
[$ns n
ode]
set nodeC
[$ns n
ode]
set nodeD
[$ns n
ode]
set lin
k0 [$ns d
uple
x-lin
k $
node
A $
nodeB
30M
b 5
0m
s D
ropTail]
set lin
k1 [$ns d
uple
x-lin
k $
node
A $
nodeC
30M
b 5
0m
s D
ropTail]
set lin
k2 [$ns d
uple
x-lin
k $
nodeC
$nodeD
30M
b 5
0m
s D
ropTail]
set lin
k3 [$ns d
uple
x-lin
k $
nodeB
$nodeD
30M
b 5
0m
s D
ropTail]
$ns r
tpro
to S
tatic
$ns r
un
30
Mb
5
0m
s
" N
etw
ork
testb
ed m
appin
g p
roble
m
#
Map v
irtu
al netw
ork
to s
hare
d p
hysic
al re
sourc
es
#
I.e
. se
lect
ha
rdw
are
on
wh
ich
to
in
sta
ntia
te n
etw
ork
exp
erim
en
ts
#
Optim
ization p
roble
m w
ith s
et of constr
ain
ts
#
Ric
ci et al: “
A S
olv
er
for
the N
etw
ork
Testb
ed M
appin
g P
roble
m”.
In
SIG
CO
MM
CC
R. 2003.
Em
ula
b:
Mo
bile
Se
ns
or
Ad
dit
ion
s
• S
eve
ral u
se
r-co
ntr
olla
ble
mo
bile
ro
bo
ts
–
On
bo
ard
PD
A, W
iFi, a
nd
att
ach
ed
se
nso
r m
ote
• M
an
y f
ixe
d m
ote
s s
urr
ou
nd
mo
tio
n a
rea
–
Sim
ple
ma
ss r
ep
rog
ram
min
g t
oo
l
–
Co
nfig
ura
ble
pa
cke
t lo
gg
ing
– !
37
OR
BIT
• O
pe
n-A
cce
ss
Re
se
arc
h T
estb
ed
for
Ne
xt-
Ge
ne
ratio
n
Wire
less N
etw
ork
s
• D
eve
lop
ed
(2
00
5)
an
d o
pe
rate
d b
y
WIN
LA
B,
Ru
tge
rs
Un
ive
rsity
OR
BIT
ch
ara
cte
ris
tic
s
• W
irele
ss testb
ed
–
Sca
le:
tota
l n
od
es ~
10
0’s
• C
ontr
olle
d e
nvironm
ent
–
Re
pro
du
cib
le e
xp
erim
en
ts
–
Use
r ca
n c
on
tro
l p
roto
co
ls a
nd
so
ftw
are
use
d o
n t
he
ra
dio
no
de
s
• M
easure
ment capabili
ties o
n r
adio
PH
Y, M
AC
and n
etw
ork
levels
•
Testb
ed is r
em
ote
ly a
ccessib
le
–
un
ma
nn
ed
op
era
tio
n
–
ab
le t
o d
ea
l w
ith
so
ftw
are
an
d h
ard
wa
re f
ailu
res
• W
ho c
an u
se it?
–
Un
ive
rsitie
s,
ind
ustr
ial re
se
arc
h la
bs
–
Bo
th U
S a
nd
no
n-U
S
• S
eem
s to b
e s
till
active b
ased o
n u
ser
maili
ng lis
t –
Se
em
s a
lso
no
t to
ha
ve
co
mp
lete
ly a
uto
ma
tic f
ailu
re h
an
dlin
g!
51
2 M
B
RA
M
Gig
abit
Eth
ern
et
(contr
ol)
Gig
abit
Eth
ern
et
(data
)
Inte
l/A
thero
s
min
iPC
I
802.1
1
a/b
/g
22.1
Mhz
1 G
hz p
wr/
reset
volt/tem
p
20
GB
D
ISK
S
erial
Console
11
0
VA
C
RJ11
NodeId
Box
+5
v s
tan
db
y
Pow
er
Supply
CP
U
VIA
C3
1G
hz
Inte
l/A
thero
s
min
iPC
I
802.1
1
a/b
/g
Blu
eto
oth
U
SB
CP
U
Rab
bit
Sem
i R
CM
3700
10 B
aseT
Eth
ern
et
(CM
)
Orb
it r
ad
io n
od
e
OR
BIT
: In
do
or
Gri
d
80 f
t (
20 n
od
es )
70 ft m ( 20 nodes )
Co
ntr
ol sw
itch
Data
sw
itch
A
pp
licati
on
Serv
ers
(U
ser
ap
plicati
on
s/
Dela
y n
od
es
/
Mo
bilit
y C
on
tro
llers
/ M
ob
ile N
od
es)
Inte
rnet
VP
N G
ate
way /
Fir
ew
all
Back-e
nd
serv
ers
Fro
nt-
en
d
Serv
ers
Gig
ab
it b
ackb
on
e
VP
N G
ate
way t
o
Wid
e-A
rea T
estb
ed
SA
1 S
A2
SA
P IS
1 IS
2 IS
Q
RF
/Sp
ec
tru
m M
ea
su
rem
en
ts
Inte
rfe
ren
ce
So
urc
es
Ru
nn
ing
ex
pe
rim
en
ts
• U
nlik
e w
ired testb
eds, difficult to isola
te e
xperim
ents
–
Se
ria
l m
od
e o
f o
pe
ratio
n
–
Qu
ickly
off
loa
d u
se
rs a
t th
e e
nd
of
the
slo
t
• S
chedulin
g s
yste
m for
requesting a
nd a
llocating s
lots
for
experim
ents
Wh
at
ab
ou
t m
ob
ilit
y in
OR
BIT
?
• V
irtu
al m
obili
ty
• E
mula
tes tra
jecto
ry b
y s
witchin
g to d
iffe
rent ra
dio
and
ante
nna p
ositio
ns a
s tim
e p
rogre
sses
• D
iscre
tized g
rid m
obili
ty
–
Virtu
al m
ob
ile n
od
e a
pp
ea
rs t
o b
e a
t lo
ca
tio
n i b
y u
sin
g r
ad
io
grid
no
de
i
–
Use
s g
rid
ra
dio
no
de
j w
he
n it
"mo
ve
s"
to g
rid
lo
ca
tio
n j
–
No
thin
g m
ove
s p
hysic
ally
Oth
er
ex
isti
ng
te
stb
ed
s
• W
irele
ss testb
eds b
uilt
on O
MF
–
Op
en
so
urc
e c
trl &
mg
mt
so
ftw
are
•
NIC
TA
an
d W
inla
b (
Orb
it)
–
NIT
OS
•
Wire
less (
80
2.1
1)
testb
ed a
t N
ICT
A
• O
rbit n
od
es a
nd
dis
kle
ss n
od
es (
45
to
tal)
• M
ulti-u
se
r e
xp
erim
en
tatio
n
–
Spectr
um
slic
ing (
channel allo
cation)
–
Ma
ny o
the
rs,
mo
stly s
ma
ll clo
se
d te
stb
ed
s
• C
ityS
ense
–
Urb
an
-Sca
le S
en
so
r N
etw
ork
Te
stb
ed
–
26
we
ath
er,
CO
2,
an
d n
ois
e p
ollu
tio
n s
en
so
r n
od
es
de
plo
ye
d in
Ca
mb
rid
ge
, M
A
• P
rog
ram
ma
ble
by u
se
rs
–
Se
em
s in
active
sin
ce
a c
ou
ple
of
ye
ars
Un
ive
rsity o
f T
he
ssa
ly's
ca
mp
us b
uild
ing
(G
ree
ce
)
Oth
er
ex
isti
ng
te
stb
ed
s (
co
nt.
)
• W
ISE
BE
D
–
“Mu
lti-le
ve
l in
fra
str
uctu
re o
f in
terc
on
ne
cte
d te
stb
ed
s o
f la
rge
-sca
le w
ire
less s
en
so
r n
etw
ork
s f
or
rese
arc
h p
urp
ose
s”
–
9 te
stb
ed
s a
rou
nd
Eu
rop
e
• In
do
or
WS
Ns
• P
HY
/MA
C:
mo
stly 8
02
.15
.4,
als
o s
om
e n
on
-sta
nd
ard
so
lutio
ns
use
d
–
We
b b
ase
d c
lien
ts to
ma
na
ge
exp
erim
en
ts
• H
ave
de
ve
lop
ed
AP
Is t
o e
xp
ose
th
e W
SN
se
rvic
es
–
EU
pro
ject
• A
lre
ad
y f
inis
he
d
• N
ot
su
re if
the
se
are
re
ally
usa
ble!
Ou
tlin
e
• E
xperim
enta
lly d
riven r
esearc
h
–
What
and w
hy?
–
Ho
w t
o d
o it?
–
Wh
at
are
th
e c
ha
llen
ge
s?
• T
estb
eds a
nd tools
–
Cu
rre
ntly a
va
ilab
le
• U
se
ca
se
s
–
Co
min
g in
th
e f
utu
re
• C
onclu
sio
ns
Pla
ne
tLa
b u
se c
ases
• E
GO
IST
[1-4
] –
Ove
rla
y r
ou
tin
g s
olu
tio
n
–
Eva
lua
ted
on
Pla
ne
tLa
b o
ve
r a
pe
rio
d o
f tw
o y
ea
rs
–
Exte
nsiv
e m
ea
su
rem
en
ts o
f p
ath
s b
etw
ee
n 5
0 P
L n
od
es
–
De
mo
nstr
ate
d
• su
pe
rio
rity
of E
GO
IST
's n
eig
hb
ou
r se
lectio
n p
rim
itiv
es o
ve
r e
xis
tin
g h
eu
ristics
• e
ffe
ctive
ne
ss u
nd
er
sig
nific
an
t ch
urn
, re
sis
tan
ce
to
ch
ea
tin
g,
an
d s
ma
ll o
ve
rhe
ad
• R
adar:
Inte
rnet to
polo
gy[5
-7]
–
"In
tern
et
rad
ar"
wh
ich
"d
raw
s"
the
In
tern
et
top
olo
gy a
s it
evo
lve
s o
ve
r tim
e
–
run
s p
erio
dic
tre
e-lik
e m
an
ne
r tr
ace
rou
tes t
o a
se
t o
f d
estin
atio
ns
• re
du
ce
s t
he
in
du
ce
d t
raff
ic a
nd
da
ta r
ed
un
da
ncy
–
15
0 P
lan
etL
ab n
od
es a
s m
on
ito
rs a
rou
nd
th
e w
orld
• O
bse
rve
s d
iffe
ren
ce
s b
etw
ee
n g
eo
gra
ph
ica
lly d
ive
rse
pa
rts o
f th
e n
etw
ork
Em
ula
b u
se c
ases
• V
OID
(V
vire
less O
nlin
e I
nte
rfe
ren
ce
De
tecto
r) [
8]
–
No
n in
tru
siv
e in
terf
ere
nce
de
tectio
n f
or
80
2.1
1 n
etw
ork
s
• A
pply
sta
tistical m
eth
ods o
n thro
ughput sum
maries a
t upstr
eam
wired r
oute
rs
• F
igure
out w
hic
h d
evic
es a
re the inte
rfere
rs -
> n
ecessary
to k
now
befo
re c
an
min
imiz
e the inte
rfere
nce
–
Sh
ow
ed
eff
ective
ne
ss w
ith
Em
ula
b’s
wire
less f
ea
ture
s
• S
ca
rle
tt [
9]
–
Ske
we
d c
on
ten
t p
op
ula
rity
ca
use
s p
erf
orm
an
ce
bo
ttle
ne
cks in
Ma
pR
ed
uce
file
syste
ms
• U
niform
data
replic
ation
• C
onte
ntion for
slo
ts o
n m
achin
es s
toring m
ore
popula
r blo
cks
–
Sca
rle
tt r
ep
lica
tes f
iles b
ase
d o
n t
he
ir a
cce
ss p
att
ern
s
• S
pre
ads o
ut to
avoid
hot spots
–
Pa
rt o
f e
va
lua
tio
n d
on
e w
ith
Ha
do
op r
un
nin
g o
n D
ET
ER
te
stb
ed (
Em
ula
b a
t U
SC
IS
I a
nd
UC
Be
rke
ley m
ain
ly f
or
se
cu
rity
re
se
arc
h)
OR
BIT
use c
ase
• E
valu
ating T
CP
Sim
ultaneous S
end P
roble
m o
ver
802.1
1 [
10]
–
Pro
ble
m:
hig
h M
AC
co
nte
ntio
n b
etw
ee
n d
ata
an
d A
CK
pa
cke
ts
du
rin
g b
ulk
tra
nsfe
r
–
So
lutio
n:
AC
K s
kip
pin
g
–
Bo
th f
rom
ea
rlie
r a
na
lytic a
nd
sim
ula
tio
n s
tud
ies
–
Use
OR
BIT
to
eva
lua
te p
rob
lem
an
d s
olu
tio
n
–
Re
su
lts
• C
on
firm
se
ve
rity
of
pro
ble
m
• C
on
firm
th
at
AC
K s
kip
pin
g a
llevia
tes p
rob
lem
• H
ow
eve
r, n
ot
as m
uch
as s
imu
latio
ns s
ug
ge
ste
d
–
Du
e t
o T
CP
im
ple
me
nta
tio
n d
iffe
ren
ce
s
Us
e c
as
e r
efe
ren
ce
s
[1] G
. S
mara
gdakis
, N
. Laouta
ris, P
. M
ichia
rdi, A
. B
esta
vro
s, J. W
. B
yers
, M
. R
oussopoulo
s. D
istr
ibute
d N
etw
ork
Form
ation
for
n-w
ay B
roadcast
Applic
ations. IE
EE
Tra
nsactions o
n P
ara
llel and D
istr
ibute
d S
yste
ms, 21(1
0),
2010.
[2] G
. S
mara
gdakis
, N
. Laouta
ris, V
. Lekakis
, A
. B
esta
vro
s, J. W
. B
yers
, M
. R
oussopoulo
s. E
GO
IST
: O
verlay R
outing u
sin
g
Selfis
h N
eig
hbor
Sele
ction.
AC
M C
oN
EX
T 2
008.
[3] G
. S
mara
gdakis
, N
. Laouta
ris, P
. M
ichia
rdi, A
. B
esta
vro
s, J. W
. B
yers
, M
. R
oussopoulo
s. S
warm
ing o
n O
ptim
ized G
raphs
for
n-w
ay B
roadcast. IE
EE
IN
FO
CO
M 2
008.
[4] N
. Laouta
ris, G
. S
mara
gdakis
, A
. B
esta
vro
s, J. W
. B
yers
. Im
plic
ations o
f S
elfis
h N
eig
hbor
Sele
ction in O
verlay N
etw
ork
s.
IEE
E IN
FO
CO
M 2
007.
[5]
A. H
am
zaoui, M
. Lata
py, C
. M
agnie
n. D
ete
cting e
vents
in the d
ynam
ics o
f ego-c
ente
red m
easure
ments
of th
e Inte
rnet
topolo
gy. P
roceedin
gs o
f In
tern
ational W
ork
shop o
n D
ynam
ic N
etw
ork
s (
WD
N),
in c
on
junction w
ith W
iOpt
2010.
[6] C
. M
agnie
n, F
. O
uedra
ogo, G
. V
ala
don, M
. Lata
py. F
ast dynam
ics in Inte
rnet to
polo
gy: pre
limin
ary
observ
ations a
nd
expla
nations. F
ourt
h Inte
rnational C
onfe
rence o
n Inte
rnet M
onitoring a
nd P
rote
ction (
ICIM
P 2
009),
May 2
4-2
8, 2009,
Venic
e, Italy
.
[7] M
. Lata
py, C
. M
agnie
n, F
. O
uédra
ogo.
A R
adar
for
the Inte
rnet. P
roceedin
gs o
f A
DN
'08: 1st In
tern
ational W
ork
shop o
n
Analy
sis
of D
ynam
ic N
etw
ork
s, in
con
jonction w
ith IE
EE
IC
DM
2008.
[8]
Cai, K
., B
lacksto
ck, M
., F
eele
y, M
. J., a
nd K
rasic
, C
. 2009. N
on-intr
usiv
e, dynam
ic inte
rfere
nce d
ete
ction for
802.1
1
netw
ork
s. In
Pro
ceedin
gs o
f th
e 9
th A
CM
Inte
rnet M
easure
ment C
onfe
rence. IM
C 2
009.
[9] G
anesh A
nanth
anara
yanan, S
am
eer
Agarw
al, S
rikanth
Kandula
, A
lbert
Gre
enberg
, Io
n S
toic
a, D
uke H
arlan a
nd E
d
Harr
is. S
carlett: copin
g w
ith s
kew
ed c
onte
nt popula
rity
in m
apre
duce c
luste
rs. In
Pro
ceedin
gs o
f th
e s
ixth
confe
rence
on C
om
pute
r syste
ms (
Euro
Sys '1
1),
2011.
[10]
Sum
ath
i G
opal and D
. R
aychaudhuri, "E
xperim
enta
l E
valu
ation o
f th
e T
CP
Sim
ultaneous S
end P
roble
m o
ver
802.1
1
Wirele
ss L
ocal A
rea N
etw
ork
s",
Pro
ceedin
gs o
f th
e A
CM
SIG
CO
MM
Work
shop o
n E
xperim
enta
l A
ppro
aches to
Wirele
ss N
etw
ork
Desig
n a
nd A
naly
sis
(E
-Win
d)
2005.
Ou
tlin
e
• E
xperim
enta
lly d
riven r
esearc
h
–
What
and w
hy?
–
Ho
w t
o d
o it?
–
Wh
at
are
th
e c
ha
llen
ge
s?
• T
estb
eds a
nd tools
–
Cu
rre
ntly a
va
ilab
le
• A
lte
rna
tive
ap
pro
ach
es
–
Co
min
g in
th
e f
utu
re
• C
onclu
sio
ns
Alt
ern
ati
ve
ap
pro
ac
hes
(or
ho
w t
o b
uild
yo
ur
ow
n t
es
tbe
d)
• In
ste
ad
of
bu
ildin
g te
stb
ed
s,
pro
vid
e c
usto
miz
ab
le b
oxe
s
• C
lick M
od
ula
r R
ou
ter
–
Exte
nsib
le t
oo
lkit f
or
writin
g p
acke
t p
roce
sso
rs
–
Wa
y t
o c
rea
te a
cu
sto
m s
oft
wa
re r
ou
ter
–
Pe
rfo
rma
nce
is o
f co
urs
e lim
ite
d!
• N
etF
PG
A
–
Ta
rge
ted
fo
r te
ach
ing
an
d r
ese
arc
h
–
Lo
w-c
ost
PC
I ca
rd w
ith
a u
se
r-p
rog
ram
ma
ble
FP
GA
fo
r p
roce
ssin
g
pa
cke
ts
–
Ba
sic
is 4
po
rts o
f G
iga
bit E
the
rne
t (o
nly!
) •
1199$ (
com
merc
ial) o
r 599$ (
academ
ic)
–
4x1
0G
E v
ers
ion
wa
s a
nn
ou
nce
d e
nd
of
20
10
•
$1,6
75 (
academ
ic)
–
Mo
re p
ow
erf
ul th
an
so
ftw
are
–
Mo
re c
om
ple
x t
o u
se
• O
penF
low
Op
en
Flo
w:
Mo
tiv
ati
on
• E
xperim
ente
rs w
ant to
try
out th
eir s
olu
tions o
n r
eal
netw
ork
ing d
evic
es
• V
endors
don’t w
ant to
open inte
rfaces insid
e their b
oxes
–
Re
liab
ility
, co
mp
etitio
n
–
Exp
erim
en
ters
dre
am
= v
en
do
rs n
igh
tma
re
• O
penF
low
sort
of str
ikes a
bala
nce
–
Le
ave
pro
du
ctio
n t
raff
ic u
nto
uch
ed
–
Bu
t a
llow
co
ntr
olli
ng
ho
w o
the
r tr
aff
ic is h
an
dle
d
–
Do
th
is w
ith
ou
t ve
nd
ors
ha
vin
g t
o o
pe
n t
he
ir b
oxe
s
Op
en
Flo
w:
Ov
erv
iew
• O
penF
low
enable
s flo
w-level contr
ol of fo
rward
ing
• Im
ple
menta
ble
on C
OT
S h
ard
ware
–
Exp
loits f
low
-ta
ble
s e
xis
tin
g in
mo
de
rn s
witch
es
• F
low
ta
ble
s a
re u
se
d n
orm
ally
by F
Ws, Q
oS
, N
AT
,!
–
No
rma
l p
rod
uctio
n tra
ffic
ha
nd
led
as u
su
ally
• M
ake d
eplo
yed n
etw
ork
s p
rogra
mm
able
• G
oal (e
xperim
ente
r’s p
ers
pective):
–
No
mo
re s
pe
cia
l p
urp
ose
te
st-
be
ds
–
Va
lida
te y
ou
r e
xp
erim
en
ts o
n d
ep
loye
d h
ard
wa
re w
ith
re
al tr
aff
ic
at
full
line
sp
ee
d
Op
en
Flo
wS
witch
.org
Contr
olle
r
OpenFlo
w
Sw
itch
PC
Op
en
Flo
w U
sag
e
De
dic
ate
d O
pen
Flo
w N
etw
ork
OpenFlo
w
Sw
itch
OpenFlo
w
Sw
itch
OpenF
low
P
roto
co
l
Jim
’s c
od
e
Rule
Sw
itch
A
ction
Sta
tistics
Rule
OpenFlo
w
Sw
itch
A
ction
Sta
tistics
Rule
OpenFlo
w
Sw
itch
A
ction
Sta
tistics
Jim
Jim
wa
nts
to
e
xp
erim
en
t w
ith
his
n
ew
ro
utin
g p
roto
co
l
1.
Se
tup
flo
w e
ntr
y t
o
forw
ard
all
tra
ffic
fro
m
his
ma
ch
ine
to
co
ntr
olle
r
2.
Wh
en
ne
w f
low
a
rriv
es p
acke
t fo
rwa
rde
d t
o c
on
tro
ller
3.
co
ntr
olle
r co
mp
ute
s
rou
te a
nd
se
ts u
p
co
rre
sp
on
din
g f
low
e
ntr
ies
4.
Flo
w p
acke
ts a
re
forw
ard
ed
acco
rdin
g t
o
ne
w f
low
en
trie
s
Flo
w T
ab
le E
ntr
y
“T
yp
e 0
” O
pen
Flo
w S
wit
ch
Sw
itch
P
ort
M
AC
src
M
AC
d
st
Eth
ty
pe
V
LA
N
ID
IP
Src
IP
D
st
IP
Pro
t T
CP
sp
ort
T
CP
d
po
rt
Ru
le
Actio
n
Sta
ts
1.
Fo
rwa
rd p
acke
t to
po
rt(s
) 2
. E
nca
psu
late
an
d f
orw
ard
to
co
ntr
olle
r 3
. D
rop
pa
cke
t 4
. S
en
d t
o n
orm
al p
roce
ssin
g p
ipe
line
+ m
ask
Pa
cke
t +
byte
co
un
ters
Op
en
Flo
w lim
ita
tio
ns
• P
er-
pa
cke
t n
etw
ork
ing
is m
ore
ch
alle
ng
ing
–
Me
ch
an
ism
is b
ase
d o
n e
xp
loitin
g f
low
-ta
ble
s
• ->
Per-
flow
handlin
g
–
Po
ssib
le t
o d
o p
er-
pa
cke
t p
roce
ssin
g
• V
ia c
ontr
olle
r (s
low
) •
Redirect flow
thro
ugh e
.g. N
etF
PG
A (
fast)
• F
orw
ard
ing
re
ma
ins t
he
sa
me
–
Ca
n’t u
se
ne
w f
orw
ard
ing
prim
itiv
es
–
Ca
n’t u
se
ne
w p
acke
t fo
rma
ts/f
ield
de
fin
itio
ns
• D
ela
y in
ne
w f
low
se
tup
–
~1
0m
s d
ela
y in
Sta
nfo
rd d
ep
loym
en
t –
Ca
n p
ush
do
wn
flo
ws p
roa
ctive
ly t
o a
vo
id d
ela
ys
–
Issu
e w
he
n d
ela
ys a
re la
rge
or
ne
w f
low
-ra
te is h
igh
• N
ee
d s
till
de
plo
ym
en
t –
Ta
rge
t is
ca
mp
us e
nviro
nm
en
ts
Ou
tlin
e
• E
xperim
enta
lly d
riven r
esearc
h
–
What
and w
hy?
–
Ho
w t
o d
o it?
–
Wh
at
are
th
e c
ha
llen
ge
s?
• T
estb
eds a
nd tools
–
Cu
rre
ntly a
va
ilab
le
–
Co
min
g in
th
e f
utu
re
• C
onclu
sio
ns
Co
min
g in
th
e f
utu
re:
GE
NI
• G
EN
I: G
lobal E
nvironm
ent fo
r N
etw
ork
Innovations
–
US
ba
se
d N
SF
fu
nd
ed
in
itia
tive
/pro
ject
–
Virtu
al la
bo
rato
ry f
or
exp
lorin
g f
utu
re I
nte
rne
ts a
t sca
le
• S
om
e d
esig
n c
oncepts
–
Pro
gra
mm
ab
ility
• R
ese
arc
he
rs c
an
co
ntr
ol h
ow
GE
NI-
no
de
s b
eh
ave
• D
ow
nlo
ad
so
ftw
are
in
to G
EN
I-n
od
es
–
Fe
de
ratio
n
• D
iffe
ren
t p
art
s o
f th
e G
EN
I su
ite
ow
ne
d a
nd
/or
op
era
ted
by
diffe
ren
t o
rga
niz
atio
ns
–
Slic
e-b
ase
d E
xp
erim
en
tatio
n
GE
NI s
tatu
s
• P
rogra
m p
roceeds in s
pirals
–
On
e s
pira
l la
sts
12
mo
nth
s
• N
ow
on s
piral 4
–
Tra
nsitio
n f
rom
a r
ap
id-p
roto
typ
ing
eff
ort
to
a "
rea
l G
EN
I" t
ha
t su
pp
ort
s n
etw
ork
re
se
arc
h e
xp
erim
en
tatio
n
–
Go
als
•
Ra
mp
up
th
e n
um
be
r o
f e
xp
erim
en
ters
usin
g G
EN
I b
y p
rovid
ing
be
tte
r to
ols
an
d s
erv
ice
s in
clu
din
g 2
4x7
su
pp
ort
• G
row
th
e s
ca
le o
f G
EN
I b
y d
ep
loyin
g G
EN
I ra
cks a
nd
by G
EN
I-e
na
blin
g c
am
pu
se
s u
sin
g O
penF
low
and W
iMA
X t
ech
no
log
ies
• C
rea
te t
he
first
rev o
f G
EN
I in
str
um
en
tatio
n a
nd
me
asu
rem
en
t syste
ms
• A
t le
ast 6 s
pirals
defined
–
Le
t’s s
ee
wh
at
co
me
s o
ut
eve
ntu
ally!
Liv
ing
Lab
s:
Sm
art
Sa
nta
nd
er
• S
mart
Santa
nder
aim
s a
t deplo
yin
g a
n Inte
rnet of T
hin
gs
infr
astr
uctu
re
–
Fo
r e
xp
erim
en
tal re
se
arc
h
–
In t
he
fra
me
wo
rk o
f a
city (
Sa
nta
nd
er,
Sp
ain
)
• K
ind o
f in
sta
ntiations o
f th
e L
ivin
g L
ab c
oncept
–
Ho
t to
pic
at
the
mo
me
nt
• In
frastr
uctu
re to s
upport
–
Re
se
arc
h c
om
mu
nity
–
En
d-u
se
rs (
inh
ab
ita
nts
, lo
ca
l a
uth
oritie
s,.
..)
–
Se
rvic
e p
rovid
ers
Sm
art
Sa
nta
nd
er
(co
nt.
)
• P
hased r
oll-
out and d
eplo
ym
ent
Sm
art
Sa
nta
nd
er
(co
nt.
)
• D
evic
e d
eplo
ym
ent
–
30
00
IE
EE
80
2.1
5.4
de
vic
es
–
20
0 G
PR
S m
od
ule
s
–
20
00
jo
int R
FID
tag/Q
R c
od
e la
be
ls
–
40
0 p
ark
ing s
en
so
rs
– !
–
Lo
ca
tio
ns
• S
tatic (
str
ee
tlig
hts
, fa
ca
de
s,
bu
s s
top
s)
• O
n-b
oa
rd o
f m
ob
ile v
eh
icle
s (
bu
se
s,
taxis
)
• U
se c
ases:
–
En
viro
nm
en
tal m
on
ito
rin
g
–
Pa
rkin
g a
rea m
an
ag
em
en
t –
Tra
ffic
mo
nito
rin
g
–
Pa
rtic
ipa
tory
se
nsin
g
– !
Sm
art
Sa
nta
nd
er
(co
nt.
)
• R
eal-w
orld e
nvironm
ent as o
pen p
latform
for
experim
enta
tion
–
Pro
toco
l e
xp
erim
en
tatio
n,
da
ta a
nd
kn
ow
led
ge
en
gin
ee
rin
g,
WS
N m
gm
t, s
erv
ice
s a
nd
ap
plic
atio
ns
• O
pen c
alls
for
experim
enta
tion
–
Ca
n a
sk f
un
din
g t
o d
o e
xp
erim
en
ts in
Sm
art
Sa
nta
nd
er
–
Se
co
nd
ca
ll o
pe
n n
ow
• E
arly s
tage c
urr
ently
–
Tim
e w
ill t
ell
ho
w s
ucce
ssfu
l it w
ill b
e
Co
nc
lus
ion
s
• T
here
is a
tim
e a
nd p
lace for
experim
enta
l re
searc
h
–
No
t n
ece
ssa
rily
th
e b
est
so
lutio
n e
ve
ry t
ime
–
Un
de
rsta
nd
th
e lim
ita
tio
ns w
rt.
oth
er
me
tho
ds
• A
na
lytica
l e
va
lua
tio
n,
sim
ula
tio
ns,
etc
.
• S
ele
ct your
tools
and testb
eds w
ell
–
Se
ve
ral ch
oic
es a
va
ilab
le
–
Exp
erim
en
ts t
ake
a lo
t o
f tim
e a
nd
eff
ort
s!
• C
an
no
t tr
y e
ve
ryth
ing
• D
esig
n e
xperim
ents
well
–
Yo
u’ll
sa
ve
tim
e b
y g
ett
ing
it
rig
ht
the
first
tim
e
• In
terp
ret your
results w
ell
and w
ith h
onesty
–
Da
ta n
ee
ds t
o b
e “
cle
an
ed
” •
Exp
erim
en
ts w
ill g
ive
yo
u a
no
ma
lies lik
e o
utlie
rs e
tc.
–
Mo
re a
bo
ut
the
se
in
oth
er
lectu
res
