1

Enhancing ASSERT: Making an Accurate Testbed Friendly

Ehsan Nourbakhsh, Ryan Burchfield, S. Venkatesan, Neeraj Mittal, Ravi Prakash

Department of Computer ScienceUniversity of Texas at Dallas

2

Motivation

• Made observations in [RFiJ] regarding experimentation

• In [ASSERT] we proposed nine main propositions, and built a testbed based on them– Advanced wireleSS Environment Research Testbed

• Our focus changed from only fidelity to also include usability– Proper usage of the testbed should not require

comprehensive knowledge of the design and implementation of the testbed.

3

Propositions

• Accuracy– accurately reflect wireless network behavior

• Controllability – provide enough control to configure topology and environment conditions

• Mobility– emulate mobility of the nodes

• Repeatability – conduct experiments that are reproducible and easily repeatable

• Cost effectiveness – be cost effective in terms of hardware, manpower, space and time requirements to set up, run experiments

on and maintain• Data collection

– provide necessary tools to collect and analyze data• Resource sharing

– be able to share the available resources to conduct multiple experiments without interfering with each other• Multi-nodal capability

– support many types of nodes• Scalability

– have the ability to scale to a large number of nodes

4

Hardware Design

• Site: unit for hardware design• Microprocessor runs Linux• Black-box view of Unit Under Test (UUT)• Control and interaction with the UUT

5

Hardware Design (contd.)

6

Hardware Design (contd.)

• RF board

7

Hardware Design (contd.)

8

Hardware Design (contd.)

9

Software Slices

• software is divided into slices– each slice implements a specific functionality

• Some of the major ones– diagnostics– user interface– experiment control– topology mapper– attenuator– UUT control

10

Software Slices (contd.)

11

Feedback Analysis

• Graduated from “functionality centric” to “user centric”– “it works” vs. “it is easy to work with”

• Ease of running experiments results in expectation of enhanced tools– easily upload their custom images to UUTs– ability to verify proper experiment start and progress – tools for debugging and investigating collected data– possibility of creating customized distribution models.

12

Capability and Flexibility

• Cable Map• Topology Maker• Topology Mapping• Distribution Cleanup• Application Parameters

13

Cable Map

14

Topology Maker

15

Topology Mapping Time

Heuristic Mean (s) Std. Dev. (s) Min (s) Max (s)

None 15.79 13.94 0.60 47.30

1 Desc. 1.45 1.45 0.10 3.30

1 Asc. 68.24 49.63 18.40 183.70

2 Desc. 19.58 28.45 0.60 97.70

1 Desc. + 2 Desc. 0.14 0.15 0.006 0.40

1 Asc. + 2 Desc. 99.17 69.04 25.6 256.00

1 Desc. + 2 Asc. 4.56 0.96 3.80 8.00

1 Asc. + 2 Asc. 0.21 0.31 0.02 1.20

16

Verification

• Log Viewer• Remote Access

17

Log Viewer

18

Task Average Time (s)Create Images 5.3Reservation 4.2Synchronizing Clocks 18Upload images to UUTs 17.1Create Attenuation Model 0.01Restarting UUTs 1.4Enabling Logs 0.9Renew Reservation 2.6Run experiment 1941.5Disable Logging 1.7Abort Reservations 1.2Total (excluding run experiment) 52.9

Run TimeAverage time for each task over three runs for a 30 minute experiment with 24 nodes

19

Data Collection

• Timestamp Adjustments in UUT Logs• Log Download• RSS Log

20

RSS Log Viewer

Unicast by 1013

ACK by 1007

Broadcast by 1007

Broadcast by 1013

Reception by 1007, 1016

21

References

• [RFiJ] Ryan Burchfield and Ehsan Nourbakhsh and Jeff Dix and Kunal Sahu and S. Venkatesan and Ravi Prakash, “RF in the Jungle: Effect of Environment Assumptions on Wireless Experiment Repeatability,” ICC 2009

• [ASSERT] Ehsan Nourbakhsh and Jeff Dix and Paul Johnson and Ryan Burchfield and S. Venkatesan and Neeraj Mittal and Ravi Prakash, “ASSERT: A Wireless Networking Testbed,” TridentCom 2010