Convergence at the Extremes –

HPDC meets Tiny Networked Sensors

David Culler Computer Science Division

U.C. BerkeleyIntel Research @ Berkeley

www.cs.berkeley.edu/~culler

8/8/2001 HPDC 2

Characteristics of the Large• Concurrency intensive

– data streams and real-time events, not command-response

• Communications-centric

• Limited resources (relative to load)

• Huge variation in load

• Robustness (despite unpredictable change)

• Hands-off (no UI)

• Dynamic configuration, discovery– Self-organized and reactive control

• Similar execution model (component-based events)

• Complimentary roles (eyes/ears of the grid)

• Huge space of open problems

...and Small

8/8/2001 HPDC 3

Emerging Microscopic Devices

• CMOS trend is not just Moore’s law

• Micro Electical Mechanical Systems (MEMS)– rich array of sensors are becoming cheap and tiny

• Imagine, all sorts of chips that are connected to the physical world and to cyberspace! LNA

mixerPLL basebandfilters

I Q

• Low-power Wireless Communication

8/8/2001 HPDC 4

Disaster Management

Circulatory Net

What can you do with them?

• Embed many distributed devices to monitor and interact with physical world

• Network these devices so that they can coordinate to perform higher-level tasks.

=> Requires robust distributed systems of hundreds or thousands of devices.

Habitat Monitoring

8/8/2001 HPDC 5

Getting started in the small

• 1” x 1.5” motherboard– ATMEL 4Mhz, 8bit MCU, 512 bytes RAM, 8K pgm flash– 900Mhz Radio (RF Monolithics) 10-100 ft. range– ATMEL network pgming assist– Radio Signal strength control and sensing– I2C EPROM (logging)– Base-station ready (UART)– stackable expansion connector

» all ports, i2c, pwr, clock…

• Several sensor boards– basic protoboard– tiny weather station (temp,light,hum,prs)– vibrations (2d acc, temp, light)– accelerometers, magnetometers, – current, acoustics

8/8/2001 HPDC 6

Emerging execution model (at large)

• Application is graph of event-driven components– robust to huge surges in demand

read headerread

headerread header

exec

read headercache

check

read header

cachemiss

read headerwrite

resp

0

500

1000

1500

2000

2500

1 10 100 1000 10000

# threads executing in server (T)

max

serv

er th

roug

hput

(S ta

sks/s

ec)

1-w ay Java

4-w ay Java

0

1000

2000

3000

4000

5000

6000

1 10 100 1000 10000

# tasks in client-server pipeline

max

serv

er th

roug

hput

(S ta

sks/s

ec)

1-way Java

4-way Java

8/8/2001 HPDC 7

...and in the small

RFM

Radio byte

Radio Packet

UART

Serial Packet

ADC

Temp photo

Active Messages

clocks

bit

by

tep

ac

ke

t

Route map router sensor appln

ap

pli

ca

tio

n

HW

SW

Example: ad hoc, multi-hop routing of photo sensor readings

8/8/2001 HPDC 8

A Operating System for Tiny Devices?

• Traditional approaches– command processing loop (wait request, act, respond)

– monolithic event processing

– bring full thread/socket posix regime to platform

• Alternative– provide framework for concurrency and modularity

– never poll, never block

– interleaving flows, events, energy management

– allow appropriate abstractions to emerge

8/8/2001 HPDC 9

Tiny OS Concepts

• Scheduler + Graph of Components

– constrained two-level scheduling model: threads + events

• Component:– Commands, – Event Handlers– Frame (storage)– Tasks (concurrency)

• Constrained Storage Model– frame per component, shared stack, no

heap

• Very lean multithreading• Efficient Layering

Messaging Component

init

Po

we

r(m

od

e)

TX

_p

ack

et(

bu

f)

TX

_p

ack

et_

do

ne

(s

ucc

ess

)RX

_p

ack

et_

do

ne

(b

uff

er)

Internal

State

init

po

we

r(m

od

e)

sen

d_

msg

(ad

dr,

ty

pe

, d

ata

)

msg

_re

c(ty

pe

, d

ata

)

msg

_se

nd

_d

on

e)

internal thread

Commands Events

8/8/2001 HPDC 10

TinyOS Execution Contexts

Hardware

Interrupts

eve

nts

commands

Tasks

8/8/2001 HPDC 11

TOS Execution Model

• commands request action– ack/nack at every boundary

– call cmd or post task

• events notify occurrence– HW intrpt at lowest level

– may signal events

– call cmds

– post tasks

• Tasks provide logical concurrency

– preempted by events

• Migration of HW/SW boundary

RFM

Radio byte

Radio Packet

bit

by

tep

ac

ke

t

event-driven bit-pump

event-driven byte-pump

event-driven packet-pump

message-event driven

active message

application comp

encode/decode

crc

data processing

8/8/2001 HPDC 12

Dynamics of Events and Threads

bit event filtered at byte layer

bit event => end of byte =>

end of packet => end of msg send

thread posted to start

send next message

radio takes clock events to detect recv

8/8/2001 HPDC 13

Event-Driven Sensor Access Pattern

• clock event handler initiates data collection• sensor signals data ready event• data event handler calls output command

• common pattern

char TOS_EVENT(SENS_OUTPUT_CLOCK_EVENT)(){

return TOS_CALL_COMMAND(SENS_GET_DATA)();

}

char TOS_EVENT(SENS_DATA_READY)(int data){

return TOS_CALL_COMMAND(SENS_OUTPUT_OUTPUT)((data >> 2) &0x7);

}

8/8/2001 HPDC 14

Tiny Active Messages

• Sending– Declare buffer storage in a

frame– Request Transmission– Naming a handler– Handle Completion signal

• Receiving– Declare a handler– Firing a handler

» automatic » behaves like any other

event

• Buffer management– strict ownership exchange– tx: done event => reuse– rx: must rtn a buffer

TOS_FRAME_BEGIN(INT_TO_RFM_frame) {

char pending;

TOS_Msg msg;

}

TOS_FRAME_END(INT_TO_RFM_frame);

. . .

ok = TOS_COMMAND(SUB_SEND_MSG)(TOS_MSG_BCAST, AM_MSG(INT_READING), &VAR(msg)))

...char TOS_EVENT(SUB_MSG_SEND_DONE)(

TOS_MsgPtr sentBuffer){...}

TOS_MsgPtr TOS_MSG_EVENT(INT_READING)(TOS_MsgPtr val){

...

return val;

}

8/8/2001 HPDC 15

Example: multihop network discovery

• message handler: – if this is a ‘new’ discover message,

» record its source as parent

» retransmit with self as source

8/8/2001 HPDC 16

Network Discovery: Radio Cells

8/8/2001 HPDC 17

Network Discovery

8/8/2001 HPDC 18

Multihop Network Topology

8/8/2001 HPDC 19

Storage Breakdown (C Code)

0

500

1000

1500

2000

2500

3000

3500

4000

Multihop Router

AM light

AM Temp

AM

Packet

Radio Byte

RFM

Photo

Temp

UART Packet

UART

i2c

Init

TinyOS Scheduler

C Runtime

3450 B code 226 B data

8/8/2001 HPDC 20

DARPA-esq demo

• UAV drops nodes along road,– hot-water pipe insulation for package

• Nodes self configure into linear network

• Calibrate magnetometers

• Each detects passing vehicle

• Share filtered sensor data with 5 neighbors

• Each calculates estimated direction & velocity

• Share results

• As plane passes by, – joins network

– upload as much of missing dataset as possible from each node when in range

• 7.5 KB of code!

8/8/2001 HPDC 21

Cory Energy Monitoring/Mgmt System

• 50 nodes on 4th floor• 5 level ad hoc net• 30 sec sampling• 250K samples to database over 6 weeks

8/8/2001 HPDC 22

Energy Monitoring Network Arch

sensor net

GW GW

802-11

control net

GW

20-ton chiller

PC

scada term

modbus

UCB power monitor net

PC telegraphMYSQL

Browser

8/8/2001 HPDC 23

Huge Space of Open Problems

• Working across levels of abstractions– ex: DC-balanced packet encoding– low-power listening– CSMA MAC for highly correlated traffic– adaptive transmission control when every node is originating

and forwarding traffic– Packets can carry time and place information– implicit network discovery

• Scale– ex: discovery for 1,000 nodes with ~30 per ‘cell’

» probabalistic flooding

• Long term management– sleep/wakeup when need to be awake to hear– in situ network programming– operating within ambient energy envelop

8/8/2001 HPDC 24

Larger Challenges

• Security / Authentication / Privacy

• Programming support for systems of generalized state machines

– language, debugging, verification

• Simulation and Testing Environments

• Programming the unstructured aggregates

• Resilient Aggregators

• Understanding how an extreme system is behaving and what is its envelope

– adversarial simulation

• Constructive foundations of self-organization

8/8/2001 HPDC 25

and of course

• Mobility

• Efficiency

• Application execution resource estimation

• Automatically interface networks of tiny devices with grid frameworks

8/8/2001 HPDC 26

To learn more

• http://www.cs.berkeley.edu/~culler

• http://tinyos.millennium.berkeley.edu/

• http://webs.cs.berkeley.edu/

• http://ninja.cs.berkeley.edu/

8/8/2001 HPDC 27

8/8/2001 HPDC 28

Typical application use of tasks

• event driven data acquisition

• schedule task to do computational portion

char TOS_EVENT(MAGS_DATA_EVENT)(int data){

struct adc_packet* pack = (struct adc_packet*)(VAR(msg).data);

printf("data_event\n");

VAR(reading) = data;

TOS_POST_TASK(FILTER_DATA);

...

• 128 Hz sampling rate• simple FIR filter• dynamic software tuning for centering the magnetometer signal (1208 bytes)

• digital control of analog, not DSP• ADC (196 bytes)

8/8/2001 HPDC 29

Tasks in low-level operation

• transmit packet– send command schedules task to calculate CRC– task initiated byte-level datapump– events keep the pump flowing

• receive packet– receive event schedules task to check CRC– task signals packet ready if OK

• byte-level tx/rx– task scheduled to encode/decode each complete byte– must take less time that byte data transfer

• i2c component– i2c bus has long suspensive operations– tasks used to create split-phase interface– events can procede during bus transactions

8/8/2001 HPDC 30

Deadline avoidance

• Pipelines transmission – transmits single byte while encoding next byte

• Trades 1 byte of buffering for easy deadline• Separates high level latencies from low level

real-time requirements• Encoding Task must complete before byte

transmission completes• Decode must complete before next byte arrives

Encode Task

Bit transmission Byte 1

Byte 2

RFM Bits

Byte 2

Byte 1 Byte 3

Byte 3

Byte 4

start …