Distributed Regression:an Efficient Framework for
Modeling Sensor Network Data
Carlos GuestrinPeter Bodik
Romain ThibauxMark Paskin
Samuel Madden
Data collection paradigm
Base StationQuery
Distributequery
Collectdata
New QuerySQL-style
query
RedoprocessGoal:
Push beyond simple data gathering devices paradigm
Example: temperature datafrom 10 nearby sensors:• Slow changes over time• Measurements correlated
4 hours of data
send 5 numbers!!(yet very good approximation)
Approximate measurements as
send 500 numbersCollect all measurements:
VS
using Regression:
Data is highly correlated
Redundancy &Structure
Redundancy &Structure
Build lower dimensional representation Compression for data transmission Provide nodes with local view of global state …
The regression problem Given, basis functions Find coeffs w={w1,…,wk}
Precisely, minimize the residual error:
N
senso
rs
K basis functions
N
senso
rs
measurements
weights
K b
asis fu
nc
Regression solution
where
k×k matrix for k basis functions
k×1 vector
Problems:
• Invert A: too expensive in one mote • “Gather” matrix A: NK2
messages
Global temperature is complex
SERVER
LAB
KITCHEN
COPYELEC
PHONEQUIET
STORAGE
CONFERENCE
OFFICEOFFICE50
51
52 53
54
46
48
49
47
43
45
44
42 41
3739
38 36
33
3
6
10
11
12
13 14
1516
17
19
2021
22
242526283032
31
2729
23
18
9
5
8
7
4
34
1
2
3540
Temperature surface is complex
Need complex basis functions?Lots of communication?
What are we missing?
SERVER
LAB
KITCHEN
COPYELEC
PHONEQUIET
STORAGE
CONFERENCE
OFFICEOFFICE50
51
52 53
54
46
48
49
47
43
45
44
42 41
3739
38 36
33
3
6
10
11
12
13 14
1516
17
19
2021
22
242526283032
31
2729
23
18
9
5
8
7
4
34
1
2
3540
Temperature surface is complex but
Lots of local structure!
Local temperature regions Do the right thing in
the overlaps
Kernel regression
SERVER
LAB
KITCHEN
COPYELEC
PHONEQUIET
STORAGE
CONFERENCE
OFFICEOFFICE50
51
52 53
54
46
48
49
47
43
45
44
42 41
3739
38 36
33
3
6
10
11
12
13 14
1516
17
19
2021
22
242526283032
31
2729
23
18
9
5
8
7
4
34
1
2
3540
Local basis functions for each region Kernels average betweenregions
Distributed algorithm for obtaining coefficients Simple communication along a spanning tree Robust to lost messages
Need global optimization to find optimal coefficients
Kernel regression Sparse matrices
0
0
sensors
basis functions
(sparse)
Sparsebasis
Kernel basis functions have local support
SERVER
LAB
KITCHEN
COPYELEC
PHONEQUIET
STORAGE
CONFERENCE
OFFICEOFFICE50
51
52 53
54
46
48
49
47
43
45
44
42 41
3739
38 36
33
3
6
10
11
12
13 14
1516
17
19
2021
22
242526283032
31
2729
23
18
9
5
8
7
4
34
1
2
3540
020
4060
80100
0
10
20
30
400
0.2
0.4
0.6
0.8
1
h1
Gaussian Elimination
A is sparse ) Efficient Gaussian elimination:
Complete system [A|b]
After Gaussian elimination,solve linear system by k simple divisions
subtract
Add messagefrom node 1
One step ofGaussian
elimination
Distributed regression
same matrices
Complete system [A|b]
Sensor 2 can locally compute w2, w3
1 2
This subsystem is enough to compute w2, w3
M12
1 2 3 4 5
.
Specifyregions.
1
Sensors compute small matrices that add up to [A|b]:
2
.
MessagePassing.
3
.
Solve localSystems.
4
Distributed Regression:Solve global kernel regression problem
with simple local communication
Communication pattern
1
2 3
6 754
High quality links may not align with kernel topology
Kernels may not form a tree structure
Kernels form a tree structure
Communication along
a spanning treeCommunication along spanning treeusing junction tree data structure
Distributed junction trees
K1, K3
K1, K2
K3, K4
K4, K6
K3, K5
K5, K6
K1 ,K2
K4 , K6
K1 ,K3
K3 ,K5 ,K6
K1 ,K2 ,K3 ,K4 ,K5 ,K6
K1 ,K2 ,K3 ,K4 ,K5 ,K6
K1 ,K2 ,K3 ,K4 ,K5 ,K6
K1 ,K3 ,K4 ,K5 ,K6
K1 ,K2 ,K3 ,K4 ,K6
K5 ,K6
K1,
, K6
, K6
1
2
4 5
3
6
Any spanning tree transformed to a junction tree
Communication along junction tree guaranteed to obtain optimal parameters
Different spanning trees lead to different junction trees with different computation and communication complexity
See Paskin and Guestrin ’04 for spanning tree optimization
Robustness Robustness is key in sensor networks
Nodes may be added to the network or fail Communication is unreliable Link qualities change over time
Distributed regression messages are robust: Lost messages correspond to lost measurements
Must make spanning tree and junction tree algorithms robust See Paskin and Guestrin ’04 for details
Locally, nodes obtain global view
SERVER
LAB
KITCHEN
COPYELEC
PHONEQUIET
STORAGE
CONFERENCE
OFFICEOFFICE50
51
52 53
54
46
48
49
47
43
45
44
42 41
3739
38 36
33
3
6
10
11
12
13 14
1516
17
19
2021
22
242526283032
31
2729
23
18
9
5
8
7
4
34
1
2
3540
020
4060
80100
0
10
20
30
4018
20
22
24
26
28
View from node 1:View from node 17:
020
4060
80100
0
10
20
30
4018
20
22
24
26
28
View from node 46:
020
4060
80100
0
10
20
30
4018
20
22
24
26
28
020
4060
80100
0
10
20
30
4018
20
22
24
26
28
Global solution:
020
4060
80100
0
10
20
30
4018
20
22
24
26
28
Global solution:
Temperature model for lab data
Convergence and robustness
0 5 10 15 200
5
10
15
20
25
Epochs
RM
S (
in C
els
ius)
0 5 10 15 200
5
10
15
20
25
Epochs
RM
S (
in C
els
ius)
0 5 10 15 200
5
10
15
20
25
Epochs
RM
S (
in C
els
ius)
Distributed regressionreliable communication
Distributed regression50% packets lost
Offline solution
Incremental changes
0 5 10 15 200
5
10
15
20
25
Epochs
RM
S (
in C
els
ius)
Distributed regression reliable communicationDistributed regression 50% packets lost
Offline solution
0 5 10 15 200
5
10
15
20
25
Epochs
RM
S (
in C
els
ius)
0 5 10 150
1
2
3
4
Epochs
RM
S (
in C
els
ius)
Initializing with noon temperatures
At 6pm, initializing fromnoon results
Residual error varies over time
Average over regions
Quadratic in time
Linear in time
Constant in time
Regression with linear spatial components:
Effect of time window
Communication complexity
Extensions and applications Adaptive sampling Outlier and faulty sensor detection Contour finding Adaptive data modeling
Basis function selection Model-based bit compression
Bounds on bit precision for Gaussian elimination applicable
Hierarchical models Unifying with wavelet-based approaches
Currently applying similar ideas to probabilistic inference, actuator control, … See Paskin and Guestrin ’04 for details
Conclusions General distributed regression algorithm
for sensor networks
Robust to node and message losses
Kernel regression is an effective model for wide range of sensor network data
Provide basis for new more complex sensor network applications
Add messagefrom node 1
One step ofGaussian
elimination
Distributed regression
same matrices
Complete system [A|b]
Sensor 2 can locally compute w2, w3
1 2
This subsystem is enough to compute w2, w3
M12