Upload
sidone
View
24
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Watchdog Confident Event Detection in Heterogeneous Sensor Networks. Matthew Keally 1 , Gang Zhou 1 , Guoliang Xing 2 1 College of William and Mary, 2 Michigan State University. Overview. Problem Statement Challenges Related Work Contributions Design Evaluation. - PowerPoint PPT Presentation
Citation preview
WatchdogConfident Event Detection in Heterogeneous Sensor Networks
Matthew Keally1, Gang Zhou1, Guoliang Xing2
1College of William and Mary, 2Michigan State University
Overview
Problem Statement
Challenges
Related Work
Contributions
Design
Evaluation
2
Confident Event Detection
Many applications for event detection have stringent accuracy requirements and demand long system lifetimes Vehicular traffic monitoring Falls in elderly patients Military/intrusion detection
Perform confident event detection Meet user-defined false positive and false negative rates in
the presence of in-situ sensing reality Reduce energy usage to extend system lifetime
3
Challenges of Confident Event Detection
How to cluster the right sensors to meet user accuracy requirements? Learn the detection capabilities of individual sensors and
clusters Use part of the detection capability to meet user
requirements and save energy
How to efficiently perform collaboration between heterogeneous sensors to meet user requirements? Difficult for modality-specific models and data fusion Need a generic solution
How to adapt detection capability to runtime observations? Easier observations and harder observations need different
detection capabilities
4
Related Work
Sensing Coverage Do not address user accuracy requirements Do not explore detection capability of deployment
Modality-specific Sensing Models and Data Fusion User requirements not met in reality Difficult to perform heterogeneous sensor fusion Do not cluster the right sensors to meet user requirements
Machine Learning Do not address user accuracy requirements Do not adapt sensing capability to runtime observations
5
Motivation: Related Work Shortfalls
Vehicle Detection: sensing irregularity Same distance, different accuracies Accuracy can increase with distance
Sensing Coverage may overdetect or underdetect events
Theoretical sensing models assume all sensors are identical
6
Motivation: Related Work Shortfalls
Different clusters (C1,C2,C3) have the same accuracy, 100%, better than individual sensors Difficult to capture for existing works: Due to lack of
knowledge of detection capability of different sensors and clusters
7
Watchdog Contributions
A confident and energy efficient event detection framework Choose the right sensors to meet user requirements Generic framework that provides heterogeneous sensor fusion
Adapt detection capability to runtime observations Easy observations: low-power sentinel sensors Hard observations: higher-power reinforcement sensors
Performance evaluation: two scenarios Monitor traffic entering and leaving computer science building Vehicle detection using Wisconsin trace data Compare against sensing coverage and signal attenuation
model
8
Watchdog Design Overview
Efficient heterogeneous collaboration
Explore detection capability of a deployment
Cluster the right sensors to meet user requirements
Adapt detection capability to runtime observations
Node
Local Aggregation
Runtime EventDetection
RequestReinforcement
Data
ClusterGeneration
Sentinel and ReinforcementSelection
Sensor
Aggregator
TrainingResults
Observations
Local AggregationCluster GenerationSentinel & Rein. SelectionRuntime Event Detection
9
Cluster Generation
Goal: determine detection capability of Individual sensors and sensor clusters A specific deployment
Method Randomly generate up to M clusters for each cluster size For each generated cluster
Step 1: Train a Hidden Markov Model for the cluster HMM is good for heterogeneous sensor fusion HMM captures time and space correlation of sensor data
Step 2: Determine cluster FP/FN based on the HMM decision and ground truth at each time interval
Local AggregationCluster GenerationSentinel & Rein. SelectionRuntime Event Detection
10
Step 2: Determine cluster FP/FN based on the HMM decision and ground truth
At each aggregation interval:
Determine event detection decision with trained HMM
Compare cluster detection decision with ground truth
Get the cluster FP/FN (accuracy)
Determine FP/FN for each possible event probability
Local AggregationCluster GenerationSentinel & Rein. SelectionRuntime Event Detection
11
Sentinel and Reinforcement Selection
Choose sentinel cluster: low detection capability
– Meets user's FN requirement
– Makes easy detection decisions
Choose reinforcement cluster: higher detection capability
– Meets both FP and FN requirements
– Used to make more difficult detection decisions
All other sensors go to sleep
Local AggregationCluster GenerationSentinel & Rein. SelectionRuntime Event Detection
12
Runtime Event Detection
Goal: adapt detection capability to runtime observations
– Easier observations and harder observations need different detection capabilities
Method:
– Sentinels and reinforcements form local observations at each aggregation interval
– Sentinels report non-default observations to the aggregator to make detection decisions
– Reinforcements requested when sentinel event probability false positive rate exceeds user requirements
– Reinforcements return non-default observation data and aggregator makes a confident decision
Local AggregationCluster GenerationSentinel & Rein. SelectionRuntime Event Detection
13
Runtime Event Detection
User requirements: u.FN = u.FP = 0.05
Local AggregationCluster GenerationSentinel & Rein. SelectionRuntime Event Detection
14
52
54
56
60
AcousticSeismic
Sentinels
Reinforcements
Aggregator
Time interval 0 1 2 3 4 5
t=1: No Event, s.FN = .01 < u.FNt=2: Event, s.FP = .02 < u.FPt=3: No Event, s.FN = .01 < u.FNt=4 :Undecided, s.FP = .45 > u.FPt=4 :Event, r.FP = 0.3 < u.FPt=5: No Event, s.FP = 0.2 < u.FP
Evaluation
App1: Wisconsin SensIT trace data
– Vehicle detection at a fixed location
– 75 nodes with acoustic, seismic, and infrared sensors
– 100ms aggregation interval
App2: Computer Science Building Traffic Monitor
– Five IRIS motes mounted on main entrance door
– MTS 310: 2-axis accelerometer, 2-axis magnetometer, acoustic, and light sensors
– Define event as when someone opens the door and walks through
– 4s aggregation interval
Compare with a modality-specific sensing model
– Distance-based signal attenuation
– Data fusion for event decisions
Compare with V-SAM, a state of the art protocol for handling sensing irregularity
– Measure data similarity between sensors
– Keep awake only sensors with dissimilar readings
15
Only a limited & discrete number of FP/FN rates supported by the deployment
For a specific FP/FN rate, a large number of clusters may be available
During runtime detection, Watchdog meets FP/FN explored during training
Exploring Detection Capability & Meeting Requirements
16
Compare with V-SAM: Accuracy
V-SAM with k-coverage and similarity coverage
Watchdog outperforms all with near perfect accuracy
17
Compare with Modality-Specific Sensing Model: Accuracy
Vehicle detection with acoustic sensors
– Select clusters with two different ranges to target location: near (<25m) and far (>40m)
Watchdog always meets user requirements
Modality-specific model ignores in-situ sensing reality
18
Compare with Modality-Specific Sensing Model: Energy
Watchdog clusters the right sensors to meet user requirements
– Meets requirements with reduced energy
Watchdog adapts its capability to runtime observations to save energy
Modality-specific sensing model uses all sensors in the cluster19
Adapting Detection Capability to Runtime Observations
Experimental setting
– Vehicle trace data and sensors from <25m
– User requires 0% false positives and false negatives
Watchdog clusters the right sensors to meet user requirements
Neither V-SAM nor the modality-specific sensing model adapts detection capability to runtime observations
Sentinel FP/FN(%)
Reinforcement FP/FN (%)
Reinforcement Requests (%)
9.5/0.0 0.0/0.0 21.0
20
Conclusions and Future Work
Existing works do not provide event detection with confidence, we need to
– Cluster the right sensors to meet user requirements
– Provide a generic approach for heterogeneous deployments
– Adapt detection capability to runtime observations
Watchdog: a confident event detection framework
– Meets user accuracy requirements
– Exceeds accuracy of existing approaches
– Uses knowledge of detection capability to save energy
Future Work
– Online and distributed detection
21
Thanks to NSF grants ECCS-0901437 and CNS-0916994
22
Compare with V-SAM: Training Length
Watchdog achieves maximum performance with a short training
V-SAM requires little training, but is less accurate
23
Local Aggregation
Allows for heterogeneous sensor fusion
Raw data is combined to form a single observation
– Use a common aggregation technique
Discrete, finite number of possible observations
– Same number for each sensor and modality
– Allow for comparison between sensors of all modalities
– We use two discrete observations
Local AggregationCluster GenerationSentinel & Rein. SelectionRuntime Event Detection
24
Event Probability Discussion
Differentiate the accuracy between different event probabilities
– Some observations are more reliable than others
– Probabilities near 0.5 are more inaccurate
Determine FP and FN for each of p probability ranges (p=10)
– Probability between .1 and .2 has zero false negatives
– Probability between .9 and 1.0 has 6% false positive rate
– Ranges with no events have 100% false positive or false negative rates
Local AggregationCluster GenerationSentinel & Rein. SelectionRuntime Event Detection
25