Embed Size (px)
Citation preview
SMART RESOURCE-AWARE MULTI-SENSOR NETWORK INTERREG IV RESEARCH PROJECT
Klagenfurt, September 2, 2011 MASSIMILIANO VALOTTO PAOLO OMERO SABRINA LONDERO
1
Autonomous complex event detection in scenarios with limited infrastructure
valo'[email protected] -‐ [email protected] -‐ [email protected] h'p://www.infofactory.it
%
2
Designing a smart resource-aware MULTISENSOR NETWORK capable of autonomously DETECTING and LOCALIZING various EVENTS such as screams, animal noise, tracks of PERSONS and more COMPLEX HUMAN BEHAVIOURS."
MAIN GOAL : SMART MULTISENSOR NETWORK
RESEARCH AREAS
%
3
1. NETWORK RECONFIGURATION
Due to limited resources, the sensors network should be able to reconfigure itself in order to limit consumes (for example switching off cameras when nothing happens in that area).
2. AUDIO/VIDEO ANALISYS Video frames and audio signals are analyzed in order to recognize objects and sounds. We can idenKfy for example the type, speed, direc2on and the coordinates of a moving object. It is possible to recognize different classes of objects such as humans, cars, dogs and cows.
3. COMPLEX EVENT DETECTION
Seman2c analysis is performed over data extracted during audio and video analysis, in order to detect complex events, such as for example <people shoo2ng to deers> <person walking in a restricted area> <dog figh2ng with person> For this purpose we use an ontological model and a rules engine.
4. MULTIMEDIA DB, RETRIEVAL & ANALYSIS The MulKMedia DB is devoted to archive the video and audio files received from sensors. Furthermore the system is consKtuted by an advanced access & retrieval & knowledge-‐discovery layer
NETWORK
ACQUISITION
ANALYSIS
COLLECTING
DATA MINING
SOUND DETECTION OBJECT RECOGNITION LOCALIZATION
SEMANTIC ANALISYS
MULTIMEDIA & EVENTS ARCHIVE COMPLEX EVENT DETECTED
SOLAR POWERED AUTO RECONFIGURABLE
VIDEO AUDIO PICTURES
1. NETWORK RECONFIGURATION
Change power mode of nodes and components Find op2mal resource alloca2on in the network Move cameras in order to follow the scene of ac2on and switch on a camera when something is expected to happen in a specific area
Operate the network at highest possible performance while minimizing resource usage."
Dynamically adapt network structure and node configura2on according to current applica2on requirements LOW ACTIVITY à exchange only status informa2on, power down as many sensors as possible HIGH ACTIVITY à exchange control and data messages, ac2vate as much sensors as needed
2. AUDIO & VIDEO ANALYSIS
Classifica2on of audio sources. Iden2fy specific sound paRerns based on characteris2c features Examples: barking dogs, shou2ng humans
3D Localization, recognition and classification of audio sources. "
Localiza2on of sound sources with 2me difference of arrival (TDOA)
waves hit the microphones at different 2me instances TDOA is related to the line of origin of the sound wave
2. AUDIO & VIDEO ANALYSIS
Detect simple paRerns of ac2vity on a ground map. Cover the paRerns with conic sec2ons represen2ng the observed zone for each video sensor
Analysis and PTZ-Cameras re-configuration. "
SOLUTION: Project real world on camera-‐based reference system The new configura2on op2mally covers the area wrt. the ac2vi2es occurring in it.
3. COMPLEX EVENT DETECTION
Define simple and complex events by means of a consistent ontology Describe the events’ context, ie., spa2al, temporal, object and event rela2onships Apply reasoning mechanisms to iden2fy complex events from low level features
Detect simple and complex events by means of a consistent ontology. "
4. MULTIMEDIA DATA BASE, RETRIEVAL & ANALYSIS
Store mul2media data, low level features, simple and complex events in a mul2media database Provide user interface for operators – High-‐level view of “what is going on“ Formulate complex queries (e.g.,all events in a certain area, the areas most frequented by bears, the sensors less ac2ve, …)
Collect multimedia data from each sensor, save events, and perform advanced analysis."
Find paRerns in data Recurring events (e,g. Visitors are used to stop in a specific area) Find rela2ons between events (event “a deer is detected in the morning in AREA 1” is ocen followed by “the deer is detected in AREA 2 in the acernoon”)à path discovery
Alert an operator Alert an operator using mobile devices.
Provide a mobile interface to access the event descrip2on and the audio/video data
The person (hunter) is detected by a camera"
"A shot is detected by a microphones array in the same area"
A camera recognizes a deer"
The position of the hunter is computed"
The network is reconfigured to look at the hunter position"
The system alerts an operator and sends the event description “a hunter shot a deer” and the audio/video data"AN
EXAM
PLE O
F THE
EVEN
T DET
ECTIO
N PRO
CESS
POWER SEARCH.
11
The user interface allows users to perform powerful retrieval operations over the collected data and advanced statistical analysis to get knowledge from the archive. The basic access metaphor used for querying the archive is a what/where/when three dimensional space.
EVENTS.
12
The search results are visualized and can be navigated following an event/place/network three dimensional approach. The events view shows the list of events resulted from the search. For each event we can see the date, the involved subjects, the action and, if defined, the zone where it happened. We can also see a map showing the exact position of the event and any related multimedia content (videos, images or audio).
DATA MINING.
13
The application offers to the user also some advanced statistical analysis, useful to get knowledge from the archive. Some examples regard the distribution of events of different types over time/in specific periods or the trend of the activity of sensors.
MOBILE ACCESS.
14
PROJECT PARTNERS
15
h'p://www.uni-‐klu.ac.at
h'p://www.eye-‐tech.it/
h'p://www.lakeside-‐labs.com/
h'p://www.infofactory.it/
