Upload
verity-charles
View
234
Download
0
Tags:
Embed Size (px)
Citation preview
Topic 5: Level 0
David L. Hall
Topic Objectives
• Introduce concepts of level 0 processing
• Identify categories of techniques for level 0 processing (e.g., signal and image processing)
• Extend the concept of traditional signal and image processing and conditioning to meta-data generation
Level 0 Processing
Comments on Level 0 Processing
• Level 0 processing concerns processing each source of data independently to obtain the most useful information possible; we want to “squeeze out” useful information
• In general, input data to a fusion system may include scalar or vector data, signal data, image data, or textual information. Each of these classes of information may entail an entire range of potentially applicable techniques (e.g., signal processing, image processing, text-based processing)
• It is beyond the scope of this lecture (and indeed this entire course) to address these subjects – however, we seek to make the student aware of the need to consider such processing
• Finally, an emerging new area is the generation of semantic meta-data (e.g., automatic semantic labeling of images) as a new powerful process to improve data fusion.
Metasensor Processing
METASENSOR PROCESSING
CA
TEG
OR
YFU
NC
TIO
N
TEC
HN
I QU
E
Sensor Validation
and Calibration
Parametric
Modeling
• Spectral analysis• Wavelet processing• Short-time Fourier transform• AR modeling• System I/D analysis
Entity
Detection
• Array processing - Synthetic aperture (SA) - Inverse SA• Moving target indicator (MTI) • Hypothesis testing• Pre-detection fusion• Distributed fusion
• Phase and gain calibration• Sensor data validation
Non-Parametric
Analysis
• Time-series analysis• Dynamical systems analysis• Image analysis• Neural networks• Trend analysis• Thresholding and integration• Energy detection
Level 0 Processing by Source Type
Scalar orVectorSensor
Signalsensor
ImagingSensor
Text-based information
yt
Semantic processing
Signal processing
Image processing
Data Conditioning
Original Source Data plus transformed & added meta-data
Source Type: Scalar & Vector Data• Representative techniques
– Scaling methods and transformations– Coordinate transformations and rotations– Smoothing, filtering, averaging– Thresholding– Feature extraction and representations
y y + z Transformation
Original data Original data Alternate representations
Source Type: Signal Data• Representative
Parametric Modeling Techniques
• Spectral analysis• Wavelet processing• Short-time Fourier
transform• Auto-Regression modeling• System Identification
analysis
• Representative Non-Parametric Modeling Techniques
• Time-series analysis• Dynamical systems analysis• Neural networks• Trend analysis• Threshold & integration• Energy detection
Transform + characterization vector, y
Original signalTransformed signal
Examples of Signal Processing• Signal processing and
enhancement are standard on most radios & receivers– High & low frequency filtering– Boost of selected frequency
ranges– “Coloring” of signals
• Active Noise Cancelation devices– Active cancellation of
background noise (e.g., airline engine noise)
Extensive commercial tools are available such as MATLAB, Mathematica, etc
Source Type: Image Data• Classification• Feature Extraction• Pattern Recognition• Projection• Multi-Scale Signal Analysis
• Principal Components Analysis
• Independent Component Analysis
• Self-Organizing Maps• Hidden Markov Models• Neural Nets
+ Meta-Data
Original Image Transformed image
ImageProcessing
An enormous amount of methods exist to process and transform image data from the individual pixel level, to object level to complete image level. Commercial tools such as Photoshop Pro make many of these available to casual users
Examples of Image Processing
http://web.uct.ac.za/depts/physics/laser/hanbury/intro_ip.html
Contrast enhancement
Image sharpening using wavelets
http://www.phasespace.com.au/wavelet_ex.htm
Deconvolution
http://www.phasespace.com.au/wavelet_ex.htm
Source Type: Text Data• Representative techniques
– Key word extraction– Name disambiguation– Link of semantics to parametric data (e.g. location)– Syntactic processing– Thesaurus processing– Semantic distance calculations– Links to other documents/
Textual input via human reports or web info.
Textual input via human reports or web info.
Meta Data - key words - lexicon data - identified parameters - links to related documents
+TextualProcessing
Concept of Semantic Meta Data Generation• Signal & images represent a data
level view of patterns, features & characteristics
• In human pattern recognition, we identify labels to represent signals & data
• New techniques are being developed to train computers to perform a similar function
Problem: How to effectively query on a very large collection of satellite imagery for semantically meaningful regions ?
Use semantic categorization combined with CBIR on small “patches” of the images (Dr. James Wang)
Query Formulation
Query “patch” – pertaining to some semantics, e.g. mountains
Satellite Image Database Ranked
ResultsPurpose ?
Geography - Find mountainous regions with snow-caps (low-level semantics).
Forestry – Find forests of a certain density, analyze deforestation (mid-level semantics).
Military – Find air-bases in certain regions of the world (high-level semantics).
Limitations of Current Approaches
• Difficulty in defining classes Continuum of variety Subjectivity
• No major improvement in classification accuracy G. G. Wilkinson, “Results and Implications of a Study of Fifteen
Years of Satellite Image Classification Experiments,” IEEE Trans. On Geoscience and Remote Sensing, 2005.
• Recognition of higher-order semantics More focus on spectral than spatial dimensions
• Scalability of complex querying and browsing
• Flexibility to handle diverse applications
Automatic Semantic Categorization
Summary of Approach
• Practical implementation on large-scale archives
• Flexibility to adapt to various applications of satellite imagery, e.g., Geography, Military, Metallurgy, Agriculture etc.
• Exploiting spectral and spatial information using a generative model for semantic categorization (supervised learning)
• Handling of untrained classes (supervised learning)
• Using a scalable CBIR system for efficient querying and browsing (unsupervised)
Overview of the System
• Classification– Generative Classifier: Two-dimensional Multi-resolution Hidden Markov
Models (2-D MHMM)
• Handling untrained classes– Discriminative Classifier: Support Vector Machines
• Querying for fast retrieval– Integrated Region Matching (IRM) measure used in SIMPLIcity system
K trained 2D-MHMMs
40 x K training patches
Query patch
Patch Class
Random samples from trained and untrained classes
K Likelihood scores Trained
biased SVM
Database search using IRM
Query Results
Details of the Architecture
Efficient database design
Pre-computation of classes using 2D-MHMM for faster response
Suitable user interface and visualization (Common CBIR issue)
Training process
Automatic Semantic Categorization
Retrieval Results
http://riemann.ist.psu.edu/image
Retrieval Results
http://riemann.ist.psu.edu/image
Topic 5 Assignments
• Preview the on-line topic 5 materials• Read Farid paper (2008)
Data Fusion Tip of the Week
Level 0 processing is a vital part of the data fusion process - in essence pre-processing or conditioning data from individual sources and sensors. This kicks off all subsequent processing. It is necessary to do the best job possible at this stage without either introducing spurious information into the observed data, nor failing to extract and enhance the data “at the source”. It is not possible to use “down-stream” data fusion processing to make up for failures at the level 0 stage.