1
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.Other requests shall be referred to AFRL/CCX, 1864 4th Street , Wright-Patterson AFB, OH 45433-7132
Information Exploitation
John Grieco CTC Lead
Information Directorate
Air Force Research Laboratory
7 June 2010
22
Information Exploitation“Info-X”
• Our Vision
Automated signal and signature exploitation for full spectrum dominance in air, space and cyberspace
• Our Mission
– Conduct fundamental research and development of advanced techniques and prototypes to detect, locate and process raw sensor data to create information
– Maximize the content that can be extracted from raw data through improved extraction, identification, and analysis of parametric information for input to information understanding, and decision-making processes
– Create force multipliers for an analyst by developing automated processes to identify, extract, analyze, correlate, sort and report actionable information
33
Info-X ChallengesMotivation
• Exploit modern transmissions - spectrum cluttering, reuse, low power, agile emitters
– Exploiting emerging technology
– Enable urban environment exploitation
• Sensor collections exceed capability – limited ability to exploit vast quantity of data
– Near real-time exploitation
– Man-on-the-Loop (MOTL) technology vs Man-in-the-Loop
• Increased demand for timely info - demand for decreasing decision timeline; low latency
– Exploitation as close to the sensor as possible
– Correlating and validating multi-source data
– Real-time exploitation
• Analysts over-tasked & under-trained - smaller force, data complexity, lack automation
– Network-centric processing & dissemination
– Autonomous trusted exploitation
44
Technology ChallengesSub-CTCs and Technology Groups
• Signal recognition and analysis technology:
– Interference cancellation & blind demodulation
– Network discovery & bitstream recovery
– Signal processing & specific emitter ID
• Spectral detection & geo technology:
– Geo-location of emitters
– Multi-static/platform geo-location
– Multi-spectral characterization
• Provenance, pedigree, & assurance technology:
– Digital signal embedding & distortion minimization
– Multimedia forensics
– Confidence measures
– Interoperable measurements
• Intelligence systems architectures technology:
– Multi-agent architectures
– Autonomous signal processing
– Cross cueing techniques
InformationProvenance, Pedigree
& Assurance
Intelligence Systems Architectures
Spectral Detection&
Geo-Location
Signal Recognition&
Analysis
Information Exploitation
Au
dio
-X
Stegan
alytics
Image
ry & M
otio
n
Image
ry-X
Electro
nics-X
Signatu
res-X
Spe
cial Signals-X
Co
mm
sIn
terce
pt-X
Fundamental Research Forum
55
Signal Processing Challenge
• Fundamental issues of the Discrete Fourier Transform
– The popular and useful DFT based spectrogram is model-based
– Assumes each signal segment repeats itself
– Results in strict mathematical trade-off between the window size and the obtainable frequency resolution
• Technical challenge
– Existing Time-Frequency (T-F) analysis techniques exhibit artifacts that are not actually present in the original signal (and can mask items of interest)
– More recent spectral re-assignment methods can provide apparent high resolution, but are inaccurate representations of the spectral content
• Research objective
– Develop a new model-based Fourier analysis approaches or other!
– Provide high resolution Spectral representation
– Allow for assessment of model accuracy and model refinement
6
Typical Spectral Representation
M-pass: 7 7 sinusoids requested; // PencilGram (top) & Spectrogram (bottom); File: Chirp
Time SegmentF
requency B
in
50 100 150 200 250 300 350 400 4500
100
200
300
400
500
Am
plit
ude (
dB
)
-20
-10
0
10
20
30
40
50
Time Segment
Fre
quency B
in
50 100 150 200 250 300 350 400 4500
100
200
300
400
500
Am
plit
ude (
dB
)
-20
-10
0
10
20
30
40
50
M-pass: 14 14 sinusoids requested; // PencilGram (top) & Spectrogram (bottom); File: faks0sa1.wav
Time Segment
Fre
quency B
in
50 100 150 200 250 300 350 400 4500
100
200
300
400
500
Am
plit
ude (
dB
)
-70
-60
-50
-40
-30
-20
-10
0
10
20
Time Segment
Fre
quency B
in
50 100 150 200 250 300 350 400 4500
100
200
300
400
500
Am
plit
ude (
dB
)
-70
-60
-50
-40
-30
-20
-10
0
10
20
77
Example Alternative Approach*
• “New” methods use instantaneous frequency and instantaneous time concepts or methods of reassignment
– Much written on the various T-F distributions; plagued by cross-terms & artifacts
– More recent works include “Sparse Time-Frequency Representations,” Timothy J. Gardner, Marcelo O. Magnasco, Proc. of the National Academy of Sciences, Vol. 103, No. 16, April 18, 2006
– Wavelet-based analyses have also been studied
• Revisit model-based spectral estimation
– See e.g., “Spectrum Analysis – A Modern Perspective,” Steven Kay, Stanley (Lawrence) Marple Jr., Proceedings of the IEEE, Vol. 69, No. 11, November 1981
– Prior emphasis was on simplifying assumptions; e.g., decay-only combined with signal segment symmetry
• The Matrix Pencil (MP) algorithm exemplifies the model-based techniques, and results in parameters that can be converted for spectral determination
– The MP allows for both decaying and growing sinusoids
– All model-based techniques can be adversely affected by inappropriate parameter selection and/or model selection, but perform remarkably well otherwise
– Iterative estimation can overcome parameter selection issues
*AFRL-RI-RS-TR-2009-55, “A Short-Segment Fourier Transform Methodology”, March 2009
8
Resolution Enhancement of
Short-Segmentation Fourier Analysis
Time Resolution Window
Right Sided SequenceLeft Sided Sequence
][nuan]1)([ Lnuan
Length L Segment
(Sum all components)
Effective Window Expansion via Left- and Right-sided Components
jeea
9
Resolution Enhancement of
Short-Segmentation Fourier Analysis
Time Resolution Window
Right Sided SequenceLeft Sided Sequence
][nuan]1)([ Lnuan
Length L Segment
(Sum all components)
Effective Window Expansion via Reduced Component-Bandwidth
jeea
Use both right- and left-sided sequences, with conjugate symmetric augmentation…
Novel methodology!
10
Alternative Spectral Representation
M-pass: 7 7 sinusoids requested; // PencilGram (top) & Spectrogram (bottom); File: Chirp
Time Segment
Fre
quency B
in
50 100 150 200 250 300 350 400 4500
100
200
300
400
500
Am
plit
ude (
dB
)
-20
-10
0
10
20
30
40
50
Time Segment
Fre
quency B
in
50 100 150 200 250 300 350 400 4500
100
200
300
400
500
Am
plit
ude (
dB
)
-20
-10
0
10
20
30
40
50
New Method (upper display); Prior Method (lower display)
64 samples per signal segment!
11
Alternative Spectral Representation
M-pass: 14 14 sinusoids requested; // PencilGram (top) & Spectrogram (bottom); File: faks0sa1.wav
Time Segment
Fre
quency B
in
50 100 150 200 250 300 350 400 4500
100
200
300
400
500
Am
plit
ude (
dB
)
-70
-60
-50
-40
-30
-20
-10
0
10
20
Time Segment
Fre
quency B
in
50 100 150 200 250 300 350 400 4500
100
200
300
400
500
Am
plit
ude (
dB
)
-70
-60
-50
-40
-30
-20
-10
0
10
20
New Method (upper display); Prior Method (lower display)
64 samples per signal segment!
1212
Feature Extraction and Identification
• Features are selected based on the signal
identification goals and limited by collection system
capabilities
• Features from high resolution Fourier analysis are
particularly useful
– Desire: high resolution in both time and frequency
– Problem: Heisenberg Uncertainty Principle
A
A
A
BB
B
C C
C
Feature Extraction
Identification
Representative Acquisition System
..
.
NotionalTransducer/Receiver/Optical Sensor
..
.
Pre-Processing Algorithms
Analog to Digital Converter
..
.
13
Way Ahead
Near Term Mid-Term Far Term
Improve performance against modern threats, actively direct exploitation, eliminate missed information, provide autonomous exploitation
Domain Specific Exploitation Coordinated Exploitation Autonomous Exploitation
- Improve against advanced threats
- Near real-time exploitation in separate domains
- Support multiple separate reports
- Discrete domain automation
Signature-X
Imagery/Motion-X
Com. Intercept-X
Electronics -X
Audio-X
Mu
ltip
le C
apab
iliti
es
Stegaanalytics
Special Signals-X
- Co-utilize disparate raw data
- Cross cue specific spectral disciplines for unique exploitation
- Complex target exploitation
- Simplified reports produced
- Multi-domain automation
Syn
ergi
stic
Exp
loit
atio
n
Cro
ss S
pec
tral
Do
mai
n S
har
ing
Coordination and
ProcessSharing
- Semi-Autonomously recognize an exploitable event
- Auto directed exploitation
- Real time complex target exploitation
- ~100% of exploitable data used
- Single report produced
- Fully automated report production
Automated, Full Spectrum,
Intelligent Exploitation
1414
Broad Agency Announcements
• Synchronized Net-Enabled Multi-Int Exploitation - BAA-10-07-RIKA – Daniel Stevens 315-330-2416
• ELINT Collection, Processing and Exploitation - BAA-10-02-RIKA – Charles Estrella 315-330-7160
• Motion Imagery & Conventional Imagery Exploitation -BAA-05-08-IFKA – Jonathan Gregory 315-330-4294
• Automated COMINT Collection and Processing - BAA-10-06-RIKA - Douglas Smith 315-330-3474
• Audio Exploitation - BAA-07-05-IFKA – John Parker 315-330-4236
• Measurement and Signatures Intelligence (MASINT) Exploitation Technology – BAA-05-09-RIKA – Bernard Clarke 315-330-2106