1

DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.Other requests shall be referred to AFRL/CCX, 1864 4th Street , Wright-Patterson AFB, OH 45433-7132

Information Exploitation

John Grieco CTC Lead

Information Directorate

Air Force Research Laboratory

7 June 2010

22

Information Exploitation“Info-X”

• Our Vision

Automated signal and signature exploitation for full spectrum dominance in air, space and cyberspace

• Our Mission

– Conduct fundamental research and development of advanced techniques and prototypes to detect, locate and process raw sensor data to create information

– Maximize the content that can be extracted from raw data through improved extraction, identification, and analysis of parametric information for input to information understanding, and decision-making processes

– Create force multipliers for an analyst by developing automated processes to identify, extract, analyze, correlate, sort and report actionable information

33

Info-X ChallengesMotivation

• Exploit modern transmissions - spectrum cluttering, reuse, low power, agile emitters

– Exploiting emerging technology

– Enable urban environment exploitation

• Sensor collections exceed capability – limited ability to exploit vast quantity of data

– Near real-time exploitation

– Man-on-the-Loop (MOTL) technology vs Man-in-the-Loop

• Increased demand for timely info - demand for decreasing decision timeline; low latency

– Exploitation as close to the sensor as possible

– Correlating and validating multi-source data

– Real-time exploitation

• Analysts over-tasked & under-trained - smaller force, data complexity, lack automation

– Network-centric processing & dissemination

– Autonomous trusted exploitation

44

Technology ChallengesSub-CTCs and Technology Groups

• Signal recognition and analysis technology:

– Interference cancellation & blind demodulation

– Network discovery & bitstream recovery

– Signal processing & specific emitter ID

• Spectral detection & geo technology:

– Geo-location of emitters

– Multi-static/platform geo-location

– Multi-spectral characterization

• Provenance, pedigree, & assurance technology:

– Digital signal embedding & distortion minimization

– Multimedia forensics

– Confidence measures

– Interoperable measurements

• Intelligence systems architectures technology:

– Multi-agent architectures

– Autonomous signal processing

– Cross cueing techniques

InformationProvenance, Pedigree

& Assurance

Intelligence Systems Architectures

Spectral Detection&

Geo-Location

Signal Recognition&

Analysis

Information Exploitation

Au

dio

-X

Stegan

alytics

Image

ry & M

otio

n

Image

ry-X

Electro

nics-X

Signatu

res-X

Spe

cial Signals-X

Co

mm

sIn

terce

pt-X

Fundamental Research Forum

55

Signal Processing Challenge

• Fundamental issues of the Discrete Fourier Transform

– The popular and useful DFT based spectrogram is model-based

– Assumes each signal segment repeats itself

– Results in strict mathematical trade-off between the window size and the obtainable frequency resolution

• Technical challenge

– Existing Time-Frequency (T-F) analysis techniques exhibit artifacts that are not actually present in the original signal (and can mask items of interest)

– More recent spectral re-assignment methods can provide apparent high resolution, but are inaccurate representations of the spectral content

• Research objective

– Develop a new model-based Fourier analysis approaches or other!

– Provide high resolution Spectral representation

– Allow for assessment of model accuracy and model refinement

6

Typical Spectral Representation

M-pass: 7 7 sinusoids requested; // PencilGram (top) & Spectrogram (bottom); File: Chirp

Time SegmentF

requency B

in

50 100 150 200 250 300 350 400 4500

100

200

300

400

500

Am

plit

ude (

dB

)

-20

-10

0

10

20

30

40

50

Time Segment

Fre

quency B

in

50 100 150 200 250 300 350 400 4500

100

200

300

400

500

Am

plit

ude (

dB

)

-20

-10

0

10

20

30

40

50

M-pass: 14 14 sinusoids requested; // PencilGram (top) & Spectrogram (bottom); File: faks0sa1.wav

Time Segment

Fre

quency B

in

50 100 150 200 250 300 350 400 4500

100

200

300

400

500

Am

plit

ude (

dB

)

-70

-60

-50

-40

-30

-20

-10

0

10

20

Time Segment

Fre

quency B

in

50 100 150 200 250 300 350 400 4500

100

200

300

400

500

Am

plit

ude (

dB

)

-70

-60

-50

-40

-30

-20

-10

0

10

20

77

Example Alternative Approach*

• “New” methods use instantaneous frequency and instantaneous time concepts or methods of reassignment

– Much written on the various T-F distributions; plagued by cross-terms & artifacts

– More recent works include “Sparse Time-Frequency Representations,” Timothy J. Gardner, Marcelo O. Magnasco, Proc. of the National Academy of Sciences, Vol. 103, No. 16, April 18, 2006

– Wavelet-based analyses have also been studied

• Revisit model-based spectral estimation

– See e.g., “Spectrum Analysis – A Modern Perspective,” Steven Kay, Stanley (Lawrence) Marple Jr., Proceedings of the IEEE, Vol. 69, No. 11, November 1981

– Prior emphasis was on simplifying assumptions; e.g., decay-only combined with signal segment symmetry

• The Matrix Pencil (MP) algorithm exemplifies the model-based techniques, and results in parameters that can be converted for spectral determination

– The MP allows for both decaying and growing sinusoids

– All model-based techniques can be adversely affected by inappropriate parameter selection and/or model selection, but perform remarkably well otherwise

– Iterative estimation can overcome parameter selection issues

*AFRL-RI-RS-TR-2009-55, “A Short-Segment Fourier Transform Methodology”, March 2009

8

Resolution Enhancement of

Short-Segmentation Fourier Analysis

Time Resolution Window

Right Sided SequenceLeft Sided Sequence

][nuan]1)([ Lnuan

Length L Segment

(Sum all components)

Effective Window Expansion via Left- and Right-sided Components

jeea

9

Resolution Enhancement of

Short-Segmentation Fourier Analysis

Time Resolution Window

Right Sided SequenceLeft Sided Sequence

][nuan]1)([ Lnuan

Length L Segment

(Sum all components)

Effective Window Expansion via Reduced Component-Bandwidth

jeea

Use both right- and left-sided sequences, with conjugate symmetric augmentation…

Novel methodology!

10

Alternative Spectral Representation

M-pass: 7 7 sinusoids requested; // PencilGram (top) & Spectrogram (bottom); File: Chirp

Time Segment

Fre

quency B

in

50 100 150 200 250 300 350 400 4500

100

200

300

400

500

Am

plit

ude (

dB

)

-20

-10

0

10

20

30

40

50

Time Segment

Fre

quency B

in

50 100 150 200 250 300 350 400 4500

100

200

300

400

500

Am

plit

ude (

dB

)

-20

-10

0

10

20

30

40

50

New Method (upper display); Prior Method (lower display)

64 samples per signal segment!

11

Alternative Spectral Representation

M-pass: 14 14 sinusoids requested; // PencilGram (top) & Spectrogram (bottom); File: faks0sa1.wav

Time Segment

Fre

quency B

in

50 100 150 200 250 300 350 400 4500

100

200

300

400

500

Am

plit

ude (

dB

)

-70

-60

-50

-40

-30

-20

-10

0

10

20

Time Segment

Fre

quency B

in

50 100 150 200 250 300 350 400 4500

100

200

300

400

500

Am

plit

ude (

dB

)

-70

-60

-50

-40

-30

-20

-10

0

10

20

New Method (upper display); Prior Method (lower display)

64 samples per signal segment!

1212

Feature Extraction and Identification

• Features are selected based on the signal

identification goals and limited by collection system

capabilities

• Features from high resolution Fourier analysis are

particularly useful

– Desire: high resolution in both time and frequency

– Problem: Heisenberg Uncertainty Principle

A

A

A

BB

B

C C

C

Feature Extraction

Identification

Representative Acquisition System

..

.

NotionalTransducer/Receiver/Optical Sensor

..

.

Pre-Processing Algorithms

Analog to Digital Converter

..

.

13

Way Ahead

Near Term Mid-Term Far Term

Improve performance against modern threats, actively direct exploitation, eliminate missed information, provide autonomous exploitation

Domain Specific Exploitation Coordinated Exploitation Autonomous Exploitation

- Improve against advanced threats

- Near real-time exploitation in separate domains

- Support multiple separate reports

- Discrete domain automation

Signature-X

Imagery/Motion-X

Com. Intercept-X

Electronics -X

Audio-X

Mu

ltip

le C

apab

iliti

es

Stegaanalytics

Special Signals-X

- Co-utilize disparate raw data

- Cross cue specific spectral disciplines for unique exploitation

- Complex target exploitation

- Simplified reports produced

- Multi-domain automation

Syn

ergi

stic

Exp

loit

atio

n

Cro

ss S

pec

tral

Do

mai

n S

har

ing

Coordination and

ProcessSharing

- Semi-Autonomously recognize an exploitable event

- Auto directed exploitation

- Real time complex target exploitation

- ~100% of exploitable data used

- Single report produced

- Fully automated report production

Automated, Full Spectrum,

Intelligent Exploitation

1414

Broad Agency Announcements

• Synchronized Net-Enabled Multi-Int Exploitation - BAA-10-07-RIKA – Daniel Stevens 315-330-2416

• ELINT Collection, Processing and Exploitation - BAA-10-02-RIKA – Charles Estrella 315-330-7160

• Motion Imagery & Conventional Imagery Exploitation -BAA-05-08-IFKA – Jonathan Gregory 315-330-4294

• Automated COMINT Collection and Processing - BAA-10-06-RIKA - Douglas Smith 315-330-3474

• Audio Exploitation - BAA-07-05-IFKA – John Parker 315-330-4236

• Measurement and Signatures Intelligence (MASINT) Exploitation Technology – BAA-05-09-RIKA – Bernard Clarke 315-330-2106