Upload
hillary-ford
View
218
Download
2
Tags:
Embed Size (px)
Citation preview
School of Electrical and Computer Engineering
A Mathematical Theoryof Automatic Target
Recognition
Aaron D. Lanterman
What Makes ATR “Harder” than Factoring Large Numbers?
• Factoring large numbers may be NP-hard, but...
• At least it’s easy to precisely specify what the problem is!
• Not so easy in ATR– Subject to controversy
Can You Build an Airplane Without a Theory of Aerodynamics?
• Sure! Without aerodynamic theory, you can do this...
• …but with a theory, you can do this!
Can You Build an Communication Systems w/out Information Theory?• Sure! Without Information Theory,
you can do this…
• …but with Information Theory, you can do this!
• Dick Blahut likens the situation to steam engines coming before the science of thermodynamics
• First steam engines build by entrepreneurs and “inventors”– Thomas Savery: 17th and 18th centuries– Necessity the mother of invention!
• Thermodynamics didn’t begin to crystallize until mid 19th century… but with it, you eventually get
Steam Engines and Thermodynamics
• Before Shannon, your boss might ask you to do the impossible, and fire you if you failed to do it!
• Your boss cannot fire your for failing to exceed channel capacity!
• You can tell your boss you need a better channel
• 1948: Claude Shannon’s “A Mathematical Theory of Communication” (1948) – Later renamed “The Mathematical Theory of
Communication”
• Found fundamental limits on what is possible, i.e. channel capacity
Shannon’s Lightning Bolt
shouldn’t
Theory and Technology• Advances in theory are not enough;
also need the technology
– Aerodynamic theory alone won’t get you a B-2;
need advances in materials, manufacturing
– Information theory along won’t get you cell phones;
need fast DSP chips, good batteries, even more theory (i.e. coding theory)
• Theory tells you what’s possible, but sometimes only hints at how to get there
– Quantum computing folks: does this sound familiar?
Info-Theoretic View of ATR
)(
)()|()|(
ˆ
yp
xpxypyxp
X
Source Channel Decoder
Hypothesis testing (LRT, GLRT) ML, Bayes, Neyman PearsonEstimation ML, MAP, M.M.S.E., Bayes
Miss, false alarm rate Confusion matrices Bias, Variance, M.S.E.
Optimality Criteria Performance Bounds
ChernoffStein’s Lemma
Cramer-Rao
Z
e )(
)(xE
xp
X
)|(
)|()|(
xyxyp
xyE
Z
e
Y ... ,Y ,Y = Y m21
Target Recognizer SceneUnderstanding
Multiple Sensors
Scene Synthesizer
Database
(Statistical Estimation-Theoretic)
CIS/MIM
What Makes ATR “Harder” than Designing a Cell Phone?
• The space of X for real-world scenes is extremely complicated
• You don’t get to pick p(x)
• Likelihood p(y|x) is difficult to formulate– The “channel” is often deliberately hostile
• Targets hiding in clutter• Using decoys and camouflage• Radars can be subject to jamming
• Geometric variability– Position– Orientation– Articulation– “Fingerprint”
• Environmental variability– Thermal variability in infrared– Illumination variability in visual
• Complexity variability– Number of objects not known
Variability in Complex Scenes
Ulf Grenander
• Student of Cramér (yes, that Cramér)
• PhD on statistical inference in function spaces (1950)
• “Toeplitz Forms and their Applications” (with Szegö)– Fundamental work on spectral estimation (1958)
• “Probabilities on Algebraic Structures” (1968)
• “Tutorial on Pattern Theory” - unpublished manuscript– Inspired classic paper by Geman & Geman (1983)
General Pattern Theory• Generalize standard probability,
statistics, and shape theory
• Put probability measures on complex structures– Biological structures
• Mitochondria• Amoebas• Brains• Hippocampus
– Natural language– Real-world scenes of interest in ATR
The 90’s GPT Renaissance
• Made possible by increases in computer power
• Michael Miller (Washington Univ., now at JHU) did a sabbatical with Grenander
• Fields Medalist David Mumford moves from Harvard to Brown; shifts from algebraic geometry to pattern theory
Composite Parameter Spaces
0
3 )3(k
kTypesSOx X
• Move away from thinking of detection, location, recognition, etc. as separate problems
kk TypesSO )3(3X
• Naturally handles obscuration• Don’t know how many targets are in the scene in advance
Applying the Grenander Program (1)
• Take a Bayesian approach
• Many ATR algorithms seek features that are invariant to pose (position and orientation)
• Grenander’s Pattern Theory treats pose as nuisance variable in the ATR problem, and deals with it head on– Co-estimate pose, or integrate it out
– At a given viewing angle, Target A at one orientation may look much like Target B at a different orientation
– “…the nuisance parameter of orientation estimation plays a fundamental role in determining the bound on recognition” - Grenander, Miller, & Srivastava
U. Grenander, M.I. Miller, and A. Srivastava, “Hilbert-Schmidt Lower Bounds for Estimators on Matrix Lie Groups for ATR,” IEEE Trans. PAMI, Vol. 20, No. 2, Aug. 1998, pp. 790-802.
Applying the Grenander Program (2)
• Develop statistical likelihood• Data fusion is natural
• At first, use as much of the data as possible– Be wary of preprocessing: edge extraction, segmentation etc.– Processing can never add information
• Data processing inequality from information theory
));(();( parametersdatafIparametersdataI
• If you need to extract features, i.e. for real-time computational tractability, try to avoid as much loss of information as possible
OCULARSGUYWITHBINMWIRLADAR LLLLL
Analytic Performance Bounds
• Estimation bounds on continuous parameters– Cramér-Rao bounds for continuous pose parameters– Hilbert-Schmidt metrics for orientation parameters
• Bounds on detection/recognition probabilities– Stein’s Lemma, Chernoff bounds– Asymptotic analysis to approximate probabilities of error– Performance in a binary test is dominated by a term exponential in a distance
measure between a “true” and an “alternate” target• Adjust pose of “alternate” target to get closest match to “true” target as seen by the
sensor system
– Secondary term involving CRB on nuisance parameters• Links pose estimation and recognition performance
U. Grenander, A. Srivastava, and M.I. Miller, “Asymptotic Performance Analysis of Bayesian Target Recognition,” IEEE Trans. Info. Theory, Vol. 46, No. 4, July 2000, pp. 1658-1665.
AnujSrivastava
Reading One of DARPA’s BAAs…
• DARPA’s E3D program seeks:
– “efficient techniques for rapidly exploiting 3-D sensor data to precisely locate and recognize targets.”
• BAA full of demands (hopes?) for different stages of the program, such as:
– “The Target Acquisition and Recognition technology areas will develop techniques to locate and recognize articulating, reconfigurable targets under partial obscuration conditions, with an identification probability of 0.85%, a target rejection rate less than 5%, and a processing time of 3 minutes per target or less”
…Leads Us to Wondering• If such a milestone is not reached, is that the fault of the algorithm or the sensor?
– How does the DARPA Program Manager know who to fire?– Without a theory, the DARPA PM may fire someone who was asked to
“exceed channel capacity,” i.e. given an impossible task
• What performance from a particular sensor is necessary to achieve a certain level of ATR performance,
independent of the question of what algorithm is used?
Perspective Projection
kk TypesSO )2(2X
Optical PSF
Poisson Photocounting Noise
Dead andSaturated Pixels
Sensor Effects
Loglikelihood
LCCD (y | ) (i)i
y(i)ln(i)i
( j) psf(ij
| j)( j)where
L(y | x) LCCD (y | render(x))
• Cascade with
render : x
• Sensor fusion natural; just add loglikelihoods
• CCD loglikelihood of Snyder et. al
Langevin Diffusion Processes
• Fix number of targets and target types• Simulate Langevin diffusion:
)())}(({)( NNXN dWXEdXN
• Distribution of • Computed desired statistics from the samples• Generalizes to non-Euclidean groups like rotations• Gradient computation
– Numeric approximations– Easy and fast on modern 3-D graphics hardware
)()( NNN xX
)}(exp{)( xEx • Write posterior in Gibbs form:
Birth Death Type-change
Jump Processes
• Gibbs style–Sample from a restricted part of the posterior
• Metropolis-Hastings style–Draw a “proposal” from a “proposal density”–Accept (or reject) the proposal with a certain probability
Jump Strategies
Example Jump-Diffusion Process
Average Static State
Average Dynamic State
Thermal Variability
Simulations from PRISM: Discretizes target surface using regions from CAD template and internal heat transfer model
CIS/MIM
Can’t Hide from Thermal VariationsProfile 8 Profile 45 Profile 75 Profile 140
Performance VariationsDue To Thermodynamic Variability
Performance Loss Due ToInaccurate Thermodynamic Information
Cooper, Miller SPIE 97CIS/MIM
Principle Component Representation of Thermal State
• Model radiance as scalar random field on surface • Compute empirical mean & covariance from
database of 2000 radiance profiles• Karhunen-Loeve expansion using eigenfunctions
of covariance on surface - “Eigentanks”• Add expansion coefficients to parameter space
– Fortunately, able to estimate directly given pose
SPIE 97 Cooper, Grenander, Miller, Srivastava
A younger, much thinner AaronLanterman
Matt Cooper(now with Xerox)
CIS/MIM
Meteorological Variation Operational Variation
Composite Mode of VariationSPIE 97 Cooper, Grenander, Miller, Srivastava
The First “Eigentanks”
Remember, we’reshowing 2-D views offull 3-D surfaces
CIS/MIM
Joint MAP Est. of Pose and Thermal SignatureReal
NVESD M60 data (courtesy
James Ratches)
SPIE 98 Cooper and Miller
InitialEstimate
FinalEstimate
CIS/MIM
“Cost” of Estimating Thermal State
MSE Performance LossComanche SNR = 5.08 dB
CIS/MIM
Ladar/IR Sensor FusionMSE Performance Bound Information Bound
LADAR (range)
FLIR(intensity)
Joe KostakisTom Green
Jeff Shapiro
CIS/MIM
LADAR & IR Sensor FusionLADAR & IR Sensor Fusion
10 11 12 13 14 15 16 17 18-28
-26
-24
-22
-20
-18
-16
-14
-12
-10
Ladar CNR (dB)
HSB Performance Curve - 15 deg error
MMSE=0.05
9 10 11 12 13 14 15 16 17 18 19 20-30
-25
-20
-15
-10
-5
Ladar CNR (dB)
HSB Performance Curve - 9 deg error
MMSE=0.05
LADAR/FLIR Hannon Curve 15 degrees error
LADAR/FLIR Hannon Curve 9 degrees error
SPIE 98 Advanced Techniques ATR III Kostakis, Cooper, Green, Miller, OSullivan, Shapiro SnyderCIS/MIM
Target Models
Panzer IILight Tank
Hull Length: 4.81 mWidth: 2.28 mHeight: 2.15 m
Sturmgeschultz IIISelf-Propelled Gun
Hull Length: 6.77 mWidth: 2.95 mHeight: 2.16 m
Semovente M41 Self-Propelled Gun
Hull Length: 5.205 mWidth: 2.2 mHeight: 2.15 m
M48 A3 Main Battle Tank
Hull Length: 6.419 mWidth: 3.63 mHeight: 3.086 m
(Info and Top Row of Images from 3-D Ladar Challenge Problem Slides by Jacobs Sverdrup)
CR-Bound on Orientation
Strum
Semo
Position assumed known
Position unknown, must beco-estimated
Interesting knee at 0.2 meters
We take a performance hit!
M48 vs. Others
M48 and Panzer have dissimilar signatures; most easily distinguished
M48 and Semo have similar signatures; most easily confused
Semovente vs. Others
At higher resolutions,Semo and M48 have most dissimilar signatures; most easily distinguished(perhaps there are nice features which only become apparent at higher resolutions?)
Semo and Sturm have similar signatures; most easily confused
At lower resolutions,Semo and Panzer have most dissimilar signatures; most easily distinguished
JosephO’Sullivan
Synthetic Aperture Radar
MichaelDeVore
•• MSTAR Data Set
• Conditionally Gaussian model for pixel values with variances trained from data
• Likelihood based classification
• Target orientation unknown and uniformly distributed over 360° of azimuth
• Joint orientation estimation and target classification
• Train on 17° depression angle
• Test on 15° depression angleSAR Images Variance Images
T72
BM
P 2
CIS/MIM
• Results using 72 variance images per target of 10° each, and using 80 x 80 pixel sub-images to reduce background clutter
• Probability of correct classification: 98%
• Average orientation error: < 10°
Supported by ARO Center for Imaging Science DAAH 04-95-1-04-94 and ONR MURI N00014-98-1-06-06
2S1 BMP 2 BRDM 2 BTR 60 BTR 70 D7 T62 T 72 ZIL131 ZSU 23 4
2S1 265 0 5 0 0 0 4 0 0 0 BMP 2 0 576 6 4 0 0 0 1 0 0 BRDM 2 2 0 259 0 0 1 1 0 0 0 BTR 60 1 0 1 193 0 0 0 0 0 0 BTR 70 2 2 0 1 191 0 0 0 0 0 D7 2 0 0 0 0 271 1 0 0 0 T 62 2 0 0 0 0 0 265 4 2 0 T 72 0 1 0 0 0 0 4 577 0 0 ZIL131 2 0 0 0 0 0 1 0 271 0 ZSU 23 4 0 0 0 0 0 3 0 1 0 270
2S1 BMP 2 BRDM 2 BTR 60 BTR 70 D7 T62 T 72 ZIL131 ZSU 23 40
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1HS Orientation Error for 72 Windows of 10 Degrees at 80x80
0
4.05
5.73
7.02
8.10
9.06
9.93
10.7
11.4
12.1
12.8
Orientation MSE effects ID!
CIS/MIM
Caveat
Do not confuse the model with reality.
Where Should Clutter Go? (1)
A “forward model,” i.e. a “scene simulator”
)""),(( noiseparamrenderfdata
• A forest might go well in the “noise” part…
non-Gaussian minimax entropy texture models by Song Chun Zhu
Where Should Clutter Go? (2)
)""),(( noiseparamrenderfdata
• …but downtown Baghdad will not “whiten”• Structured clutter is the most vexing• May need to go in here, and directly manipulate the
clutter
…or a bit of each
• Where to draw the line?
Acknowledgments• Much of the work described here was funded by the ARO
Center for Imaging Science
• Also ONR (William Miceli) and AFOSR (Jon Sjogren)
• Slides with CIS/MIM tag were adapted from slides provided by Michael Miller