Upload
phungnga
View
252
Download
0
Embed Size (px)
Citation preview
Supported by:Supported by: ARO Center for Imaging Science DAAH 04ARO Center for Imaging Science DAAH 04--9595--1049410494
ONR MURI N00014ONR MURI N00014--9898--11--0606--0606
Boeing FoundationBoeing Foundation
ATR Theory and ATR Performance ATR Theory and ATR Performance
Analysis and PredictionAnalysis and Prediction
Joseph A. OJoseph A. O’’SullivanSullivanElectronic Systems and Signals Research LaboratoryElectronic Systems and Signals Research Laboratory
Department of Electrical EngineeringDepartment of Electrical Engineeringjaojao@@eeee..wustlwustl..eduedu
Michael D. Michael D. DeVoreDeVore andand NataliaNatalia A.A. SchmidSchmid
Washington University in St. LouisWashington University in St. LouisSchool of Engineering and Applied ScienceSchool of Engineering and Applied Science
22
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Invitation from Fred GarberInvitation from Fred Garber
“…“… givegive ‘‘invited paperinvited paper’’ addressing the subject addressing the subject
matter of the day.matter of the day.
The subjects of Thursday's session are:The subjects of Thursday's session are:
ATR Performance EvaluationATR Performance Evaluation,, Theoretical Approach to ATRTheoretical Approach to ATR,,
andand ATR Performance Prediction.ATR Performance Prediction.””
My vision of ATR theory and ATR performance analysis.My vision of ATR theory and ATR performance analysis.
33
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ATR Theory and PerformanceATR Theory and Performance
•• ATR Systems of InterestATR Systems of Interest
•• Training and Testing ParadigmTraining and Testing Paradigm
•• Some System Design IssuesSome System Design Issues
•• Information Theory and ATR Information Theory and ATR
•• System Implementation IssuesSystem Implementation Issues
•• ConclusionsConclusions
aa=T72=T72
SARSAR
PlatformPlatform
rr
TargetTarget
ClassifierClassifier
OrientationOrientation
EstimatorEstimator
ââ=T72=T72
=45=45°°^
ModelModel
DatabaseDatabase
OutlineOutline
44
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ATR Systems of InterestATR Systems of Interest
•• Imaging SensorImaging Sensor
•• Problem DefinitionProblem Definition
•• Algorithm for Algorithm for
–– ClassificationClassification
–– Parameter estimationParameter estimation
•• System Resource ConstraintsSystem Resource Constraints
–– Database sizeDatabase size
–– Processor speedProcessor speed
–– Communication speedsCommunication speeds
–– ArchitectureArchitecture
aa=T72=T72
SARSAR
PlatformPlatform
rr
TargetTarget
ClassifierClassifier
OrientationOrientation
EstimatorEstimator
ââ=T72=T72
=45=45°°^
ModelModel
DatabaseDatabase
Parameters:Parameters:
•• PosePose
•• VelocityVelocity
•• ““FeaturesFeatures””
55
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ATR System Design: Training Paradigm ATR System Design: Training Paradigm
ParameterParameter
ExtractionExtraction
ScoreScore
FunctionFunction InferenceInference
Scene and SensorScene and SensorPhysicsPhysics
Training DataTraining Data
ProcessingProcessing
ââ=T72=T720 50 100 150 200 250 300 350 400
7.4
7.6
7.8
8
8.2
8.4
8.6
8.8x 10
4
Azimuth (degrees)
Raw HRR Raw HRR
DataData
SAR ImageSAR Image ScoreScore
functionfunction
66
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ATR System Design: Training Paradigm ATR System Design: Training Paradigm
•• Likelihood functions Likelihood functions
parameterized by functionsparameterized by functions
•• TrainingTraining
–– Function estimationFunction estimation
•• InferenceInference
–– Hypothesis testingHypothesis testing
–– Parameter estimationParameter estimation
FunctionFunction
EstimationEstimation
LL((rr||aa,, )) InferenceInference
Scene and SensorScene and SensorPhysicsPhysics
Training DataTraining Data
ProcessingProcessing
ââ=T72=T720 50 100 150 200 250 300 350 400
7.4
7.6
7.8
8
8.2
8.4
8.6
8.8x 10
4
Azimuth (degrees)
Raw HRR Raw HRR
DataData
SAR ImageSAR Image LogLog--likelihoodlikelihood
functionfunction
77
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ATR Theory and PerformanceATR Theory and Performance
•• ATR Systems of InterestATR Systems of Interest
•• Training and Testing ParadigmTraining and Testing Paradigm
•• Some System Design IssuesSome System Design Issues
•• Information Theory and ATRInformation Theory and ATR
•• System Implementation IssuesSystem Implementation Issues
•• ConclusionsConclusions
aa=T72=T72
SARSAR
PlatformPlatform
rr
TargetTarget
ClassifierClassifier
OrientationOrientation
EstimatorEstimator
ââ=T72=T72
=45=45°°^
ModelModel
DatabaseDatabase
88
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ModelModel--Free versus ModelFree versus Model--Based ApproachesBased Approaches
•• ModelModel--Based ApproachesBased Approaches
–– ConditionalConditional likelihoodslikelihoods for datafor data
derived from understanding physicsderived from understanding physics
•• ModelModel--Free ApproachesFree Approaches
–– Processing architecture fixedProcessing architecture fixed——
no model for data assumedno model for data assumed
–– Examples:Examples:
»» Neural networksNeural networks
•• Intermediate ApproachesIntermediate Approaches
–– Use models when knownUse models when known
–– Use constrained architectures for restUse constrained architectures for rest
»» MSE on logMSE on log--magnitudesmagnitudes
»» MSE on quarter powerMSE on quarter power
»» Most featureMost feature--based classifiersbased classifiers
rrp(r|a,p(r|a,
rr ffp(f|a,p(f|a,featurefeature
99
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Performance Analysis and PredictionPerformance Analysis and Prediction
•• Clear problem statementClear problem statement
–– Hypothesis testingHypothesis testing
–– Estimation problemEstimation problem
•• Known distributionsKnown distributions
–– Information boundsInformation bounds
»» ChernoffChernoff, Rate functions, Rate functions
»» Fisher Information, CRLBFisher Information, CRLB
–– LaplaceLaplace approximationsapproximations
––Monte Carlo techniquesMonte Carlo techniques
•• Unknown distributionsUnknown distributions
––MinimaxMinimax boundsbounds
•• Partially known distributionsPartially known distributions
Achievable PerformanceAchievable Performance
InformationInformation--TheoreticTheoretic
BoundsBounds
1010
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Issues in Function EstimationIssues in Function Estimation
•• Statistical Tradeoffs:Statistical Tradeoffs:
–– Approximation errorApproximation error——
Estimation errorEstimation error
–– BiasBias——VarianceVariance
–– OvertrainingOvertraining
•• Learning Theory BasisLearning Theory Basis
•• Current InformationCurrent Information--
Theoretic View:Theoretic View:
–– Complexity regularizationComplexity regularization
–– MDL BasisMDL Basis
Moulin, Yu, Barron, Moulin, Yu, Barron, RissanenRissanen
-- LLR(f) + LLR(f) + Complexity(f)Complexity(f)
0 1 2 3 4 5 6 7-6
-4
-2
0
2
4
6
8
10
12
14
Lo
g s
quare
d e
rro
r
Complexity: Log Dimension
Approximation and Estimation Error
Log integrated squared errorLog integrated squared error
ISE=App. Error + Est. ErrorISE=App. Error + Est. Error
Individual errors exponential Individual errors exponential
in dimension in dimension
1111
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Regularization for Function EstimationRegularization for Function Estimation
Tikhonov
Grenander’s Sieves
Prior Likelihoods
Constraint Sets
Penalty Functionals
Complexity Regularization
..
.
fS
F
F1
F2
f2
f1
12
..
1212
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Robust Conditionally Robust Conditionally GaussianGaussian ModelModel
J. A. OJ. A. O’’Sullivan and S. Jacobs, IEEESullivan and S. Jacobs, IEEE--AES 2000AES 2000
Model each pixel as complex Model each pixel as complex GaussianGaussian plus uncorrelated noise:plus uncorrelated noise:
i
NaK
r
i
Ai
i
eNaK
ap 0
2
,
0
,,
1,r
R
aBayes r argmaxa
maxkp r k ,a
ˆHS r,a argmax
k
p r k ,a
GLRT Classification and MAP Estimation:GLRT Classification and MAP Estimation:
J. A. OJ. A. O’’Sullivan, M. D. Sullivan, M. D. DeVoreDeVore, V. , V. KediaKedia, and M. Miller, IEEE, and M. Miller, IEEE--AES to appear 2000AES to appear 2000
1313
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ConditionallyConditionally GaussianGaussian ModelModel
Model each pixel Model each pixel ii as independent, zero mean, complex conditionally as independent, zero mean, complex conditionally GaussianGaussian
pR ,A,C
2 r ,a,c2 1
c2 i2 ,a
e
ri2
c2 i2 ,a
i
Where:Where: ii22(( ,,aa) = variance function over pose and class) = variance function over pose and class
cc22 = constant over all pixels to account for power fluctuation = constant over all pixels to account for power fluctuation
a, ˆ, c2
argmaxa, ,c2
lnp r a, ,c2
p r 2i
Ii (a, )
Recognition by maximizing the logRecognition by maximizing the log--likelihood ratiolikelihood ratio**
Where:Where: 22 = average clutter variance= average clutter variance
IIii = mask function= mask function
**SchmidSchmid & O& O’’SullivanSullivan ““ThresholdingThresholding Method for Reduction of DimensionalityMethod for Reduction of Dimensionality””
1414
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Normalized Conditionally Normalized Conditionally GaussianGaussianResultsResults
1515
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
PerformancePerformance--Complexity LegendComplexity Legend
Forty combinations of number of piecewise constant
intervals and training window width
1616
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ATR Performance and ComplexityATR Performance and ComplexityComparison in terms of:
• Performance achievable at a given complexity
• Complexity required to achieve a given performance
Information Theory Basis: Rate-Distortion Theory
Rate-Recognition Theory
1717
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Image Segmentation Image Segmentation Target ExtractionTarget Extraction
InformationInformation--Theoretic ApproachTheoretic Approach•• Hypothesis Test:Hypothesis Test:
–– pixels on target vs. on clutterpixels on target vs. on clutter
•• PixelwisePixelwise measure of information for discriminationmeasure of information for discrimination
D(pD(pii||p||p00))
ConditionallyConditionally GaussianGaussian
•• Segmentation ComplexitySegmentation Complexity
–– LikelihoodsLikelihoods on snakes (contours)on snakes (contours)
–– Complexity of regionComplexity of region
2
2
2
2
ln1ii rr
Top 5Top 5 Top 50Top 50 Top 100Top 100 Top 300Top 300
1818
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
System Design Issues:System Design Issues:
DynamicallyDynamically ReconfigurableReconfigurable AlgorithmsAlgorithms
•• Information Theory ContributionsInformation Theory Contributions
–– SuccessivelySuccessively RefinableRefinable ModelsModels
»» EffrosEffros, Cover and , Cover and EquitzEquitz,, RimoldiRimoldi
»» J. Shapiro, Said and J. Shapiro, Said and PearlmanPearlman
»» R.R. DeVoreDeVore, A. Cohen, , A. Cohen,
»» I.I. DaubechiesDaubechies, D. , D. DonohoDonoho,, ……
–– SuccessivelySuccessively RefinableRefinable RecognitionRecognition
»» RateRate--distortiondistortion RateRate--recognitionrecognition
»» LogLog--time, logtime, log--spacespace RateRate
1919
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
SuccessivelySuccessively--RefinableRefinable Sensor ModelsSensor Models
Consider decreasing interval Consider decreasing interval
widthswidths
dd11=2=2 ,, dd22== ,, ……,, ddmm=2=2 /2/2mm--11
••••••
Search over Search over kk in level in level ii ordered by the most likely pose at level ordered by the most likely pose at level ii--11
˜d,i2
k ,a1
di2,a d
2 kNd
d2
2 kNd
d2
Divide azimuth into Divide azimuth into NNdd nonnon--
overlapping intervals of width overlapping intervals of width dd
2020
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
FourFour--Class ExampleClass Example
Classification error as a function of number of Classification error as a function of number of
bits passed between the database and processorbits passed between the database and processor
•• Eventually, search covers all Eventually, search covers all
possibilitiespossibilities
•• BreadthBreadth--first search quickly first search quickly
finds good combinations of (finds good combinations of ( ,,aa))
•• Method for modeling target Method for modeling target
reflectivity statistics from sample reflectivity statistics from sample
imagesimages
•• Target models used to estimate Target models used to estimate
conditional sensor output conditional sensor output
statisticsstatistics
SuccessivelySuccessively--refinablerefinable sensor models yield successivelysensor models yield successively--refinablerefinable decisionsdecisions
2121
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
System Design IssuesSystem Design Issues
•• ATR PerformanceATR Performance
•• RefinableRefinable ComputationsComputations
•• ParallelizableParallelizable
•• System Resource System Resource
ConstraintsConstraints
Result Quality vs. Complexity
2222
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ATR as a ATR as a ParallelizableParallelizable OperationOperation
•• MaximizingMaximizing ppRR|| ,,AA is equivalent to maximizing the logis equivalent to maximizing the log--
likelihood,likelihood, ll((r|r| ,,aa)) == kk ++ lnln ppRR|| ,,AA
l r ,a ln i2,a
ri2
i2,ai
•• Each measured value, Each measured value, rrii , undergoes operations of the , undergoes operations of the
same form for all pixels, orientations, and target classessame form for all pixels, orientations, and target classes
2323
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ATR as a ATR as a ParallelizableParallelizable OperationOperation
ATRATR aa11rr1
••
••
••
aa22rr2 ATRATR
aamm
rrm ATRATR
aamaxmax
ll((rr|| 1,, aa1))^
maxmax ll((rr|| ,, aa1))
••
••
••
maxmax ll((rr|| ,, aa2))
maxmax ll((rr|| ,, aat))
ll((rr|| 2,, aa2))^
ll((rr|| t,, aat))^
••
••
••
maxmax
ll((rr||355355 ,,aa))
ll((rr||55 ,,aa))
ll((rr||00 ,,aa))
ll((rr|| ,,aa))^
rr
22(( ,, aa))
gg gg gg
gg gg gg
gg gg gg
•• •• ••
•• •• ••
•• •• ••
••
••
••
ll((rr|| ,, aa))••
••
••
••
••
••
2424
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ATR IllustrationATR Illustration
•• QualityQuality -- Probability of erroneous classificationProbability of erroneous classification
•• ThroughputThroughput -- Target images processed per secondTarget images processed per second
•• ResourcesResources -- Processors, memory and I/O bandwidth, etc.Processors, memory and I/O bandwidth, etc.
aa=T72
SARSAR
PlatformPlatform
rr
TargetTarget
ClassifierClassifier
OrientationOrientation
EstimatorEstimator
ââ=T72=T72
=45=45°°^
For classification/estimation components we relate:
2525
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ExampleExampleT2=T1 with prefetch 16 KB/SAR Image (4B floats)
1 GHz clock M=10 targets
Varying target model complexity
(L templates/target and N pixels/template)
1 Gb/s Interconnection Network 10 Gb/s Interconnection Network
2626
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
ConclusionsConclusions
•• ATR Performance Bounds ATR Performance Bounds Problem StatementProblem Statement
–– Information Rate Functions for DetectionInformation Rate Functions for Detection
–– Fisher Information for EstimationFisher Information for Estimation
–– Approximation ErrorApproximation Error——Estimation ErrorEstimation Error
•• ModelModel--Based Approaches:Based Approaches:
Known DistributionsKnown Distributions
•• Successive RefinementSuccessive Refinement
•• Implementation ConsiderationsImplementation Considerations
2727
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Factor InterrelationshipsFactor Interrelationships
•• ATR systems are explicitly or implicitly based on models of ATR systems are explicitly or implicitly based on models of
targets with some complexity targets with some complexity CC
•• More complex target models require more computation but can More complex target models require more computation but can
yield better results; Pr(error)=yield better results; Pr(error)=ff((CC,, SARSAR))
•• Target model complexity and computational power determine Target model complexity and computational power determine
overall system throughput; overall system throughput; TTCHIPCHIP==hh((CC,, COMPCOMP))
•• Given an architecture, both result Given an architecture, both result qualityquality, Pr(error), Pr(error),, andand
throughputthroughput,, RR=1/=1/TTCHIPCHIP, are parameterized by target model , are parameterized by target model
complexitycomplexity
2828
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Quality of Results and ComplexityQuality of Results and Complexity
Model complexity Model complexity
resolution in resolution in
approximation of approximation of 22(( ,,aa))
Coarse model of aT62 tank,
1 template with 16K floats
Fine model of a T72 tank (1/5 relative scale),
72 templates totaling 1.1M floats
2929
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
MotivationMotivation
OutlineOutline
1.1. ConditionallyConditionally GaussianGaussian Model for SAR imageryModel for SAR imagery
2.2. Likelihood Based Approach to RecognitionLikelihood Based Approach to Recognition
3.3. Target Model Estimation & SegmentationTarget Model Estimation & Segmentation
4.4. SuccessivelySuccessively--RefinableRefinable Sensor ModelsSensor Models
5.5. ExampleExample
ATR from CAD ModelsATR from CAD Models Template Based ATRTemplate Based ATR Model ExtractionModel Extraction
Combine sensor output prediction with training dataCombine sensor output prediction with training data
3030
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Target Model EstimationTarget Model EstimationGiven N registered training images qj of a target with pose j , estimate
variances over Nw windows of width d
wk2 k
Nw
d
2,2 k
Nw
d
2ˆ 2 k ,a
1nk
q j2
j: j wkwhere
Variance estimate for an Variance estimate for an
unregisteredunregistered image image rr with pose with pose
formed by transforming the formed by transforming the
estimate from the closest estimate from the closest wwkk
Registered Variance ImagesRegistered Variance Images
Transformed EstimatesTransformed Estimates
TT00°° TT9090°° TT180180°° TT270270°°
3131
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Target Model SegmentationTarget Model Segmentation•• Pixel information relative to nullPixel information relative to null--hypothesis used for target recognitionhypothesis used for target recognition
•• Retain pixels Retain pixels ii that are informative relative to the nullthat are informative relative to the null--hypothesis:hypothesis:
Sa i :1
NwD p ; ˆ i
2k ,a p ;
2
k
Top 5Top 5 Top 50Top 50 Top 100Top 100 Top 300Top 300
•• Segmentation of target models, not of imagesSegmentation of target models, not of images
•• Ordering of pixels by their empirical information relative to nuOrdering of pixels by their empirical information relative to nullll--hypothesis.hypothesis.
For nullFor null--hypothesishypothesis 22=0.0028=0.0028 -- approximate background variance approximate background variance -- pixels on illuminated pixels on illuminated
side of target are deemed most informative.side of target are deemed most informative.
3232
ATR Theory and PerformanceATR Theory and Performance OO’’Sullivan, SPIE SAR 2001Sullivan, SPIE SAR 2001
Computational ModelsComputational Models
Chip processing rate Chip processing rate RR=1/=1/TTCHIPCHIP
Assumptions:Assumptions:
•• Each CPU optimizes over a region of the search spaceEach CPU optimizes over a region of the search space
•• MultiMulti--issue CPU with 2 instructions/clock cycleissue CPU with 2 instructions/clock cycle
•• 6 instructions per pixel6 instructions per pixel
TCHIP sec/SAR Image L templates/target
T1 sec/clock cycle M targets
T2 sec/template memory read N pixels/template
T3 sec/SAR Image load P processors
TCHIP 3LMN
PT1
LMN
PT2 T3