Upload
yamal
View
76
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Status of AiTR/ATR in Military Applications. James A. Ratches CERDEC NVESD January 2007. UMDAROATR. Outline. Definition Importance & Scenarios Performance Assessment Problem Statement Way Forward Summary & Conclusions. Military Definition. - PowerPoint PPT Presentation
Citation preview
Status of AiTR/ATRin
Military Applications
James A. RatchesCERDEC NVESD
January 2007
UMDAROATR
Outline
• Definition • Importance & Scenarios• Performance Assessment• Problem Statement• Way Forward• Summary & Conclusions
• Generic term to describe automated/semi-automated functions carried out on imaging sensor data to perform operations ranging from cuing a human observer to complex fully autonomous object acquisition and identification• Machine function: - Detection - Classification - Recognition - Identification - Friend or Foe
• Aided Target Recognition (AiTR) - Machine makes some level of decision and annotates the image - Human makes higher level decision. e.g. to identify and fire
• ATR is fully autonomous - No human in-the-loop after weapon firing, e.g. fire-and-forget seeker
• ATR/AiTR may use information from other sensors to make decision by fusing information
AiTR (aided) ATR (autonomous)
Military Definition
Scenarios Where AiTR Essential
Urban terrain; 360 degree situationalawareness, short ranges, human intent,
transmission limitations
Rapid wide area search for close combat in high clutter, against difficult targets
(occlusion, defilade, CC&D) and variabletarget signatures
UAV & UGV transmissionsover limited bandwidthBDA
1830
road
road
Tree trunks
debris
New object1030 Detection of
Dismounts & intent,& bunkers
Scouts in Overwatch-Objects of interest and scene changes
Missile Scenarios Where ATR Essential
Fire UnitsFire UnitsFire UnitsFire Units
1st1stWaypointWaypoint
(Tower)(Tower)
1st1stWaypointWaypoint
(Tower)(Tower)
Field Of View
Engagement
EngagementAreaArea
1 K
m
- Power Up
- Computer Initialization
- Intelligence Preparation of
Battlefield
- Plan Missile Routes
if necessary
-Receive Target
Information
Through C2 Network
- Verify Target Selection
- Route & Salvo Selection-Launch Missile(s)
- In-Flight Intelligence
- Target Marking
- Target Reporting
to C2 Network
- Start Search (Wide FOV)
- Locate Target (Narrow FOV)
- Lock On
- Aimpoint Update
Warhead Function
On Impact
4 Km
Navigate To
Emplacement Site
Missile Auto-Navigate To
Target Search Point(Enroute Recon)
Detect, Recognize &
Identify Target(Engage Autotracker)
ObstacleObstacle 2ndWaypoint(Mountain)
2ndWaypoint(Mountain)
Target ofOpportunity
Target ofOpportunity
-Acquire GPS Satellites
-Update GPS Position
-Calibrate the Inertial System
- Navigate to Target Area
Which pixels in image correspond to targets?
AiTR Annotates Images – Not Maps
Hunter Ligget
Yuma
Grayling
Clutter levels:
High Hunter Ligget
Medium Yuma
Low Grayling
Effects of Clutter
WIDE AREA TARGET CUEING WITHIN 4 SECONDSCTRS
+
H MD VIS
H MD M ODE
H MD W P
NAV U PDT
NA V WP
NAV APLT
SCRN BRT
+
+
29
N
S E
W
NB 5200 2250NB 2320 445 6TFXY
AB1
TS D HO ME
T SD SCLE
TSD TAC
TSD NAV
TSD CNTR
T SD W NDW
SLAVE
Manual FLIR (300 FOR) Search Time > 60 Sec.
Aided Target Search Time less than 4 Sec.
ROC Curves
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
False Alarms per Square Degree
Pro
bab
ilit
y o
f D
etec
tio
n
Algo 1 - In Open
Algo 1 - Occluded
Algo 2 - In Open
Algo 2 - Occluded
Effects of Occlusion
Lab/Field Measured Performance
COMMON SENSORCOMMON SENSORLong Range Scout Surveillance SystemLong Range Scout Surveillance System22ndnd Gen FLIR Modified f/ Gimbaled Scan Gen FLIR Modified f/ Gimbaled Scan
AiTD/R•Assess Maturity Ground Based AiTD/Rs in Varied Environmental Conditions
Gimbaled Scan FLIR
•Long Range Target Detection• 2nd Gen B-Kit (LWIR)
SWIR CAMERA•Long Range Target Identification•Leverage ACT II LIVAR and CETS Program - EBCCD Technology •4.5” Aperture
MTI Radar•Utilize AN/PPS-5D
Laser Illumination/Designation
AiTR/ATR Continues to Be Tested in Realistic Environments
Target Acquisition Sensor Suite (TASS)
• Assess maturity of SOA AiTR in gimbal scanning mode in the field • SOA single color/ shape LWIR based algorithms from multiple sources• Include urban bkgds and man targets
Evaluations yieldROC curves
Overall Assessment
1. DOD investment in AiTR has resulted in quantifiable level of performance documented in ROC curves2. Performance measured under favorable conditions3. Order of magnitude improvement in search time with AiTR over human only4. Discrimination levels above detection have not been vigorously pursued*5. Detection performance can have degradation for sub- optimal conditions*
- high clutter - low contrast - obscuration - extended ranges
6. Training target sets have been typically for < 10 targets7. There are no human detection algorithms
SOA AiTR Algorithms Have Known Limitations
* Especially for ground-to-ground
Need for Robust AiTR/ATR
For future combat scenario must be robust - High false alarm rate renders aid useless and operator
will turn it off (AiTR)- Ground-to-ground presents high clutter - Target variability increases complexity- Low signature targets can be expected- Partial occlusion & defilade obscures the target - CC&D need to be mitigated- Detect human threats in urban terrain- Final ID can be man-in-the-loop (AiTR)
Robust AiTR/ATR Critical for Ground-to-GroundClose Fight Manned-shoot first Unmanned-autonomous
operation
The AiTR/ATR Problem
• ~$100M investment to realize SOA AiTR• Humans can still do better than SOA AiTR (Except for speed)
• Robust AiTR required - Potential target set is large with wide range of environmental and operational variations -AiTR for humans and urban terrain
• New university concepts have not migrated to industry and military developers
ARL-SEDDDATA
AiTR/ATR Cannot Do As Well As The Human
Alone-However, It Can Do It Faster
-Improvement that approaches human performance will be an enabling force multiplier
00.10.20.30.40.50.60.70.80.9
1
0 20 40 60Search Time (sec)
Aided
Manual
Pro
bab
i lit y
of
Det
e ct i
on
Perceived Impediments to ATR/AiTR
• Required computational power• High cost, power and size• Proprietary issues• Tactical scene complexity• Required to be better than human• New CONOPS will be needed to fully utilize benefits of ATR/AiTR
Real Limitation Is The Lack of An Image Science-What Is Important in An Image?
Possible Paths to Improvement
• 3D LADAR – if can cover/search field of regard- Otherwise, use for higher level discrimination
• Multi/hyper-spectral/look/mode sensor and Sensor Fusion• Untried University “New Ideas”
- Recognition by parts- Advanced eye-brain understanding- Gradient index flow and active contour analysis- Frame-to-frame correlations- Spatial contextual intelligence- Hierarchical imaging - Category theory
• Off-board sensor features data via low bandwidth tactical networks• Validated synthetic image generation to stress algorithms during formulation• Investment in Image Science
ARO/Duke University WorkshopComputational Sensors for Target Acquisition and Tracking
Beaufort, NC December 2-4, 2003
Representative Recommendations
• Different approach to applying eye-brain understanding to AiTR needed - Does not necessarily mean that we need to mimic that process
• Artists may give a unique insight into minimalist representation• Poor performance of AiTRs relative to humans suggest there are better features than have been found by AiTRs• The perspective of clutter rejection rather that object feature extraction may present a different set of opportunities
Eye-Brain Understanding Can Still Be A Fertile Groundof Investigation for AiTR Concepts
1. Statistical a. no range ΔT, size, perimeter, etc. Comanche - target in open & in center of FOV
- ~ 10 target set in low clutter - baseline performance
b. w/range same + target window size - reduce search time (10X) - reduce FAR (10X)
2. Template Matching comparison of ROI to stored SAIP - expand target set, e.g. aspect,target templates articulation, dirurnal/seasonal, etc.
2. Model Based comparison of ROI to stored MSTAR - increase target set with stored datatarget model set reduction
4. Multi-spectral pixel value=f(λ,Δλ) MFS3 - penetrate camouflage - reduce FAR
5. Multi-look target indications at GPS Dynamic - reduce FAR (~ 10-100X) by coords from off-board Variable by correlating target detectssensors Threshold - detect obscured, defilade targets
- missed target reduction (~2X)
6. Multi-mode non-imaging sensor indications ASM - mitigate CC&D(sound, vibration, magnetics) algorithms - reduce FAR
ALGORITHM FEATURE EXAMPLE CAPABILITY METRIC
*
*
*
*
*
*
* Classified data on false alarms and Pd exist for these algorithms
Progression of Algorithms
ALGORITHM FEATURE EXAMPLE CAPABILITY METRIC
7. Geographic Contour Maps terrain slope DTED - FAR reduction (potential ~ 75%)
8. Advanced Eye-Brain synapse maps NN, holographic NN, - intelligent search & detect Understanding & wavelets - FAR reduction Representation - reduce search timelines
9. Recognition-by-Parts target subelements - detect partially obscured targets detected - missed target reduction
10. Gradient Index Flow & 2D chips of humans - determine human intent Active Contour Analysis
11. Frame-to-Frame pixel change MTI - detect changes in scene Correlations correlations - reduce search times
12. Spatial Contextual target forbidden terrain - reduce search time Intelligence by reducing search area
13. Artists Insights hierarchical scene - reduce search time by characteristics focus on search area
14. Hierarchical Imaging activate/retard signals - bandwidth reduction by evaluating information before transmission
15. Category Theory sensor report & - geolocation accuracy locations improvement
Sf
S1
S2
ScSw
*
Progression of Algorithms (con’d)
• Different features have different ROC curves
– Range dependent
• Features from different sensors and platform can be passed over the network (low bandwidth information)
• Performance gain proportional to ROC curves
• Pick 2 features as example– local variation– wavelet
10X FAR reduction
Theoretical Basis for Multi-Look
00
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1ROC Curves for 2100m, 2500m, and Fused
Pd
Pfa
Fused Best: Pd Indep., Pfa Indep.
Fused: Pd Indep., Pfa Fully Correl.
Fused Worst: Pd, Pfa Fully Correl.2100m
2500m
Features from Off-Board Sensors Can Improve On-BoardSensor AiTR/ATR
R1
R3R2
V2
R4 R2
V1V2
False Alarms Uncorrelated between Sensors Ridge Sensor
Valley Sensor
Side View
Most false alarms
for LWIR-2Most false alarms
for LWIR-1
LWIR-1
LWIR-2
Plan ViewMost false alarms
for LWIR-1Most false alarms
for LWIR-2
LWIR-1LWIR-2
Category Theory is a mathematically sound framework:-Designed for network applications-Describes information fusion systems and processes in an elegant language - Captures commonality and relationships between objects
Specifications S: S1. S2 data from sensors 1 & 2 Sc real world stimuli Sw ground truth Sf registration transformation between S1 & S2Morphisms: arrowsFunctors: Relationships with other categories
Sf
S1 S2Sw
ScExample of a
category
Gunbarrel
Hot spot
Turretgeon1
geon2
geon3
Tracks & wheelsgeon4
Engine exhaust
geon5
FLIR Image
T-72 tank
Recognition is basedon recognition of critical sub-componentscalled geons
Biderman (USC)Kokar (Northeastern)
Recognition-by-Parts Category Theory
Library of Geons for targets of interestforms the basis for recognition
Network supplies opportunity for sophisticatedfusion techniques to be applied to AiTR
O1 O2 O3 O4a b c
cxaxc=(cxb)xa=cx(bxa)Composition operation that is associative
“Image Science” Based Algorithms
SOA algorithms attempt to recognize static targets in single frames: Need to consider more image-based, e.g.
parameters e.g., image temporal-spatial relationships.
Sensor-Scene Dynamics
Context
Algorithms Must Extract More Contextual Information
Change detection & MTI
Gradient of intensity (x, y)
Gradient Vector Flow (GVF)
• Higher level process or user initializes any curve close to the the object boundary (indication of a region of interest)• The parametric curves (snakes) then starts deforming and moving towards the desired object boundary• In the end it completely “shrink-wraps” around the object
Zucker (Yale) & Xu & Prince (JHU)
(Active Contour Analysis)
Eye-Brain Understanding Must Be Applied Faithfully
GVF field is defined to be a vector field X [x(s), y(s)]for s in [0,1]Solve Euler equation αx''(s) - β x''''(s) - Eext = 0to minimize energy functional E = ∫0
1 ½ (α│x'(s)│2 -β│x''(s)│2) + Eext (x(s))ds (α and β user defined constants)
Human intent
This painting shows how Van Gogh was able to transmit detailed Information about a person (20-year old woman) to the viewer Using Only ~10 brush strokes for her face.
Artists Unique Insight
From Falco (U of AZ)
Hierarchical Imaging &Target Representation
Elements of Network Make Localized Decisions Rather ThanSimply Sending Raw Data to A Central Processor
• Sensors Sample n Parameters• Network becomes large scale sensor• Hierarchical decisions - Local decisions determine relevant information - Global decisions develop global model - Each node is a virtual point detector at the next level - Algorithms determine what is to be shared/when
The Network Becomes The Sensor & AiTR
Conclusions
• SOA ATR/AiTR has attained a level of performance that has some level of military value
- Targets in the open- Low to medium clutter- Target set ~ 10-15- No obscuration or camouflage- No humans or human intent- No high value targets, e.g. bunkers
• Major new innovations are needed to get a leap ahead in performance under operational environments
- New university concepts- Network information
Back Up Slides
Aided Target Recognition for Intelligent Search
M35 @ 0°, scale = 1.0
ZSU @ 165°, scale = 0.85
M60 @ 180°, scale = 0.85
M35 @ 270°, scale = 1.0
M35 @ 195°, scale = 1.0
Prescreener FeatureExtraction Registration Recognition
NeuralNet
Inner
outer
M-35
Range x
Sensor
Representative Configuration of SOA AiTR
3D from Optical Flow
Crater
Above Surface Mound
• Subtle motion provides substantial depth information• Memory/ processing advances permit harvesting of depth information (Target/Sensor motion)• Algorithms have been developed that amplify motion vectors and present them in a binocular display in real time to create “hyperstereo” using advances in microlens technologies
Courtesy of FOR 3D, Santa Rosa, CA
Processing Motion Information Can Provide Depth (Range)
Passive Ranging with DTED Data
Lines of constant range superimposed on FLIR image where earth is flat at Yuma P.G.
DTED Data
Lines of constant rangesuperimposed on FLIR
image at Hunter-Liggett
Flat Earth Approximations Cannot Be Used for All ScenariosOf Interest
Work on passive ranging and imagery by Raytheon.
Passive Ranging• Accurate range estimation can reduce false alarm rates in AiTR• Permits estimation of target size• Active ranging potentially reveals position• Most AiTR algorithms make a flat earth approximation• Passive ranging may provide more adequate accuracy
“ Despite often heard claims to the contrary, without range data there is no wayof knowing if the target is 100- times smaller than a pixel or 1000 times largerthan the image as a whole. – Northrop-Grumman
Optical flow
Eye motion Near fieldobjects Far field
objects
DTED w/GPS
DTEDoverlaid
on imagery
GPS
El & Az of gun known
Lines ofconstant
range