Embed Size (px)
Citation preview
AN OPTIMISED FRAME WORK FOR MOVING TARGET DETECTION FOR
UAV APPLICATION
Md. Shahid, Pooja HR#
Aeronautical Development Establishment(ADE), Defence Research and development Organization(DRDO),
Bangalore - 560075
# Siddaganga Institute of Technology(SIT), Tumkur
Agenda
ADE: UAV Scenario
Introduction
Problem Statement
Acquisition and Tracking
Proposed Frame work
Evaluation Criteria
Results Analysis
Conclusion
Scope of future work
Discussion
ADE: UAV Scenario
LAKSHYA – Aerial target NISHANT – TUAV RUSTOM-1 UAV
Introduction
Electro Optics (EO) Payload
DTV Camera
FLIR Camera
Electronics package
Unmanned Aerial Vehicles (UAVs) are increasingly being used for reconnaissance and surveillance.
GROUND CONTROL STATION (GCS)
Problem Statement
Acquisition: – Moving target detection under platform disturbances and delays.
Tracking: – Tracking target/vehicle independent of speed/maneuvering.
– Not limited to number of targets.
Target Acquisition
Target Tracking
Manual Automatic
Why moving target detection?
Acquiring moving targets from airborne platform is difficult task due to associated delays.
– Video Downlink: 200 mSec
– Commands uplink: 400 mSec
– Object displacement: ~ 100 pixels (in acquisition)
Ground Control Station(GCS) Antenna Vehicle
Video downlink (200 mSec)
Command uplink
(400 mSec)
Acquisition Problem
Tracking
Proposed Frame work for Acquisition
Input
Video
Moving targets
detected video
* MATLAB 2013a and its tool boxes
Interest point Detection
(Eligibility Criteria)
Registration (Scene lock)
Background Subtraction
Clutter reduction
Interest Point Detection
Definition: – Local image structure around the interest point is rich in terms of
local information contents.
Examples : Corner, blob, ridge, edge etc.
Corner Detection Techniques:
– Harris Detector.
– Moravec Operator.
– Features From Accelerated Segment Test (FAST).
– Median Method.
Interest point Detection
(Eligibility Criteria)
Registration (Scene lock)
Background Subtraction
Clutter reduction
Corner detection results
(a) (b) (a) (b)
(c) (d)
Figure : Corners detected by (a) Harris Detector (b) Moravec Operator (c) FAST algorithm (d) Median algorithm
Complexity Analysis
Complexity in terms of computations and memory (per pixel), is as follows
ALGORITHM
Multiplication operation
Arithmetic operation
Division/ Comparision
operation
Memory requirement
Robustness
Harris Detector
49 43 - 18
Excellent
Moravec Operator
72 136 - 1
Moderate
Fast Algorithm
1 18 - 2
Good
Median Algorithm
- 6 1/26 1 Moderate
Eligibility Criteria
A3(2x2)
A2(2x2)
A1(2x2)
A0(2x2)
Restricts the eligible candidates(pixels) to be under process for further corner detection.
Considering the pixel as a centre for its 5x5 size block A and its four sub-blocks.
Sub-blocks are of 2x2 size each
Threshold : D2 > Mean(A)
Reduces computational burden significantly.
3210 2 AAAAH
0213 2 AAAAV
222 VHD
Robustness of eligibility criteria
Eligibility criteria is robust under
– Translation
– Rotation
– Scaling
– Noisy environment
Computational saving
Algorithm/ Technique
Processing time (all pixels)
Processing time (eligible pixels)
Computational Saving
Harris Detector
~1.5458 sec ~0. 5825 sec 62%
Moravec Operator ~0.5449 sec
~0.2075 sec 61%
FAST Algorithm
~0.1059 sec
~0.0399 sec
62%
Results of eligibility criteria
(a) (b) (c) (d)
(e) (f) (g) (h)
Figure : (a) Original image of size 256x256. (b) Corners detected by Harris Detector. (c) Rotated image (-100).
(d) Corners detected for rotated image. (e) Image affected by Gaussian noise. (f) Corners detected for Image affected by Gaussian noise. (g) Original image resized to 128x128. (h) Corners detected for resized image.
Results of eligibility criteria(UAV image)
(a) (b) (c) (d)
(e) (f) (g) (h) Figure : (a) Original image of size 256x256. (b) Corners detected by Harris Detector. (c) Rotated image (-100). (d)
Corners detected for rotated image. (e) Image affected by Gaussian noise. (f) Corners detected for Image affected by
Gaussian noise. (g) Original image resized to 130x130. (h) Corners detected for resized image.
Registration
Why Registration?
– Arresting the background against platform movement
– Required for moving or dynamic platform
Assumptions
– Background forms most part of the scene
– Background interest points moves slower
than foreground
Multiple target tracking
– Circularization and correlation matching
– Restricting to least movement, r2 = x2 + y2
– Proportional weightage X = (4*x1 + 3*x2)/7,
Y = (4*y1 + 3*y2)/7
– Discarding poor target & Best target weigh more
Unregistered
video clip
Registered
video clip
Interest point Detection
(Eligibility Criteria)
Registration (Scene lock)
Background Subtraction
Clutter reduction
Background Subtraction
Background subtraction
– Key aspect of the frame work
Type of backgrounds
– Dynamic backgrounds
– Gradual illumination changes
– Sudden illumination changes
– Moved object
– Shadows
Various methods
– Pixel or region based methods
– Parametric or nonparametric methods
– Recursive or non-recursive methods
Interest point Detection
(Eligibility Criteria)
Registration (Scene lock)
Background Subtraction
Clutter reduction
Background Subtraction Methods
Bayesian histogram
Morphological filtering
Sigma-delta(∑-∆) motion detection
Visual Background Extractor (ViBe)
Static platform
(a) (b) (c)
(d) (e) (f)
Figure : a) Input video frame of static camera. b) Ground truth. c) Bayesian histogram. d) Morphological Filtering. e) ∑-∆ motion detection. f) ViBe method. Courtesy: video sequence Highway II (available at http://cvrr.ucsd.edu/aton/shadow/data/highwayII-raw.avi)
Moving platform
(a) (b) (c)
(d) (e) (f)
Figure : (a) Input video frame of UAV. (b) Ground truth. (c) Bayesian histogram. (d) Morphological Filtering. (e)∑-∆ motion detection. f) ViBe method. (Note: Clutter has not been removed)
Clutter
Clutter reduction algorithm
Background subtracted image
Remove the area containing less than 200 pixels
Find boundaries
Find, A o = Area occupied by object
& A B = Filled area of bounding box
Fill Ratio, A = A o/ A B
Aspect Ratio
A r =Width/Height
( A > 0.5) ?
Clutter
No
Moving Objects
(3.5 > A r > 1 )? No
Yes
Yes
Background Subtraction 1
2
Interest point Detection
(Eligibility Criteria)
Registration (Scene lock)
Background Subtraction
Clutter reduction
Evaluation Criteria
Subjective: Visual inspection
Objective :
– Percentage of Correct Classification(PCC)
– False Positive Rate (FPRate)
Where,
True positives(TP): Number of correctly detected foreground pixels
False positives(FP): Number of background pixels incorrectly classified as foreground
True negatives(TN): Number of correctly classified background pixels
False negatives(FN): Number of foreground pixels incorrectly classified as background
FNFPTNTP
TNTPPCC
TNFP
FPFPRate
Results Analysis
Conclusion
MATLAB helped us all the way to develop this frame work for real time UAV application
Extensively utilized following MATLAB Tool boxes
– Computer vision system tool box
– Image Processing tool box
– DSP System Processing tool box
– Statistics tool box
Quick study of various methodologies
Not limited to number of moving targets
Complexity is independent of target speed
Reduced time to develop this framework
Scope of future work
Immediate:
– Target merged to clutter
– Fill ratio criteria
(a) (b)
Figure : (a) Targets merged with clutter. (b) Target failing to fill ratio criteria.
Next:
– Clutter is more due to rolling
– Replacement of Morphological operation
References
• O. Barnichand M. Van Droogen broeck, "ViBe: A Universal Background Subtraction Algorithm for Video Sequences", IEEE Transactions on image processing, Vol 20, no.6, June 2011.
• A Robust and Computationally Efficient Motion Detection Algorithm Based on ∑-∆Background Estimation. A. Manzanera J. C. Richefeu. ENSTA/LEI, 32 Bd VictorF-75739 PARIS CEDEX 15, july 6, 2011.
• “FASTER and better: A machine learning approach to corner detection” in IEEE Trans. Pattern Analysis and Machine Intelligence, Edward Rosten, Reid Porter and Tom Drummond, vol 32, pp. 105-119, 2010.
• Saad Ali and Mubarak Shah, COCOA - TRACKING IN AERIAL IMAGERY. Computer Vision Lab,School of Computer Science,University of Central Florida, 2006.
• A Background Subtraction Model adapted to Illumination changes. Julio Cezar Silveira Jacques Jr., Claudio Rosito Jung and Soraia Raupp Musse, IEEE Transactions, 1-4244-0481-9/06/$20.00 C 2006.
• C. Harris and M. Stephens. A Combined Corner and Edge Detector. Proc. Alvey Vision Conf., Univ. Manchester, pp. 147-151, 1988.
• H. P. Moravec. Towards Automatic Visual Obstacle Avoidance. Proc. 5th International Joint Conference on Artificial Intelligence, pp. 584, 1977.
Acknowledgement
Aeronautical Development Establishment (ADE),
Defence Research and Development Organization (DRDO),
Bangalore, India.
Department of Electronics & Communication Engg.,
Siddaganga Institute of Technology (SIT), Tumkur, India.
CoreEL Technologies, Bangalore, India.
Discussion
