Upload
jackeline-bodwell
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
Computational Intelligence Research in ECETarget detection and
Recognition
Linguistic Scene Description
Sketch Understanding
What’s Coming?
Jim Keller, Marge Skubic, Dominic Ho and others
Robot Spatial Reasoning
Scene Matching
Computational Intelligence Technologies
• Fuzzy Set Theory and Fuzzy Logic
• Neural Networks
• Probabilistic Reasoning – Including Evolutionary Computation and Genetic Algorithms
• To Build Intelligent Systems for:– Image/Signal Processing
– Pattern Recognition
– Robotics
• Strong Applications Orientation
Object Detection and Recognition
Large long-term effort in Landmine Detection
Forward-looking GPR
Seismic/Acoustic Sensing
Technology Transfer from Basic Research to Fielded Systems
Army Research Office MURIBasic Research
TechnologyTransfer
HSTAMIDS GSTAMIDS
Robotic Tripwire Detection
Little Bob
Morphological Shared Weight Neural Networks for Object Recognition
First Developed to Find Object (Blazer) in Visible Imagery
Original Frame
Output Plane
Target Aim Point Selection
Final Output
All Twelve Targets Detected with no
False Alarms
Target Detection in SAR Imagery
Application to Tank Detection in (Processed) LADAR Range Images
Trained on 2 frames from one sequence (8 instances)
Tested on Different Flight Sequence
Face Recognition (Homeland Security)
Typical Training Image of Bob
Examples of “Bob” Detection
Even with Glasses on
Automated Amblyopia Screening Assistant
1. Input Image (from video sequence)
2. Locate Iris Pair 3. Locate Eyelids
4. Locate Pupil Pair 5. Locate Hirschberg Points / Estimate Fixation
6. Extracting Features(e.g., Crescents)
Screenshot of Program User InterfaceSelect Patient Step through images
Process all images of a patient
Get summaryof sequence
Intermediateimages to view
Intermediate image - irises found
Intermediate image - pupils found
Text feedback
Result of all images Sequence summary dialog
Equine Gait AnalysisMotion capture Data analysis
Classification
Animation for visualization
Database
Pre-processand store
normal dysfunctional
Scene Description
• Natural scene understanding is an important aspect of computer vision
• Spatial relations among image objects play a vital role in the description of a scene
2 3 41
F
A B
( )rA B
v
A
B
Histogram of forces
gravitational
constant
-
-
3
2
1
1
3. A system of 27 fuzzy rules and meta-rules allows meaningful linguistic descriptions to be produced.
1. Each histogram gives its opinion about the relative position between the objects that are considered.
2. The two opinions are combined. Four numeric and two symbolic features result from this combination.
Linguistic Scene Description
Linguistic Scene Description
There are 5 missile launchers (1, 2, 3, 6, 8)They surround a center vehicle (4)The image includes a SAM siteA convoy of vehicles (5, 7, 9, 10) is BelowRight of the SAM site
Scene Description and Recognition
Loosely ABOVE-LEFT.
The system here describes the relative positionof the red object(s) with respect to the group of buildings (in blue).
Do These Images Contain The Same Power Plant ?Do These Images Contain The Same Power Plant ?
Image 1
Image 2
Another Question : If they are indeed the same power plant, which building(s) that appear on Scene 1, also appear on Scene 2 ? (labeling problem)
DIRECT MATCHINGIS DIFFICULT
overhead view
?
overhead view
?
THIS WOULDBE EASIER
Scene Matching Using Fuzzy Regions
The only true matching got the highest matching degree
3,628,800 ways to match the two scenes.
Scene Matching and Recovery of View Parameters
Human/Robot Dialog• Spatial Reasoning incorporated into NRL’s Natural
Language Understanding System for mobile robots
• Sensed data results in a “grid map” that displays occupancy of cells (doesn’t need to be binary)
Grid map after component labeling – robot heading towards Object 5
DETAILED SPATIAL DESCRIPTIONS for 6 OBJECTS:
•Object 1 is mostly behind me but somewhat to the right (the description is satisfactory). The object is very close.
•Object 2 is behind me (the description is satisfactory) The object is very close.
•Object 3 is to the left of me but extends to the rear relative to me (the description is satisfactory). The object is very close.
•Object 4 is mostly to the right of me but somewhat forward (the description is satisfactory). The object is very close.
•Object 5 is in front of me (the description is satisfactory). The object is very close.
•Object 6 is to the left-front of me (the description is satisfactory). The object is close.
Scene 1
High-Level Description
There are objects in front of me and behind me.
Object number 3 is to the left of me.
Object number 4 is mostly to the right of me.
What do you see, Roby?
PATH DESCRIPTION GENERATED FROM THE SKETCHED ROUTE MAP1. When table is mostly on the right and door is mostly to the rear (and close) Then
Move forward2. When chair is in front or mostly in front Then Turn right3. When table is mostly on the right and chair is to the left rear Then Move forward4. When cabinet is mostly in front Then Turn left5. When ATM is in front or mostly in front Then Move forward6. When cabinet is mostly to the rear and tree is mostly on the left and ATM is mostly in front Then Stop
Understanding Sketched Route MapsUnderstanding Sketched Route Maps
Sketch-Based NavigationSketch-Based Navigation
The sketched route mapThe robot traversing the sketched route
Sketch-Based NavigationSketch-Based Navigation
The digitized sketched route map
The robot traversing the sketched route
Identification of Spatial Regions
“GORT, go behind object #3”
Region shown in greenBased on Histograms of Forces
Centered here
Future work: combine ATR with spatial language
Cognitive Robotics:Collaborative Research with Vanderbilt Univ.
Funded by NSF
What’s Coming? (Here, Actually)
Robot Skill Acquisition: Teaching Robonaut New Skills
What’s Coming? (Proposed)
What’s Coming? (We Hope)
Sensor-loaded (Sensor nets)
Intelligent Action (Reasoning)
Cooperative Behavior
Power Issues
Communication
Hardware and Systems Concerns
Mini-, Micro-, Nano- and Bio-Scales
Potential Cast of Characters:
Jim, Marge, Henry, Lex, Shubhra, Randy, Mike, Rusty, Dominic, Yi (CS), Sheila (BAE), …
Mobile Robot Teams Research Laboratory