View
217
Download
0
Category
Tags:
Preview:
Citation preview
GM-Carnegie Mellon Autonomous Driving CRL
Title Automated Image Analysis for Robust Detection of Curbs
Thrust Area Perception
Project Lead David Wettergreen, CMUWende Zhang, GMInna Stainvas, GM
Contributors JongHo Lee, CMU
1
GM-Carnegie Mellon Autonomous Driving CRL
ScheduleCurb location Sensor location Methodology Date
On the side Bottom of the side mirror Visual appearance ~ Jan. 2014
In front The front bumper Geometric structure ~ May. 2014
In front The front bumper Appearance + Geometry with production camera
~ Nov. 2014
DeliverablesDemonstration: In-vehicle curb detection
Annual reports
2
GM-Carnegie Mellon Autonomous Driving CRL
Objectives
Develop reliable methods of detecting, localizing, and classifying features associated with curbs using in-vehicle, low-cost, monocular vision sensor
Localize curbs within a range of 5 meters with 99% accuracy
3
GM-Carnegie Mellon Autonomous Driving CRL
Approaches for Curb Detection Using Mono Camera Images
• Appearance-based image analysis (~ Nov. 2013)• Extract features
• Evaluate performance
• Geometry-based image analysis (~ May. 2014)• Structure-from-motion to estimate camera motion
• Multi-resolution plane sweeping algorithm to create 3-D point cloud
• Plane fitting to detect curb
• Combine appearance and geometric analysis (This Review)
4
GM-Carnegie Mellon Autonomous Driving CRL
• Appearance-based image analysis (~ Nov. 2013)• Extract features
• Evaluate performance
• Geometry-based image analysis (~ May. 2014)• Structure-from-motion to estimate camera motion
• Multi-resolution plane sweeping algorithm to create 3-D point cloud
• Plane fitting to detect curb
• Combine appearance and geometric analysis (This Review)
Approaches for Curb Detection Using Mono Camera Images
5
GM-Carnegie Mellon Autonomous Driving CRL
Appearance-based image analysis
6
GM-Carnegie Mellon Autonomous Driving CRL
Edge Detection
7
GM-Carnegie Mellon Autonomous Driving CRL
Detect Curb Using HOG* Feature
* Histogram of Oriented Gradients
Input image HOG imageCurb model
8
GM-Carnegie Mellon Autonomous Driving CRL
• Appearance-based image analysis (~ Nov. 2013)• Extract features
• Evaluate performance
• Geometry-based image analysis (~ May. 2014)• Structure-from-motion to estimate camera motion
• Multi-resolution plane sweeping algorithm to create 3-D point cloud
• Plane fitting to detect curb
• Combine appearance and geometric analysis (This Review)
Approaches for Curb Detection Using Mono Camera Images
9
GM-Carnegie Mellon Autonomous Driving CRL
Geometry-based image analysis
Input image Depth image
Ground plane estimation
3-D point cloud
10
GM-Carnegie Mellon Autonomous Driving CRL
Plane Fitting
11
GM-Carnegie Mellon Autonomous Driving CRL
• Appearance-based image analysis (~ Nov. 2013)• Extract features
• Evaluate performance
• Geometry-based image analysis (~ May. 2014)• Structure-from-motion to estimate camera motion
• Multi-resolution plane sweeping algorithm to create 3-D point cloud
• Plane fitting to detect curb
• Combine appearance and geometric analysis (This Review)
Approaches for Curb Detection Using Mono Camera Images
12
GM-Carnegie Mellon Autonomous Driving CRL
Appearance at t+1
Schematic Overview
Input at t+1
Input at t Appearance at t
Geometry
Candidate regions
Annotate curb region
GM-Carnegie Mellon Autonomous Driving CRL 14
Appearance
- For each image, divide intom x n grids- m: image height / grid
size (pixels)- n: image width / grid size
(pixels)
Image at t
GM-Carnegie Mellon Autonomous Driving CRL 15
Appearance
- For each grid, classify among two classes (road, curb)- uniform Local Binary
Pattern (LBP)
Image at t
GM-Carnegie Mellon Autonomous Driving CRL 16
Local Binary Pattern
64 63 52
85 152 227
189 167 205
0 0 0
0 1
1 1 1
threshold
Binary: 00011110Decimal: 30
0 1 2 3 255
GM-Carnegie Mellon Autonomous Driving CRL 17
Appearance
- Once all the grids of two images are classified, get the intersection of them
Output at t+1Output at t Intersect
GM-Carnegie Mellon Autonomous Driving CRL
Geometry
- Green lines shows the vectors from the interesting points of image at time t (blue dots) to those of image at time t+1 (red dots)
- Calculate the 3-D points using camera matrix
GM-Carnegie Mellon Autonomous Driving CRL
Appearance + Geometry
- For each grid,- Fit the best plane using 3-D
points - Compute the normal vector- Determine the normal vector
is a road surface or a curb surface
GM-Carnegie Mellon Autonomous Driving CRL
Appearance + Geometry
GM-Carnegie Mellon Autonomous Driving CRL
Extend Curb Region
- If the appearances are similar, extend the curb region- Calculate the distance of LBPs using chi-square
GM-Carnegie Mellon Autonomous Driving CRL
Extend Curb Region
GM-Carnegie Mellon Autonomous Driving CRL
Track Curb Region
- For the next frames, tracking the appearance of the curbs- When tracking, keep checking the geometry constraint to
remove the false positives if exist
Input at t+2 Input at t+3 Input at t+4
GM-Carnegie Mellon Autonomous Driving CRL
Curved curb case
Extend Curb RegionCombine Analyses
GM-Carnegie Mellon Autonomous Driving CRL
Curb Detection using Production Camera
Image size : 480 by 640FOV: 180 degree
GM-Carnegie Mellon Autonomous Driving CRL
Test Curb Detection
Image Size: 640 x 480 (pixels)ROI: 640 x 160 (pixels)Size of grid: 20 x 20 (pixels)Number of grids: 32 x 8
Output of the appearance-based curb detection
GM-Carnegie Mellon Autonomous Driving CRL
Test Curb Detection
Remove outliers based on cluster size
Find edges using Canny operator inside candidate region
GM-Carnegie Mellon Autonomous Driving CRL
Test Curb Detection
- Fit polynomials to each segments, and check lines for similar curvatures (blue), and remove high curvatures (red) Annotate curb region on the
original input image
GM-Carnegie Mellon Autonomous Driving CRL
Future WorkApplication: Operate real-time curb detection in vehicle
~ May 2015
29
GM-Carnegie Mellon Autonomous Driving CRL
Thank you
Questions ?
30
Recommended