In Situ Real-Time Vision-Based Lane Detection on Raspberry Pi using 1D Haar Wavelet Spikes

Vladimir KulyukinDepartment of Computer Science

Utah State University

IMECS 2017

Outline

IntroductionLane Detection with 1D Haar Wavelet Spikes EvaluationConclusions

Introduction

Self-Driving Cars

Pros– Enhanced mobility for the elderly and disabled– Reduction in traffic accidents

Cons– Major job losses in driving and transportation industries– Loss of privacy and increased risk of hacking attacks

For now, human drivers are still indispensable

Related Work

S. Mallat and W. L. Hwang. “Singularity Detection and Processing with Wavelets.” IEEE Trans on Information Theory, vol. 38, no. 2, pp. 617 – 643, March 1992

Y. Wang, E. Teoha, D. Shen. “Lane detection and tracking using B-snake.” Image and Vision Computing, vol. 22, pp. 269-280, 2008

Z. Kim. “Robust lane detection and tracking in challenging scenarios.” IEEE Trans. on Intelligent Transportation Systems, vol. 9, no. 1, pp. 16 – 26, Mar. 2008

P. Hsiao, C. Yeh, S. Huang, L. Fu. “A portable vision-based real-time lane departure warning system: day and night.” IEEE Trans. on Vehicular Technology, vol. 58, no. 4, pp. 2089 – 2094, May 2009

P. Mandlik, A. Deshmukh. “Raspberry-pi based real time lane departure warning system using image processing.” International Journal of Engineering Research and Technology, vol. 5, issue 06, June-2016, pp. 755 – 762

Lane Detection with

1D Haar Wavelet Transform Spikes

Up-Down Spikes

Up-down spikes

Down-Up Spikes

Down-up spikes

Hardware

The hardware consists of an RPi 3 Model B ARM v8 1GB RAM computer, an RPi Camera Board v2, an RPi Night Vision Camera, and a 7 inch RPi touchscreen

The hardware is placed inside a Jeep Wrangler with a camera attached to the windshield from the inside and is powered through a regular 12V-to-5V car charger

As the car moves along a road, the detected lanes are drawn in real time in a small bottom right window on the touchscreen

HaarSpiker: Spike Detection Algorithm

HaarSpiker is an algorithm that uses 1D Haar Spikes to detect lanesThe algorithm is implemented in Python 2.7.9 with OpenCV 3.0.0The algorithm consists of three phases: pre-processing, spike detection, line fitting

Preprocessing: Cropping a Region of Interest (ROI)

A 360 x 240 PNG image is taken and a 56 x 200 ROI is cropped in the bottom center portion of the image where road is likely to be

Preprocessing: Grayscale, Blur, ThresholdGrayscale cropped ROI

Preprocessing: Grayscale, Blur, ThresholdGrayscale cropped ROI

Blur with Gaussian 7x7 kernel

Preprocessing: Grayscale, Blur, ThresholdGrayscale cropped ROI

Blur with Gaussian 7x7 kernel

Threshold with Otsu

Spike Detection

- Take image row segments on both sides of the image- Each segment is 64 pixels long (but this is a parameter)- Apply 2 iterations of Haar Wavelet Transform to each row segment- Detect up-down spikes in each row and compute the means of the flat segments for each detected spike- Adjust the start and end row segments on the basis of where the mean is detected in the previous row

Spike Detection

- Do you go top to bottom or bottom to top? Bottom to top, because you want to detect what is closest to the car - Do you process every row? No, go up in increments of 5 rows. This is a parameter that can be adjusted

Line Fitting

1D polynomial line fitting is applied to the (x, y) spike positions The lines are filtered by inclination angles:

- Left line threshold values are -60° to -30°- Right line threshold values are 30° to 60°

The final left and right lines indicate left and right lanes

Ideal Case of Lane Detection

Evaluation

Data

All the images taken from a Jeep Wrangler going at a speed of 60 mph on State Route 30 in Northern Utah

Sample 1: October 12th, 2016, a sunny day, consists of 1000 PNG images

Sample 2: November 12th, 2016, a cloudy day, consists of 1000 PNG images

Sample 3: January 6th, 2017, a snowy night, consists of 1000 PNG images

Sample 4: January 7th, 2017, a snowy day, consists of 775 PNG images

Image Size: 360 X 240 pix

Results

Sample Num. of Images Both Lanes Detected (%) At Least 1 Lane Detected (%)

1 1000 61.90 91.20

2 1000 34.10 77.40

3 1000 16.90 64.10

4 775 15.74 57.03

Results

All true negatives in sample 1 are caused by shadows cast by other vehicles or trees and changes in road surface texture

The algorithm’s accuracy on sample 2 which is 74.1% is worse due to faded road lanes on a cloudy day

Results

The algorithm’s accuracy drops to 64.10% on sample 3 on a snowy night

The algorithm’s accuracy deteriorates to 57.03% on sample 4 on a snowy day

ConclusionsThe algorithm processes 22 images/second in situ at a speed of 60 mph

The system’s hardware can be placed inside a car, next to the windshield, and can be powered through a regular 12V-to-5V car charger, i.e. the power requirement is 10W

The hardware and software components of the presented algorithm can be replicated with off-the-shelf hardware components and open source software

The algorithm performs well in fair weather; the performance declines at night and on snowy roads

Data and Source Code Availability

Images used in this study are available at https://www.dropbox.com/sh/yqpq0adt42n54dt/AAA6m5OF4s_C2KYYDLI4kc2ra?dl=0

Py code is available at https://github.com/VKEDCO/PYPL/tree/master/haar_spiker

Thank You!