Computer Vision - Lane Line Detection

By Jonathan Mitchell

Credit: Udacity’s Self-Driving-Car Nanodegree Program and Community

ContentA) Image as tensorsB) GrayscaleC) Gaussian BlurD) Canny Edge DetectionE) Masked ImageF) Hough TransformG) Overlay Hough Image over Original

A) Images as TensorsTensors are matrices with additional dimensions. (Scalar -> Vector -> Matrix -> Tensors)

Example a 2x3x2 Tensor

Tensor[0,0,0] = 2Tensor[0.0,1] = 3Tensor[1,1,0] = 4

Image = mpimg.imread(‘test_images/’ + imagefile.png)

Image is now a tensor. (960 x 540 x 3)

This means 960 rows, 540 columns, and 3 elements per each unit. (R,G,B)

These elements contain brightness values [0 to 255] for each corresponding pixel.

You can think of a pixel as a unit containing 3 elements.

Image[:,:,0] => red_values Image[:,:,1] => green_values Image[:,:,2] => blue_values

A) Images as Tensors

[R,G,B]Pixel list

A) Visualizing Images as TensorsImage.shape => (258, 320, 3)

Y[0,258]

X[0,258]

Y X

Image[:,:,0] => red_channel Image[:,:,1] => green_channel

[R,G,B]

We can isolate the Red, Green, and Blue Channels

B) Grayscale● We convert our image to grayscale so that we can abstract away from a Tensor

to a Matrix and deal with raw pixel values. (960,540,3) -> (960,540)● Treat yellow or white lane lines the same● Pixel list = color channel● Average color channel values within the pixel list (R + G + B / 3)

gray = cv2.grayscale(image)

Input = (960,540,3)

TENSOR

output = (960,540)

MATRIX

C) Gaussian Blur● Apply Gaussian Blur to reduce image noise and

detail. ● Smooth out the image before applying Canny edge

detection so we do not detect faint edges● Blur_gray = cv2.GaussianBlur(src, ksize, std_dev)● Src: image to modify● Ksize = kernel size. Larger kernel size implies

averaging/ smoothing over a larger area. (sample size for convolution)

● Return a new matrix Blur_gray If std_dev = 10

D) Canny Edge Detection- Used to detect boundaries of an image- Uses differential of pixel values- Three regions:- A: Pixel higher than upper threshold = edge (kept)- B: Pixel between High and Low is accepted if connected to edge- C: Pixel below Low rejected- Recommend 1:2 or 1:3 Low:High threshold ratio

High Threshold

Low Threshold

A

B

C

Process so far:

Original Color Image(960,540,3)

Grayscale(960,540)

Gaussian Blur

Canny Edges

A

B

C

D

We’re about halfway

Remember to display gray color map when plotting gray imagesplt.imshow(gray, cmap=”gray)

E) Masked Image● We want to narrow down our analysis to a

section of the image.● Apply a trapezoidal mask over edges● Use bitwise AND to return image only where

mask pixels are non-zero.

left_apex right_apex

right_bottomleft_bottom x

y

Apply over edges

Original Color Image(960,540,3)

Grayscale(960,540)

Gaussian Blur

Canny Edges

A

B

C

D

Masked ImageE

F) Hough Transform● Goal: To identify straight lines● Process: For each pixel at (x,y) the hough transform algorithm determines if

there is enough evidence of a straight line at that pixel● Lines = cv2.HoughLinesP(canny_edges, rho, theta, threshold,

min_line_length, max_line_gap)● Rho: Distance of P(pixels). Minimum value of 1. Higher Rho = Increased

distance resolution● Theta: angular resolution. Higher theta = higher angular resolution● Min_line_length: minimum length of a line in pixels you will accept in the

output ● Max_line_gap: max distance (in pixels) between segements that you allow to

be connected into a single line

F) Hough Transform● Theta = pi/180● Rho = 1.89● Threshold = 75● min_line_len = 25● max_line_gap = 125● (values dependent on features)

Apply Hough Transform then draw_lines

Onto original image

Original Color Image(960,540,3)

Grayscale(960,540)

Gaussian Blur

Canny Edges

A

B

C

D

Masked ImageE

Hough ImageF

There was a Draw Lines process in between E and F

G) Overlay Hough Image on Original● cv2.addWeighted(initial_img, α, img, β, λ)● initial_img: original img to write over● Img: img to overlay on top of initial_img

Original Color Image(960,540,3)

Grayscale(960,540)

Gaussian Blur

Canny Edges

A

B

C

D

Masked ImageE

weighted ImageG

Hough ImageF

Thank you

Email me if you have questions/ feedback: jmitchell1991@gmail.com

Our github: https://github.com/Self-Driving-Cars-LosAngeles/Computer-Vision-Lane-Detection/blob/master/SDC-Lane-Detection-Meetup1.ipynb