Comparison of Pedestrian Detection SystemsSeminar: Mobile Human Detection Systems

Felix Stern, 3142747

Universitat HeidelbergInstitut fur Technische Informatikfelix.stern@stud.uni-heidelberg.de

03. Februar 2017

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Overview

1 Motivation

2 Problem Formulation

3 Solution Approach

4 Methods

5 Results

6 Conclusion

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 2 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Motivation

[2]

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 3 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Problem Formulation

[2]

Given: (Stereo-)camera on robot, car, ”Google Glasses”

Required: Bounding boxes of detected pedestrians

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 4 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Solution Approach

Single frame of pedestrian areasLower image quality (motion blur, artifacts, light conditions)Possible reflectionsPartially-occluded pedestrians

Two approaches considered:

1 Using mono-camera:

Create Histograms of Oriented Gradients (HOG)

Train Support Vector Machine (SVM)

Use binary classifier for detecting humans in image blocks

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 5 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Solution Approach

Single frame of pedestrian areasLower image quality (motion blur, artifacts, light conditions)Possible reflectionsPartially-occluded pedestrians

Two approaches considered:

1 Using mono-camera:

Create Histograms of Oriented Gradients (HOG)

Train Support Vector Machine (SVM)

Use binary classifier for detecting humans in image blocks

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 5 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Solution Approach / 2

2 Using additional depth information (stereo camera):

bounding boxes of detected objects (pedestrians) are requiredidentify ground plane in the depth mapuse ground plane to evaluate pedestrian hypotheses

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 6 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Histograms of Oriented Gradients

directional change in intensity / color

dx = I (c + 1, r)− I (c − 1, r) dx = 127− 0 = 127

dy = I (c , r − 1)− I (c , r + 1) dy = 255− 0 = 255

gradient orientation: θ = tan−1( dydx ) ∗ 180

π θ ≈ 63.5

gradient magnitude:√dx2 + dy2

√2552 + 1272 ≈ 285

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 7 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Histograms of Oriented Gradients

directional change in intensity / color

dx = I (c + 1, r)− I (c − 1, r) dx = 127− 0 = 127

dy = I (c , r − 1)− I (c , r + 1) dy = 255− 0 = 255

gradient orientation: θ = tan−1( dydx ) ∗ 180

π θ ≈ 63.5

gradient magnitude:√dx2 + dy2

√2552 + 1272 ≈ 285

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 7 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Histograms of Oriented Gradients

directional change in intensity / color

dx = I (c + 1, r)− I (c − 1, r) dx = 127− 0 = 127

dy = I (c , r − 1)− I (c , r + 1) dy = 255− 0 = 255

gradient orientation: θ = tan−1( dydx ) ∗ 180

π θ ≈ 63.5

gradient magnitude:√dx2 + dy2

√2552 + 1272 ≈ 285

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 7 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Histograms of Oriented Gradients / 2

64x128 pixel detection window

window is divided into cells and blocks:

cell: 8x8 pixelblock: 2x2 cells

blocks have 50% overlap

block cells normalized by L2-Hys norm

gradient orientation quantized into 9 bins(each 20◦), magnitude added to bin

1D feature vector by concatenatinghistogram values

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 8 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Histograms of Oriented Gradients / 2

64x128 pixel detection window

window is divided into cells and blocks:

cell: 8x8 pixelblock: 2x2 cells

blocks have 50% overlap

block cells normalized by L2-Hys norm

gradient orientation quantized into 9 bins(each 20◦), magnitude added to bin

1D feature vector by concatenatinghistogram values

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 8 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Histograms of Oriented Gradients / 3

[1]

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 9 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Support Vector Machine

classifier to predict a class

labeled training data are needed

construct maximum-margin-hyperplane

Source: https://upload.wikimedia.org/wikipedia/commons/f/f2/Svm_intro.svg

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 10 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Stereo Vision

orthogonal depth can be calculated using the intercepttheorem

Figure: f : focal length, z : orthogonal depth, X : reconstructed point,x/x ′: vertical image position, O/O ′: camera originSource: http://docs.opencv.org/3.1.0/dd/d53/tutorial_py_depthmap.html

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 11 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Stereo Vision Depth Map

Source: http://www.360doc.com/content/14/0512/16/17164701_376968762.shtml

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 12 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Multiple Pedestrian Detection

ground plane helps in constraining object detection tomeaningful locations

[2]

hypotheses: bounding boxes around pedestrians (used: ISMdetector)

depth cues: evaluate depth inside bounding box of hypothesis

depth map evidence: probability that the ground planegenerated the depth map (Belief Propagation)

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 13 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Multiple Pedestrian Detection / 2

Figure: ground plane in depth maps could be missing (right)

[1]

using ground plane, bounding box and depth cue to verify ifthe hypothesis is valid

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 14 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Results (HOG)

[1]

best cell size for detecting humans: 8x8, block size 2x2

extremities of pedestrians are about 8px wide

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 15 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Performance Comparison

test datasets and metricsdiffer (FP/W vs. FP/I)

used image size in [2]:640x480

in [1], this would result in73x45 detection windows

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 16 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Conclusion

two pedestrian detection systems compared

only single images/frames needed (no series)

simple approach: Histograms of Oriented Gradients

very accurate on tested datasetsdependent on block and cell size

advanced approach: using additional depth information

first extract ground plane, using Belief Propagationuse those information to validate detections

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 17 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Critique

Approach 1 (HOG):

only one pedestrian per window

undefined maximum occlusion rate for detection

pedestrians have all the same dimensions

Approach 2 (additional depth information):

dependent on pedestrian detector

explanation of the ground plane’s effect missing

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 18 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Critique / 2

[2]

Figure: white boxes: true positives, red boxes: false positives

problems with reflections still existing

faraway pedestrians not detected (> 25m)

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 19 / 21

1. Motivation 2. Problem Formulation 3. Solution Approach 4. Methods 5. Results 6. Conclusion

Sources

Paper 1: Histograms of Oriented Gradients for HumanDetection, Navneet Dalal, Bill Triggs

Paper 2: Depth and Appearance for Mobile Scene Analysis,Andreas Ess, Bastian Leibe, Luc Van Gool

ISM detector: Pedestrian detection in crowded scenes, Leibe,Seemann, Schiele

Image p.3, p.4, p.13, p.16, p.17: Paper 2

Image p.7, p.8, p.14, p.15: Paper 1

Image p.9: https://upload.wikimedia.org/wikipedia/commons/f/f2/Svm_intro.svg

Image p.10: http://docs.opencv.org/3.1.0/dd/d53/tutorial_py_depthmap.html

Image p.11: http://www.360doc.com/content/14/0512/16/17164701_376968762.shtml

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 20 / 21

Stereo Calculation

ϕ = tan(w/2f ), B = B1 + B2 = Z ∗ tan(ϕL) + Z ∗ tan(ϕR)

x−w/2w/2 = tan(ϕL)

tan(ϕ) ,x ′−w/2w/2 = tan(ϕR)

tan(ϕ)

→ Z = B∗w2∗tan(ϕ)∗(x−x ′)

Source: http://dsc.ijs.si/files/papers/s101%20mrovlje.pdf

03. Februar 2017 Felix Stern Comparison of Pedestrian Detection Systems 21 / 21