Human tracking and counting using the KINECT range sensor based on Adaboost and Kalman Filter

ISVC 2013

Problem

• Human tracking

Avoid occlusion

Human Detection

• Observations:– There is an empty space in the front and back of

head– The right side of right shoulder and the left side of

left shoulder are also empty– There is a height difference between the head and

the two shoulders

How to describe the spatial information of 3D HASP

• Those criteria can be formulated as the difference of two pixel areas in the depth map – Haar-like feature

• Adaboost is introduced to construct a strong classifiers from those weak criteria

Human Detection

Human Detection by Adaboost

• Framework

Spatial feature

• Processing window– 20 redefined sub-windows

Spatial feature

• Four Haar-like features

Depth integral image

• The sum of rectangle pixel values from the top-left corner to a pixel in depth image– To speed up the computation of Haar-like features

• All pixel intensity values of D:( ) (4) (3) (2) (1)areaValue D dd dd dd dd

Adaboost algorithm

• Construct a strong classifier by a weighted linear combination of weak classifiers

1, * ( ) *

0, * ( ) *

1, * ( )H( , , , )

1,

j j

j j

H jF

H j

wherep h x p

h x potherwise

Our Classifier

• Challenge– Human can stand and face all directions with many

postures

• Solutions– Combine a horizontal strong classifier and a

vertical strong classifier

( ) ( ) | ( )C hor verwin win winF F F

Horizontal Strong Classifier

• Formulation

1, * ( ) *( )

0, * ( ) *j j

horj j

H jwin

H jF

Vertical Strong Classifier

• Formulation

1, * ( ) *( )

0, * ( ) *j j

verj j

H jwin

H jF

Training

• Took many depth maps of each object by rotating a certain degree

• 720 positive images + 288 negative images

Results

• Testing on three datasets:– Dataset 1: only one human object standing in

different directions– Dataset 2: Two human objects– Dataset 3: three or more human objects

Results (Dataset 1)

Results (Dataset 2)

Results (Dataset 3)

Choice of window sizes

Limitation

• Fails if detected humans are standing two very close to each other– Improve tracking accuracy by incorporating

Kalman Filter, since the closing time is short in real tracking application.

Conclusion

• We construct a real-time human detection based the depth image from Kinect sensor

• Head and Shoulder Profile described by some Haar-like features is incorporated into Adaboost algorithm to detect human objects.

• Detection time for each image is about 33 ms.