Marker-less motion capture systems using depth cameras,

such as Microsoft Kinect Sensor, recently became a hot issue

in auto industry for their size, price, performance, and

portability. The marker-less motion and face tracking system

is expected to replace the typical optical motion capture

systems and reduce set-up and motion capture time greatly.

The advantages of size and portability of the depth sensor

will enable testing it in an actively running vehicle. In this

study, the Microsoft Kinect v2 depth camera was used to

gather 2.5-dimensional color and depth (RGB-D) data along

with hypothesized skeletal linkage data in real time.

In order to improve the accuracy and performance of the

obtained data, various techniques were implemented in the

developed software, including background removal, face

recognition, noise reduction, and model-based joint location

prediction. Various motions were tested to validate the

system with and without a vehicle mockup structure while

compared to the golden reference data obtained from Vicon

system.

Authors: Hwan Lee, Nevin Mital, Christopher Atkins; Matthew P. Reed, PhD; and Byoung-Keon D. Park, PhD

CONCLUSION

RESULT

11:24:31:850 11:24:38:515 11:24:45:379 11:24:52:92 11:24:58:756

2.0

2.2

2.4

2.6

2.8

3.0

3.2

3.4

A

11:24:31:850 11:24:38:515 11:24:45:379 11:24:52:92 11:24:58:756

1800

2000

2200

2400

2600

2800

3000

3200

3400

C2

0

A

0 500 1000 1500 2000 2500

0

100

200

300

400

500

Head x

0 500 1000 1500 2000 2500

200

300

400

500

600

700

Head y

0 500 1000 1500 2000 2500

2000

2100

2200

2300

2400

2500

Head z

0 500 1000 1500 2000 2500

-100

-50

0

50

100

150

200

250

300

350

400

Left Shoulder x

0 500 1000 1500 2000 2500

100

150

200

250

300

350

400

450

500

550

600

Left Shoulder y

0 500 1000 1500 2000 2500

2100

2200

2300

2400

2500

Left Shoulder z

0 500 1000 1500 2000 2500

200

250

300

350

400

450

500

550

600

650

700

Right Shoulder x

0 500 1000 1500 2000 2500

0

50

100

150

200

250

300

350

400

450

500

Right Shoulder y

0 500 1000 1500 2000 2500

2050

2100

2150

2200

2250

2300

2350

2400

2450

2500

Right Shoulder z

ABSTRACT

FUTURE WORK

ACKNOWLEDGEMENT

This research was supported by Ford

Motor Company and the Center for Child

Injury Prevention Studies (CChIPS)

Fragment of original joint depth data (top) and

interpolation processed depth data (bottom)

Comparision of aligned skeleton

data from Vicon and Kinect

SOFTWARE DEVELOPMENT

Kinect sensor tracks 25 joints in real time. Missing joints are marked in red color.

Marker-less motion capture system with data refinement software

is reliable and can be used for variety of motion analysis and

safety vehicle system including ingress/egress, human machine

interaction (HMI), child backseat safety, doze driving warning,

and crash motion analysis, etc.

Kinect sensor’s portability benefits variety of testing environment

including testing in active running vehicles which marker-based

motion capture systems cannot achieve.

Kinect sensor range was adequate for vehicle motion capture.

Noise reduction, interpolation, and segment fixing technique

enhanced data quality.

Marker-less motion capture depth sensors do not require any

marker set attached on human body and calibration process

which saves a lot of testing time.

Single Kinect sensor is limited to track hidden joints occluded by

other body parts or obstacles.

Kinect sensor raw data has non-linear spatial depth-resolution problem. Post data

processing software was developed to reduce noise and interpolate missing points.

To overcome Kinect sensor resolution issue,

joint location refinement software was

developed.

Processed Kinect data (red) vs Vicon data for rising arms motion

Processed depth data quality depends on the original depth data

obtained from Kinect sensor. Depends on Kinect direction and

subject orientation, joint locations could be reliable/unreliable.

To overcome this issue, multiple Kinect real-time

synchronization development will be considered.

Using depth data and Vicon data, software will be developed to

improve prediction of poorly tracked motion by matching

motions from motion database pool containing numerous

postures and behaviors of humans.

Monitoring and Modeling Vehicle Occupant Motions