Object Tracking for Autonomous Mobile Robot based

on Feedback of Monocular-vision Xiaogang Guo, Changhong Wang, Zhenshen Qu

Space Control and Inertial Technology Research Center, Harbin Institute of Technology

Xi Da Zhi Street 92, Harbin, 150001, People’s Republic of China

Email: edward_gxg@126.com

Abstract - Object tracking is the key issue for autonomous

mobile robot navigation. In this paper, a method of mobile robot

object tracking is put forward based on feedback of monocular-

vision. This method implements object tracking through

identifying the special sign. And this method can also simulate

the spacecraft RVD. Experiments show: the method is effective

for object tracking of autonomous mobile robot navigation, as

well as robustness against environment disturbance.

Index Terms - Mobile Robot, Monocular-vision, Object

Tracking, Image processing, RVD

I. INTRODUCTION

An autonomous mobile robot(AMR) is an intelligent

machine system, which can run autonomously in the room or

the outside. Now the research on AMR is still the hot field in

the world since it is widely used in many fields such as

military, space exploration, dangerous environment and other

fields. The identifying and tracking of an object is one of the

key tasks for an autonomous mobile robot. Many different

kinds of sensors are used to mobile navigation system such as

infrared ray, laser telemeter, sonar, and visual systems. The

method of vision-based navigation is widely used now for

mobile robot[5]. It has higher distinguishability for space and

gray, the scope of detecting abroad, higher precision[2, 3, 4].

For a vision-driven mobile robot, it has a function of a vision

system to provide most of navigation information. The

conventional approach to vision-based object tracking is

stereovision and computation of range from image disparity.

Adoption of the monocular vision requires only a single

imager and exploits mobility to perform this task, thus

dramatically reducing the complexity of the image processing

system[8]. The measurement of monocular gets the passive

imaging of object through single camera, it implements

measurement making use of the prior information of object

and the model of geometric line[1]. In the system of object

tracking for autonomous mobile, the method of monocular-

vision predigests the structure of system maximally; it

achieves the algorithm of object tracking easily, and it

guarantees the real time of object tracking.

Space craft rendezvous and docking(RVD) is a complex

system of engineering. And further, we can use this mobile

robot object tracking system to simulate the RVD system. In

this system, the object can run freely, so we can use the object

to replace the target spacecraft and the mobile robot for the

chaser. So the spacecraft RVD can be easily replaced by the

course of object tracking for autonomous mobile robot.

Though this system is simpler than the RVD, it can be used

for the first step of study. In the following research, we will

put emphases on this.

The remainder of this paper is organized as follows: In

Section II, the component of object tracking system is

presented, and each of four layers is explained detailedly.

Section III is focused on the object identifying so that the

changes in the system can be tracked more effectively. The

algorithm of object tracking is discussed. In Section IV, the

object tracking is carried out and the fuzzy control method is

used. In Section V, it is the system implement and

experiment, an AS-R robot is used for the flat of experiment.

Finally, conclusions are drawn in Section VI.

II. COMPONENT OF OBJECT TRACKING SYSTEM

Component of the autonomous tracking experiment system

is shown in the picture below. This system includes four

parts: the bottom control system of mobile robot, object

identifying system, autonomous control system, and remote

console with wireless communication.

Fig. 1. architecture of object tracking system for autonomous mobile robot

The bottom control system is embedded in the mobile robot.

It consists of the mechanical system and a motor control

adapter. The robot is controlled by the motor control adapter.

The speed of the left and right wheels at any time may be

detected by counting the pulse from the left and right of micro

sensors which detect the slots in the encoder disks.

The object identifying system is made up of a monocular

CCD camera and a Matrox II video card. The real-time image

of object is captured by the CCD camera, and then it is sent to

the autonomous control system through the conversion of

Matrox II card.

467

1-4244-0737-0/07/$20.00 c©2007 IEEE

The autonomous control system is a most important part in

the system. It calculates the status information of the mobile

robot, and according to the information it gives commands to

the motor control adapter. The course of controlling is a

closed-loop control.

The remote wireless console system includes a remote

computer control center, a wireless access point and two

wireless adapters. The wireless console can control robot

manually.

III. OBJECT IDENTIFYING

A. the character of object

The object is the special pattern for image processing and

identifying, which is shown in the picture below.

Fig. 2. the mark of object

In the experiment system, we design the object that is a

target with three different radial concentric circles. Using the

fixed different color to distinguish the different circle, we

adopt black, white and black in this system, which has such

advantages. This object can expand or contract comparatively

with the center circle, and it doesn’t influence the algorithm

of image processing.

B. Algorithm of object tracking

As shown in the picture below, the CCD camera is fixed at

the upside of mobile robot. The background is complexity in

the visual field of camera, and various disturber as the

illumination exchange or the shadow of people and object[9].

Therefore, at the beginning of algorithm, it needs to pop out

the object for identifying. In this paper, the identifier of robot

above is a foundation for standing out the object.

Fig. 3. object identifying

To stand out the object, the common method is that we

transform gray image to binary image[7]. But the fixed value

of threshold is unacceptable, for the environment of mobile

robot changing. We choose adaptive threshold algorithm

based on average gray-scale of each block. This method can

give different threshold according to different background, in

order to stand out object and to eliminate local disturber.

The course of algorithm processing is shown in the picture

below.

Fig. 4. algorithm flow of adaptive threshold based on blocks of

average gray level

Firstly this algorithm divides the collection of 394*288

gray-image into m*m blocks, then calculates the average gray

level of each block and gives the corresponding binary

threshold. The value of m can be confirmed by itself.

In the experiment system, the global average value is

effective to filter the disturbance of the local, and the average

value of blocks is effective to stand out the object. But the

disturbance of the local influences the stability and rapidity of

algorithm severely. Thus in this system, we adopt the

approximate global average value in order to ensure the

stability of system.

Fig. 5. algorithm of ellipse drawing up

468 2007 Second IEEE Conference on Industrial Electronics and Applications

Binary image can be acquired through the algorithm of

prior processing. Then it is necessary to draw up the curve in

binary image.

Because the ellipse is a special circle, and the circle can be

changed into the ellipse in the condition of none-envisage[6].

Based on the above considering, we choose the algorithm of

ellipse drawing up in the experiment system.

The course of processing is shown in the Fig.5.

C. Data of vision feedback

The algorithm of mobile robot control needs the references

including the center image coordinate of object (pixel) and

the radial of three different circles. These data present the

position and orientation information of object in general

conditions. In complex conditions, for instance, the angle is

partial between the visual angle of robot and the object, the

control program can give the special algorithm of searching

for tracking.

IV. OBJECT TRACKING

When the algorithm of tracking doesn’t need to have the

higher precision of location, it requires that object tracking be

quickly and steadily[10]. In this system, we choose the

monocular-vision so that the algorithm of distance measuring

is simple. So the system can not get the precise distance

information of the object. But the system gets the fuzzy

distance information through the radial variety of object. The

controlled variable is calculated by the polling list of

fuzzification. And it is chosen through the experiment at a

great level.

The basic structure of algorithm is shown in the picture

below.

Fig. 6. algorithm structure of tracking

When the object enters into the visual field of mobile robot,

namely vision feedback presents the data of location, the

algorithm of control implements the course of stable tracking.

The system acquires the precise position and fuzzy distance

information of object through processing the data of vision

feedback. So it can give the fuzzy control field: LEFT, MID,

RIGHT, TOOFAR, FAR, NEAR, in order to control robot

with closed-loop. According to the difference of control field,

the different algorithm of control is presented. So the robot

control is smooth, and the tracking is stable.

V. SYSTEM IMPLEMENT AND EXPERIMENT

In this system, the experiment object we choose is an AS-R

robot that it is the research version of the Power Strom robot

produced by the Shanghai Grandar Robotics Co. Ltd. Using

the CCD gray camera upside the mobile robot to locate the

object, we can get the position and orientation information of

robot. At the same time, we send the real-time data of position

and orientation to the control program of robot tracking, then

the algorithm of fuzzy control can control the position and

orientation of robot according to the different situation, at last

complete the experiment by tracking the object.

In the experiment, the object is placed 4 meters far from the

robot. By the remote control center, the robot was

commanded to implement the course of tracking. Firstly, if

the object was not in the visual scope of the robot, the robot

would rotate slowly to look for the object. Until the object

was found, the distance and visual angle between the robot

and the object was calculated. According to the information

of distance and orientation, the robot reaches to the object and

tracking it. The robot was controlled by the fuzzy control

method in this course.

VI. CONCLUSION

In this paper we present a method of object tracking for

autonomous mobile based on the feedback of monocular-

vision. This method can acquire the precise orientation and

fuzzy distance information of object, so as to implement the

object tracking for mobile robot through identifying the

special mark and processing the data of vision feedback. And

further, this system is the basic study for the behind research

of RVD. Proved by our experiments, this method can track

the object with real-time successfully, and it can also quickly

and precisely locate the object. So the simulation method of

mobile robot object tracking is effectively for the RVD

system. This method, which is able to restrain noise, is robust.

Thus, it is a reference for real-time tracking of autonomous

mobile robots system in complex environment.

REFERENCES

[1] M. Bertozzi, A. Broggi, and A. Fascioli, “Vision-based intelligent

vehicles: state of the art and perspectives,” Robotics and Autonomous

Systems, 2000 (3): 1 16Proceedings of 6th Conference on Optical 3-D

Measurement Techniques[C], Zurich: ETH, 2003.

[2] W. Choi, C. Ryu, and H. Kim, “Navigation of a mobile robot using

mono-vision and mono-audition,” Proceedings of the IEEE International

Conference on Systems, Man and Cybernetics, pp. IV-686 - IV-691, v4,

1999.

[3] Davidson, C. James, Hutchinson, and A. Seth, “Recognition of

Traversable Areas for Mobile Robotic Navigation in Outdoor

Environments,” IEEE International Conference on Intelligent Robots and

Systems, pp. 297-304, v1, 2003.

[4] R. Gregor, M. Lutzeler, and M. Pellkofer, “EMS-Vision: a perceptual

system for autonomous vehicles,” IEEE Transactions on Intelligent

Transportation Systems, pp. 48-59, 3 (1), 2002.

[5] X. G. Guo, Z. S. Qu, and B. Q. Xi, “Research on Location Method for

Mobile Robots Formation based on Global-camera,” ISSCAA2006, pp.

347-349, 2006.

[6] L. J. John, C. H. Declan, and P. W. Karim, “Vision-based navigation for

rendezvous, docking and proximity operations,” [R]. AAS, 99-102, 1999.

[7] Pettersen, Alf, Rotvold, Oyvind. Method and system for point

measurement of spatial coordinates [P]. U. S. Patent: 5440392, 1995.

2007 Second IEEE Conference on Industrial Electronics and Applications 469

[8] M. Poteser, K. Kral, “Visual Distance Discrimination Between Stationary

Targets in Praying Mantis: an Index of The Use of Motion Parallax,” The

Journal of Experimental Biology 198, pp. 2127-2137, 1995.

[9] M. Subbaro, G. Surya, “Depth from defocus: a spatial domain approach,”

[J]. Int J Computer Vision, pp. 271 -294, 13 (3), 1994.

[10]H. Zhang, J. P. Ostrowski, “Visual motion planning for mobile robots,”

[J]. IEEE Transactions on Robotics and Automation, pp. 199-208, 18 (2),

2002.

470 2007 Second IEEE Conference on Industrial Electronics and Applications