ORIGINAL ARTICLE

C.-C. Wang (*)Department of Electronic Engineering, Chienkuo Technology University, No. 1, Chieh Shou N. Rd., Changhua City, Taiwan, ROCe-mail: jasonccw@ctu.edu.tw

S.-F. LienGraduate School of Engineering Science and Technology (Doctoral Program), National Yunlin University of Science and Technology, Yunlin, Taiwan, ROC

K.-H. HsiaDepartment of Management Information System, Far East University, Tainan, Taiwan, ROC

J.-P. SuDepartment of Electrical Engineering, National Yunlin University of Science and Technology, Yunlin, Taiwan, ROC

This work was presented in part at the 14th International Symposium on Artifi cial Life and Robotics, Oita, Japan, February 5–7, 2009

Artif Life Robotics (2009) 14:95–100 © ISAROB 2009DOI 10.1007/s10015-009-0636-7

Chun-Chieh Wang · Shao-Fan Lien · Kuo-Hsien Hsia Juhng-Perng Su

Image-guided searching for a landmark

by a piezo-driven microrobot with multiple inner sensors,2 and localization and error correction for a mobile robot with an image sensor,3 etc., all deal with positioning prob-lems via machine vision. However, as regards image-guided control, few researchers have investigated fl ying robots. In 2007, Kei Watanabe et al.4 presented an image-based visual servo control for a microhelicopter. A stationary camera is placed on the ground, and this obtains the image features of the helicopter. This visual servo system enables the heli-copter to enter a stationary hover procedure. However, this is still not available for all helicopter aprons.

In this work, we place a stationary camera beneath the helicopter so that it can arrive at the correct place on any apron by image-guided searching. To illustrate the effec-tiveness of the design, the proposed method will be verifi ed through a practical test on an experimental set-up called the ball and plate system. The experimental results validate the superiority and practicality of the image-guided search for landmarks.

2 Image-guided algorithm

To obtain the relative locations of the landmark and the helicopter, an image-guided search algorithm is used to fi nd the center of the landmark and the moving vector. These procedures for image processing are shown in Fig. 1.

2.1 Image preprocessing

The image preprocessing includes binarization and fi ltering with the aim of distinguishing the object from the background. To reduce the amount of data, the three-dimensional RGB-image information is transformed into a one-dimensional colour axis. Unfortunately, the regulation of the threshold value usually produces noise. To improve the image quality, the median and area fi lters will be used to reduce this noise.

Abstract A novel image-guided structure is proposed for searching a helicopter landing apron. The key feature of the structure is the introduction of a stationary camera placed beneath the helicopter so that it can arrive at the correct place on any apron by image-guided searching. The validity of the proposed image-guided structure is verifi ed by means of a practical test on an experimental ball and plate appa-ratus. The experimental results validate the superiority and practicality of the image-guided search for landmarks.

Key words Image-guided searching · Landmark · Ball and plate platform

1 Introduction

Over the past 20 years, machine vision has been successfully used for robotic development and servo-position control. For example, a pneumatic robotic arm controlled by on–off valves for automatic harvesting based on vision localiza-tion,1 an automatic micro-indentation and inspection system

Received and accepted: April 19, 2009

96

2.1.1 Sobel fi lter

To detect the edge of a binarized picture, a Sobel fi lter is the most common tool and is very useful. The following two matrices are used as the fi lter operators:

Gx =−−−

⎡

⎣

⎢⎢⎢

⎤

⎦

⎥⎥⎥

1 0 1

2 0 2

1 0 1

(1)

Gy =− − −

⎡

⎣

⎢⎢⎢

⎤

⎦

⎥⎥⎥

1 2 1

0 0 0

1 2 1 (2)

where Gx and Gy are used for edge detection in the x-direction and the y-direction, respectively. These two 3×3 convolution masks are applied to each pixel. The result of each convolution is treated as a vector representing the edge through the current pixel. If the magnitude of the sum of these two orthogonal vectors is greater than any user-specifi ed threshold, the pixel is marked in black as an edge. Otherwise, the pixel is set to white.

2.1.2 Moving-edge detection

To extract the background automatically, a large enough number of successive structures must be available for pro-cessing. The procedures for moving-edge detection are

shown in Fig. 2. First, three successive structures (It, It+1, It+2) will be taken from CCD, and then the differences D[It+2 − It+1] and D[It+1 − It] can be calculated. The Logical AND operation can be used to evaluate the difference product. Hence, the moving vector of It+1 can be obtained.

2.1.3 Moving-target shifting

As shown in Fig. 3, to overcome image-delay, the moving-target shifting method will be used to get the new landmark M2. By comparing D[It+2 − It+1] with M1, the shift values Xd and Yd can be estimated as

X C C A A

Y B B D Dd

d

= −( ) + −( )= −( ) + −( )

⎧⎨⎩

1 2 1 2

1 2 1 2 (3)

2.2 The central point of a landmark

In this article, 0 and 1 gray-level images are regarded as the background and the landmark, respectively. If the maximum and minimum values of projection are found on the x-axis and y-axis, respectively, then the center point can be obtained (see Fig. 4).

3 Ball and plate apparatus5

To justify the validity of the proposed algorithms experi-mentally, a test device, called a ball and plate apparatus (Fig. 5a) was constructed. This is a dynamic system with two inputs and two outputs. Its system confi guration is shown in Fig. 5b. The system consists of a plate pivoted at its center such that the slope of the plate can be manipulated in two perpendicular directions. A servo system consisting of a

Binary Image Median Filter Area Filter

Moving Edge Detection

Moving Target Shifting

Target Position Estimation

Former Image Input

Present Centerof Target

Image Preprocessing

Moving Target DetectionCenter of Target

Fig. 1. Procedures for image processing

1[ ]t tD I I+ − 2 1[ ]t tD I I+ +−

1Moving Vector of tI +

1tI + 2tI +tI

×

Fig. 2. Moving-edge detection

Fig. 3. Moving-target shifting

97

Fig. 6. Flowchart of image-guided search for the landmark

Fig. 7. Schematic diagram of the structure of the CCD

Fig. 4. Central point of the landmark

a b

Fig. 5. The ball and plate apparatus. a Actual picture. b Schematic diagram

Fig. 8. The relationship between the motion of the swashplate and the fl ight direction of the helicopter

motor controller card and two stepper motors is used for tilting the plate. The ball position and target location are measured by a vision system which is composed mainly of a CCD camera and an FG 201 Structure Grabber PC add-

on card. Afterward, the PWM signal will be transmitted by an MF614 multifunction card. The time period for updating the ball position measurement is set at 40 ms. The basic control task is to control the position of a ball freely rolling on the plate.

4 Experimental design

To show the effectiveness of the design completely, a situ-ation fl owchart has to be planned, as shown in Fig. 6. Once

98

we fi nd the coordinates of the landmark via the image-guided algorithm, the fl ight direction of the helicopter will be changed by tuning the swashplate.

Step 1. As shown in Fig. 7, let A and B be regarded as the landmark and the location of the projection of the heli-copter, respectively. Once A and B are overlapping each other, the helicopter will be steered to arrive at the correct place on the landmark.

Step 2. Calculate the moving vector and the central coordi-nates of the landmark.

Step 3. To further control the servo motors of the swash-plate, the central coordinates and the moving vector will be transferred to the control signals. Figure 8 shows the relationship between the motion of the swashplate and the fl ight direction of the helicopter.

The swashplate can be regarded as the X–Y plane and the ball, freely rolling on the plate, an apron. In addition, the arbitrary set-point can be viewed as the projection of the helicopter. The experimental goal is to control the servo motors of the swashplate such that the ball can arrive at the assigned point as rapidly as possible.

5 Experimental results

The location picture, taken from CCD, is shown in Fig. 9. The range of the X–Y coordinates is between −1 and +1. To simulate the different tracking situations, 5 set-points are dispersed over the plate. The initial point of the ball is set to be (0.79, 0.91) owing to the restrictions of the physical size of the ball and the edge-wall of the plate.

Figures 10–14 show the tracking responses, where the targets are arbitrary but have the same initial point. The corresponding actual responses are shown in Figs. 15–19, respectively, where the point in each structure is the default target.

Fig. 9. Locations of the set points

Fig. 10. Tracking responses from (0.79, 0.91) to (0,0)

Fig. 11. Tracking responses from (0.79, 0.91) to (−0.79, −0.81)

Fig. 12. Tracking responses from (0.79, 0.91) to (0.4, 0.4)

Fig. 13. Tracking responses from (0.79, 0.91) to (0.58, 0.61)

6 Conclusion

In this article, a novel image-guided structure has been proposed for searching a helicopter apron. The key feature

99

Fig. 14. Tracking responses from (0.79, 0.91) to (0.61, −0.73)

a

c

b

d

Fig. 15. Actual responses from (0.79, 0.91) to (0, 0)

a b

c d

Fig. 16. Actual responses from (0.79, 0.91) to (−0.79, −0.81)

a b

c d

Fig. 17. Actual responses from (0.79, 0.91) to (0.58, 0.61)

a b

c d

Fig. 18. Actual responses from (0.79, 0.91) to (0.61, −0.73)

a b

c d

Fig. 19. Actual responses from (0.79, 0.91) to (0.4, 0.4)

of the structure presented is the introduction of a stationary camera placed on the helicopter so that it can arrive at the correct place on any apron by image-guided searching. To verify the effectiveness of the design, the proposed method

100

was verifi ed through practical tests on an experimental setup called the ball and plate system. The experimental results validate the superiority and practicality of the image-guided search for landmarks.

References

1. Carducci G, Foglia M, Gentile A, et al (2004) Pneumatic robotic arm controlled by on–off valves for automatic harvesting based on vision localization. IEEE International Conference on Industrial Technology, Hammamet, Tunisia, pp 1017–1022

2. Fuchiwaki O, Tobe N, Aoyama H, et al (2005) Automatic micro-indentation and inspection system by piezo-driven micro robot with multiple inner sensors. Proceedings of the IEEE International Con-ference on Mechatronics and Automation, Niagara Falls, Canada, pp 83–88

3. Nagai I, Tanaka Y (2006) Localization and error correction for mobile robot with an image sensor. SICE–ICASE International Joint Conference, Busan, Korea, pp 5373–5377

4. Watanabe K, Yoshihata Y, Iwatani Y, et al (2007) Image-based visual PID control of a micro helicopter using a stationary camera. SICE Annual Conference, Kagawa, Japan, pp 3001–3006

5. Horácek P (1996) Ball and plate apparatus: user’s technical manual, 3rd revision, Humusoft, Czech Republic