Abstract— Since the intricate and dangerous working environment of steam generator, it’s risk for workers to complete fastening bolts of blocking plate. The paper introduces a fastening bolt method based on image recognition. The mechanical structure and control system were proposed gradually to be a carrier for the method. Then the image recognition adapted the Canny algorithm to realize the image binaryzation and draw the image counter. After the polygonal curve algorithm was utilized to get the hexagon, the center coordination values of the hexagon was obtained by the hexagonal spatial moment. In addition, the fastening bolt method was displayed, which adapted the torque and time control method. Finally the precision and bolt experiment validated the procedure of the image recognition and fastening bolt.

I. INTRODUCTION S the development of nuclear power and automation technology, the robot technology has been widely used

to protect the staff working with the dangerous environment in the special industry, which has radioactive material [1-2]. After the event in Fukushima, Japan, nuclear safety is concerned by more governments and organizations. During the maintenance period of the nuclear power plant, the inlet and outlet of the steam generator need to block in order to protect the maintenance staffs and not damage other steam generators. The blocking plates and capsules are used to seal water inlet and outlet of steam generator.

In traditional way, the aluminum blocking plate which included three components was delivered into the steam generator one by one and the worker needed to place the blocking plates and capsules on the water inlet of steam generator as shown in Fig 1.

This work was supported by the National High Technology Research

and Development program of CHINA (863 Project) under Grant No. 2012AA041606, Grant No. 2011AA040201 and Beijing National Science Foundation No. 7132132.

Xiangguang Duan is with the Intelligent Robotics Institute, Key Laboratory of Biomimetic Robots and Systems, Ministry of Education School of Mechatronical Engineering, Beijing Institute of Technology (e-mail: duanstar@bit.edu.cn).

Yonggui Wang is with the Intelligent Robotics Institute, Key Laboratory of Biomimetic Robots and Systems, Ministry of Education School of Mechatronical Engineering, Beijing Institute of Technology. (corresponding author to provide phone: +86-18801156488; fax: 010-68915420; e-mail: wangyg1025@163.com)..

Meng Li is with the Intelligent Robotics Institute, Key Laboratory of Biomimetic Robots and Systems, Ministry of Education School of Mechatronical Engineering, Beijing Institute of Technology (e-mail: limenglll@163.com).

Xiangzhan Kong is is with the Intelligent Robotics Institute, Key Laboratory of Biomimetic Robots and Systems, Ministry of Education School of Mechatronical Engineering, Beijing Institute of Technology (e-mail: ailiyakong@126.com)

Fig.1. Blocking plate and fastening the bolt

Then the staff used a long pole which installed the camera and fastening device to fasten bolts, this process needs much time and the staff is under the radiation environment for a long time [3-5]. The bolt of blocking plate is designed technically and hexagon socket shape in Fig. 2.

Fig.2. The bolt of blocking plate

To the environment of steam generator, the paper promotes a fastening bolt method. The blocking plate manipulation robot system is presented in the part 2, including the mechanical structure and control system. Then the image recognition is introduced carefully in the part 3. Next part illustrates the method of fastening bolt. The experiment is conducted to validate the image recognition and the procedure of fastening bolt.

II. SYSTEM OVERVIEW

A. System Components The blocking plate manipulation robot system includes

three components, the manipulation robot, control system and image acquisition and processing system.

The mechanical structure of manipulation robot is designed for fastening bolts, as shown in Fig.3. It has three degrees of freedoms (DOFs) and mainly holds two functions, in order to move precisely to necessary position and fasten a number of bolts to a certain torque. The blocking plate manipulation robot chooses a typical cylindrical coordinate system, due to compact structure, flexible and quick motion and high location precision. By the motor 1 and motor 2 corresponding to the joint 1 and joint 2, the blocking plate manipulation robot could move to

A Fastening Bolt Method Based on Image Recognition Xingguang Duan, Yonggui Wang, Meng Li, Xiangzhan Kong, Syed Amjad Aliy

A

978-1-4799-2744-9/13/$31.00 ©2013 IEEE

Proceeding of the IEEEInternational Conference on Robotics and Biomimetics (ROBIO)

Shenzhen, China, December 2013

2709

the certain position, the motor 3 controls the fastening device in the vertical direction. as shown in Fig 3.

Fig. 3. Mechanical structure of manipulation robot

Considering the complex working condition in nuclear power plant, a safe and reliable control system has to be provided. Therefore, distributed control system based on CAN-bus is selected. Because of reliable communication, real-time, flexibility, strong robustness, long transmission distance etc., CAN-bus is widely applied in all kinds of industry control field. The control system is shown in Fig.4 [6-7].

Fig. 4. Control system of manipulation robot

The image acquisition and processing system play an important role in entire system. As the blocking plate is consist of three parts, each bolt position of blocking plate is uncertain. After acquiring the image from the camera which installed in the robot arm in order to keep the appropriate distance from the camera to the surface of blocking plate, the image acquisition and processing system could acquire the bolt center coordinates. Then convert and transfer the center coordinates to the blocking plate manipulation robot so that the manipulation robot can complete fastening the bolts.

B. Working Procedure Combining the three subsystems, the working procedure

for blocking plate manipulation robot system is illustrated as follows:

--First, the staff delivers the blocking plate and capsules into the steam generator, mounting the manipulation robot by using fast locking mechanism and pre-tightened bolts on the inlet and outlet of the steam generator.

--Second, start the automatic fastening bolts operation. The camera driven by motor 1 and motor 2 in circumferential and radial direction moves to a place above

the bolt and capture images. The bolt center coordinate values of the images could be obtained by the image acquisition and processing system.

--Third, the control system processes the center coordinate values and drives the fastening device just above the bolt. The fastening device and motor 3 complete fastening or loosening bolts.

In this procedure, accessing to the bolt center coordinates is a crucial step through image acquisition and processing system. After obtaining these center coordinate values, control precisely the fastening device to the corresponding position and complete fastening the bolt by the torque and time control method.

III. IMAGE RECOGNITION

The image acquisition and processing system aims to capture the image and extracts the bolt center coordinate values. As the structure of bolts is special, identifying hexagon is more effective than the circle. Because the blocking plate could have a number of circular structure and it is prone to recognize other center coordinate values rather than the bolts. The camera is special designed for radiation protection due to complex working environment.

During acquiring real-time bolts image using the camera, the light, uneven exposure and other factors brought to the image noise. Without noise removal, image segmentation, image enhancement, image analysis and judgment would be affected. So the corresponding approach is applied in order to realize the image recognition and acquire the center coordinate value [8-9].

A. Image Recognition Process The procedure of image recognition is presented as

follows: 1) Capture the gray image through CCD camera due to the

gray image is easy to be processed in Fig. 5.A. 2) Enlarge image and set the bolt location as the region of

interest so as to image recognition in Fig. 5.B. 3) Through Gaussian convolution filter, take sample

downwardly and upwardly in order to filter out image noise in Fig. 5.C.

4) The Canny algorithm realizes the image binaryzation, for the purpose of finding the image edges and displaying them in the image in Fig. 5.D.

5) Expanse the image contours obtained by the Canny algorithm and calculate contour number in binary image.

6) Use the polygonal curve to approach image contour and judge the number of edge of the polygonal curve in Fig. 5.E.

7) Through the analysis of the hexagonal spatial moment, calculate the center values of the hexagonal.

2710

A B

C D

E

Fig. 5. The process of image recognition Image recognition aims to acquire the center coordinate

values of bolts. Through the above procedure, the center values could be got. In this procedure, the image binaryzation and identification center become key steps.

B. Image Recognition The image binaryzation determines the later steps, like

identifying the hexagonal and center point. Different image recognition algorithms have been experimented for edge detection in Fig. 6.

A. experiment sample B. Sobel Operator

C. Laplace Operator D. Canny Operator

Fig. 6. The different image recognition algorithms Through the analysis of experiment data, Canny operator

could obtain optimum efficiency of edge detection. High signal noise ratio, better positioning performance and only a

single response to a single edge are the advantages of Canny algorithm. The basic principle of Canny algorithm is to calculate the gradient magnitude and direction of the Gaussian filter impulse response [10-11].

Supposing gradient vectors is ( , )

( , )( , )

x

y

f i jf x y

f i j⎡ ⎤

∇ = ⎢ ⎥⎣ ⎦

(1)

boundary strength is 2 2

, ( , ) ( , )i j x yf i j f i j∇ = + (2)

and the gradient direction angle is

( , )

arctan( , )

y

x

f i jf i j

θ ⎛ ⎞= ⎜ ⎟

⎝ ⎠ (3)

However, geometrical figure is incomplete, which will make it difficult to find the center of bolt. Besides that, the light direction and illumination have a great influence on image recognition. Non-maximum suppression method and two threshold judgment are applied in edge detection. The Fig.7 shows image binaryzation under the different thresholds which include two thresholds, one threshold controls edge connector and the other one controls the initial segmentation of strong edge. In the following Fig, the one threshold is three times as much as another.

A. threshold=(40,120) B. threshold=(60,180)

C. threshold=(80,240) D. threshold=(100,300) Fig. 7. Image binaryzation under the different thresholds

After the adjustment of threshold, a satisfy result can be obtained, as shown in Fig.8.

Fig. 8. The result of image recognition

Through the analysis of the hexagonal spatial moment, the

2711

center values of the hexagonal could be calculated by

1

1

( , )

( , )

n

i i ii

n

i ii

x p x yx

p x y

=

=

=∑

∑1

1

( , )

( , )

n

i i ii

n

i ii

y p x yy

p x y

=

=

=∑

∑ （4）

IV. FASTENING BOLT METHOD The fastening bolt method includes two procedures, one

procedure controls the fastening device to arrive just on above the bolt, the other completes fastening or loosening bolts.

A. Data Conversion After getting the center coordinate values of the bolt,

fastening device needed to be driven to the appropriate position. So the conversion relationship from the center coordination values to certain angles and distances rotated by motor 1 and motor 2. Because of the error of matrix transformation, an easy method was adapted to complete the matrix transformation.

When the camera moved to the place just above the bolt, captured the image and recognized the center coordinate values. In our manipulate robot, the center coordinate values is (500, 540) in the image using the image acquisition and processing system. Then the corresponding relationship between the image pixels and angle of motor 1 and motor 2 could be verified. Controled the motor 1 rotate 0.1o each time, and recoded the center coordinate values of bolts, the same to motor 2. The results show in the following Fig. 9.

Pixels——Angle of motor 1

Pixels——Angle of motor 2

Fig. 9. The relationship between pixel and motor

The above figure says that the relationship between the angles of two joints and the pixels is linear. The 0.1 degree of joint 1 corresponds to two pixels, the 0.5 millimeter of joint 2 to 3 pixels.

In addition, if the result of image recognition is (500, 540), namely the camera is located just above the bolt, the angle of joint 1 from this position to fastening bolt location should be calculated [12-13]. Seen from the mechanical dimensions,

R=508mm, b=85mm, the angle bθ could be calculated.

Fig.10. The mechanical dimensions

So the angle and the distance of joint 1 and joint 2 from the camera position to the fastening bolt locations could be analyzed. If the center coordination values from the acquisition and processing system is (x, y), the angle and distance could be obtained as follows:

=(x-500) 0.1/2+( 540) *0.5 / 3

b

l yθ θ×

= − (5)

B. Fastening Bolts The fastening bolts use torque and time control method

and the procedure of fastening bolt is completed by the joint 3 and fastening device. When the fastening device moves to the bolt above, the motor 3 drives the fastening device to go down. In order to accurate operation, the end of the fastening device is designed according to the requirement and the bolt shape as shown in Fig. 11.

Fig. 11. The end of fastening device and bolt

Even though the end of fastening device could not drop into the bolt hole once or drop onto the surface of the bolt, rotating fastening device could guarantee the end of fastening device to drop into the bolt hole. The gap between the bolt hole and the end of fastening device is designed reasonably to complete to fasten blot and reduce errors.

The procedure of fastening bolt adapts the torque and time control method in Fig.12. When the end of fastening bolt goes down to the surface of bolt, the entire process would enter into the bolt cap process in which the end of fastening bolt drip into the bolt hole. In the first stage, the bolt is loose. So the end of fastening bolt is in a high speed revolution and the smaller torque. In the second stage, the rotate speed of

2712

the end of fastening bolt decreases gradually and the torque increases until the limiting value. If it has the other problems, the time controls to stop the fastening bolt.

A. The torque variation in the procedure

B. The rotational speed variation in the procedure

Fig. 12. The change of torque and speed in the procedure of fastening bolt After the bolt is tightened, the fastening device needs to

pull up from the bolt hole. Because the big torque was existed, it is difficult to pull up directly from the bolt hole. So the fastening bolt needs to reverse a little so as to eliminates the torque.

Compared with the process of fastening bolts, the process of loosening bolts is similar. But the process of the loosening bolts adapts time control since it is difficult to control the torque.

V. EXPERIMENT

A. Precision Experiment The precision experiment aims to validate the image

acquisition and processing system and affirm the scope of taking image. Because it is difficult for the camera to stop just above the bolt in the image recognition procedure, we need to ensure the scope of taking image so that the fastening device completes the tightening and loosening bolt.

The procedure of precision experiment is presented as follows:

1）Move the camera to just above of the bolt and recode the center coordination values of the bolt, such as (500, 540). So when the joint 1 moves bθ , the fastening device could be rotate to the just above of the bolt.

2）The joint 1 rotates 0.2 degree each time along the clockwise direction and recodes the center coordination values of bolt by the image recognition.

3) Along the counterclockwise direction, the joint 1 rotates 0.2 degree each time and recodes the values, as shown in Fig. 13.

A. The different position of visual identification

B. The relationship between the pixel and joint 1

Fig. 13. The result of precision experiment The result shows in the Fig.13 that the relationship

between the camera location and center coordination values is linear in certain scope. If the location of the above bolt is the zero point, the scope of taking image is (-2.70, 5.69).

B. Bolt Experiment The experiment aims to verify the fastening bolt

according to the above theory and method. The fastening bolt adapts the torque and time control method and the loosening bolt adapts the time control method. The experiment process of fastening bolt is illustrated as follows: 1) According to the above experiment conclusion, control the robot to the appropriate location in Fig.14.a.

2) Acquirement the image and do the image recognition to obtain the center coordination values of the bolt hexagon.

3) Control of the fastening device to move just above bolt. Then joint 3 is driven the fastening device to descend until the surface of the bolt in Fig.14.b and c.

4) Tighten the bolt until the torque reaches to the limiting value in Fig. 14.d.

5) Reverse the fastening device in order to eliminate the torque in Fig. 14.e.

6) Go up the fastening device to the initial position. Then prepare the next bolt in Fig. 14.f.

2713

a b

c d

e f

Fig. 14. The process of fastening bolt The process of loosening bolts is similar to the fastening

bolt. The experiment validates the image recognition and fastening bolt method.

VI. CONCLUSION After a lot of experiments were conducted, the image

acquirement and processing system and fastening bolt method has been proved to be effective. But the image recognition is not enough stable because of the light intensity. So it still has a lot of work to do to make the system better and higher reliability.

ACKNOWLEDGMENT This work was supported by the National High

Technology Research and Development program of CHINA (863 Project) under Grant No. 2012AA041606, Grant No. 2011AA040201 and Beijing National Science Foundation No. 7132132.

REFERENCES [1] Hogan, M.K. “Anti-radiation clothing,” Technical Textiles

International.20, 1995.

[2] Johnstone, G.A, Fryer, J, Smith, J.M., Barker, R.L. “Improve protective clothing and reduce radwaste,” 1995 Winter Meeting of American Nuclear Society. 96 vol.73, 1995.

[3] Hayasaka, Y. “R&D of advanced robot for nuclear power plant facilities,” Robot. May 1988, 0387-1940.

[4] Maki, H. “The concept of the advanced robot for nuclear power plants,” Proceedings of '85 International Conference on Advanced Robotics, 1985, pp: 497-505.

[5] Rohrabacher, A. Carlton, R. Gelhaus, F. “Considerations in the development and implementation of a maintenance robot for nuclear power facilities,” 13th Biennial Conference on Reactor Operating Experience International Meeting on Nuclear Power Plant Operation, 1987, pp. 186.

[6] Thomas Herpel, Kai-Steffen Hielscher, Ulrich Klehmet, and Reinhard German, “Stochastic and deterministic performance evaluation of automotive CAN communication,” Computer networks, 2009, 53(8).

[7] Xinhua Yu. Shaozhong Cao, “The Communication of Can Bus Used in Synchronization Control of Multi-Motor Based on DSP,” 2011 IEEE International Conference on Cloud Computing and Intelligence Systems, 2011.

[8] Chengxian Zhou, Wei Fu. “A Study of Robot Control Technology Based on Stereo Vision,” 2011 International Conference on Electronics and Optoelectronics, January 2011.

[9] Kroger, T, Padial, J. 2012 IEEE International Conference on Robotics and Automation, 2012 4862-9.

[10] http://en.wikipedia.org/wiki/Canny_operator [11] Karen A. Panetta, Sos S. Agaian, Shahan C. Nercessian, Ali A.

Almunstashri. “Shape-dependent canny edge detector,” Optical Engineering, 2011, 50(8).

[12] Qiang Zhan, Shouren Huang, Jia Wu. “Automatic navigation for a mobile robot with monocular vision,” 2008 IEEE Conference on Robotics, Automation and Mechatronics, pp. 1005-10, 2008.

[13] Roger Y. Tsai, Reimar K. Lenz. “A New Technique for Fully Autonomous and Efficient 3D Robotics Hand/Eye Calibration,” IEEE Transactions on Robotics and Automation, Vol.5, NO.3, June 1989.

2714