EISWIER Automation in Construction 5 (1997) 393406
Programming construction robots using virtual reality techniques
R. Navon a92, A. Retik b33 a Technion. Department of Civil Engineering, National Building Research Institute, Technion City, 32000 Haifa, Israel
b Department of Civil Engineering, University of Stratchylde, Glasgow G4 ONG, UK
The paper describes a new approach to programming construction robots, using virtual reality (VR) techniques. The new approach is needed because both traditional and new methods of programming industrial robots, described in the paper, have specific drawbacks, which become crucial in the construction arena. This is because of the ever-changing environment of construction and its nature, a prototype or one-of-a-kind, industry. As a result, construction robots need much more programming than their industrial counterparts, which is labor intensive using known methods and is not compensated by mass production. The VR approach is demonstrated with the Multi-Purpose Interior Finishing Robot (MPIR) for a masonry task, accompanied by a detailed description of the VR-based programming model and approach.
Keywords: Automation; 13uilding; Construction; Programming; Robot; Virtual reality
The objective of this paper is to suggest a new method for robot programming with the aid of vir- tual reality (VR) technology, which allows the fol- lowing to be achieved: 1. Learn geometry and spatial arrangements of a
Discussion is open until August 1997 (please submit your discussion paper to the Editor on Construction Technologies and Engineering, M.J. Skibniewski).
Tel: + 972-4-8292600; Fax: + 972-4-8324534; e-mail: cvronie@ tx.technion.ac.il; Internet: http://techunix.technion.ac.il/ - cvronie.
3 Tel.: + +44-141-552-4400; fax: + +44141-552-0112; e- m ail: a.retik@ stratch.ac.uk; Internet: http//www.strath.ac.uk,/Departments/Civeng/conman/cm.html.
Explore the best way for a robot to perform a task using human reasoning. Check and verify structural and organizational safety issues of a robot, including location of materials. Train the robot for its designated tasks, in the same way as workers would have been trained to perform new tasks. Robots are considered as flexible manufacturing , .
machines because they can easily be (re)programmed to perform different tasks. Yet, often the complexity of the programming process limits more widespread use of robotic technology [ 11. Researchers and practi- tioners alike are constantly looking for new methods of programming robots to save programming time and costs. This section reviews some common and novel programming techniques used or developed for
0926-5805/97/$17.00 Copyright 0 1997 Elsevier Science B.V. All rights reserved.
394 R. Nauon, A. Retik/Automation in Construction 5 (1997) 393-406
manufacturing robotics. These techniques are exam- ined for their potential use for construction robots, which operate in an ever-changing environment, pro- ducing one-of-a-kind or at best small-batch products.
1. I. Native programming languages
Currently, most commercial robots are pro- grammed in textual robot programming languages (RPLs) similar to BASIC, PASCAL, etc. Popular examples include VAL, RAIL and AML . These languages include positioning instructions, process parameters and other instructions . RPLs contain primitives to express manipulator motion in world coordinates as a location (x, y, z> and orientation (roll, pitch, yaw) 141. Most languages also contain higher level control commands.
A program in RPL can be written when the robot is in its designated work location (on-line program- ming). The advantage of doing so is that program- ming is relatively easy, but during programming the robot and the dependent equipment are idle, thus increasing programming costs significantly. Off-line programming is supposed to solve this problem but it is very difficult to carry out without visual aids. Lees and Leifer  state that describing a robots motion through space is not the strength of RPLs (in both the on- and off-line modes). It is difficult for a human to envisage the position and orientation of a robot by merely looking at a string of numbers and it is all but impossible to envisage the robots spatial relationship with the rest of the objects in its workspace. Since the basic components of a robot programming language are so counter-intuitive, a complete manipulation program is even more so.
This technique is most unsuitable for program- ming of construction robots due to the complex nature of the construction work. In order to program a construction robot using this technique, the pro- grammer would have to visualize in his/her mind the entire project, including the location of all robot workstations, the locations of the raw materials, etc. In addition, he/she would have to visualize the robots reaction to various commands and its interac- tion with the changing environment. No less impor- tant is the fact that programming this way is labor intensive - because of the one-of-the-kind nature of the construction work - and consequently expensive.
1.2. Programming with the aid of graphic simulators
Off-line programming with simulators enables the programmer to write a program without recourse to the robot itself. It is done in a computer-graphic environment, enabling the programmer to preview, debug and verify the program before the robot is installed, or even bought. Simulators enable the de- velopment of typical tasks, examine different altema- tives to perform them, test different control strate- gies, explore raw material supply alternatives, etc. Once a program is written, the programmer can visualize the robots task being performed, check the planned paths suitability for the performance of specific tasks, check that the robot does not collide with other components in the work cell, etc. When the programmer is satisfied with the program, it is translated into the robots native language and down- loaded to it.
One of the most important characteristics of graphic simulators is that they are able to measure cycle times, which is an important parameter influ- enced by the selected paths, by the control algo- rithms, by the material supply method, etc. Thus higher quality programs can be developed with the aid of graphic simulators, taking into account robot productivity as well. However, simulators still re- quire the users to work with text-based languages and consequently only solve part of the problems associated with native robot programming languages
La. Graphic simulators also aid in selecting the best
robot for a given task. Robot selection is based on a number of parameters, such as cycle time, work envelope, path limitations, joint singularities, re- quired floor area, etc. Most of those parameters cannot be checked with a reasonable level of accu- racy and certainty without a graphic simulation sys- tem.
Construction robot programming using graphic simulators offers a meaningful improvement to RPL techniques by enabling the programmer to see both the environment and the robot during programming. While this solves the visualization problem, it still does not decrease the amount of work involved in programming the robot because the commands re- main very basic (positioning instructions, manipula- tor motion instructions, end effector operations, etc.).
R. Nauon, A. Retik/Automation in Construction 5 (1997) 393-406 395
Consequently, this technique is not suitable for con- struction robot programming.
1.3. Additional programming methods
prehensive solution to the complex problem of pro- gramming construction robots, it is merely suggest- ing an additional idea, which can be used either on its own or with other methods, some of which were reviewed above.
Programming by example, or by human demon- stration, is an intuitive method. The programmer demonstrates how the task is performed using a human/robot teaching device that measures the hu- mans forces and positions [l]. The data gathered from the human is used to generate the robots program.
2. VR iu building construction
The goal in automatic robot programming is to get the robot to perform a task by telling it what needs to be done, rather than by explicitly program- ming it [51. An approach to automatic robot program- ming was suggested by Rogalinski .
Visual robot programming is a term applied to systems which allow programming in a two- or three-dimensional manner . A visual demonstra- tion of the task is performed not in the real world but within the digitized images that the robots vision system sees. The programmer indicates directly in the digitized images (using a mouse) robot actions, constituent parts, grip points, relative orientations, approach routes, etc. Complex task-planning by the robot is avoided by keeping the human in the control loop, thus the human problem solving skills support the robots understanding and execution of the task.
VR is an advanced computer graphics technology dealing with visualization . As a computer graph- ics application it can be used for a wide variety of uses, e.g. [9-121. The use of object-oriented tech- niques for creating virtual environments was the key for a breakthrough in the credibility and applicability of VR technology [ 131. Various definitions exist: one which is both descriptive and short is by Pimentel and Teixeira [ 141 who define VR as the place where humans and computers make a contract. Similarly, Larijani 1131 defines VR as the conver- gence of computer simulation and visualization that attempts to eliminate separation between a user and a machine. From their viewpo