Virtual Reality based Paint Spray Training System

Ungyeon Yang Gun A. Lee Seonhyung Shin Sunyu Hwang Wookho Son

Electronics and Telecommunication Research Institute

ABSTRACT We present a virtual reality (VR) system that simulates the

situation of ship block spray painting. The system was developed under demands from ship building industry for training purpose. We designed an immersive stereo display platform, a realistic spray painting rendering technique and intuitive user interface to match with the real working environment. The system is currently under user-test in the ship building company with receiving positive responses.

Keywords: VR system, training, ship painting, immersive

display, intuitive interface Index Terms: I.3.7 [COMPUTER GRAPHICS]: Three-

Dimensional Graphics and Realism – Virtual Reality, H.5.2 [INFORMATION INTERFACES AND PRESENTATION]: User Interfaces - User-centered design

1 INTRODUCTION The virtual reality technology can provide virtual experiences

that are impossible in (or even better than) the real situation, in terms of safety and cost. Hence, it is widely used for training purposes in medical, military and industrial field, to overcome limits of real training environment.

In this paper, we present a virtual training system for practicing paint spraying which is one of the main processes in ship construction. Painting process affects the whole ship construction schedule and quality; therefore, skilled experts are necessary. However, due to the poor working environment there are a lot of changes of employment, hence an efficient training course is highly required to secure highly skilled experts. The state of the art of paint spray training course is practicing to paint a real ship block model with the real paint. But, there are certain problems with using real materials for practicing:

Waste of materials (a lot of expensive paints and waters are consumed)

Air pollution with poisonous paints (hard to continue practicing for a long period)

Hard to give instructions while practicing (hard to communicate with each other while wearing masks)

Hard to practice continuously (need to dry and clean up the painted structure after practicing)

Limited space for practicing (limited number of trainees are allowed to practice at a time)

We are mainly focusing to overcome these problems in the current training course with real materials, by applying virtual reality technologies.

2 RELATED WORKS Since the main purpose of a training system is to help users (or

trainees) to acquire skills used in the real work place, it is important to make harmonization between visualizing realistic

scenes of the real working space and providing the same working interfaces used in the real situation [1] and make them fit into the teaching/training methods.

There are couple of previous works that simulated spray painting work, such as [2], however, they were limited to painting simple planar structures and were missing supports for measuring paint film thickness, which is an important measure of skill developed by training. There are also some works in computer graphics to visualize paintings in photorealistic fashion [3], however, these are not suitable with our virtual training system in terms of their non-real-time processing performance and using different painting tools (e.g., a paint brush) for artistic expressions.

3 VR-BASED TRAINING SYSTEM

Figure 1. Overall System Architecture

Fig.1 presents the overall structure of the Virtual Paint Spray

Training Simulator. This system is designed to support various VR interfaces and displays to actively response to the needs in different training scenarios. The detail specification and design of the system were decided according to the demands from professional paint spray instructors in the real work field.

In order to provide the same experience with the real working environment, we considered the following guidelines for deciding

161 Gajeong-dong, Yuseong-gu, Daejeon, 305-700, KOREA {uyyang; endovert; estee; prosyh; whson}@etri.re.kr

289

IEEE Virtual Reality Conference 2007March 10 - 14, Charlotte, North Carolina, USA1-4244-0906-3/07/$20.00 ©2007 IEEE

the display platform. First, the display system must meet the space and cost requirements. A number of display systems must be installed for multitudes of trainees, and therefore, they should require small spaces and costs. Second, for the trainee, stereo visualization and head tracking is necessary for correct understanding of the target object which has three dimensional structures. Under this notion, we selected two types of display configuration: the wall display (W:150cm x H:200cm) for using in the lecture room, and the FMD (face mounted display) for the practice room.

In addition, we’ve also developed another version with LMD1 [4,5] which takes advantages between two display systems to supplement each other. To support various display configurations and tweaking visualization parameters according the users physical characteristics (e.g., inter-pupilary distance), we also developed a display simulation tool which can visualize the display configuration and simulate the resulting images on the display surface.

To cover the user’s view and to make the screen surface and the surface of the virtual structure identical2, the users were asked to stand in front of the screen with about 1 meters distance. For providing stereo image, we used circular polarizing filters that are relatively cheaper than active shutters, while keeping users able to turn their heads freely.

FMD provides fully immersive views to the user with relatively small sized device. While the wall display has limitation with the users viewing direction, FMD provides omni-directional view to the user by tracking the user’s head movement. Since the field of view of the working mask is about 110°, we are trying to develop a FMD with HD-level image resolution and wide field of view which provides near to 130°, similar to normal eye glasses. The main difficulty of enlarging the field of view of the new FMD is to design and to manufacture a micro-optical system that can be produced with small enough errors.

Real-time visualization of paint spray is one of the important features of our virtual training system. In addition to visualizing the paint spray pattern correctly, it is important to simulate the thickness of the paint accumulated on the surface of the target structure. The thickness is the main measurement for testing the trainee’s skill level. The most precise method will be using CFD/FEM(Computational Fluid Dynamics/Finite Element Method) for physical simulation of paint spray particles. However, this approach is not suitable for real-time processing due to the limitation in performance of PC platform. Instead, as a heuristic approach, we measured a set of real paint spray patterns with different angles and distances between the spray gun and the surface, and constructed a database with the measurement data. According to these measurements, we developed an approximated

1 Layered Multiple Display: An integration of stereo images presented on multiple heterogeneous display devices 2 Users tended to paint on physical surfaces

mathematical model for calculating the thickness of the paint spray pattern in real-time. We’ve constructed a database of 2 types of spray gun tip (which mainly affects the shape of the pattern) and 2 types of paints. According to this model, the texture maps of the painted structures are updated interactively. A simple ray casting method is used for detecting the collision between the spray volume and the target structure surface in real-time. Due to the system performance and the quality of visualization, currently we are using textures with 192x192 resolutions for calculating the thickness of the spray pattern painted on a 1m2 sized target surface.

We adopted the same spray gun used in the work field as the main user interface and attached electronic buttons to sense triggering. We also connected an air compressor to the spray gun to provide force and sound feedback of spraying. The air pressure was adjusted to provide a force similar to the real paint spray.

The training system also provides a digitized lecture material (see lower left of Fig.1) which shows an animation of a motion captured standard work process. The user can choose different scenarios and different viewpoints to understand the motion before practicing.

4 CONCLUSION AND FUTURE WORK In this paper, we reported the current development status of our

virtual paint spray training simulator. Currently, we are planning to make following improvements to our system.

Although we’ve received a positive response from paint spray instructor from ship building industry, we surely need to verify the usability of our system in a formal user test by comparing learning effects or our system and the current training course using real materials.

While currently we are using a virtual model as a paint target structure, we are also developing an augmented reality version of our system which can paint virtually on a physical target structure. Providing olfactory feedbacks is another possible improvement to improve presence and make users to feel more realistic. Besides simulating the virtual environment as real as possible, providing additional information, such as virtual guides for user’s motion, is another topic to improve our training system [6]. This might help trainees to correct their faults and try in a more standard way.

ACKNOWLEDGEMENT This paper is a result of the ‘Intelligent Manufacturing System’

project: a joint research project between Korean Ministry of Information and Communication and Fraunhofer IGD, Germany.

REFERERNCE [1] www.osc.edu/research/video_library/ford.shtml [2] MY Lim, and R Aylett. MY virtual graffiti system, Multimedia and

Expo, 2004. ICME'04. 2004 [3] T Van Laerhoven, and F Van Reeth. Real-time simulation of watery

paint, Proceedings of CASA’05, 2005 [4] Gun A. Lee, Ungyeon Yang, and Wookho Son. Layered Multiple

Displays for Immersive and Interactive Digital Contents. Int. Conf. of Entertainment Computing, UK, 2006

[5] Ungyeon Yang, Dongsik Jo, Wookho Son, and Hyunbin Kim. Visual Interface for Presenting Multiple Mixed, Stereo Image. US Patents, applied no. 11/223066. 2005

[6] Ungyeon Yang and Gerard Jounghyun Kim. "Just Follow Me": An Immersive VR-based Motion Training System.Presence: Teleoperators and Virtual Environments, MIT Press, 11(3), USA, 2002

Figure.2 Mixed Reality Face Mounted DisplayLeft: Anti-gas mask for ship painting Mid.: Intermediate version (FOV65°, 1280x1024) Right: Final version in development (FOV128°

by micro-lens projection, 1920 x1080)

290