VR-based Safety Evaluation of AutomaticallyControlled Machine Tools

Rainer Herpers∗†‡, David Scherfgen∗, Moritz Vieth∗, Timur Saitov∗,Sandra Felsner∗, Thomas Hofhammer∗, Michael Schaefer§ and Michael Huelke§

∗Institute of Visual Computing, Bonn-Rhein-Sieg University of Applied Sciences, Sankt Augustin, Germany†University of New Brunswick, Fredericton, Canada

‡York University, Toronto, Canada§IFA – Institute for Occupational Health and Safety of the German Social Accident Insurance (DGUV),

Sankt Augustin, Germany

Abstract—Although extensive safety measures and safe work-ing procedures have been applied to improve and secure metalworking machines, they still put their operators at risk. Theserisks might often result from manipulation errors, in particularif safety measures are ignored. In this contribution, a safetyevaluation strategy has been developed that applies VR andmixed reality technologies to investigate the usability of workingmachines. An automatically controlled machine tool was sim-ulated and connected to a real input panel, usually employedin industrial settings. However, Human-Machine Interfaces aresometimes built in a way that does not prevent the operatorfrom cognitive misinterpretations which in turn might result inmistakes. To take that into account, a control program for a lathewas altered by hiding a typical programming mistake in the linesof code. Subjects were given the task to evaluate the program insingle step mode and to report abnormalities while running thesimulated lathe, comparable to new control program checks atreal machines. An evaluation of the study demonstrated that evenexperienced metal workers accepted the simulation and reactedas if the given task was real. Behavioural data of consideredsubjects showed comparable profiles and most subjects rated theVR-based approach as a reasonable means for investigating worksafety problems.

I. INTRODUCTION

The area of work safety suffers from a methodologicalproblem. It is challenging to improve work safety and reducethe risks for operators and workers, because when developerscreate a new safety measure it is difficult, or even impossible,to evaluate the new measure in real procedures without puttingtest subjects at risk. So, because of these ethical reasons in-depth evaluation of design concepts and working procedurescannot be undertaken although the desire to do so is very high.Moreover, it has been shown that the application of machinesand machine tools is becoming more and more complex [1]and the Human-Machine-Interaction needs to be investigatedin more depth to identify physical risks and hazards duringcomplicated work processes.

Therefore, one objective of the underlying research is toprovide new evaluation options to the work safety area byapplying VR and mixed reality technologies so that classicalusability investigations can be carried out [2]. First trialapplications have been developed in work safety research labsin France, Finland and Poland [3], [4], [5].

In the study described in this contribution, VR-based tech-nologies are investigated to determine if they allow directed

usability evaluations at working machines to be conducted,avoiding any real threat to subjects. Therefore, an interactiveVR-based simulation environment was developed in which theusability of technical safety means can be manipulated andthen systematically investigated and optimized. An automat-ically controlled machine tool is simulated and connected toa real input panel (see Fig. 2), usually applied in industrialsettings. However, HMIs are sometimes built in a way thatdoes not prevent the operator from cognitive misinterpreta-tions which in turn might result in mistakes. Although thereare initial approaches to consider usability aspects duringthe design and construction of machines [6], a systematicalinvestigation of the usability of work processes at machinesmight be beneficial in order to avoid safety measures beingignored to accelerate work processes [7]. In the presentedapplication a control program for a lathe was developed and aclassical programming error was hidden in the lines of code.The task given to subjects was to evaluate the program insingle step mode and to report anormalities while running thesimulated lathe, comparable to new control program checks atreal machines.

In the next sections the applied methodology and theevaluation strategy are introduced. In section IV the resultsare presented, followed by conclusions and discussions.

II. MATERIALS AND METHODS

The material and methods used include the realisation ofa 3D simulation of a lathe within an immersive visualizationenvironment [8], [9] as well as dynamic sound and backgroundnoise generation. An industrial machine display and controlpanel were reconstructed and attached to the simulation tocontrol the virtual lathe using an interface that closely matchesthe one in the real world.

A. Simulation of the Lathe

All the physical and dynamic components of a lathe weremodeled in 3D and appropriate textures collected from realmachines and preprocessed so that they fit the models (seeFigure 1 and Figure 5). The visual simulation of the turningmachine was based on the graphics engine Ogre3D [10] andwas written in C++. High level scenarios, like motion of themachine tools according to particular inputs of the controlsystem, changes in a work piece’s 3D shape during operation

978-1-4799-0965-0/13/$31.00 ©2013 IEEE

and a simulation of material particles flying out (see Figure 5)were also implemented in C++.

The 3D visualization part of the system is computed on astandard PC (Intel Core i7 860, 8 GB RAM, Radeon HD 5870,Windows 7 x64). The control panel simulation was realizedon a separate machine using Linux. The visualization andthe control panel computer communicated via TCP/IP overEthernet. An overview of the system set-up within in the 3-wall VR visualization environment “Immersion Square” [8],[9] is given in Figure 1.

B. Dynamic Simulation of Sound and Background Noise

Typical sounds of a lathe in different kinds of operationsequences were recorded, digitized and analyzed. The chal-lenge of this subtask is to provide synthesized sound samplesfor each and every rotation velocity of the lathe’s rotationcomponents. The sound output has to match the conditionsof the machine tool and the work pieces, which might changedynamically. Sounds have to be very realistic because prestud-ies with subjects have shown that operators expect appropriatesound changes as soon as the operation mode is changed,and that they also use auditory cues to determine whetherthe machine is working as expected. Therefore, demands onrealistic, seamless and dynamic synthesis of sound were high.To enhance the realism, background noises typical for metalworking environments were included as well.

Fig. 1. Mixed reality simulation set-up of a lathe in the immersive three-wallvisualization environment “Immersion Square”. On the right side the physicaldisplay and control panel mounted on a stand can be seen.

C. Control Panel

In order to achieve high degrees of immersion and realism,a physical control panel with a display was developed. Itconsists of two components: a control module and a display.

The control module used as a basis the Siemens SinumerikKB 483C full CNC keyboard and the Siemens Sinumerik MCP483C machine control unit. As a PC interface module, themicrocontroller from a Logitech Gamepad F310 was used,which allowed input signals to be tranferred to the PC usinga standard USB connection. The front panel of the controlmodule is shown in Figure 2. The current control moduleprovided selective access to the key functions of the controlpanel, such as two rotation dials for spindle speed and feedrate, buttons for switching between manual and automatedmode, start/stop buttons and emergency stop.

Fig. 2. The control module is based on the real industrial control panels fromthe Siemens Sinumeric product line, with an integrated gaming controller foreasy interfacing to the simulation PC.

The display module was based on a NEC MultiSync LCD1860nx display. The visual simulation of a user interface on thedisplay module was based on the Fanuc Oi-TC control display(see Figure 3). It is able to display G-code of the currentlysimulated program (G-code is the programming language thatis interpreted by the lathe to control its motion and theselection of tools), show positional coordinates of the tool,spindle speed and feed rate as well as the control dials’ currentpositions.

The mechanical design of the panels included severalspecial features. The universal mounting points allow fixationof the panels in any reasonable configuration: side-by-side orone on top of each other.

Fig. 3. Screenshot of a simulated dynamic user interface based on the FanucOi-TC control display. The subwindow on the right shows parts of the machinecode to be evaluated. Further visualizations show standard parameters of amachine control panel.

III. EVALUATION STRATEGY

An experimental study was designed and conducted toevaluate if our mixed reality application allowed for usabilitytesting in general. A program to produce a standard aluminumwork piece was developed in machine code. Subjects wereasked to produce a cylindrical work piece with several rims andgrooves based on the drawing shown in Figure 4. The programincluded numerous different subtasks (turning, trimming and

polishing) while exchanging and applying different tools. Theentire manufacturing process would take 5–7 minutes in anautomated series production mode, but in single step mode,25–35 minutes would have to be estimated to evaluate theprogram.

Fig. 4. Technical drawing of an aluminum work piece, which should beproduced in the lathe simulation.

A. Subjects

The first subject sample consisted of 10 male CNC machineoperator apprentices of a nearby training school. Most of themwere less than 20 years old; only 3 subjects were between 21and 25 years old. Accordingly, they had limited experiencewith CNC lathes, but most of them had 1–5 years experiencewith conventional lathes. A second subject sample used 4experienced machine tool operators with an age range of 26–35 years and either 5–10 years or more than 10 years ofexperience.

B. Procedure

Each subject was asked to first run a short training programon the virtual lathe to get used to the virtual environment. Thistook about two minutes and showed the trimming of a virtualbowl.

Afterwards they were shown details of the technical draw-ing of the aluminum work piece to be manufactured (Figure 4).The instruction they were given was as follows: “A colleaguehas written the program to fabricate the work piece shown.Your task is to evaluate it in single step procedure anddetermine if the program is correct before it enters seriesproduction. If there are any inconsistencies or mistakes, pleasereport them”. This kind of instruction compares to the standardprocedure in a realistic production set-up.

After this instruction the investigator stepped aside andthe subject started the program using the control panel. Theprogram contained a mistake which appeared, dependent onthe rotational speed and the feed rate chosen, after about 20minutes. It consisted of a missing program instruction whichresulted in a major crash of the machine. The missing programinstruction would have correctly positioned the machine’s toolbefore switching to a different tool. Its absence caused thelathe to move the tool along a direct path through the workpiece, which was rotating at high speed. Mistakes of this kindhave led to fatalities in the past, because the work piece wasexpelled from the lathe and hit the worker standing next toit. Having consulted with experts beforehand, it was expectedthat this mistake would be difficult to find, in particular forless experienced subjects.

Fig. 5. 3D model of the aluminum work piece which is produced in thelathe simulation, together with one of the tools.

In a final step all the subjects were given three surveys:first a demographic one, second a survey to assess symptomsof simulator sickness and third a survey to evaluate the qualityof the study.

C. Evaluation

In the survey, subjects were asked to report back howmuch they agree to a set of 19 statements, on a discretescale with 5 options. The answers were analyzed with re-spect to four different dimensions (which were not known tothe subjects): presence, quality, fascination and acceptance.Presence expresses the degree to which the user feels thathe/she is part of the virtual environment [11]. Quality refersto the overall quality of the simulation, graphics and audio.Fascination describes how fascinated the user is by the entireVR experience. Finally, acceptance is determined by whetherthe subject thinks that the system is appropriate for work safetystudies.

A sample statement in the dimension of presence was“While working with the virtual lathe, I acted with the samecaution as with a real one”. For the dimension of quality,a sample statement was “I was able to identify the relevantcomponents of the lathe”. “The journey into the virtual worldwas fun” was one of the statements addressing the dimensionof fascination. For the dimension of acceptance, one of thestatements that had to be evaluated was “I think that VR is areasonable means for work safety studies”.

IV. RESULTS

Results were analyzed separately for behavioural data anddata collected with the three surveys.

A. Behavioural Data

The mistake in the program was, as expected, hard tofind, in particular for the less experienced subjects. Only oneout of ten found the mistake and did not proceed with thesingle step mode, thus avoiding the crash. None of the fourmore experienced metal workers detected the mistake. Oralfeedback from the more experienced subjects indicated that themistake was very well hidden even for experienced operators.The sample size of both subject groups might have to beincreased to obtain more valid statistical data. However, the

high correlation with the appraisal of the experts leads us tothe educated guess that working on our virtual lathe can beconsidered as comparable to working on a real lathe wheninvestigating human factors within this context.

B. Results from the Surveys

None of our subjects reported any symptoms of simulatorsickness. Figure 6 shows the results from the evaluation survey.

Fig. 6. Results from the evaluation survey. Shown are the mean values forevery dimension: presence, quality, fascination and acceptance (scale: 1–5).Higher values mean better evaluations.

Results from the dimension presence varied dependingon the item. Especially transitional statements, for example“Sometimes I found it hard to determine where the real worldended and where the virtual world started”, were rejectedduring the pre-experiment questionnaire. Other statements, forexample “I felt as if I was working on a real lathe”, receivedbetter evaluations. The quality of the lathe simulation was ratedmedium to good. Negative assessments mostly resulted fromthe (low) resolution and rigidity of the immersive display.However the dimension fascination shows that the subjectswere really enthusiastic about the VR technology. Among alldimensions, fascination received the highest scores. Accep-tance was rated good as well. Most subjects mentioned thatvirtual reality provided a reasonable means for investigationswithin the field of work safety.

V. DISCUSSION AND CONCLUSIONS

In the current implementation, the program that is pro-cessed by the lathe is hard-coded and cannot be changed easily.The 3D animation of the lathe producing the work piece hasbeen created manually. In order to conduct further experimentsthat involve different programs (or even completely user-defined programs for training purposes), it would be advisableto integrate a G-code interpreter into the simulation and alsosimulate the milling process. A voxel-based approach appearsto be promising for the visualization part of the task. Thegraphics quality can be further improved by using displayswith a higher resolution and stereo 3D. A head tracking systemwould allow the user to move around the lathe and observe itsoperation from different points of view. It is expected that thiswould increase the user presence. An alternative would be tobuild a mock-up of a lathe and replace the viewing windowwith an (auto-)stereoscopic display. By replacing the physical

lathe control interface with a touch screen, usability aspects ofsuch interfaces could be evaluated in more detail. Users wouldthen have to complete typical tasks with different interfacelayouts (for example, different button configurations) in orderto determine what constitutes an “optimal” layout.

Results from our research show that it is possible toconduct investigations in the field of work safety by applyingVR and mixed reality technology within a virtual environment,and that this can in fact be recommended. Important factorsfor a successful study are a high quality of the visualizationenvironment, display and sound. It turned out to be crucial tomake the virtual world as realistic as possible. It is anticipatedthat future virtual reality applications will be much morewidespread and become a common means for investigationsand trainings in various fields of science and working life.

Interactive VR scenarios have demonstrated that usabilityand work safety evaluations can be conducted under realisticconditions without exposing the operator to a risk, while in re-ality these kinds of investigations would have been impossiblefor ethical reasons. Further research might target the develop-ment of serious games within these kinds of environments, inwhich safe working procedures including inherent risks couldbe trained.

ACKNOWLEDGMENT

The authors would like to acknowledge funding in partby DGUV (German Social Accident Insurance) research grantFP279.

REFERENCES

[1] E. Flaspaler, D. Reinert, and E. Brun, Expert Forecast onEmerging Physical Risks Related to Occupational Safety andHealth. European Agency for Safety and Health at Work, 2005,http://europa.eu.int/publications/factsheet/60.

[2] S. Heinsen and P. Vogt, Usability praktisch umsetzen. Hanser Verlag,Munchen, 2003.

[3] J. Marsot, J. Ciccotelli, and J. Marc, “Dynamical indexation of riskin a virtual work environment.” International Journal of ProductDevelopment, vol. 4, no. 6, pp. 570–586, 2007.

[4] J. Viitaniemi, K. Helin, S. Aromaa, I. Kulmala, and A. Saamanen,“Usage of indoor simulation data in virtual environment,” 2005, VirtSafeWorkshop (CIOP-PIB).

[5] J. Ciccotelli and J. Marsot, “Virtual reality and prevention. Contributionsand tendencies.” Hygiene et securite du travail – Cahiers de notesdocumentaires, no. 199, pp. 99–111, 2005, Points de repere – PR 17(Landmarks in occupational safety).

[6] A. Glockle, “Druck- und Papierverarbeitungsmaschinen: Integrationsicherheitstechnischer und ergonomischer Aspekte bei der Konstruktionund dem Design,” 2006, Intern. Symposium on Design Process andHuman Factors Integration, Nice, France.

[7] “HVBG-Report Manipulation von Schutzeinrichtungen an Maschinen,”Sankt Augustin, 2006, Hauptverband der gewerblichen Berufsgenossen-schaften (Hsrg.).

[8] R. Herpers and F. Hetmann, “Dreiwandprojektionssystem,” Patent,2001, Deutsches Patentamt, Az. 101 28 347.4.

[9] R. Herpers et al., “Multimedia sensory cue processing in the fivissimulation environment,” Multiple Sensorial Media Advances and Ap-plications: New Developments in MulSeMedia, IGI Glubal, pp. 217–233, 2011.

[10] “Ogre3D,” http://www.ogre3d.org/.

[11] D. R. Mestre, “Immersion and presence,” 2005,http://www.ism.univmed.fr/mestre/pub/Pres 2005.pdf.