Automated design process modelling and analysis using immersive virtual reality

  • Published on

  • View

  • Download


<ul><li><p>Keywords:Virtual realityDesign task analysisCable harness designUser loggingDesign rationale and knowledge capture</p><p>research which demonstrates how the detailed logging and analysis of an individual designers actionsin a cable harness virtual reality (VR) design and manufacturing system permits automated design taskanalysis with process mapping. Based on prior research, which utilised user-logging to automaticallyanalyse design activities and generate assembly plans, this work involves the automatic capture ofextracted design knowledge embedded within the log files and subsequently represented using IDEF0diagrams, DRed graphs, PSL, XML, annotatedmovie clips and storyboard representations. Using this designknowledge, an online help system has been demonstrated which helps users to carry out design taskssimilar to those performed previously by expert users. This is triggered by monitoring the designersactions and functions in real time and pushes knowledge and advice to the user which was capturedfrom experts and subsequently formalised during earlier design sessions.</p><p> 2009 Elsevier Ltd. All rights reserved.</p><p>1. Introduction</p><p>Within product design andmanufacture, the cost of virtual real-ity (VR) tools is reducing and, in the near future, it is envisaged thatthey will become widely used throughout industry as a major partof the product life cycle process. Due to this anticipated expansion,there is a need to investigate how such VR tools could have an im-pact on the design andmanufacturing processes. In addition, somenew tools which may help increase the future use of VR need tobe researched, such as user-interface analysis and the automaticgeneration and analysis of product engineering information. Dueto the nature of VR systems, and themanner inwhich they are pro-grammed, it is straightforward to implement routines which allowuser actions to be logged unobtrusively. By analysing the informa-tion embedded in the logged data, useful design knowledge can beextracted and used as a basis for an interactive help system to trainnew users and store knowledge for future analysis and use.This paper focuses on utilising and expanding the capabili-</p><p>ties of a well tried and tested cable harness virtual aided design</p><p> Corresponding author. Tel.: +44 131 451 4569; fax: +44 131 451 3129.E-mail addresses: (R.C.W. Sung),</p><p>(J.M. Ritchie), (G. Robinson), (P.N. Day), (J.R. Corney), (T. Lim).1 Tel.: +44 141 548 2254.</p><p>(VAD) system as a tool for the analysis of industrially equivalentharness routing and assembly planning tasks. This tool has beensuccessfully compared with a number of CAD systems and foundto provide considerable productivity benefits over traditionalcomputer-based techniques [1]. Prior research, presented in [2],demonstrated that the user-logged data has been successfully usedto automatically generate assembly sequence plans and IDEF0 as-sembly diagrams. This has subsequently laid the foundations fordesign rationale capture through identifyingmethods for recognis-ing signature patterns relating to design activities for cable harnessrouting and assembly planning. In this paper, a prototype informa-tion push system is proposed which offers automated assistanceto users during a design task by combining the monitoring of useractivities and utilising design knowledge that has been previouslycaptured. Aswell as allowing expert design knowledge to be taughtto new users, it is envisaged that such a system can be used to trainengineers in maintenance-related tasks.The immersive VR apparatus is detailed in Section 2 whilst the</p><p>experimental methodology used to investigate this design domainis presented in Section 3. In Section 3, the various formal repre-sentations that have been used to represent the design knowledgeextracted from the log files are presented while, in Section 4, anoverview of how the design knowledge has been used to developan online help system is detailed. Finally, a discussion of the resultsis presented in Section 5 before ending with some conclusions.Computer-Aided Design</p><p>Contents lists availa</p><p>Computer-A</p><p>journal homepage: www</p><p>Automated design process modelling andRaymond C.W. Sung a,, James M. Ritchie a, Graham RTheodore Lim aa Scottish Manufacturing Institute, Heriot-Watt University, Edinburgh, EH14 4AS, UKb University of Strathclyde, Design, Manufacture &amp; Engineering Management, 502b James</p><p>a r t i c l e i n f o</p><p>Article history:Received 23 July 2008Accepted 29 September 2009</p><p>a b s t r a c t</p><p>The capture of engineering ddifficult due to the high overhused in industry, usually invthem to remember how a d0010-4485/$ see front matter 2009 Elsevier Ltd. All rights reserved.doi:10.1016/j.cad.2009.09.00641 (2009) 10821094</p><p>ble at ScienceDirect</p><p>ided Design</p><p></p><p>analysis using immersive virtual realityobinson a, Philip N. Day a, J.R. Corney b,1,</p><p>Weir Building, UK</p><p>esign processes and associated knowledge has traditionally been extremelyead associatedwith current intrusive and time-consumingmanualmethodsolving interruption of the designer during the design task and relying onesign solution was developed after the event. This paper presents novel</p></li><li><p>R.C.W. Sung et al. / Computer-Aid</p><p>1.1. Using immersive virtual reality in engineering applications</p><p>VR takes many forms with a wide array of technologies classi-fied as being virtual environments (VE) in one form or another. VRhas, and is, being used in the engineering of products using a widevariety of technology [3]. In this paper, the focus is on a virtual envi-ronment where users wear a head-mounted display (HMD), ratherthan looking at a 3D screen or using augmented reality (AR); there-fore, users are completely immersed in the virtual environmentwith a wide field-of-view. VR systems are altering the manner inwhich engineers are developing products, concepts and planningmanufacturing processes [3,4] with a substantial number of appli-cations now being evident within the public domain. For exam-ple, [5] presents a virtual design environment for aircraft whichallow several users to collaborate together during the design stage.In [6] virtual prototypes are used for the sound quality design ofautomobiles while in [7] models of underwater vehicles are gener-ated in a virtual environment and then simulations can be run totest the design. As an example of the use of virtual tools to aid theproduction planning process [8] presents a tool which uses VR andAR technologies tomodel a factory and then run simulations to op-timise the manufacturing processes involved. Varga et al. [9] haveinvestigated hand motion as a means of creating conceptualisedgeometry for designs in a virtual environment as well as identi-fying the benefits that this form of interface brings to the designtask. Holt et al. [10] demonstrated that interactive immersive VRdoes have a role to play in the design process by being more costand time-efficient as well as giving users the tool support that isrequired in creating an effective design.As can be seen, engineering applications utilising VR are ex-</p><p>panding and are too numerous to mention in this paper. It is en-visaged that the use of VR tools will increase, so it is importantto carry out research which investigates how engineers will op-erate and function in such environments in the future. In relationto extracting design andmanufacturing information and engineer-ing knowledge, VR tools allow a new type of capture. One key as-pect of this is to take advantage of monitoring the user; this allowsboth the acquisition of geometry and system interaction informa-tion (e.g. menu choices) which, when combinedwith the users be-havioural data, extracts what is implicitly embedded within thetask-associated activities. This knowledge can then be made ex-plicit, formalised and, if required, passed onto other users or storedin a product lifecycle management (PLM) system. This importantavenue of research is discussed more fully in the next section.</p><p>1.2. User logging and design knowledge capture</p><p>The capture and identification of design knowledge is an im-portant aspect of product engineering at all stages of the productlife cycle because this means that vital historical product knowl-edge relating to engineering solutions can be stored and is not lostshould engineers leave the company or retire [8]. This is partic-ularly the case for long-life cradle-to-grave projects which coverlong periods of time where product engineering knowledge andinformation generated early on in the products life cycle can in-variably be lost [11]. Also, if this is possible, engineering knowledgecan be captured from experienced engineers and subsequently beused to train and educate inexperienced engineers on design andmanufacturing methods. This problem is compounded by the factthat existing CAD systems often have proprietary file formats thatdo not allow embedding of knowledge information [12]. This isreaffirmed in [13], which states that: Current documentary ap-proaches are not sufficient to capture activities and decisions in their</p><p>entirety and can lead to organizations revisiting and in some casesreworking design decisions in order to understand previous designed Design 41 (2009) 10821094 1083</p><p>episodes. In addition, traditional methods of manual knowledgecapture are labour-intensive [14] and interruptions to a users ac-tivity may alter the pattern of how they carry out their task [15].Using these methods, a lot of information is lost or forgotten bythe time an engineer attempts to formalise their design method orsolutions, they tend to be carried out in a non-standard fashion,and failed or less suitable alternatives (which are just as importantto engineering knowledge capture) are forgotten about or missedout. To overcome these problems, automated and unobtrusivecomputer-based logging of engineers could be utilised to generateand formalise knowledge and information aswell as the associatedprocesses. Furthermore, using automated tools can help reduce er-rors to occur during the design task, since it has been found that:Using (automated) tools to do tedious, exacting, or uninterestingtasks helps eliminate human errors by releasing people to concen-trate on more interesting work for which human intelligence is es-sential and most valuable [16]. From past research experience, theopen and accessible development platforms associated with vir-tual aided design (VAD) environments provide an ideal arena inwhich to investigate the practicality of this concept. The authorsalso contend that immersiveVRprovides considerable potential forthe non-intrusive analysis of design tasks and, through the recog-nition of associated patterns of an engineers behaviour recognisedin the context of the task being carried out, this will be even morethe case in the well-structured downstream manufacturing plan-ning activities, such as assembly planningwhere productivity gainshave been demonstrated using both non-haptic and haptic VR [17].However, once design knowledge has been captured, it needs to</p><p>be presented in a formal and structuredmanner to allow the infor-mation to be quickly and easily understood and accessed by engi-neers [11]. Research has found that engineers can spend asmuch as30% of their time looking for and viewing various sources of designinformation and, quite often, they are unable to find useful data intheir organisation, partly because what is required is not in a suit-able format [18]. In [19], a system called Process Data Warehouse(PDW) is presented that allows the capture and re-use of designknowledge during a design of a chemical reactor. PDW performssemi-automated knowledge capture by manually importing doc-uments into the system, which then get automatically translatedinto an XML representation. Currently, the capturing of work pro-cesses and decisions is not supported and the current system canonly handle a small amount of instance data. The research in thispaperwill attempt to remedy these shortcomings by automaticallytranslating the logged user actions into XML and then automati-cally generating further, understandable formal representations ofthe work processes and decisions.Twoknowledge capture systems, calledHyper-Object Substrate</p><p>(HOS) and PHIDIAS, are detailed in [20]. PHIDIAS allows the captureof CAD designs, text, sketches, audio and video; the system thenallows the searching of the captured data, and related links canbe manually added to related items. HOS is a system used forcomputer networkdesign and it contains a store for all the differenttypes of media, such as text, line drawings, composites and spatialarrangements. To provide some structure to the captured data, textanalysis is performed on the information, such as imported e-mailsand USENET messages, to look for possible links between differentpieces of data. Another component present in the two systemsconsists of critics which observe a users actions and displaysrelevant information to help improve the design. However, the twosystems presented neither automatically represent the captureddesign knowledge and information in a formalised and easilyunderstood manner, nor push this at the user.An architectural analysis method called SAAM (Software Archi-</p><p>tecture Analysis Method) is detailed in [21] where a tool called</p><p>SAAMPad is used to automatically capture and retrieve design ra-tionale during a SAAM session. This involves designing using an</p></li><li><p>1084 R.C.W. Sung et al. / Computer-Aid</p><p>electronic whiteboard while an audio and video of the session isrecorded, together with the whiteboard contents. Each recordedaction is time-stamped and then used to generate electronic textsummaries and visualisations of the architecture. A limitation ofthe current system is that it cannot be used to communicate withother software architecture tools.The capability of VR for design task analysis was demonstrated</p><p>by COVIRDS (COnceptual VIRtual Design System) which showedimmersive virtual design [22] using voice commands and trackingof the hand. In this paper, the authors proposed that, during theconceptual design stage, the need to create detailed models isnot as important, so the COVIRDS system uses a shape modellingfunctionality to quickly create initial designs.Exploiting user-logged data created during a VR session, Ritchie</p><p>et al. [23] investigated the use of data in log files to automate as-sembly plans. Wyatt et al. [24] analysed the log files from a VRgeotechnical laboratory to help create a more interactive designtool. Brough et al. [25] and Schwartz et al. [26] presented a vir-tual environment, called Virtual Training Studio (VTS), where usersare trained to perform assembly tasks, and assistance is offered tothem if requested or if errors are made. Whilst users perform thepreferred assembly task, their actions are logged and then analysedto identify problems that need to be rectified by further training.VTS is one of the...</p></li></ul>


View more >