ADMISSION INFORMATION AND PROCESS - Laramie County Community

1 Copyright © 2006 by ASME

Proceedings of IDETC/CIE 2006 ASME 2006 International Design Engineering Technical Conferences

& Computers and Information in Engineering Conference September 10-13, 2006, Philadelphia, Pennsylvania, USA

DETC2006-99541

INCLUDING HUMAN BEHAVIOR IN PRODUCT SIMULATIONS FOR THE INVESTIGATION OF USE PROCESSES IN CONCEPTUAL DESIGN: A SURVEY

Wilhelm F. (Wilfred) van der Vegte Faculty of Industrial Design Engineering,

Delft University of Technology, Landbergstraat 15, NL-2628 CE Delft

The Netherlands [email protected]

Imre Horváth Faculty of Industrial Design Engineering,

Delft University of Technology, Landbergstraat 15, NL-2628 CE Delft

The Netherlands [email protected]

ABSTRACT

In this paper, approaches for behavioral simulation of humans and human-artifact systems are reviewed The objective was to explore available knowledge for the development of a new method and system for the simulation of use processes of consumer durables in conceptual design. A key issue is to resolve the trade-off between minimizing the modeling and computing effort on the one hand, and maximizing the amount of valuable information obtained from simulations to facilitate improving the product. After drawing up review criteria, we reviewed existing simulation approaches, which we characterized based on the simulation models. We found that the surveyed approaches can only address limited, largely unconnected subsets of the various behaviors that can be simulated. For the most advanced approaches, the subsets can be clustered into three main groups: (i) kinematics and rigid-body kinetics simulated with non-discretized object models, (ii) mechanical-deformation behavior and non-mechanical physical behavior simulated with discretized object models and (iii) interpreted physical behavior (information processing) simulated with finite-state machines. No clear-cut solutions for integrated behavioral simulation of use processes have been found, however, we could identify opportunities to bridge the gaps between the three groups of behavior, which can help us to resolve the aforementioned trade-off. In the first place, it seems that the possibilities for using discretized models in kinematics simulation (especially with consideration of the large deformations that are common in biomechanics) have not been fully explored. Alternatively, a completely new uniform modeling paradigm, possibly based on particles, might also

help to resolve the gap between the two distinct groups of physical behaviors. Finally, hybrid simulation techniques can bridge the gap between the observed physical behaviors and interpreted physical behaviors. Here, the combination with the object models commonly used for simulations in group (i) and (ii) seems to be largely unexplored.

Our findings offered valuable insights as a starting point for developing an integrated method and system for modeling and simulating use processes. We expect that other researchers dealing with similar issues in combining seemingly disconnected simulation approaches could benefit as well.

1. INTRODUCTION

During product design, behavioral simulation is frequently used to gain insight into the course of processes in which the product is involved. In a product’s life cycle one of the key processes is the use process, in which the product is actually applied for its purpose. Behavioral simulations of use processes can provide a valuable contribution to ‘designing-for-use’ (or DfU) approaches. After all, simulations (behavioral or non-behavioral) have been defined as experiments performed on models [1], which means that they make it possible to investigate life-cycle processes such as use before a product is available in its final form. In industrial product design, the simulation model of a product is typically called prototype, i.e., a physical prototype, a virtual prototype or an augmented prototype. Our research focuses on behavioral simulation in the beginning of the design process, where virtual prototypes are preferred because they are easier to create than physical or augmented ones. A virtual prototype is a non-real, digital prototype modeled and visualized using a


computer [2]. The behavior of a virtual prototype can be studied to verify the functions of the product in advance, i.e., to predict whether the behavior will be as intended. But a use process involves more than behavior of the product. In the literature the consensus is that, when investigating use processes, a larger system of three main components should be taken into account consisting of three main components: the human user, the product and the surrounding environment [3]. These components interact through mutual exchange and transformation of matter, energy and information. In this paper, the system will be referred to as the human-product-surroundings system, for short HPS system or HPSS. Our assumption is that simulation of HPS systems can be a valuable addition to the currently available methods and tools to support designing for use. While traditionally, the objective of behavioral simulation in engineering is to study the intended and unintended behaviors of a product, integrated behavioral simulation of HPS systems adds the opportunity of additionally investigating intended and unintended behavior of the human user as well as that of the surroundings of the product.

1.1. Objectives and scope of the survey

This survey is part of the knowledge exploration for the development of a new computer-based simulation approach that can be applied in conceptual design to investigate use processes and predict the behavior of HPS systems. In the investigated literature, no existing approach that fulfills this purpose was found. Only a scattered collection of separate approaches partially covering the area of interest appears to be available. Ideally, an integrated approach should take advantage of the available scattered simulation knowledge. With that objective in mind, a comprehensive survey would have to cover approaches for simulation of artifacts and simulation of humans.

This paper forms the second part of a comprehensive survey of simulation approaches for use processes. The first part [4], focused on simulation approaches for artifacts, which are commonly used in engineering design. This second part is written in such a way that it can be read independently, but for specific details the reader may want to consult the first part. The literature search for this survey started out as an inventory of available simulations of human behavior as a supplement to the already analyzed approaches to artifact simulation. During the search, it turned out that some of the human-simulation approaches offered a possibility to include artifact behavior. Although the application area was typically unrelated to use processes of consumer durables, it is assumed that it is an advantage if a possibility for linking human and artifact behavior is already offered. Thus, the focus of this paper is on simulation of the behavior of humans and human-artifact systems and on how they can be deployed to predict and investigate use processes. We will keep closely to the interpretation of simulation as ‘performing experiments on virtual models’, in that the whole HPSS is represented virtually, including the human. This means that most of the human simulations in the survey rely on autonomous virtual humans, who conduct themselves in the virtual space [5]. So-called

human-in-the-loop simulations have not been surveyed extensively.

The survey focused on research achievements published in scientific literature but relevant commercial solutions have also been included; these are referenced in footnotes. Priority was given to achievements and examples related to the use of consumer durables. Contributions from other fields have been included if no search results had been obtained from the focus area.

1.2. Assessment criteria

The various approaches to simulation will be assessed based on their potential contribution to investigation of use processes. The following criteria will be applied: • Range of behaviors covered. Simulation should cover as

many types of human and artifact behavior as is reasonably possible. In the next subsection a scheme is introduced to classify the various behaviors.

• Relevance of the scope. The overlap between the scope of a simulation approach and the scope of the application area, use of consumer durables, should be as large as possible.

• Ease of preparation. The amount of time needed to set up a simulation should be kept at a minimum. In particular, this concerns the inclusion of product models that designers have created, in the simulation. Our assumption is that most designers of consumer durables use solid-modeling CAD packages. If these models cannot directly be used as a virtual prototype, a second-best option is that available CAD models can be converted to simulation models in an automated way.

• Speed and computability. The time needed for a simulation to run on common hardware should be as short as possible.

• Ease of interpretation. Traditionally simulation output is numerical, e.g., tables or graphs show the course of values in time. Our assumption is that, especially to designers, spatial 3D animation of the simulated system is a valuable addition to numerical output.

• Fidelity of the outcomes. The outcomes of the simulation must sufficiently correspond to real behavior.

1.3. Types of behavior in simulation

In [4], the possible behaviors of artifacts have been subdivided into observed physical behavior and interpreted physical behavior. The effects of all behaviors can be observed as flows and transformations of energy and matter. Interpreted physical behavior substitutes observed physical behavior based on abstractions, like, for instance, the string ‘100111’ may represent a series of output voltages. The distinction of interpreted physical behavior was necessary because certain simulation approaches disregard the physical background of information. Observed physical behavior has subcategories according to the fields of physics (mechanical behavior, acoustic behavior, optical behavior, etc.) which can be simulated individually or simultaneously (i.e., multiphysics). If a system is interpreted as purely informational, it is characterized as discrete and if it is investigated through its observed behavior it is said to be


continuous [6]. If it is investigated by observing and interpreting behaviors it is called hybrid [7]. The subdivision into observed physical behavior (with its subfields) and interpreted physical behavior is also applied to characterize the range covered by simulation of the artifact part in simulation approaches for human-artifact systems.

To typify human behaviors we propose a simplified reasoning model about human behavior as presented in Figure 1 and explain it as follows. Based on elementary functions that can be recognized in the human body relating to the processing of matter, energy and information we distinguish metabolic behavior, perceptual behavior, cognitive behavior, control behavior, actuator behavior, kinetic behavior, kinematical behavior and non-mechanical physical behavior. Metabolic behavior involves the chemical processes that convert matter taken in as food, drink and air to energy [8]. Our assumption is that the relevance of metabolic behavior to product use is limited to (i) aspects of human endurance and fatigue, as they are investigated in the design and development of military equipment [9], work environments [10] and sports equipment [11] but not in the design of typical consumer durables, and (ii) aspects of ingesting and processing consumables [12] rather than durables. Therefore it is disregarded in this survey. Perceptual behavior is the behavior of sensory cells and systems that convey the impingement of electromagnetic, mechanical, and chemical changes in energy [13], which is passed on by the central nervous system to the brain. Cognitive behavior relates to information processing by the brain, which includes attention, remembering, producing and understanding language, solving problems, and making decisions [14]. Control behavior involves conscious (or high-level) control of human action by the brain and subconscious (or low-level) control by the central nervous system [15]. Actuator behavior relates to human-initiated output of mechanical energy, i.e., producing motion at joints through contraction and relaxation of muscles [16]. Kinetic behavior involves all mechanical behavior concerned with the effects of forces on the motion of body parts and other objects, with the exception of actuator behavior performed by muscles. Statics is treated as a special case of kinetics with zero resultant

forces. Typical body parts involved in kinetic behavior are the skin, subcutaneous tissue, the skeleton and muscles. Although traditional rigid-body kinetics can be applied to certain parts of the human body (in particular, the bones), large deformations typically play a crucial role in human-body kinetics [17]. Finally, kinematical behavior relates to the movement of bones and joints caused by muscles and external influences. It does not include the energy aspects included in kinetics. Omitted from Figure 1 but included in the review is non-mechanical physical behavior that can be subdivided according to the fields of physics. It has been left out of Figure 1 to reduce its complexity.

The highest-level subdivision into observed and interpreted behavior that is applied in the characterization of artifact behavior can also be applied to the various human behaviors: those behaviors that are characterized in Figure 1 as information flows can be treated as interpreted human behavior. As will be shown in the next sections this typically applies to cognitive behavior and conscious control behavior. 1.4. Structure of the survey

The discussion of the various approaches to simulation is structured based on the theoretical basis of the models that are used for simulation. The left-hand side of Figure 2 gives the taxonomy of model types that we chose to use in his survey. At the highest level, behavioral models and object models are distinguished. These models represent the behavior of the object (human or artifact) or the object itself, respectively. Behavioral models can be subdivided into control models and processing models, which represent the ‘whys’ and the ‘hows’ of simulated behaviors, respectively. Control models are logic-based or laws-based and processing models are algebraic, algorithm-based or animation-oriented. There are two types of object models: relationship models and entity models. Relationship models describe logical or spatial relations. Entity models can be abstract or concrete. Abstract entity models are based on 2D-graphics or 3D-schematics. Concrete entity models are typically boundary models, simplified boundary models, volumetric models or simplified volumetric models.

types

of

behavio

rbody p

art

s and

org

ans

involv

ed sense

organsbrain central

nervoussystem

muscles skin,subcu-taneoustissue,etc.

kine-maticalbehavior

cognitivebehavior

perceptualbehavior

controlbehavior

actuatorbehavior

bowels,lungs,heart,bloodvessels,etc.

metabolicbehavior

flow of energy flow of informationflow of matter

centralnervoussystem

kineticbehavior

tendons.bones,joints

interface withthe outside world

Figure 1 Typification of human behavior based on elementary functions, including the relations with the involved body

parts and organs.


As the survey results will show, most simulations are based on combinations of model types. The right-hand side of Figure 2 shows how model types appear combined in the investigated simulation approaches, and how the survey has been organized accordingly. For a detailed discussion on the fundamentals of the individual approaches, the reader is referred to [4], which largely follows the same subdivision. Yet, the reader may already be familiar with basic simulation approaches such as block diagrams, bond graphs and finite elements. After the actual review, section 4 follows with the conclusions, including a comparative overview of the analyzed approaches, and addresses which open issues remain and how they can be dealt with in future work.

2. SIMULATION APPROACHES PRIMARILY BASED

ON BEHAVIORAL MODELS In simulation of humans, two types of behavioral-model

based simulation are distinguished: simulations (purely) based on algebraic descriptions (2.1) and simulations based on algorithms (i.e., algebra combined with logic), in particular qualitative reasoning and finite state machines (2.2).

2.1. Simulations based on algebraic descriptions

From the 1970s on, systems of equations from Newtonian dynamics have been applied to develop models of (parts of) humans in the area of biomechanics [18, 19]. Multon et al. [20] introduce a dynamical model of the human arm that describes control behavior, actuator behavior, kinematical behavior and rigid-body kinetic behavior. The model is based on Lagrange mechanics. Numerical simulation output is used as input data for the animation of 3D human models. Griffin [21] presents several models based on equations that describe the rigid-body kinetics of the human body as a mass-spring-damper system to investigate vibrations. To simulate subconscious control behavior with algebraic models as closed-loop negative feedback control (i.e., as an analog, continuous system) Powers introduced perceptual control theory in the 1970s [22]. A different application of algebraic models is presented by Job et al. [23]. For clinical purposes, they modeled the induction of electric current in the retina,

which is a subsystem of visual perceptive behavior, and simulated it based on equations. Other examples of algebraic modeling of non-mechanical physical behavior, namely emission and absorption of heat or electricity through the skin can be found in papers by Mochnacki and Majchrzak [24] and Panescu et al. [25], respectively. Algebraic descriptions have also been applied to modeling and simulation of human-artifact systems. An example is the simulation of external impact on a human head covered with a helmet presented by Therrien and Bourassa [26].

The described examples are only a small selection from the algebraic simulation models of human behavior that we found in the literature. The only types of human behavior for which we could not find references using algebraic simulation models are cognitive behavior and conscious control behavior. The forerunning survey of artifact-simulation approaches showed that algebraic descriptions focus on observed physical behavior without limitation to a particular type of behavior [4]. This seems to apply to human simulation as well.

Regarding the human-behavior representations, the investigated models have in common that they are not explicitly limited to one particular human or a particular group of humans, except where behavior of humans with specific disorders is concerned (e.g., [23]). Apparently, the models are assumed universally valid for all ‘healthy’ humans. Typically they contain variables for which the values can be changed, thus offering the necessary versatility in dealing with different individuals while the equation itself remains in the same form (e.g., [20]). This is different from the situation in artifact simulation, where a newly designed class of artifacts often requires new dedicated differential equations to model observed physical behavior. This disadvantage also applies to human-simulation models that include artifacts (e.g., [26]).

The output of algebraic-description based human simulation is typically numerical. Direct 3D animation is only possible by manually defining the links between computer-generated output and an animatable object model.

qualitative models

finite state machines

block diagrams

bond graphs

3D schematic models

Guided 3D volumetric models

particle models

algebraic descriptions

A

SimulationModels

Behavioralmodels

Objectmodels

Entitymodels

Relationshipmodels

Processingmodels

Controlmodels

algebraic

algorithmbased

animationoriented

laws-based

logic-based

graphical (2D)

schematic (3D)

simplifiedboundary

spatial

logical

volumetric

boundary

concrete

abstract

B

A includes B simplifiedvolumetric

(semi) autonomous3D volumetric models

Models on which simulationapproaches are based

2D graphicalobject models – 3.1

algorithm-based models – 2.2

3D volumetricobject models – 3.3

finite-element models

boundary-element models

skinning models mes

h-b

ased

3D discretizedobject models – 3.4

2.1

3.2

3. Fo

cus

on

ob

ject

model

s2

. Fo

cus

on

be-

hav

iora

lm

odel

s

subchapterstaxonomy of model types

production rules

Figure 2 Taxonomy of model types (left side) and structure of the survey (right-hand side)


2.2. Simulations based on algorithms The human simulation approaches based on algorithmic

behavioral models that we found in the literature can be subdivided into approaches based on (i) qualitative reasoning, (ii) finite state machines (FSMs) and (iii) production rules.

Qualitative simulation. Qualitative simulation is often

based on the qualitative reasoning theory [6] that can be used to create and simulate incomplete models. A qualitative approach to algorithm-based simulation has been developed by Šuc and Bratko [27]. They discuss a dedicated qualitative model describing human ‘strategy’ in bicycle-riding behavior derived from quantitative measured data. The modeled strategy includes subconscious control behavior, actuator behavior and kinematical behavior of the human and rigid-body kinetic behavior of the bicycle. By ‘cloning’ the strategy back on measured data, quantitative numerical output is obtained (but no method is given to obtain animated output). The bicycle model was developed for this specific problem; it cannot easily be replaced by another artifact model converted from, for instance, CAD. The other qualitative simulation approaches or humans we found in the literature fall outside the scope of this review, since they focus on metabolic behavior [28].

Simulation approaches based on finite state machines.

FSMs are visually enhanced algorithms such as state transition diagrams and Petri nets, which are typically used to model and simulate discrete systems. Fogel and Moore [30] modeled steering behavior (i.e., subconscious control behavior) of pilots in aeronautics using a state transition graph combined with evolutionary algorithms. Rauterberg et al. [31] developed a Petri-net model of cognitive behavior in human-computer interaction. Human decision-making is modeled based on a log file of someone using particular software, i.e. from observed behavioral sequences. Based on the results of multiple users, distinct user groups with similar decision-making strategies could be identified [32]. Liu and Salvucci [33] present a model of conscious control behavior based on a hidden Markov chain (a particular type of state transition diagram), which was used to simulate human driving tasks. In all the above cases, virtual or real artifacts are assumed to be available. They provide input to the simulation and receive and process its output. These artifacts are not included in the simulation models.

Simulation approaches based on production rules. The

assumption behind rule-based simulation of cognitive and control behavior is that humans make decisions based on what they perceive around them and that these decisions can be modeled as production rules. Such models are often applied in cognitive psychology [34]. Architectures for rule-based human-behavior simulation are, among others, AM, EURISKO [35] and SOAR [36]. The rules are typically based on findings from psychological experiments on human subjects. The models have originally been developed for simulation of purely cognitive behavior, e.g., puzzle-solving. Later developments, such as ACT-R and EPIC [37] also include

modules to represent of body parts interacting with artifacts, in particular sense organs (eyes, ears) to perceive information, and hands and speech organs to communicate information to artifacts [38] (Figure 3). The simulation is restricted to interpreted behavior (e.g., entering ‘abcd’ on a keyboard or reading ‘1234’ from a monitor), and the observed physical behavior that is involved in interaction (moving fingers and capturing photons with the eye, respectively) is bypassed, as if the human’s information processing is directly connected to that of the artifact. As Figure 3 shows, subsets of rules are grouped into functional modules. This produces a representation that can be considered an object model rather than a behavioral model.

3. SIMULATION APPROACHES PRIMARILY BASED

ON OBJECT MODELS OF THE HUMAN In simulation of humans, four basic types of object-model

based simulation are distinguished: simulations based on 2D graphical object models (3.1), 3D schematic object models (3.2), rigid 3D volumetric models (3.3) and simulations based on discretized 3D models (3.4).

3.1. Simulations based on 2D graphical object

models Simulation approaches based on 2D graphical object

models representing the human body or part of it, are based on block diagrams or bond graphs.

Block-diagram simulation of human behavior is

typically applied to subconscious control behavior. We distinguish models that are based on (i) conventional control loops and (ii) neural nets. Examples of behavior simulated as a conventional control loop are eye-hand coordination in vehicle control [39] and interactive compensation of machine behavior by McRuer [40]. In McRuer’s loop, the machine reacts to human input and to external disturbances that the human needs to compensate. In both approaches, the physical transfer of information entered by the human into the machine – e.g., by operating an interface element – is a step that is skipped.

Cognitive ProcessorLong TermMemory

Short TermMemory

AuditoryProcessor

VisualProcessor

OcularMotor

Processor

VocalMotor

Processor

ManualMotor

Processor

Production RuleInterpreter

WorkingMemory

TactileProcessor

Figure 3 The EPIC architecture at its highest level of

decomposition (adapted from [38])


Artificial neural nets (ANNs) are particular class of block diagrams [41], which can be considered discretized models of human body components, that is, of biological neural nets1. ANNs differ from conventional simulation algorithms because of their ability to learn and self-organize, to generalize from training data, and to process information in other ways normally thought of as intelligent [42]. An example describing control behavior is steering behavior of airplane pilots in wind shear [43]. Like in the rule-based simulations described in 2.2, the human’s information processing is considered to be directly connected to that of the plane. Kim & Hemami [44] use an algorithm combining an ANN for subconscious control behavior in combination with nonlinear equation-solving to simulate kinematical and rigid-body kinetic behavior of the head and torso. The ANN acts on the output of a ‘desired trajectory generator’, which represents cognitive behavior that decides about motion of the head. This unit is left out of the simulation.

Bond-graph based simulations have been introduced by

Paynter in the 1950s as an alternative to block diagrams for systems that can be considered as built up from discrete components [45]. The human body, which is largely built up from continuum structures, is typically simplified in order to make modeling as discrete components possible. Margolis [46] developed a bond-graph model to simulate the response of the human body as a whole to external vibrations. To model and simulate musculoskeletal structure and function, Wojcik [47] proposes modular bond graphs. She acknowledges that the appearance of the resulting models is rather cumbersome (Figure 4), but nevertheless suggests that the use of bond graphs could promote standardization in human-body modeling. Pop et al. [48] apply bond graphs to create a partial model of the human body to simulate walking. It includes subconscious control behavior, actuator behavior, rigid-body kinetic behavior and kinematical behavior. Hubbard [49] presents a bond-graph model of a human using a vaulting pole. The simulation includes kinematics and rigid kinetics of the whole human-artifact system, as well as dynamic deformations of the pole.

A disadvantage is that, although bond graphs represent objects and although they can be used to model humans and artifacts together, no approach could be found in the literature to automatically derive bond-graph models from CAD models. To include a product in a bond-graph model, the designer has to re-model it with bond graph elements. Also, the standard output of bond-graph simulations is numerical and the graphical representation of a bond graph cannot be animated to visualize the results.

1 However, usually there is no one-to-one correspondence between the

components in an ANN and the components of a biological neural net. There-fore, it could be argued that they belong to the algorithm-based simulations discussed in 2.2

3.2. Simulation approaches based on 3D schematics of humans

3D schematic simulation models of humans are typically some kind of skeleton-like model. They have been developed for motion simulation and posture prediction [50], i.e., simulation of kinematical behavior. The models mostly represent the human skeleton, but skeleton-like models have also been generated based on anatomical landmarks [51]. Jung and Choe [52] present a skeleton-like model that is used to predict the reach envelope of the upper limbs. Webb and Aggarwal [53] developed an AI-based method to generate skeleton-like models from filmed motion sequences performed by human subjects. The models are used for kinematical motion simulation. A more comprehensive approach to simulate motion, which includes actuator behavior with muscles modeled as linear actuators, is presented by Park and Fussell [54]. An alternative approach to modeling actuator behavior is inverse dynamics, which can be interpreted as reasoning backwards. Instead of modeling muscles as actuators and calculating their effects (also known as forward dynamics), the resulting motion of the end effector (e.g., a hand) is taken as the starting point [55]. If needed, muscle actuation patterns are calculated based on the required motion pattern. To generate motion patterns for inverse-dynamics simulation of their skeleton-based model of the human arm,

Figure 4 Schematics of the dynamics of human sagittal

balance when subjected to horizontal support surface perturbation, with bond graph model (Wojcik, 2003)


Yang et al. [56] applied Kane’s method, which is based on linearization of generalized velocities. The objective was to investigate actuator behavior and kinematical behavior for the assessment of comfort in reaching tasks. Subic and Preston [57] use the same approach to simulate actuator behavior, rigid-body kinetics and kinematics in sports (e.g., jumping and diving). Kim et al. [58] predict motion patterns with joint trajectories based on B-splines, and demonstrate the approach on various human activities such as walking, pushing, and climbing. Typically the simulation results of the approaches discussed in this subsection can be visualized as animations. Inclusion of artifacts in simulations is more common with the volumetric models discussed in the next two subsections.

3.3. Simulation approaches based on rigid 3d

volumetric models of humans 3D volumetric models that are used for simulation of

human behavior are usually a simplified representation of human geometry. To the bones and joints, which are typically modeled as connected rods, volumetric entities have been added to represent the surrounding tissues. We can distinguish (i) guided 3D volumetric models for analyzing motions and reach, and (ii) autonomous and partially autonomous models, in which simulated control behavior, and in some cases cognitive behavior, determines human motion.

Guided 3D volumetric human models do not move

autonomously but have to be manipulated by the user. Since the 1970s various commercial software packages have been on the market to offer support for the assessment of possible postures and fields of view, e.g., SAMMIE, HECAD, COMBIMAN and CHESS [59]. Artifact models can typically be imported from CAD to investigate virtual products with virtual humans in virtual surroundings [60].

Simulation based on autonomous and semi-

autonomous 3D volumetric human models. These approaches mostly involve 3D models based on simplified volumetric and/or skeleton-like elements. Mechanical simulation based on algorithms is combined with simulation of subconscious and conscious control behavior, and cognitive behavior. It is visualized as an animation. In computer graphics, the continuous simulation of subconscious control and mechanics is known as local control and the discrete simulation of conscious control and cognition is known as global control of animations [61]. Based on the algorithms used for the simulation of the interpreted behavior, we distinguish agent-based simulations, ANN-based simulations and FSM-based simulations.

Agent-based modeling and simulation is applied to human perception in a simulation approach by Bordeux et al. [62]. Perception is simulated by agents acting as filters, passing through the information from the surroundings that is needed for decision-making. The included cognitive behavior covers short-time memory of perceived objects, and searching the surroundings for objects. Jung et al. [63] present an agent-based simulation technique for human motion planning through spaces populated with objects. It covers cognitive

behavior, control behavior, actuator behavior and kinematics. As is the case with the approaches discussed in 2.2, it is assumed that objects in the field of view are always seen, and therefore perception is not simulated. Another agent-based human simulation package is JACK [64]. JACK is a semi-autonomous virtual human that simulates perception, subconscious and conscious control based on agents. Decision-making is based on commands entered by the user. Additionally it simulates actuator behavior, rigid-body kinetic behavior and kinematical behavior. Artifacts are imported from a CAD environment into JACK’s surroundings, where they can be moved by the manikin. Weights are assigned to objects for quasi-static simulation of loads on JACK’s joints.

Neural networks have been used to model subconscious control behavior in 3D volumetric models. Reil and Husbands [65] present a simulation approach for human walking based on evolutionary selection principles that culminate into a best-performing network. They acknowledge that a lack of included biomechanical knowledge in the model makes it less realistic. The approach has been further developed to the commercial software package Endorphin by NaturalMotion Ltd.2, which is successfully being applied in the entertainment industry [66]. Simulation of the virtual humans in NaturalMotion covers neither conscious control nor cognition. In artifact simulation, Endorphin has the advantage over Jack that it also covers rigid-body kinetics (Figure 5).

By combining kinematical 3D human modeling with FSMs, De Lima Bicho et al. [67] created a hybrid model to simulate human gait patterns. The discrete behavior is represented by a Petri net and the continuous behavior is simulated with inverse dynamics. A similar approach is followed by Laszlo et al. [68], who use FSMs to model low-level human motion patterns. The user only defines high-level actions such as ‘take next step’ while the Petri net decides whether this is a right or a left step and it autonomously controls the corresponding detail motions. Both of the hybrid-model based approaches discussed above focus on control behavior and kinematical behavior. A different FSM-based

2 naturalmotion.com

Figure 5 Simulation of a human falling from a balcony,

created with Endorphin (NaturalMotion Ltd.)


approach is taken by Martins et al. [69]. They developed an animation system called AGA-J that is applied to 3D models of humanoid characters and artifacts built up from solid blocks and joints. Decision-making is simulated based on state transition graphs. The course of the animation varies depends on the fulfillment of conditions, for instance related to starting positions, dimensions or orientations of characters and artifacts. The continuous behavior of humans and artifacts is simulated in a rather primitive way: it is based on sets of 3D animation frames predefined for shape or posture changes. For instance, an animation sequence between the postures ‘standing’ and ‘sitting’ can be defined that takes into account the kinematical constraints. Figure 6 shows an example of a human-product system simulated with AGA-J. The system aims at the entertainment industry.

Those simulations based on autonomous and semi-autonomous 3D volumetric human models that have been developed to include artifacts (as indicated above) allow direct or indirect3 import of CAD models.

3.4. Simulation approaches based on discretized

models of humans Two types of human simulations based on discretized

models are distinguished: (i) simulations based on mesh models and (ii) simulations based on particle models. Simulation outcomes are typically visualized as animations. Prostheses are the only examples we could find in the literature of artifacts that are simulated together with humans concerned (e.g., [70]). Yet, most of the approaches discussed here are also used for artifact-only simulations.

Simulations based on mesh models of humans.

Common approaches for mesh-based modeling for the simulation of physical phenomena are based on the finite-element (FE) and boundary-element (BE) methods.

In biomechanics, where FE-based simulations have been applied from the second half of the 1970s, we can distinguish static and dynamic application of FE models. In the literature, several publications focus on modeling the complex material properties of different tissue types. Based on measurements from in-vivo or in-vitro samples, approximations of the elastic and viscoelastic deformation behavior as first-order, second-order or even third-order have been put forward (e.g., [71]). Chow & Odell [72] apply a static FE model with bilinear elastic tissue properties to predict body deformations of sitting persons. Cheung et al. [73] developed a static model for the prediction of stress in a foot during standing. They assumed

3 For instance, via STL or DXF format

the bones and ligaments to be linearly elastic and the soft tissues to be hyperelastic. Bandak et al. [74] apply a dynamic finite-element calculation to study the deformational effects of axial impact on the human foot, in which the deformation of bones is modeled as linear viscoelastic. Koch et al. [75] include actuator behavior by applying an equation-based model of muscle contraction in their dynamical FE model of facial-expression forming. In this model, tissue deformation is considered linear elastic. A non-linear FE model of muscle contraction and passive muscle stretching using a NURBS-based geometric representation is presented in [76]. Sun et al. [77] simulate perception behavior with an FE model of the human ear, assuming linear viscoelastic behavior of the cochlear fluid and linear elastic behavior of the other ear components.

The BE method is typically used for the simulation of non-mechanical physical behavior of humans: Thiebaut and Lemonnier [78] present an application to human heat transfer. Bradley and Pullan [79] use the BE approach to simulate human electrical-potential behavior. Another mesh-based modeling technique which is used in human-body modeling is skinning [80]. It is similar to the BE approach in that only the surface of the simulated object is meshed. In this case the mesh is solely used to simulate changes in the shape of the surface during posture changes, typically for computer-graphics animation. Deformations are not calculated based on forces and energy but on surface fairing and interpolation techniques [81]. Typically only the skin is modeled as a mesh [82], which is often combined with an internal skeleton model to simulate the kinematical behavior. An example of commercial application is shown in Figure 7. To simulate kinematics, Seo and Magnenat-Thalmann [83] developed a

4 “Skeleton and mesh skinning”, okino.com/conv/skinning.htm

Figure 6 Screen captures from an animation with AGA-J

Figure 7 Combination of a mesh-based deformable skin

model with a kinematical skeleton-like model in a computer game (Okino computer graph-ics, Inc. 4)


skin-mesh model that is combined with volumetric internal elements.

Simulation approaches based on particle models. In

human simulation, particle-based models are typically used to model and simulate mechanical behavior based on mass-spring models with the objective to create animations in computer-graphics applications. Zhang et al. [84] present a three-layer mass-spring model representing the epidermal, dermal and hypodermal layers of the facial skin for dynamic simulation of deformations. Nedel and Thalmann [85] applied mass-spring particle modeling to muscle contraction in a larger-scale model of the human body, combined with skinning and skeleton modeling. The simulation covers kinematics and rigid-body kinetics of the skeleton, deformations and inverse dynamics-based actuator behavior of the muscles, and deformation of the skin based on skinning. Mechanical deformation of soft tissues other than muscles is not taken into account. Similar mass-spring models, mostly applied to the musculature of animals and artificial life forms, are discussed in a survey by Cerezo et al. [86].

4. DISCUSSION OF THE EXAMINED HUMAN

SIMULATION APPROACHES In the preceding subsections a variety of human

simulation approaches has been discussed. Returning to the criteria listed in 1.2, the objective of this subsection is to assess to what extent the various approaches can be deployed for simulation of use processes and/or to what extent they can contribute to a new approach to be developed:

Range of behaviors covered. Table 1 gives an overview

of the types of behavior covered by the analyzed simulation approaches. It has to be noted that this overview is based on what was found in the literature and in descriptions of commercial software. Simulation approaches may exist that cover some of the blanks in the table. It seems that in general the capabilities of the reviewed approaches are scattered among various simulation tools individually covering different limited sets of behaviors, instead of offering a ‘complete picture’ of human (and human-artifact system) behavior. Together, the various hybrid 3D volumetric models cover a wide spectrum but even in this category the capabilities are scattered over different tools and approaches.

3D Discretized model-based approaches offer support for the simulation of a wide range of observed physical behaviors in both humans and artifacts, but they seem to be underrepresented in integral simulations in which human and artifact behavior are investigated together. So far, discretized-model oriented research and development for human applications and for artifactual applications appears to take place in separate scientific communities. For designers, the lack of discretized models reduces the options for integrated simulation with complexly-shaped artifacts and investigation of certain behaviors, such as flexible-body mechanics and non-mechanical behaviors. These are typical application areas for discretized models.

Tools for autonomous simulation that include cognitive behavior together with observable physical behavior are scarce. The only simulation approach that was found to support it to some extent is AGA-J. The approach to model human decision-making as a finite state machine is appealing from the viewpoint of increasing the autonomy, because it allows inclusion of predictive knowledge about what a human will do under given circumstances.

Relevance of the scope. Some simulation approaches,

such as AGA-J and NaturalMotion, have been specifically developed for application in computer graphics, i.e., for the entertainment industry (movies, computer games, etc.). Because the objective of a simulation in this area is typically not to test a virtual product (but, for instance, to produce attractive visual effects), these simulations are not necessarily suitable for application to use processes of novel products.

Ease of preparation. This criterion pertains both to

human models and to artifact models (if the latter can be included). Where human models are concerned, the main preparation task is to instantiate models of individuals with different characteristics. Since the range of simulation approaches discussed varies from proof-of-ideas type academic developments to matured commercial packages, it is difficult to make a fair and quantitative comparison. Therefore the ease of preparing human models has not been investigated. Regarding the preparation of artifact models, we can say that the various approaches based on volumetric 3D models offer the best opportunities to incorporate CAD models that designers typically have available.

Speed and computability. This criterion could not be

evaluated. The fact that some approaches are still under development while others have matured and are commercially available makes it difficult to compare the performance.

Ease of interpretation. Most of the various approaches

based on 3D object models directly produce animations of simulated behavior. Although the output of other approaches can possibly be connected to 3D representations to provide animations, this would require manual efforts.

Fidelity of the outcomes. Here, again, the fact that some

simulation approaches have been developed for computer-graphics animation has to be mentioned: if a simulation produces realistic-looking behaviors, which tends to be sufficient in entertainment, these results may not be useful as input for design decisions.

5. CONCLUSIONS

A large number of computer-based approaches are available to model humans, products and surroundings and to simulate their operation and behaviors. As was shown in the survey of artifact simulation approaches [4], none of the many approaches that can be used to simulate the behavior of products and surroundings is able to offer complete simulation of all the different types of behavior together. Of the


approaches surveyed in this paper, none is capable to simulate human behavior comprehensively – not to mention the behavior of humans and artifacts together. Additionally, application in a design environment requires that behavioral simulation (i) must be applicable to a variety of products, (ii) should be embedded in the computer support for product design and (iii) should produce output that is helpful for designers in terms of improving product concepts.

Since existing simulation approaches appear to offer solutions that cover different patches of the problem area, it seems worthwhile to investigate whether a combination of these approaches can be deployed to make integral simulation of use processes in conceptual product design possible. Combining existing simulation approaches can be considered from a methodological, computational or application viewpoint. Since there are large differences between the reviewed simulation approaches when it comes to the level of elaboration and maturity, the following discussion focuses on the methodological aspects of integration.

Current state of the art. To some extent, fortunately, it is

possible to combine different simulation approaches. In artifact simulation, the two main issues of integration are (i) bringing together the various observed physical behaviors into multiphysics simulation and (ii) combining observed and interpreted physical behavior into hybrid simulation [4]. The highest potential for multiphysics is offered by the various 3D discretized model-based approaches. Solutions for hybrid simulation have also been proposed but these require case-specific models of the observed-physical behavior. Such dedicated models are difficult to embed in a design environment, because they require preparation of a new dedicated model from scratch for every different product.

In human simulation, multiphysics is also a largely unresolved issue However, simulation of non-mechanical physical behavior in use processes is not frequently addressed in the studied literature, and is perhaps not considered to be of great importance. Conversely, simulation of mechanical behavior is a recurring topic in the literature. As is the case with artifacts, simulating mechanical deformation is possible with discretized-model based approaches, while kinematics and rigid-body kinetics is generally simulated with non-discretized object models. There are artifact-simulation approaches that allow integrated investigation of these behaviors by running FE-based simulation on individual system components in parallel to the rigid-body simulation5. In that case, the effects of deformations on the geometry are ignored in the kinematical simulation. In the investigated literature, we could not find examples of this combined approach to human mechanical behavior. A plausible reason is that the role of large deformations is so important in human-body mechanics that they cannot be ignored. A more favored integration approach for mechanics in human simulation is to couple parts that are each simulated in a different way (e.g., muscles are modeled with particles because their deformation is simulated, while the skeleton is modeled as rigid, because its kinematics are simulated [85]) The issue of how to combine large deformations with kinematics and rigid-body kinetics in the simulation of one object (or part of an object) seems to be largely unaddressed. We could not find indications in the literature that including kinematics directly in simulations with discretized models is fundamentally impossible. However, the possibilities seem to be unexplored, which is

5 For instance, MSC visualNastran 4D [87] and CosmosMotion (solidworks.com/Pages/products/cosmos/cosmosmotion.html)

Table 1 Types of behavior covered by human simulation approaches

per

ceptu

al b

ehav

ior

cogn

itiv

e beh

avio

r

consc

ious

contr

ol beh

avio

r

subco

nsc

ious

contr

ol beh

avio

r

actu

ator

beh

avio

r

rigid

-body

kinet

ics

flex

ible

-body

kin

etic

s

kin

emat

ics

non-m

ech

anic

al p

hys

ical

beh

avio

r

rigid

-body

kinet

ics

flex

ible

-body

kin

etic

s

kin

emat

ics

non-m

ech

anic

al p

hys

ical

beh

avio

r

inte

rpre

ted p

hys

ical

beh

avio

r

a a a a a a a a a a a

a a a a a a a a

b b b b b b b b

c c c c

c c c c

d d e

guided 3D volumetric models f e e

a a a a a a a a a

a a a a a a a

Types of behavior covered in the investigated literature

obje

ct-r

epre

senti

ng block diagrams

type of model on which the simulation approach is based

3D schematics

(semi) autonomous3D volumetric modelsdiscretized 3D volumetric models

algebraic descriptions

beh

avio

ral

artifact behaviorhuman behavior

bond graphs

production rules

qualitative algorithms

finite state machines

notes:

a various simulation tools individually covering different sets of behaviors

b possible through combination with 3D volumet-ric models

c discrete-component systems only

d through inverse dynamics

e applies to 3D schematic/volumetric artifact-only models

f field of view only

supportedsupported; discussed in artifact-simulation reviewsupported with limitations (see notes)bypassed in loop of interpreted behaviorunsupported


possibly because the aforementioned two integration approaches with rigid-body simulation are considered adequate in the respective application areas of engineering and computer-graphics animation.

Hybrid simulation of humans is a combination of the simulation of cognitive and/or conscious control behavior with simulation of observed physical behavior of the human. This appears to be less problematic than it is for artifacts, probably because there is more uniformity between human individuals than between products. Therefore, putting effort into the creation of dedicated models is more rewarding. Since the processes involved in human thinking are extremely complex, unpredictable and not fully understood, simulation models or human cognitive processes are based on oversimplifications. Completely deterministic simulations are not to be expected as a goal. In fact, the whole human body and the way it functions are so complex that simulations of most human behaviors, including those of mechanical behaviors, must be based on extreme simplifications. Such simplification is often applied to certain chains or loops of behavior that can be distinguished in the human behavior as a whole, in which energy and/or information is passed on from one body part to another. The simplification is to skip particular behaviors (and corresponding body parts) in the chain. A good example is the chain that is typically active in interactions: external input → perception → cognition → control → actuation → kinetic and kinematical action (cf. Figure 1). This chain can be simplified by skipping perception, control and actuation. Simulation of perception can be skipped by assuming that any external input is detected by the brain. Simulation of both control behavior and actuator behavior can be skipped by assuming that a posture change and force exertion simply will take place as commanded by the brain. In many common situations the simplifications can be considered realistic, for instance if the external input is clearly perceivable and if the necessary mechanical action does not require impossible posture changes or forces.

Researchers and commercial enterprises have produced only a modest number of integrated approaches to simulate behavior of humans and artifacts together. Current integrated approaches only cover selected human behaviors and artifact behaviors. Guided virtual humans still play an important role in integrated simulations. In the context of application in product design, the most promising achievements so far in integrated simulation have been (i) integrated simulation of rigid-body mechanical behavior of humans and artifacts with semi-autonomous volumetric models, and (ii) simulation of human decision-making as an FSM, which enhances semi-autonomous behavior of virtual humans with simple decision-making.

Towards a new approach for simulating product use

processes. The major issue in use-process simulation is to resolve the trade-off between minimizing the efforts of model-making and computation on the one hand, and maximizing the amount of information obtained from simulations for improvement of the product and its use interaction on the other hand. Minimizing the efforts of model-making can be

achieved by reducing the need for creating specialized simulation models, i.e., by using simulation algorithms that can work with the product models used in design (currently CAD models) and with human models that do not have to be created by the designer, but which can be instantiated from a prepared library of human-body models. Minimizing the computation effort can be realized by using simple or simplified models, or by using advanced simulation algorithms. Resolving this issue is not comprehensively addressed in this paper because of its focus on methodological issues. Maximizing the amount of useful information obtained from simulations can be realized by integrating as many as possible aspects that are currently covered by separate modeling and simulation approaches. In the following exploration of opportunities to develop a new approach we have focused on the integration issues, while keeping in mind the modeling effort required for the designer.

In the studied literature and web sources, no attempts have been reported to develop an integrated approach for human-artifact simulation specifically for use-process investigation. Should we want to create a use-process simulation system supporting conceptual design of computer durables based on the current state of the art, the most attractive combination of approaches appears to be: • Simulation of artifacts based on discretized 3D-geometric

models to enable investigation of mechanical behavior, and perhaps additionally one other type of observed physical behavior (if the state of the art in multiphysics simulation allows it).

• Simulation of humans based on volumetric virtual hu-mans similar to the models used in Endorphin or Jack, controlled by decision-making based on an FSM, or some AI-based simulation as is used in ACT-R and EPIC. Consid-eration and modeling of perception, control and actuator behavior is perhaps not needed. The combination brings together the most advanced

simulation approaches for both humans and artifacts, but there will still be limitations in the behavioral simulation capabilities, namely, (i) true multiphysics simulation of artifacts, (ii) simulation of interpreted physical behavior of products (embedded software, electronics) and (iii) combined simulation of large deformations and kinematics in humans would not be possible.

A more fundamental problem lies in bringing together the different types of models into one integrated simulation environment. Combining FSM simulation with continuous simulation – commonly known as hybrid simulation – has been done before, although the combination with the object models commonly used for the continuous simulations discussed in 3.3 and 3.4 (and also CAD-based models) seems to be scarcely explored, as is the application of hybrid simulation to use processes. However, what seems to be more difficult is to combine the different types of geometric models of humans and artifacts, i.e., volumetric and simplified volumetric (discretized). Of course the solution could be to use one of the two representations only, for modeling both humans and artifacts. Exclusive use of non-discretized


models, even for artifacts, would rule out simulation of objects with complex shapes and reduce the variety of observed physical behaviors that can be simulated. Exclusive use of discretized models would call for the development of a discretized model of the complete human body. However, current deployment of discretized models in human simulation appears to be limited to behaviors that are local to a specific area of the human body, and possibly a whole-body model might become too complex and thus too computationally intensive. Another disadvantage is that the investigation of kinematical behavior, which is important in human-product interaction, does not appear to be a strong point of current simulation approaches based on discretized models. Adapting simulation approaches for discretized models so that kinematics can be included and simulated together with large-deformation behavior of soft tissues seems to be an unexplored opportunity to overcome this bottle-neck.

Alternatively, modeling based on a completely new paradigm might open up new opportunities, but what is this paradigm? A novel uniform object modeling approach could bridge the incompatibility between the distinct types of models. A suitable candidate could be nucleus-based modeling [88]. This conceptual-design oriented modeling approach, which is currently at an early stage of development, allows multiphysics simulation (including kinematics) using particle-based discretized geometries and, if needed, it can also incorporate non-discretized modeling elements.

REFERENCES 1. Korn, G.A., Wait, J.V., 1978, Digital continuous system

simulation, Englewood Cliffs: Prentice-Hall, Inc.. 2. Eggert, R.J., 2005, Engineering Design, Upper Saddle

River: Pearson Prentice-Hall. 3. Roozenburg, N.F.M., Eekels, J., 1995, Product design:

fundamentals and methods, Chichester: John Wiley & sons.

4. Van der Vegte, W.F., 2006, "A survey of artifact-simulation approaches from the perspective of application to use processes of consumer durables", Proceedings of the TMCE, Ljubljana SI.

5. Magnenat-Thalmann, N., Thalmann, D., 2004, "An overview of virtual humans", In: Magnenat-Thalmann, N., Thalmann, D. (Eds.), Handbook of virtual humans. Hoboken: Wiley, pp. 1-25.

6. Bobrow, D.G., 1984, "Qualitative reasoning about physical systems: an introduction", Artificial Intelligence, Vol. 24, pp.1-5.

7. Zeigler, B.P., Praehofer, H., Kim, T.G., 2000, Theory of modeling and simulation - integrating discrete event and continuous complex dynamic systems, 2nd edition, San Diego: Academic press.

8. Voet, D., Voet, J.G., Pratt, C.W., 2006, Fundamentals of biochemistry, New York: Wiley.

9. Anonymous, 2004, "Modeling muscle fatigue", In: Abdel-Malek, K. (Ed.), End-of-year technical report for project digital human modeling and virtual reality for future combat systems. Iowa City: University of Iowa.

10. McLeod, D., 2000, The rules of work: a practical engineering guide to ergonomics, New York: Taylor & Francis

11. Strauss, R.H., 1979, Sports medicine and physiology, Philadelphia: Saunders

12. Hui, Y.H., 2006, Food biochemistry and food processing, Ames: Blackwell Publishing

13. Day, R.H., 1969, Human perception, Sydney: Wiley. 14. Robinson-Riegler, G., Robinson-Riegler, B., 2004,

Conitive psychology: applying the science of the mind, Boston: Pearson.

15. Rasmussen, J., 1986, Information processing and human-machine interaction, New York: North-Holland.

16. Oatis, C.A., 2004, Kinesiology - the mechanics & pathomechanics of human movement, Philadelphia: Lippincott Williams & Wilkins.

17. Silver, F.H., 1987, Biological materials: structure, mechanical properties, and modeling of soft tissues, New York : New York University Press

18. Huston, R.L., Passarello, C.E., 1982, "The mechanics of human body motion (due to impulsive forces during occupational manoeuvres and jumping)", In: Ghista, D.N. (Ed.): Human body dynamics: impact, occupational and athletic aspects. Oxford: Clarendon Press.

19. Hughes, R.E., An, K-N., 1999, "Biomechanical models of the hand, wrist and elbow in ergonomics", In: Kumar, S. (Ed.): Biomechanics in ergonomics, pp. 179-198. London: Taylor & Francis Ltd..

20. Multon, F., Nougaret, J.-L., Arnaldi, B., Hégron, G., Millet, L., 1999, "A software system to carry out virtual experiments on human motion", Proceedings of IEEE Computer Animation 1999, pp. 16-23.

21. Griffin, M.J., 1990, Human vibration, London: Academic Press.

22. Bourbon, W.T., Powers, W.T., 1999, "Models and their worlds", Int. J. Human-Computer Studies, Vol. 50, pp. 445-461.

23. Job, H.M., Keating, D., Evans, A.L., Parks, S., 1999, "Three-dimensional electromagnetic model of the human eye: advances towards the optimisation of electroretinographic signal detection", Medical & Biological Engineering & Computing Vol. 37, pp. 710-719.

24. Mochnacki, B., Majchrzak, E., 2003, "Sensitivity of skin tissue on the activity of external heat sources", Computer Modeling in Engineering and Sciences, Vol. 4, pp. 431-438.

25. Panescu, D., Webster, J.G., Stratbucker, R.A., 1994, "A nonlinear electrical-thermal model of the skin", IEEE Transactions on Biomedical Engineering, Vol. 41, No.7, July 1994.

26. Therrien, R.G., Bourassa, P.A., 1982, "Mechanics application to sports equipment: protective helmets, hockey sticks, and jogging shoes", In: Ghista, D.N. (Ed.): Human body dynamics: impact, occupational and athletic aspects. Oxford: Clarendon Press.


27. Šuc, D., Bratko, I., 2000, " Qualitative trees applied to bicycle riding", Linköping Electronic Transactions on Artificial Intelligence, Vol. 4, Section B, pp. 125-140.

28. Lucas, P. (Ed.), 2003, Model-based and Qualitative Reasoning in Biomedicine, Working notes of the workshop held during The 9th European Conference on Artificial Intelligence in Medicine, AIME'03 Protaras, Cyprus.

29. Heitmeyer, C., Mandrioli, D., 1996, Formal methods for real-time computing: an overview, Chichester: John Wiley & sons.

30. Fogel, L.J., Moore, R.A., 1968, Modeling the human operator with finite-state machines, Washington DC: National Aeronautics and Space Administration (NASA contractor report No. 1112).

31. Rauterberg, M., Fjeld, M., Schluep, S., 1997, "Goal setting mechanism in Petri net models of human decision making", Proceedings of the IEEE Int. Conference on Systems, Man and Cybernetics, Orlando FL, pp. 2696-2701.

32. Fjeld, M., Schluep, S., Rauterberg, M., 1999, "Action-driven quantification of task-solving behaviour", In: Harris, D. (Ed.), Engineering psychology and cognitive ergonomics, Vol. 4., pp. 253-261. Hampshire: Ashgate.

33. Liu, A., Salvucci, D., 2001, "Modeling and prediction of human driver behavior", Proceedings of the 9th Int. Conference on Human-Computer Interaction, Los Angeles CA, pp. 1479-14839.

34. Eysenck, M.W., Keane, M.T., 2000, Cognitive Psychology (4th Edition), Hove, UK: Psychology Press.

35. Lenat, D.B., Brown, J.S., 1984, "Why AM and EURISKO appear to work", Artificial Intelligence, Vol. 23, pp. 269-294.

36. Laird, J.E., Newell, A., Rosenbloom, P.S., 1987, "SOAR: an architecture for general intelligence", Artificial Intelligence, Vol. 33, pp. 1-64.

37. Anderson, J.R., Bothell, D., Byrne, M.D., Lebiere, C., 2004, "An integrated theory of the mind", Psychological Review, Vol. 111, Vo. 4, pp. 1036-1060.

38. Kieras, D.E., Wood, S.D., Meyer, D.E., 1997, "Predictive engineering models based on the EPIC architecture", ACM Transactions on Human-Computer Interaction, Vol. 4, No. 3, pp. 230-275.

39. Alexík, M., 2000, "Modelling and identification of eye-hand dynamics", Simulation Practice and Theory, Vol. 8, pp. 25-38.

40. McRuer, D., 1980, "Human dynamics in man-machine systems", Automatica, Vol. 16, pp. 237-253.

41. Karayanakis, N.M., 1995, Advanced system modelling and simulation with block diagram languages, Boca Raton: CRC Press.

42. Gurney, K., 1997, An introduction to neural networks, London: UCL Press.

43. Martens, D., 1998, "Neural networks as a tool for the assessment of human pilot behaviour in wind shear", Aerospace science and technology, 199, no. 1, pp. 39-48.

44. Kim, J., Hemami, H., 1998, "Coordinated three-dimensional motion of the head and torso by dynamic

neural networks", IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol. 28, No. 5, pp. 653-666.

45. Paynter, H.M., 1961, Analysis and design of engineering systems, Cambridge, MA: The MIT Press.

46. Margolis, D.L., 1975, "An introduction to bond-graph techniques for biomedical system modeling", Simulation, Vol. 44, pp. 22–26.

47. Wojcik, L.A., 2003, "Modeling of musculoskeletal structure and function using a modular bond graph approach", Journal of the Franklin Institute, Vol. 340, pp. 63-76.

48. Pop, C., Khajepour, A., Huisson, J.P., Patla, A.E., 1999, "Application of Bond graphs to Human Locomotion Modeling", Proceeding of the HKK Conference and Symposium, University of Waterloo, Canada June 13-16, pp. 85-90.

49. Hubbard, M, 1980, "Dynamics of the pole vault", Journal of Biomechanics, Vol. 13, pp. 965-976.

50. Horváth, I., Zhang, B., Moes, C.C.M., Molenbroek, J.M., 2005, "Status of digital human body modeling from the aspect of information inclusion", Proceedings of ASME IDETC/CIE, Long Beach CA.

51. Zhang, B., 2005, Using artificial neural networks for prediction of human body postures based on 3D landmarks, Delft: Delft University of Technology; dissertation.

52. Jung, E.S., Choe, J., 1996, "Human reach posture prediction based on psychophysical discomfort", International Journal of Industrial Ergonomics, Vol. 18, pp. 173-179.

53. Webb, J.A., Aggarwal, J.K., 1982, "Structure from motion of rigid and jointed objects.", Artificial Intelligence, Vol. 19, pp.107–130.

54. Park, J., Fussell, D.S., 1997, "Forward dynamics based realistic animation of rigid bodies", Computers and Graphics, vol. 21, No. 4, pp. 483-496).

55. Boulic, R., Baerlocher, P., 2004, "Body motion control", In: Magnenat-Thalmann, N., Thalmann, D. (Eds.), Handbook of virtual humans. Hoboken: Wiley, pp. 99-118.

56. Yang, F., Ding, L., Yang, C., Yan, X., 2005, "An algorihm for simulating human arm movement considering the comfort level", Simulation Modelling Practice and Theory, Vol. 13, pp. 437-449.

57. Subic, A.J., Preston, S.B., 1996, "Symbolic dynamics for motion analysis in sports", Proceedings of the first International Conference on the Engineering of Sport, Sheffield.

58. Kim, H.J., Horn, E., Xiang, Y., Abdel Malek, K., Arora, J.S.2004, "Dynamic motion prediction of gait and lifting", In: Abdel-Malek, K. (Ed.), End-of-year technical report for project digital human modeling and virtual reality for future combat systems. Iowa City: University of Iowa.

59. Sanders, M.S., McCormick, E.J., 1992, Human factors in Engineering and Design, Singapore: McGraw Hill.

60. Feyen, R., Liu, L., Chaffin, D., Jimmerson, G., Joseph, B., 2000, "Computer-aided ergonomics: a case study of


incorporating ergonomics analyses into workplace design", Applied Ergonomics, Vol. 31, pp. 291-300.

61. De Camargo, J.F., Magalhães, L.P., Raposo, A.B., 1995, "Local and global control in computer animation", Proceedings of IEEE SIBGRAPI, pp. 151-157.

62. Bordeux, C., Boulic, R., Thalmann, D., 1999, "An efficient and flexible perception pipeline for autonomous agents", Eurographics, Vol. 18, No. 3.

63. Jung, M-R., Badler, N., Noma, T., 1994, "Animated human agents with motion planning capability for 3D-space postural goals", Journal of Visualization and Computer Animation, Vol. 8, pp. 225-246 .

64. Badler, N.I., Phillips, C.B., Webber, B.W., 1993, "Simulating humans - computer graphic animations and control", New York: Oxford University Press.

65. Reil, T., Husbands, P., 2002, "Evolution of central pattern generators for bipedal walking in a real-time physics environment", IEEE Transactions on Evolutionary Computation, Vol. 6, No. 2, pp. 159-168.

66. Paulson, L.D., 2003, "Researchers automate the digital animation process", Computer, Vol.36, No. 4, p. 24.

67. De Lima Bicho, A., Raposo, A.B., Magalhães, L.P., 2001, "Control of articulated figures animations using Petri nets", Proceedings of IEEE SIBGRAPI, pp. 200-207.

68. Laszlo, J., Van de Panne, M., Fiume, E., 2000, "Interactive control for physically-based animation", proceedings of SIGGRAPH, New Orleans, pp. 201-208.

69. Martins, J.D.F., Nedel, L.P., Menezes, P.F.B., Accorsi, F., 2004, "An automata-based animation engine to control articulated characters", Proceedings of SBC Symposium on Virtual Reality, São Paulo, pp. 39-50.

70. Chu, T.-M., Reddy, N.P., Padovan, J., 1995, "Three-dimensional finite element stress analysis of the polypropylene ankle-foot orthosis: static analysis", Medical Engineering & Physics, Vol. 17, pp. 372-379

71. Les, C.M., Keyak, J.H., Stover, S.M., Taylor, K.T., 1997, "Development and validation of a series of three-dimensional finite element models of the equine metacarpus", Journal of Biomechanics, Vol. 7, pp. 737-742.

72. Chow, W.W., Odell, E.I., 1977, "Deformations and stresses in soft body tissues of a sitting person", Journal of Biomechanical Engineering, May 1978, Vol. 100, pp. 79-87.

73. Cheung, J. T.-M., Zhang, M., Leung, A. K.-L., Fan, Y.-B., 2005, "Three-dimensional finite element analysis of the foot during standing - a material sensitivity study", Journal of Biomechanics, Vol. 38, pp. 1045-1054.

74. Bandak, F.A., Tannous, R.E., Toridis, T., 2001, "On the development of an osseo-ligamentous finite element model of the human ankle joint", Int. Journal of Solids and Structures 38, pp. 1681-1697.

75. Koch, R.M., Gross, M.H., Bosshard, A.A., 1998, "Emotion editing using finite elements", Eurographics, Vol. 17, No. 3, pp. 295-302.

76. Lu,J., Zhou, X.2004, "Skeletal muscle modeling using a NURBS-based finite element method", In: Abdel-Malek, K. (Ed.), End-of-year technical report for project digital

human modeling and virtual reality for future combat systems. Iowa City: University of Iowa.

77. Sun, Q., Gan, R.Z., Chang, K.-H., Dormer, K.J., 2002, "Computer-integrated finite element modeling of human middle-ear", Biomechanics and Modeling in Mechanobiology, Vol. 4, pp. 190-199.

78. Thiebaut, C., Lemonnier, D., 2002, "Three-dimensional modelling and optimisation of thermal fields in a human body during hypothermia", International Journal of Thermal Sciences, Vol. 41, pp. 500-508.

79. Bradley, C., Pullan, A., 2002, "Application of the BEM in biopotential problems", Engineering Analysis with Boundary Elements, Vol. 26, No. 5 ,pp. 391-403.

80. Aubel, A., 2004, "Body deformations", In: Magnenat-Thalmann, N., Thalmann, D. (Eds.), Handbook of virtual humans. Hoboken: Wiley, pp. 140-160.

81. Lewis, J.P., Cordner, M., Fong, N., 2000, "Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation", Proceedings of SIGGRAPH, pp. 165-172.

82. De Rose, D., Kass, M., Truong, T.G., 2000, Texture mapping and other uses of scalar fields on subdivision surfaces in computer graphics and animation, United States Patent No. 6,037,949.

83. Seo, H., Magnenat-Thalmann, N., 2004, "An Example-Based Approach to Human Body Manipulation", Graphical Models, Vol. 66, No. 1, pp. 1-23,.

84. Zhang, Y., Prakash, E.C., Sung, E., 2004, "Face alive", Journal of Visual Languages and Computing, Vol 15, pp. 125–160.

85. Nedel, L.P., Thalmann, D., 2000, "Anatomic modeling of deformable human bodies", The Visual Computer Vol. 16, pp. 306-321.

86. Cerezo, E., Pina, A., Serón, F.J., 1999, "Motion and behaviour modelling: state of the art and new trends", The Visual Computer Vol. 15, pp. 124-146.

87. Wang, S.-L., 2001, "Motion simulation with working model 2D and MSC.visualNastran 4D"; Journal of Computing and Information Science in Engineering, Vol. 1, pp. 193-196.

88. Horváth, I., 2004, "On some crucial issues of computer support of conceptual design: what to consider in order to be successful", In:Talabă, D., Roche, T. (Eds.), Product Engineering: eco-design, technologies and green energy. Dordrecht: Springer.

Documents

ADMISSION INFORMATION AND PROCESS - Laramie County Community