5Architecture of Virtual Reality Systems
The creation of VR systems to support virtual environments (VE) is a chal-lenging problem requiring diverse areas of expertise, ranging from networksto psychology. Developing VEs is a very expensive task in terms of time andnancial and human resources. VEs can be applied in a broad range of areas,such as scientic visualization, socializing, training, psychological therapy, andgaming (for more details, see the Applications part of this book). Such adiversity of applications produces a set of requirements that make it very dif-cult, if not impossible, to build a single system that ts all needs. The resulthas been the creation of monolithic systems that are highly optimized to aparticular application, with very limited reusability of components for otherpurposes.
According to Oliveira et al. , the problem of lack of reusability isdue to the current trend in the VE community: developing a new VE systemfor each dierent application. The reinventing the wheel and not inventedhere syndromes limit the innovation and delay the use of VEs in wider areasfor the general public.
Monolithic systems such as DIVE , MASSIVE , NPSNET ,SPLINE , and dVS/dVISE , among others, proliferated in the pastdue to the lack of system exibility for a particular application . Theintroduction of more modular architectures led to the emergence of toolkitssuch as WorldToolkit , Avocado , VR Juggler , VHD++ ,and Virtools (http://www.virtools.com). These software suites have dierentdegrees of exibility. Frameworks like VHD++ dier from others due to itsspecialized skills in a particular domain, such as virtual humans simulationtechnologies. All of them are based on a hierarchical representation of thevirtual environment: a scene graph.
108 5 Architecture of Virtual Reality Systems
5.1 Scene Graph-Based Systems
A scene graph is an abstract logical access structure, a tree. It is used torepresent objects comprising the environment (scene data) and the relation-ships between them. The scene graph consists of parent nodes, child nodes,and data objects. The parent nodes, also called group nodes, organize and, insome cases, control how to interpret their descendants. Group nodes serve asthe glue that holds a scene graph together. Child nodes can be either groupnodes or leaf nodes. Leaf nodes have no children. They encode the core seman-tic elements of a scene graph, for example, what to draw (geometry), whatto play (audio), how to illuminate objects (lights), or what code to execute(behavior). Leaf nodes refer to data objects. These objects are not scene graphnodes, but they contain the data that leaf nodes require, such as the geometryto draw or the sound sample to play.
Scene graphs should not be confused with data structures used to do vis-ibility culling or collision queries such as octrees, BSP, ABT, KdT. Scenegraphs are used to connect game rules, physics, animation, and AI systems tothe graphics engine.
Popular implementations of scene graph programming interfaces (APIs)include: Cosmo3D (SGI), Vega Scene Graph,1 Java3D,2 OpenSceneGraph,3
and OpenGL Performer4. All of them were designed for creating real-timevisual simulations and other performance-oriented 3D graphics applications.
OpenGL Performer is a commercial toolkit that evolved from OpenInventor, which is corrently an open-source project.5 Open Inventor is consid-ered the archetypical example of a scene graph library. It presents an object-oriented programming model based on a 3D scene database (scene graph) thatprovides a higher layer of programming for OpenGL.
Java 3D gained popularity as the main scene graph-based API for develop-ing 3D applications with Java. It is frequently used for developing Web-basedapplications enhanced with real-time 3D graphics ,,. It is also avery representative example of a scene graph-based 3D toolkit.
Scene graph implementations are based on a hierarchical spatial represen-tation of the objects in the scene, for example, a terrain contains a house, andinside the house there is a person holding a hammer in her right hand.
Usually, the semantic information that is encoded in the scene graph cor-responds mainly to visualization aspects, geometry to draw, and associatedeects such as a sound sample. Only elemental relationships between objectscan be specied, for example, smart objects  (simple tools and objectssuch as a hammer or a drawer) that contain information describing how they
1 http://www.presagis.com2 http://java3d.dev.java.net3 http://www.openscenegraph.org4 http://www.sgi.com/products/software/performer5 http://oss.sgi.com/projects/inventor
5.2 Semantic Virtual Environments 109
can be grasped by a virtual human and a predetermined behavior to perform(see Section 4.4.1).
Eorts aimed at enhancing the adaptability and reusability of VE applica-tions and entities within them have focused on designing software componentframeworks to manage the resources and building blocks of a VE system.Such is the case of the Java Adaptive Dynamic Environment (JADE) ,which permits dynamic runtime management of all components and resourcesof a VE system. While this kind of initiative is successful in terms of pro-viding reusability, and interoperability at the level of the code source, theydo not address the fundamental problem of reusing the virtual entities thatparticipate in the VE application. The use of scene graphs as hierarchical spa-tial representations is not questioned. As a result, source code implementinganimation, visualization, and interaction algorithms can be reused to someextent. But the knowledge associated with a virtual entity remains dicultto reuse. In  Gutierrez proposed the semantic virtual environments ap-proach, which considers virtual entities as complex items with dierent typesof data and knowledge associated with them.
5.2 Semantic Virtual Environments
The main idea behind semantic virtual environments (SVE)  is that ahigher-level semantic representation of virtual environments can enhance thereusability and adaptation capabilities of the virtual entities participatingin a VE. The semantics-based representation that is proposed builds on theconcept of a scene graph, with the dierence that it does not focus on visualor spatial relationships between entities but on higher-level semantics.
The SVE approach is based on the semantics of the virtual entities and noton their geometry, as is the case in most of the VE applications. A semanticmodel provides a way to specify alternative geometric representations for eachentity and a range of functionalities. The entities are called digital items andcan be not only virtual objects to be rendered as part of the scene, but alsoindependent user interfaces and controls for animation or interaction.
A semantic model of a virtual environment can be dened keeping inmind that the main attribute of a virtual entity (objects or characters) is its3D shape. Virtual environments are geometry-based applications. However,depending on the context, the shape of the entity can change. Moreover,there are contexts in which multiple shapes representing the same virtualentity must be manipulated and synchronized. For instance, when displayinga virtual character on a large projection screen, we require information forrendering a high-resolution (hi-res) 3D shape. Nevertheless, the hi-res shapecould be controlled by the user interactions performed through a dierentrepresentation, either a simplied shape or an abstract manipulator: a menu,a slider, or any other GUI control. Using independent representations foreach case is costly in terms of data synchronization. The semantic model
110 5 Architecture of Virtual Reality Systems
encapsulates the set of geometric and/or abstract representations belongingto each virtual entity and associates descriptors to inform the rendering andinterface systems about how to handle each object.
Figure 5.1 shows a general class diagram of the semantic model for the VEand the tools that can be used to control and interact with it.
Fig. 5.1: Semantic representation of an interactive virtual environment
The scene is the main container of the VE model; it has references to thedigital items contained in the VE. It is the data repository used by digitalitems and other software tools such as viewers or interaction devices.
A semantic descriptor provides human- and machine-readable informationabout a particular digital item (we also call them virtual entities). They arethe entry points for the scene controller to choose the correct geometry andinterface to present. The semantic descriptor is the placeholder for any infor-mation describing how the digital item is to be used and how it is related toother items in the scene.
The geometric descriptor of a digital item species the type of shape as-sociated with the entity: a deformable mesh, an articulated body (joints andsegments), and the like. For instance, hierarchical structures for skeleton-basedanimation can be dened by the geometric descriptors, such as H-Anim,6 thestandard specication of a hierarchical structure for humanoid character an-imation. Moreover, alternative skeletons with dierent levels of detail on thenumber of joints and/or the complexity of the segments (3D shapes associatedwith each joint) can be specied as well.
Handling alternative geometric representations for each virtual entity inthe environment is not enough to provide a complete representation of a vir-tual world. The model reects also the relations between entities. The semantic
5.3 Generic System Architecture for VR Systems 111
descriptors characterize each object or character in the scene and