Open Wonderland: An Extensible Virtual World Architecture

Virt

ual W

orld

Arc

hite

ctur

es

38 Published by the IEEE Computer Society 1089-7801/11/$26.00 © 2011 IEEE IEEE INTERNET COMPUTING

V irtual world technology is on the verge of a phase change from an interesting experiment to a large-

scale phenomena. Although today’s most popular virtual worlds such as Second Life (http://secondlife.com) and Active Worlds (http://activeworlds.com) have fallen short of expectations for collaboration and education, we predict that in the future, most Internet sites will engage visitors with 3D experi-ences. We base this belief on factors such as broadband pervasiveness, the advent of voice over IP (VoIP) for home users, and the popularity of massively multiplayer online games, which dem-onstrate the power of real-time collab-oration in 3D environments.

Open Wonder land (ht t p:// openwonderland.org), an open source toolkit for creating 3D virtual worlds, along with a few other systems such as Open Croquet (http://opencroquet.org) and

OpenSimulator (http://opensimulator.org), represent a new genre of virtual world technology that has the potential for large-scale deployment in which orga-nizations will host their own virtual worlds that will be federated together into an enhanced 3D Web. Open Wonder-land follows a large body of work on collaborative virtual environments, starting with research systems from the early 1990s such as Diamond Park1 and the Distributed Interactive Virtual Environment (DIVE).2

The Open Wonderland architecture defines a common foundation for build-ing a diverse ecosystem of such worlds, each with different features and capa-bilities. The Open Wonderland project, which began at Sun Microsystems in 2007 as Project Wonderland, has been completely community-driven since January 2010. Although the initial moti-vation for creating the toolkit was to

Open Wonderland is a toolkit for building 3D virtual worlds. The system

architecture, based entirely on open standards, is highly modular and designed

with a focus on extensibility. In this article, the authors articulate design goals

related to collaboration, extensibility, and federation and describe the Open

Wonderland architecture, including the design of the server, the client, the

communications layer, and the extensibility mechanisms. They also discuss the

trade-offs made in implementing the architecture.

Jonathan Kaplan and Nicole YankelovichOpen Wonderland Foundation

Open Wonderland: An Extensible Virtual World Architecture

IC-15-05-Yank.indd 38 8/18/11 12:38 PM


SEPTEMBER/OCTOBER 2011 39

support business collaboration, the project’s mis-sion quickly broadened to encompass education, training, simulation, and visualization. Here, we examine Open Wonderland’s architecture and design.

Design GoalsIn designing the Open Wonderland architecture, we had three main goals: enabling collabora-tion with a focus on synchronous interaction, providing an extensible toolkit based on open standards, and putting in place the infrastruc-ture for federation to enable the 3D Web.

CollaborationOur goal with regard to collaboration was to enable all the types of synchronous collabora-tion possible with Web-based tools while add-ing the benefits inherent to 3D interaction. In particular, we wanted to support informal collaboration. Many of the same features that sup port formal collaboration, such as immer-sive audio, also apply to informal interaction. One important advantage of a 3D space is that it provides an intuitive way to organize multiple, simultaneous conversations, something not pos-sible with current audio- or video-conferencing technology.

Immersive audio coupled with the visual 3D context also enhances collaboration by pro-viding a strong sense of other people’s pres-ence in the virtual world.3 As we know from our research,4 audio is perhaps the single most important factor in successful remote collabora-tion. Given this, we aimed to create an architec-ture that treats high-fidelity, immersive audio as a core toolkit component.

We made it a design priority to support real work activities with both legacy applications and collaboration-aware applications designed specifically for multiple users. If an application is in the world, it is shared, unless a user speci-fies otherwise. To make sharing as seamless as possible, we wanted to enable users to drag-and-drop content and automatically launch the correct application to display that type of data in the world.

Lastly, we wanted to provide enterprise-class security and authentication. For business and education applications, users must know people’s identity. It is also important to secure objects in the world so that unauthorized users can’t change important documents, delete crucial

data, or otherwise disrupt the real work taking place in the virtual world.

ExtensibilityOur goals for collaboration led us to focus the technical design on extensibility. While we could identify certain features — such as audio — that were relevant to all collaborations, making the environment useful for real work required that it be customizable for different tasks. Each use case we looked at benefited from new interactive applications, visualizations, and integration with different data (see Figure 1). By building an extensible toolkit rather than a fixed-feature environment, we aimed to let developers quickly build highly customized worlds with task-specific applications.

To enable this broad range of extensions, we focused on a modular architecture based on open source Java components. We structured the project with a small set of core services that manage the 3D world, including authentica-tion, networking, content management, and cli-ent rendering. Beyond these core services, we implemented most of the features in modules.

Our extensive use of modules to implement core features — including avatars, audio, and shared applications — meant that we needed a comprehensive set of extension points. We knew we would require extension at many dif-ferent levels, from adding new menus in the

Figure 1. Example extensions. By building an extensible toolkit rather than a fixed-feature environment, we aimed to let developers quickly build highly customized worlds with task-specific applications.

Virtual piano

Animated code editor CMU Alice integration Marble rollercoaster

MIT TEALSim physics Hospital privacy screen

IC-15-05-Yank.indd 39 8/18/11 12:38 PM

Virtual World Architectures

40 www.computer.org/internet/ IEEE INTERNET COMPUTING

client to pluggable authentication mechanisms in the server to integrating new services such as Extensible Messaging and Presence Protocol (XMPP; http://xmpp.org) chat servers.

Our final extensibility goal was to enable integration with external data. We started by choosing a set of well-supported open stan-dards, including Collada (Collaborative Design Activity; http://collada.org) for graphics and the Session Initiation Protocol (SIP; http://ietf.org/rfc/rfc3261.txt) for audio. We also wanted to make sure that developers could integrate data from other sources — for instance, from open Web services to proprietary databases. It was especially important that developers be able to use existing Java libraries to access these services.

FederationOur long-term goal for the Wonderland toolkit is to enable a new type of 3D Web. We imag-ine a set of loosely connected servers — like the World Wide Web — each presenting worlds

with different purposes, features, and code. Cli-ent browsers will let users easily move between servers, downloading both content (3D artwork) and behavior (mobile Java code). Unlike the Web, these worlds’ focus will be on synchro-nous communication, and as such, they’ll need richer, more extensible programming interfaces and network protocols, which can handle 3D visualization, rich presence information, real-time application sharing, and full multimedia collaboration.

Wonderland ArchitectureFigure 2 shows Open Wonderland’s var i-ous components and how they communicate. Wonder land uses a client–server model to create collaborative virtual worlds. In practical terms, a world is a virtual space with its own 3D coordi-nate system that clients can connect to in order to collaborate. Wonderland is written entirely in the Java programming language. The cli-ent provides a browser that turns these shared services into a 3D view of the environment.

Figure 2. Open Wonderland network diagram. We show communication between the system components. The Wonderland client communicates via HTTP with the Web server. Using a number of task-specific protocols, the client communicates with other services including the game server and voice bridge.

Web server

Modulemanager

Singlesign-on

Assetstorage

RESTful Web service APIs

Servicemanager

Web-based management

Service nodes

Darkstarserver

Web administration

Sharedapp server

Voicebridge

Control channels (TCP)

HTTP

Wonderland client

MT game

Communications

JMonkeyEngine

CellAvatars HUD

AudioSecurityDnD

Input/events Collision Physics

Rendering

Coreservices

Networking

HTTPDarkstar

(TCP)App data

(TCP)SIP/RTP(UDP)

IC-15-05-Yank.indd 40 8/18/11 12:38 PM



This includes rendering graphics, downloading and caching content, responding to user inter-actions, and reacting to server messages.

The client and server communicate using several networking protocols optimized for dif-ferent data types:

• Web services for authentication, download-ing code, and world assets such as 3D mod-els and textures;

• custom TCP-based protocols for communi-cating world data such as object properties and position;

• SIP and RTP for audio; and• multimedia streaming protocols for video,

application sharing, and screen sharing.

Using multiple communications channels allows each protocol to be optimized for the type of data being sent between the client and the server.

Server ComponentsThe Wonderland server is based on a set of four cooperating services. Each service is a separate Java application with its own networking and storage mechanisms. Designing these as sepa-rate services enables increased flexibility and scalability: typically, we deploy all services on a single machine, but Wonderland admin-istrators can spread services across multiple machines to increase scalability.

The Web administration server is the main coordination point for the various services. This server is based on the open source Glass-fish Java EE Application server (http://glassfish.java.net). The core Wonderland features such as authentication and asset management are implemented as Java EE Web services. The Web server acts as a central management console, providing Web-based management of all ser-vices in the system, regardless of which server they are running on. Another important service is a token-based single sign-on mechanism. After users authenticate to the Web server, they receive a token that they can give to other services. Those services then use the token to authenticate the client when it connects over different channels.

The Darkstar server is based on the Project Darkstar technology, also developed at Sun. (Project Darkstar has subsequently become a community project known as RedDwarf Server

at http://reddwarfserver.org). Darkstar pro-vides a server platform specifically designed for online games, including “serious games” such as the Wonderland environment. Unlike a Web server, it is optimized for low latency rather than high throughput. The Darkstar server divides all actions into short tasks that it executes within a transaction. It immediately writes out the results to an internal database, guaranteeing that no state is lost even during server crashes. Wonderland uses the Darkstar server to track the frequently updated state of live objects in the world. This includes prop-erties such as the location for each object and avatar. Darkstar also provides an abstract com-munication mechanism, allowing a client to send simple messages to the server and the server to send messages to any subset of clients connected to that same server.

JVoiceBridge (http://tinyurl.com/jvoicebridge) is a pure Java audio-mixing application that provides server-side mixing of high-fidelity, immersive audio. It runs as a separate Wonder-land server that mixes SIP audio for multiple users, based on where in the virtual space they are. Objects in the world, such as microphones and cones of silence, can also affect audio. JVoiceBridge communicates directly with the Darkstar server over a private channel to keep all the audio in sync with the world’s state as users move around or are added and removed.

The shared application server (SAS) is the final standard server component. The SAS runs on Linux or Solaris systems to allow server-hosted application sharing (see Figure 3). In this model, an unmodified X Windows application, such as Firefox or Open Office, runs inside a custom X Windows server. This server broad-casts application updates in the form of images to each Wonderland client with an avatar in the application’s range. Clients reconstruct these images into a local view of the application that users can see and interact with. These legacy applications are designed for a single user, so a control-passing system ensures that only one user makes changes to the application at a time. This is necessary only for legacy applications. Multiuser collaboration-aware applications written specifically for Wonderland run locally on each client and send change events through the Darkstar server, allowing multiple users to interact simultaneously while using minimal bandwidth.

IC-15-05-Yank.indd 41 8/18/11 12:38 PM



Client DesignThe Wonderland client is a single application that acts as a browser for connecting to differ-ent Wonderland servers. As with the server, the client provides several core services based on existing open source components.

The client’s rendering layer consists of two separate projects. JMonkeyEngine (http:// jmonkeyengine.com) is a popular rendering framework for writing OpenGL-based applica-tions in Java. It provides the basic scene graph

and rendering framework but is limited to working on a single thread at a time. MT Game is a subproject of Open Wonderland that adds multiprocessor capabilities to JMonkeyEngine by breaking computation into separate process-ing and rendering phases.

The core services layer provides the features that the Wonderland modules use. These services include the position of objects in the 3D world, the ability to move objects, and collision detection. Extended core services, such as the ability to load models, calculate real physics, and enforce secu-rity, are layered on top of the core as modules.

CommunicationThe Wonderland client’s communications layer is implemented in a combination of built-in Wonder-land features and module extensions. The built-in features support authenticating to the Web server and communicating with the Darkstar server. Other communications, such as audio and shared application channels, are specified in modules. This demonstrates the toolkit’s ability to support new network protocols entirely in modules.

Wonderland ExtensibilityThe Open Wonderland toolkit provides the framework for building a collaborative 3D envi-ronment, but extensions create the world the user sees. To enable this extensibility, we cre-ated a core modular architecture with several well-defined extension points. We also designed mechanisms for integrating with external data.

Extension PointsThe Wonderland toolkit provides developers with a number of standard extension points and patterns. New object types are the most common type of extension. An object in the 3D world is referred to in the Wonderland code as a cell (because the word “object” is already used in most programming languages). A cell is simply a volume of 3D space, and any cell can contain other cells to form a cell tree.

Each cell in Wonderland is an independent Java object that can have both client and server behavior. Examples of client behavior include rendering a 2D or 3D object, reacting to user input, or sending and receiving messages from the server. Examples of server behavior include storing persistent properties, receiving mes-sages from clients, and sending messages to groups of clients. Figure 4 shows a Wonderland

Figure 3. Sharing applications. The Open Wonderland platform supports both legacy 2D X11 applications and 2D and 3D Java applications written specifically for multiple users.

2D shared XII applications

2D and 3D collaboration-aware apps

NetBeans Firefox

Sticky notes Whiteboard Audio recorder

Figure 4. A world divided into cells. A cell is a volume of 3D space. Any cell can contain other cells to form a cell tree.

Cell treeRoom cell

App cell

Avatar cell

Bed cell

WorldRootcell

IC-15-05-Yank.indd 42 8/18/11 12:38 PM



world represented as a cell tree. Note how each object in the world, including the room, the 2D application, and the avatars, are all variations of the basic cell. Cells have a well-defined life cycle that includes the ability to save them as XML for long-term storage.

Another important extension point is a capability, or a feature that can be dynamically added to any cell. Example capabilities include a placemark, which adds an item to users’ placemark menu so they can jump to a par-ticular cell, and a clickable link, which opens a Web browser to a particular page whenever a user clicks on an object. When building a world, users can add capabilities to any cell to augment its functionality. A capability has the same life cycle as a cell and is almost identi-cal except that each instance of a capability is associated with a particular cell.

Both cells and capabilities relate to items that have a particular location in the world. Developers can add other extensions that aren’t spatial in nature via plug-ins, which are avail-able to users no matter where they are in the world. Thus, they’re useful for features such as text chat and inventory that must always be available. Like cells, plug-ins can have func-tionality in both the client and server, so the client plug-in can send messages that the server plug-in must process. The server plug-in can also save its state in persistent storage.

Plug-ins might also use custom connec-tions. A connection is a particular data chan-nel between any number of clients to the server. The connection’s type defines the format of the data the plug-in will send over the channel. Custom connections are useful for adding new data channels for features such as text chat or administrator tools. Developers can also employ custom connections to connect to special- purpose applications other than the Wonderland client to form a bridge.

The last major extension point is the ability to add custom Web applications. This lets develop-ers add functionality to the Web administration user interface or entire new Web services. These extensions are provided as standard Java EE applications that are deployed to the Wonder-land Web server.

Module SystemWonderland modules are the mechanism for pack-aging extensions, including objects, capabilities,

plug-ins, connections, and Web applications. A module is a specially formatted Java archive (JAR) file. In addition to the standard JAR attri-butes, a module contains metadata including the module name, version number, and depen-dencies on other modules.

The bulk of a Wonderland module is in the data. We divide module data up by type, with each type represented as a top-level direc-tory within the module. The module system handles each type using a deployer that is in charge of unpacking the data and making it available to the correct subsystem. Example deployers in the Wonderland core include artwork, which is unpacked into a directory in the Web server where clients can down-load it; client code, which is also made avail-able to clients via the Web server; server code, which is installed in the Darkstar server; and Web administration modules, which are deployed to the Web server using standard Java EE mechanisms. The set of deployers in the module system itself is even extensible; Developers can use a new deployer contained in a module to deploy custom content in other modules.

Design Trade-OffsThe Wonderland architecture has been in use for close to four years, having undergone two complete rewrites in that time. Here we discuss some of the major design decisions we made and the advantages and disadvantages we found for each approach.

Simulation ModelWonderland is based on a hybrid computa-tion model between the client and server. In this model, the server maintains objects’ states primarily by reacting to client requests. The server doesn’t handle objects’ graphical states but rather their properties, such as name or position. The client does most of the work in rendering the object on the screen as well as responding to user input and property changes the server sends.

This approach falls somewhere in between comparable systems; OpenSimulator performs more computation — including physics — on the server and shares fine-grained state with the client (see http://opensimulator.org/wiki/OpenSim:Introduction_and_Definitions). Open Croquet, on the other hand, uses a peer-to-peer

IC-15-05-Yank.indd 43 8/18/11 12:38 PM



model in which most computation is replicated between servers (see www.opencobalt.org/about/synchronization-architecture). Wonder-land is flexible in that developers can employ e ither model as needed; h igh ly interac-tive tasks can be simulated on the client with the understanding that synchronizat ion might not be perfect between different users. Tasks with stronger synchronization require-ments can run on the server, with the trade-off of higher latency and therefore less frequent updates.

Scalability and InteractivityIn many cases, we’ve found the need to choose between scalability and interactivity. The basic trade-off is simple: a world that’s more inter-active changes more frequently, requiring more bandwidth and computation to keep all the cli-ents up to date. A world that changes less fre-quently, or is static (as in many videogames), can support more users with less communi-cation required per user. This same decision applies to almost every feature of the environ-ment. For example, using more graphically rich avatars provides a better sense of presence but requires more resources from the video card, limiting the number of avatars that a Wonder-land world can display.

For our Wonderland collaboration use case, we targeted small work groups of fewer than 20 people, putting more emphasis on interactivity than on large numbers of users. This target was based on research related to meeting behavior in which we found that the typical meeting had between two and 16 participants.4 The current version of Wonderland supports up to 50 users in a single space, allowing room for multiple simultaneous groups to interact in the same space. Larger groups must be divided into multi-ple spaces. Different trade-offs might be made in a world designed for giving large presenta-tions, with much less interactivity but scaling to many more users.

Modularity and ComplexityThe last major trade-off is between a modular architecture and software complexity. We’ve already described many of a modular architec-ture’s advantages, including extensibility and manageability. Some downsides exist as well. Developing in a modular fashion introduces much more fragmentation to the code. Rather than

developing features in the core, interfaces are designed in the core and implemented in modules. Figuring out which module implements which feature can be difficult. Furthermore, because Wonderland administrators can add, update, and remove modules individually, module depen-dencies and versions become a management challenge.

Despite this complexity, a modular architec-ture lets us build an ecosystem of extensions around the Wonderland toolkit. We provide a Module Warehouse where developers can share their modules with others, and we host module repositories so they can share code.

T he Open Wonderland toolkit is in active use all over the world for projects in education,

collaboration, and simulation. Our main focus is on improving the current version’s collabora-tion features, stability, and scalability.

One key area of future development is increased server federation — that is, the ability to connect multiple servers. We’ve developed our client as a browser, enabling a single cli-ent to connect to many servers with different features. We’d like to enhance this ability — for example, to let a client connect to multiple serv-ers simultaneously — to simulate large, continu-ous environments. Another extension would be to cluster servers so that a group of servers share common resources such as authentica-tion scope, content repositories, and presence information.

As we start expanding support for multiple servers and data types, we must also think about interoperability. As a first pass, many groups are working together to define com-mon artwork formats and presence mecha-nisms that different virtual worlds could use. Eventually, as with the Web, we expect to see large-scale standardization of virtual environ-ments. This will require standardization not only of content but also of behavior, so that a user can access interactive, collaborative virtual spaces that work the same no matter which browser they use. Although predicting what this standard model will look like is dif-ficult, the Open Wonder land architecture can be a starting point for this standardization effort.

Open Wonderland is a highly exten-sible toolkit for building vir tual worlds.

IC-15-05-Yank.indd 44 8/18/11 12:38 PM



In its current form, we can deploy it to sup-port a wide range of collaboration use cases. Due to our focus on extensibility, it is also an ideal platform for experimentation and research into new virtual world features and applications.

References1. D.B. Anderson et al., “Building Multi-User Interactive

Multimedia Environments at MERL,” IEEE Multimedia,

vol. 2, no. 4, 1995, pp. 77–82.

2. W. Broll, “Interacting in Distributed Collaborative Vir-

tual Environments,” Proc. Virtual Reality Ann. Int’l

Symp., 1995, pp. 148–155.

3. J. Andreano et al., “Auditory Cues Increase the Hippo-

campal Response to Unimodal Virtual Reality,”

CyberPsychology & Behavior, vol. 12, no. 3, 2009,

pp. 309–313.

4. N. Yankelovich et al., “Meeting Central: Making Dis-

tributed Meetings More Effective,” Proc. ACM Conf.

Computer Supported Cooperative Work (CSCW 04),

ACM Press, 2004, pp. 419–442.

Jonathan Kaplan is an architect for the Open Wonderland

Foundation and the CTO of WonderBuilders. He is the

original software architect of the Wonderland plat-

form, a project he cofounded at Sun Microsystems Lab-

oratories. Kaplan has an MSE in computer science from

the University of Pennsylvania. He is the coauthor of

J2EE Design Patterns (O’Reilly and Associates, 2003).

Contact him at [email protected].

Nicole Yankelovich is the executive director of the Open

Wonderland Foundation and CEO of WonderBuilders.

She cofounded the Wonderland project in 2007 as prin-

cipal investigator of the Collaborative Environments

research program at Sun Microsystems Laboratories.

She’s also a visiting scientist at the Massachusetts

Institute of Technology Center for Educational Com-

puting Initiatives. Yankelovich holds seven patents

and has published in the areas of collaborative envi-

ronments, speech applications, and hypertext. Contact

her at [email protected].

Selected CS articles and columns are also available for free at http://ComputingNow.computer.org.

A new publication model that will provide subscribers with features and benefits that cannot be found in traditional print such as:

• MoreRapidPublicationofResearch• OnlineAccesstotheCSDL• Interactive Disk and a Book of

Abstracts• LowerPrice

Available Transactions Titles by 2012:

• TDSC• TMC• TPAMI• TPDS• TVCG

For more information about OnlinePlus™, please visit http://www.computer.org/onlineplus.

IC-15-05-Yank.indd 45 8/18/11 12:38 PM

Documents

Open Wonderland: An Extensible Virtual World Architecture