From visual simulation to virtual reality to games

  • Published on

  • View

  • Download


  • 0018-9162/05/$20.00 2005 IEEE September 2005 25

    P E R S P E C T I V E S

    P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y

    From VisualSimulation toVirtual Realityto Games

    During the past two decades, the virtual reality community hasbased its development on a synthesis of earlier work in inter-active 3D graphics, user interfaces, and visual simulation.1Doing so let developers create a more open technology thanthe visual simulation community could, increased the numberof people working in 3D, and developed a science, technology, and lan-guage considerably beyond that of the earlier field.

    Beginning in 1997 with the publication of an NRC report titled Modelingand SimulationLinking Entertainment and Defense,2 the video game com-munity has pushed into spaces previously the domain of the VR community.Clearly, the VR field is transitioning into work influenced by video gamesand thus now influences that industry as well. Because much of the researchand development being conducted in the games community parallels the VRcommunitys efforts, it has the potential to affect a greater audience.

    Given these trends, VR researchers who want their work to remain rele-vant must realign to focus on game research and development. Research inthe games arena affects not just the entertainment industry but also the gov-ernment and corporate organizations that could benefit from the training,simulation, and education opportunities that serious games provide.

    DEFINING SERIOUS GAMESPeople respond differently to the emotionally charged term game depend-

    ing on whether they played or did not play video games while growing up.This is basically a generation-gap issue because children who have grownup since the 1980s have been exposed to video games their entire lives.

    Before we can seriously tackle the issue of what a games research agendamight be, we must define what the term means. Dictionaries tend to definea game as a physical or mental contest, played according to specific rules,with the goal of amusing or rewarding the participants. When seeking a def-inition of the more specific term video game, we are likely to encounter adescription such as a game played against a computer, which would moreaccurately be worded as a game played with a computer. To fully fleshout this definition, we might propose the following: Video game: a mentalcontest, played with a computer according to certain rules for amusement,recreation, or winning a stake.

    Leveraging technologyfrom the visual simulation and virtualreality communities,serious games provide a delivery system fororganizational videogame instruction andtraining.

    Michael ZydaUSC Information Sciences Institute

  • 26 Computer

    Developing a science of games opens up a hugepotential for the wider application of games in gov-ernmental and corporate arenas. The formal defi-nition might read as follows: Serious game: amental contest, played with a computer in accor-dance with specific rules, that uses entertainmentto further government or corporate training, edu-cation, health, public policy, and strategic commu-nication objectives.

    Bing Gordon, chief creative officer of video andcomputer games developer Electronic Arts, oncetold me that he defines video games as story, art,and software.

    When designing a game, the development teamblends these elements into a finished product. Thedesign team crafts the story, which provides thegames entertainment component. The art teamprovides the games look and feel. The program-ming team develops the code that implements storyrequirements, interface features, networking, Webconnectivity, scoring systems, AI scripting, gameengine changes, and just about anything technicalor programmatic that the entire development effortrequires.3

    Serious games have more than just story, art, andsoftware, however. As Figure 1 shows, they involvepedagogy: activities that educate or instruct,thereby imparting knowledge or skill. This addi-tion makes games serious. Pedagogy must, how-ever, be subordinate to storythe entertainmentcomponent comes first. Once its worked out, thepedagogy follows.

    A human performance engineering team worksclosely with the design team to oversee this peda-gogy insertion. The teams lead is part instructional

    scientist and part subject-matter expert for thedomain around which the teams are building theserious game.

    Building serious games takes more than simplyhanding their development to a traditional gameteam, however. The team must interact with theinstructional scientists and subject matter expertsthat comprise the human performance engineeringteam. To thrive, this new organization must have auniversitys facilities and support or similar researchorganization. A research agenda that supports seri-ous games also benefits the entertainment industry,one of the USs largest economic sectors.

    CREATING A SCIENCE OF GAMESThe development and wide release of the

    Americas Army game began a revolution in think-ing about the potential role of video games fornonentertainment domains. It also sparked a dis-cussion about how to advance game technologysstate of the art to support future entertainment andserious games.4 This, in turn, promoted interest increating a science of games and an allied educa-tional program. Experiences with digital-gamenativesthose who have grown up playinggamesindicated that a game-centered researchand educational program could offer many posi-tive benefits. While much speculation regardingthese benefits is anecdotal, substantive evidenceshows that game experiences affect digital-gamenatives positively. If researchers construct and per-form their studies carefully, they may be able to har-ness these positive effects for societal gain.

    The announcement of Americas Army, shownin Figure 2, at the 2002 Electronics Entertainment

    Figure 1. From gameto serious game.Unlike their enter-tainment-only coun-terparts, seriousgames use pedagogyto infuse instructioninto the game playexperience.


    to story




    Human performanceengineering team









  • Expo prompted the US Army to commission astudy of the game to see if it could be used for train-ing. In the summer of 2002, a US Army captainfrom Fort Benning, who was tasked with spendingabout a week playing the game, reported that hewas unimpressed. Although he deemed the game agood recruiting tool, he indicated that there wasinsufficient fidelity in the game for it to be of anyuse in training.

    Cyberpunk author William Gibson once notedthat the street finds its own uses for things.5 We sawthis effect in action in October 2002, when a staffsergeant from Fort Benning approached theAmericas Army booth at the Armys annual AUSAConference in Washington, D.C. An enthusiasticgamer who had played Americas Army from dayone, he told the staff, We love this game atBenning. We use it for training.

    The sergeant had bypassed the Armys require-ments documents and formal studies and deployedthe game on his own initiative. He went on toexplain that when new recruits had trouble withthe rifle range or the obstacle course, his team hadthose recruits play Americas Army and requiredthem to complete those levels in the game, as shownin Figure 3. When the recruits did so, they thenwent back to the range and usually passed the rangetests. This made us wonder what other applicationswould benefit from game-based instructionespe-cially after we developed a revised version ofAmericas Army with new levels for a variety ofDepartment of Defense, Army, and Secret Servicetraining needs.

    Six to nine months after its release, motherswould meet me and complain that my son is play-ing Americas Army five to six hours a day, sevendays a week. What is going to become of him? Iwould usually answer that these children would betwice as likely to consider a career in the US Armyas those who didnt play the game, something theArmy counts on with respect to the games recruit-ing mission. When I asked the mothers if their chil-dren knew a lot about the US Army, the mothersusually responded that they know everythingabout the Army, having learned it from the game.Wouldnt it be nice if playing games could teachthem something more useful?

    These comments led us to wonder how much ofK-12 science and math education could be taughtvia games and how we might exploit studentscapability for collateral learningthe learning thathappens by some mechanism other than formalteaching. Ultimately, we wondered if we couldincorporate all K-12 science and math education

    in a highly immersive, highly addictive gamewecalled this our first-person education grand chal-lenge, a play on the phrase first-person shooter.6

    Given the many anecdotes about digital-gamenatives being better surgeons and displaying excep-tional business skills, it became clear that creatinga science of gamesa scientific and engineeringmethod for building them and understanding andanalyzing game playhad become essential.

    September 2005 27

    Figure 3. Training simulator. Despite the initial evaluators skepticism, AmericasArmy proved to be an effective military training simulator. Soldiers who playedthe rifle range segment of the game, for example, earned improved scores on the real-life rifle range.

    Figure 2. Americas Army. The most widely used and successful serious game todate, this title initially served as a recruiting tool.

  • 28 Computer

    GAME PRODUCTION CHALLENGESToday, game production poses some

    incredible challenges. Teams come togetherto build a single game, then dissolve. Trainingtimes on game engine toolsets increasesteadily. Code modules, crafted for specificgames, offer less than 30 percent reuse in sub-sequent efforts. First-person shooters are cur-rently the only games that have reusableengines. These engines are proprietary, how-ever, used for a few games only, licensed atexorbitant expense, cost more than $6 mil-lion, and take more than a year to develop.Game production times increase steadily asgraphics cards provide more visual capabil-

    ities and the game-playing public demands moreinteractivity and verisimilitude.

    The demand for better computer characters andstory increases with the complexity of visual dis-plays and with the release of each new, more com-plex-than-ever game. Game play innovation isbecoming a competitive necessity. The big hits havebeen sports, first-person shooters, adventure, andSims-type games. For the game industry to continueto grow, additional genres must become moresophisticated, with better backstories and thor-oughly researched, developed, and deployed foun-dational game-play technologies.

    The game-playing public is becoming ever moreenamored of portable entertainment platforms, andcomputing speeds are becoming ever faster. Thesetrends pose the critical challenge of providing well-designed interfaces, ever-increasing storage capac-ity, and expanded wireless network bandwidth.Efficient streaming of content to wireless portabledevices and for game downloads from the Web willalso become priorities.

    GAMES RESEARCH AGENDATo influence the future of both serious and enter-

    tainment games, developers must create a researchand development agenda that transforms the gameproduction process from a handcrafted, labor-intensive effort to one with shorter, more pre-dictable production timelines that still manages toprovide innovations and increased complexity. Thisresearch agenda has three components: the infra-structure, cognitive game design, and immersion.

    InfrastructureThe underlying software and hardware necessary

    for developing interactive games include massivelymultiplayer online game architectures, gameengines and tools, streaming media, next-genera-

    tion consoles, and wireless and mobile devices.Many application domains use massively multi-

    player online game architectures, including the mil-itary, security and homeland defense, and onlineeducation. MMOGs pose the fundamental researchquestion of how to develop dynamically extensibleand semantically interoperable software architec-tures. This functionality requires building game orsimulation clients that can connect to a runningMMOG, download the appropriate code for dis-play and interaction, then operate with other onlineplayers. This work, of interest to gaming in gen-eral, has special relevance for the large govern-mental game-based simulation sector.

    Currently, only static solutions that dramaticallydrive up the cost of large-scale simulation and gam-ing have been developedno dynamic solutionsare available. Developers must solve the MMOGarchitecture problem not just for game clients butalso for large-scale computational architecturessuch as grid computing. Game engines and toolswill be vital to researchers bent on attacking thelack-of-reuse problem in gaming. They can alsoprovide the technology for moving games fromcrafted systems built by the game industry gnomesto engineered systems used widely in the govern-ment and corporate sectors.

    The Americas Army project has made someattempts to broaden first-person-shooter gameengine reuse to other domains, but all such useshave suffered from major limitations.3 For example,many game engines lack support for large terrainboxes and can only handle a 1 km 1 km area eventhough most real-world applications require muchlarger spaces. Other limitations include onerousand expensive game-engine licenses and the gen-eral unavailability of these engines for R&D andto the serious games community at large.

    Thus, developers need an open source gameengine that includes a development toolset aswidely available and utilized as Linux. With anopen source engine, developers could explore addi-tional capabilities, including a larger terrain box,dynamic terrain, physical modeling, and otherrequirements that the entertainment world ignores.In addition, an open source engine would make fea-sible exploring many other directions, including themodeling and simulation of computer characters,story, and human emotion.

    Streaming media will play a prominent role inthe delivery of dynamic content to PC-based gamesand mobile devices. This will gain more importancein games as computing becomes smaller, faster, andmore capable. In turn, these developments will

    Americas Army hasmade some attempts

    to broaden first-person-shooter

    game engine reuseto other domains,but all such uses

    have suffered frommajor limitations.

  • encourage research into the proper interfaces forsuch devices and in determining how to best deploythem for serious and entertainment purposes.

    Cognitive game designTaking a cognitive approach to game develop-

    ment will give developers the tools to create theo-ries and methods for

    modeling and simulating computer characters,story, and human emotion;

    analyzing large-scale game play; innovating new game genres and play styles;

    and integrating pedagogy with story in the inter-

    active game medium.

    Computer-generated autonomy involves model-ing human and organizational behavior in n...


View more >