From visual simulation to virtual reality to games

Embed Size (px)

Text of From visual simulation to virtual reality to games

  • 0018-9162/05/$20.00 2005 IEEE September 2005 25

    P E R S P E C T I V E S

    P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y

    From VisualSimulation toVirtual Realityto Games

    During the past two decades, the virtual reality community hasbased its development on a synthesis of earlier work in inter-active 3D graphics, user interfaces, and visual simulation.1Doing so let developers create a more open technology thanthe visual simulation community could, increased the numberof people working in 3D, and developed a science, technology, and lan-guage considerably beyond that of the earlier field.

    Beginning in 1997 with the publication of an NRC report titled Modelingand SimulationLinking Entertainment and Defense,2 the video game com-munity has pushed into spaces previously the domain of the VR community.Clearly, the VR field is transitioning into work influenced by video gamesand thus now influences that industry as well. Because much of the researchand development being conducted in the games community parallels the VRcommunitys efforts, it has the potential to affect a greater audience.

    Given these trends, VR researchers who want their work to remain rele-vant must realign to focus on game research and development. Research inthe games arena affects not just the entertainment industry but also the gov-ernment and corporate organizations that could benefit from the training,simulation, and education opportunities that serious games provide.

    DEFINING SERIOUS GAMESPeople respond differently to the emotionally charged term game depend-

    ing on whether they played or did not play video games while growing up.This is basically a generation-gap issue because children who have grownup since the 1980s have been exposed to video games their entire lives.

    Before we can seriously tackle the issue of what a games research agendamight be, we must define what the term means. Dictionaries tend to definea game as a physical or mental contest, played according to specific rules,with the goal of amusing or rewarding the participants. When seeking a def-inition of the more specific term video game, we are likely to encounter adescription such as a game played against a computer, which would moreaccurately be worded as a game played with a computer. To fully fleshout this definition, we might propose the following: Video game: a mentalcontest, played with a computer according to certain rules for amusement,recreation, or winning a stake.

    Leveraging technologyfrom the visual simulation and virtualreality communities,serious games provide a delivery system fororganizational videogame instruction andtraining.

    Michael ZydaUSC Information Sciences Institute

  • 26 Computer

    Developing a science of games opens up a hugepotential for the wider application of games in gov-ernmental and corporate arenas. The formal defi-nition might read as follows: Serious game: amental contest, played with a computer in accor-dance with specific rules, that uses entertainmentto further government or corporate training, edu-cation, health, public policy, and strategic commu-nication objectives.

    Bing Gordon, chief creative officer of video andcomputer games developer Electronic Arts, oncetold me that he defines video games as story, art,and software.

    When designing a game, the development teamblends these elements into a finished product. Thedesign team crafts the story, which provides thegames entertainment component. The art teamprovides the games look and feel. The program-ming team develops the code that implements storyrequirements, interface features, networking, Webconnectivity, scoring systems, AI scripting, gameengine changes, and just about anything technicalor programmatic that the entire development effortrequires.3

    Serious games have more than just story, art, andsoftware, however. As Figure 1 shows, they involvepedagogy: activities that educate or instruct,thereby imparting knowledge or skill. This addi-tion makes games serious. Pedagogy must, how-ever, be subordinate to storythe entertainmentcomponent comes first. Once its worked out, thepedagogy follows.

    A human performance engineering team worksclosely with the design team to oversee this peda-gogy insertion. The teams lead is part instructional

    scientist and part subject-matter expert for thedomain around which the teams are building theserious game.

    Building serious games takes more than simplyhanding their development to a traditional gameteam, however. The team must interact with theinstructional scientists and subject matter expertsthat comprise the human performance engineeringteam. To thrive, this new organization must have auniversitys facilities and support or similar researchorganization. A research agenda that supports seri-ous games also benefits the entertainment industry,one of the USs largest economic sectors.

    CREATING A SCIENCE OF GAMESThe development and wide release of the

    Americas Army game began a revolution in think-ing about the potential role of video games fornonentertainment domains. It also sparked a dis-cussion about how to advance game technologysstate of the art to support future entertainment andserious games.4 This, in turn, promoted interest increating a science of games and an allied educa-tional program. Experiences with digital-gamenativesthose who have grown up playinggamesindicated that a game-centered researchand educational program could offer many posi-tive benefits. While much speculation regardingthese benefits is anecdotal, substantive evidenceshows that game experiences affect digital-gamenatives positively. If researchers construct and per-form their studies carefully, they may be able to har-ness these positive effects for societal gain.

    The announcement of Americas Army, shownin Figure 2, at the 2002 Electronics Entertainment

    Figure 1. From gameto serious game.Unlike their enter-tainment-only coun-terparts, seriousgames use pedagogyto infuse instructioninto the game playexperience.


    to story




    Human performanceengineering team









  • Expo prompted the US Army to commission astudy of the game to see if it could be used for train-ing. In the summer of 2002, a US Army captainfrom Fort Benning, who was tasked with spendingabout a week playing the game, reported that hewas unimpressed. Although he deemed the game agood recruiting tool, he indicated that there wasinsufficient fidelity in the game for it to be of anyuse in training.

    Cyberpunk author William Gibson once notedthat the street finds its own uses for things.5 We sawthis effect in action in October 2002, when a staffsergeant from Fort Benning approached theAmericas Army booth at the Armys annual AUSAConference in Washington, D.C. An enthusiasticgamer who had played Americas Army from dayone, he told the staff, We love this game atBenning. We use it for training.

    The sergeant had bypassed the Armys require-ments documents and formal studies and deployedthe game on his own initiative. He went on toexplain that when new recruits had trouble withthe rifle range or the obstacle course, his team hadthose recruits play Americas Army and requiredthem to complete those levels in the game, as shownin Figure 3. When the recruits did so, they thenwent back to the range and usually passed the rangetests. This made us wonder what other applicationswould benefit from game-based instructionespe-cially after we developed a revised version ofAmericas Army with new levels for a variety ofDepartment of Defense, Army, and Secret Servicetraining needs.

    Six to nine months after its release, motherswould meet me and complain that my son is play-ing Americas Army five to six hours a day, sevendays a week. What is going to become of him? Iwould usually answer that these children would betwice as likely to consider a career in the US Armyas those who didnt play the game, something theArmy counts on with respect to the games recruit-ing mission. When I asked the mothers if their chil-dren knew a lot about the US Army, the mothersusually responded that they know everythingabout the Army, having learned it from the game.Wouldnt it be nice if playing games could teachthem something more useful?

    These comments led us to wonder how much ofK-12 science and math education could be taughtvia games and how we might exploit studentscapability for collateral learningthe learning thathappens by some mechanism other than formalteaching. Ultimately, we wondered if we couldincorporate all K-12 science and math education

    in a highly immersive, highly addictive gamewecalled this our first-person education grand chal-lenge, a play on the phrase first-person shooter.6

    Given the many anecdotes about digital-gamenatives being better surgeons and displaying excep-tional business skills, it became clear that creatinga science of gamesa scientific and engineeringmethod for building them and understanding andanalyzing game playhad become essential.

    September 2005 27

    Figure 3. Training simulator. Despite the initial evaluators skepticism, AmericasArmy proved to be an effective military training simulator. Soldiers who playedthe rifle range segment of the game, for example, earned improved scores on the real-life rifle range.

    Figure 2. Americas Army. The most widely used and successful serious game todate, this title initially served as a recruiting tool.

  • 28 Computer

    GAME PRODUCTION CHALLENGESToday, game production poses some

    incredible challenges. Teams come togetherto build a single game, then dissolve. Trainingtimes on game engine toolsets increasesteadily. Code modules, crafted