Embed Size (px)
Citation preview
Will Undergraduate Students Play Games toLearn How to Conduct Library Research?by Karen Markey, Fritz Swanson, Andrea Jenkins, Brian Jennings, Beth St. Jean,Victor Rosenberg, Xingxing Yao and Robert FrostAvailable online 16 June 2009
This exploratory study examines whetherundergraduate students will play games to
learn how to conduct library research.Results indicate that students will play
games that are an integral component ofthe course curriculum and enable them toaccomplish overall course goals at the same
time they learn about library research.
Karen Markey is Professor, School of Information, University ofMichigan, 304 West Hall, 1085 South University Ave., University
of Michigan, Ann Arbor, MI 48109-1107, [email protected];
Fritz Swanson is Lecturer, English [email protected];
Andrea Jenkins is Programmer, School of [email protected];
Brian Jennings is Programmer, School of [email protected];
Beth St. Jean is Graduate Student Research Assistant,School of [email protected];
Victor Rosenberg is Associate Professor, School of [email protected];
Xingxing Yao is Research Assistant, School of [email protected];
Robert Frost is Associate Professor, School of [email protected].
The Journal of Academic Librarianship, Volume 35, Number 4, pages 303–3
THE PROBLEM
Research libraries are leading the revolution from paper-based collections to online full-text resources. The“online” advantage is convenience—library users cansearch, study, and use online full texts at anytime andfrom anywhere. The “online” disadvantage is context—nolonger can library users rely on the physical nature of thebricks-and-mortar research library to reveal importantdistinctions in the form and function of library resources.In the past, library users could sense these distinctions byphysically navigating the research library—its referencerooms, departments, branches, reading rooms, stacks,etc.—or ask a nearby public services librarian what re-sources they should use to further their search forinformation. Today's online research library is fast becom-ing a list of hyperlinks, menu choices, tabs, and icons thatneither reveals form-and-function distinctions of availableresources nor places a librarian at arm's reach to helpstudents navigate the maze. Incoming undergraduatestudents need a “blueprint” to navigate both physicaland online components of today's research library.
“Today's online research library is fastbecoming a list of hyperlinks, menu
choices, tabs, and icons that neither revealsform-and-function distinctions of availableresources nor places a librarian at arm's
reach to help students navigate the maze.”
Students can turn to various non-credit venues tolearn how to conduct library research. Librarians areexemplary in terms of assisting students who want tolearn, offering workshops, short courses, virtual refer-ence assistance, web-based instruction pages, walk-upservice at information desks, in-class lectures, etc.1
Unfortunately, these opportunities enable librarians toreach only a fraction of the students who really needassistance. Because the teaching faculty are primarilyconcerned with disciplinary coverage, they are reluctantto cede valuable in-class time to librarians.2 Thetremendous growth in e-learning—academic programs
13 July 2009 303
that are administered almost entirely online—alsomakesit imperative that librarians identify effective strategiesto help students develop research skills so they canbecome successful learners in an online environment.3
THE SOLUTIONGames hold much promise for teaching incomingstudents how to conduct library research and developinformation literacy skills. Gaming has many featuresthat facilitate good learning. For example, game playersget results by trial and error, they stumble across things,follow hunches, repeat actions over and over until theyget them perfect, and assume new identities, projectingtheir hopes, values, and fears onto their new identityinstead of shouldering the burden on their real-lifeidentity.4 Game play that takes place online could booststudents' participation and acceptance because theycould play anytime and anywhere.
This purpose of this paper is to describe exploratoryresearch that determines whether undergraduate stu-dents will play games to learn how to conduct libraryresearch. A research project teamdeveloped aweb-basedboard game that was designed to give game playersexperience with library-research content during gameplay. The team deployed the game in an undergraduateclassroom,monitoring the extent of student participationin online game play. After game play ended, the teamanalyzed game-play logs and interviewed students toanswer research questions that revealed whether theywill play games to learn how to conduct library research.
LITERATURE REVIEW
There is no shortage of spokespersons who advocateonline games for learning academic concepts andsubject matter. Prensky embraces online game-basedlearning because it “meets the needs and learningsystems of today's and the future's generations oflearners, [it] is motivating, it is fun, [and it] isenormously versatile, adaptable to almost any subject,information, or skill to be learned, and when usedcorrectly, is extremely effective.”5 Johnson tells howpeople learn by playing: “you're supposed to figure outwhat you're supposed to do. You have to probe thedepths of the game's logic to make sense of it, and likemost probing expeditions, you get results by trial anderror, by stumbling across things, by followinghunches.”6 Game scholar James Paul Gee breaks probingdown into the four-part “prove, hypothesize, reprobe,rethink cycle” which he considers “to be the basis ofexpert reflective practice in any complex domain.”7
“There is no shortage of spokespersonswho advocate online games for learningacademic concepts and subject matter...As a result, a handful of librarians have
undertaken game development tocreate a platform specifically centered
on information literacy content.”
304 The Journal of Academic Librarianship
Urging librarians to get involved with games, main-stream spokespersons tell them to add games tocollections, host tournaments, create in-library zonesto accommodate open play, and use games to teachlibrary users about information literacy.8 Circulatinggames and hosting tournaments are fast becomingacceptable practices at school, academic, and publiclibraries.9 At Syracuse University, Scott Nicholson hasestablished the Library Game Lab to explore the effec-tiveness of games and gaming activities in libraries andto help librarians make informed selection decisions.10
Van Leer describes how the popular World of Warfaregamemeets four ACLR Information Literacy Standards.11
Branston, Doshi and Leach and Sugarman exhorttheir fellow librarians to experiment with games forinformation literacy.12 Except for the three-dimensionalvirtual world Second Life,13 there are no commercialgames into which librarians can retrofit their informa-tion literacy goals. As a result, a handful of librarianshave undertaken game development to create a plat-form specifically centered on information literacycontent. At Arizona State University's Fletcher Library,staff are developing Quarantined: Axl Wise and theInformation Outbreak, a single-player game environ-ment that simulates the complex processes of selecting,using, evaluating, and synthesizing multiple sources ofinformation.14 Austin Community College authored theonline Info Game that issues questions and scoresanswers to test questions based on the content of atext-based tutorial.15 On the University of NorthCarolina-Greensboro's Info Lit game board, playersclick an electronic die (e-die) to move their gamepiece forward and score points for their correct answersto the game's library-use questions.16
When we initiated our games investigation, hardly ahandful of games with library-user education contentwere operational. We were skeptical about these gamesbecause they appeared to be testing players about whatthey already knew about libraries and library resources.To test our hypotheses about the potential of games forteaching students about conducting library research,we would first have to engage in game design anddevelopment, establishing a new game genre thatwould inspire game players to do library research tomake progress and improve their standing in the game.
MethodologyOur investigation of games for library-user instruc-
tion had these 3 major phases:
1. Choose a research process model that studentswould learn as a result of game play.
2. Design, develop, and pretest an online game thatteaches students the research process model.
3. Evaluate the online game with undergraduate stu-dents who are enrolled in a class in which libraryresearch is required to complete class assignments.
GenSpec: A Research Process Model for BuildingUnderstanding and Depth on a Topic of InterestThe project team selected the General-to-Specific
(GenSpec) Research Process Model for conducting
library research. Devised by Thomas Kirk at EarlhamCollege, this model tells students where to start theirlibrary research for a topic of interest, how to build on agood start, and how to achieve depth on their selectedtopic.17
The project team updated the model adding Googlesearching as the first step in the discovery processbecause they assumed that students are set in theirways and already search Google and the web to achievea basic understanding of their topics of interest. TheGenSpec Model then advises students to continue theirresearch with broad overview tools such as general anddiscipline-specific encyclopedias, handbooks, and his-tories so they round out their basic understanding oftheir chosen topics. The model advances students tofinding tools—bibliographies, abstracting and indexingsources, and catalogs—for specific information on theirtopics upon which they can build a solid foundation ofunderstanding. Finally, the model advances the fewstudents who want to specialize and achieve depth intheir chosen topics to forward-chaining tools—citationindexes—to find the latest cutting-edge research.
Research findings on how undergraduate studentsconduct library research demonstrate a heavy relianceon Google and the web almost to the exclusion of allother resources.18 By following the GenSpec Model,students could proceed in lockstep fashion from basicresources like Google and the web to scholarlyresources to achieve greater depth and sophisticationin their knowledge about their chosen topics.
The Web-based Board Game: Design, Development,and Pretesting
Designing the game was a difficult task because wehad to envision how players would experience theGenSpec Model through game play. Two ideas achievedprominence during the project's design phase: (1) agame inwhich players engaged in bibliography buildingand (2) a game that enlisted the board-game genre togive players practice navigating, searching, and findingdigital resources that answered substantive questionson a scholarly topic. We chose the latter because itappeared to be doable given the project's limited timeand staff resources.
The web-administered Defense of Hidgeon boardgame was primarily written in Ruby on Rails alongwith HTML and CSS. Game action takes place in themiddle of the 14th century at the height of the BlackDeath's sweep through Europe. The objective of thegame is to be the Duchy's richest, fastest, and mostaccurate research team. To accomplish this, teams of 4players play the game. They must land on each of the 6different monastery libraries and give correct answersat least 3 times to the questions that are posted at theselibraries. Correct answers earn teams a golden scroll.Teams are required to collect all 18 scrolls and urged toamass as much gold and property as they can duringgame play.
Monastery libraries specialize in certain types ofresources, and their placement on the Defense ofHidgeon game board corresponds to GenSpec Model.Players roll an e-die, and the first libraries on which
they are likely to land issue them questions for whichthey must search the web to find correct answers. Nextcome monastery libraries specializing in online ency-clopedias that issue questions for which players mustsearch these encyclopedias to find correct answers.Then come libraries specializing in books for whichplayers must search the online catalog, and so on. Theorder of monastery libraries adheres to the GenSpecResearch Model because it is the underlying frameworkfor the game and, ultimately, what the project teamhoped students would learn as a result of game play.When teams answer monastery library questionscorrectly, they earn a golden scroll and the opportunityto purchase exclusive licenses to monastery libraries.
A roll of the e-die may land game pieces on spacesthat require different actions. For example, landing onthe Fox Hunt space puts teams in the Hospital spacewhere they must remain until they complete a task thatdemonstrates their fitness to continue researching.Teams landing on the Well space throw their preciousgold into the well to make a wish. Landing on theGarrison space puts teams in a win-gold or lose-goldscenario. Teams can engage in challenges to improvetheir standing in the game. Generally, teams are urgedto do the research, strive for accuracy, and maximize onopportunities to become rich by earning gold, obtainingexclusive licenses, and engaging in challenges to wrestcontrol of monastery libraries from opponents.
Gamedesign anddevelopment took7months andwasfollowed by a 2-month pretesting period to eliminatebugs and streamline game play. A YouTube moviedemonstrates game play.19 The research team's finalreport describes and illustrates the game's full function-ality and explains its complex scoring algorithm.20
EVALUATING THE GAME
To determine whether undergraduate students willplay games to learn how to conduct library research, theproject team designed an evaluation that answeredthese research questions:
1. What levels of engagement do students have withthe game?
2. Do instructors need to give students incentives toplay the game?
3. What types of questions are the most difficult ones?Why?
4. What do students learn as a result of game play?5. Do students prefer games or traditional approaches
to learn about library research?6. How should games be configured into the course
curriculum?
To answer these questions, the project team deployedthe game in the undergraduate class named “Introduc-tion to Information Studies” that is administered in theSchool of Information (SI) and cross-listed in theDepartment of Sociology. SI 110 is taught by this paper'seighth author, and enrolls a total of 75 undergraduatestudents at all levels from a wide range of majors.
To gauge student enthusiasm on the game itself,Professor Frost mentioned game play to SI 110 students
July 2009 305
in passing at the beginning of the semester, but he didnot list it on the course syllabus or other documenta-tion. On October 30, 2007, Frost introduced Markey to SI110 students. She kept her remarks about the gamebrief, wanting students to develop their own ideasabout what the game was teaching themwhile playing.She demonstrated game play, cited game objectives,announced prizes to first- ($400), second- ($200), andthird-place ($100) teams, encouraged them to thinkabout what they were learning while playing the game,and gave notice that she would visit their discussiongroups after game play ended to ask them about theirexperiences. Game play lasted about 4 weeks, begin-ning on November 3, and ending on November 29, 2007.While students were playing the game, they wereresearching a 5- to 7-page term paper on how thedesign and structure of information systems influencethe information people find. Students were expected todocument their papers with footnotes and a bibliogra-phy. To avoid biasing research results, Frost or Markeydid not point out connections between the game andterm-paper assignment to students.
“Successful teams were the 6 teamsthat met the criteria for the instructor'sincentive, answering the 18-questionquota with a 40% accuracy rate, andunsuccessful teams were the 7 teamsthat failed to meet the incentive.”
Three times a day, a Perl script captured each team'splay-by-play data into consecutive columns of a Micro-soft Excel workbook. These data included questionsattempted and questions answered correctly andincorrectly by type, scrolls by type, time elapsed sincethe start of the game, amount of gold, scrolls by type,and library licenses owned by type. Upon the game'sconclusion, these data were copied into Excel foranalysis.
When game play ended, project team membersattended 3 regularly-scheduled SI 110 discussion groupswhere they had access to all students registered in theclass to ask whether teams played together or indivi-dually, what students learned as a result of playing thegame, why some teams that signed up failed to play thegame, why some students failed to sign up to play thegame, what improvements students wanted made tothe game, and what they thought about using games tolearn about library research and academic topicsgenerally.
RESULTS
The discussion of results is subdivided according to thisproject's 6 research questions on levels of engagement,incentives, easy and difficult questions, learning fromgame play, students' preferred learning approaches, andconfiguring games into the course curriculum.
306 The Journal of Academic Librarianship
Incentives and Levels of EngagementOf the 75 students enrolled in SI 110, 29 students
signed up on 8 teams that ranged from 2 to 4 students.Project team members scrutinized daily logs of gameplay activity. Only one team played the game over thefirst weekend, answering 12 of 14 questions correctlyand acquiring 8 of the game's 17 exclusive licenses tomonastery libraries. Concerned about the low level ofgame play, Markey and Frost discussed strategies thatwould induce students to sign up and play the game.Frost volunteered to give a half-letter grade increase tostudents who answered 40% or more questionscorrectly in the course of collecting all 18 golden scrollsand announced it in class. In response, an additional 20students signed up on 5 new teams to play the game.Overall, 49 (65%) of the 75 students in the class signedup on 13 teams to play the game.
After game play ended, the project team scrutinizedlogs and determined that “successful teams”were the 6teams that met the criteria for the instructor's incentive,that is, answering the 18-question quota with 40%accuracy rate, and “unsuccessful teams” were the 7teams that failed to meet the incentive.
The Game Play of Unsuccessful TeamsThe Best and Hail teams were dropouts, never
signing onto the game. The RedNinjas and Blue teamssigned onto the game once or twice, answered 1 and 3questions, respectively, and never signed on again. TheConquerors and Wolverines teams were somewhatmore active, signing onto the game 3 and 5 times,respectively, and answering as many as 7 questions inone sitting. With the exception of the Warriors team,most unsuccessful teams appeared to be testing thewaters, that is, trying to determine whether they shouldinvest time and effort in game playing. The Warriorsplayed in spurts with most of their activity occurringfrom the middle to end of November. Their 50%accuracy rate exceeded the instructor's criteria for theincentive. Because the teamwas unable to sustain gameplay for the length of time needed to earn 18 scrolls,they did not earn the incentive.
Unsuccessful teams answered 35.7% of questionscorrectly. Omit the quasi-successfulWarriors team fromthe analysis and the percentage of correct answersdipped to 21%, about 9 percentage points below whatwould have been expected by chance. The onlyunsuccessful team to issue a challenge, the Warriorschallenged the second-place Heroes team 4 times andwon 1 time. The Warriors also purchased exclusivelicenses to 2 monastery libraries.
The Game Play of Successful TeamsThe first-place InfoHunters team played on the first
day of game play, November 3. In fact, the InfoHuntersplayed alone through much of the first half ofNovember. For the rest of the month, the InfoHuntersplayed sporadically to maintain their leader status andrespond to challenges. The Heroes, Victors, and Maizeteams “tested the waters,” answering a few questionsright after the instructor announced the incentive inclass on November 6 and 7. The Victors signed onto the
game 3 more times with the lion's share of their activityoccurring on November 21when they answered over 60questions. Signing onto the game 8 more times, theMaize team never answeredmore than 10 questions persign on; however, its low-level activity was sufficient formeeting the instructor's incentive.
The Heroes became active on November 7 answering1 or more questions over a dozen days. This teamanswered over 20 questions to earn their final scrollduring the game's final week. The Authorities andValiant teams earned a handful of scrolls from time totime then rushed tomeet the criteria for the instructor'sincentive during the game's final week.
The InfoHunters team played the game to win,initiating game play on the same day that the gamebegan, earning all scrolls and owning all exclusivelicenses on the “Hidgeon” game board at the same timetheir major competitors—teams Heroes, Victors, andMaize—were testing the waters. Teams Valiant, Autho-rities, andMaize exhibited game-play behavior that wasconnected to only meeting the instructor's incentive.Although second and third places were up for grabs, theHeroes team was easily able to assert itself because itplayed on the low-key “Plague” game board where theunsuccessful Warriors were an irritating but ineffectiveopponent and team Valiant was only intent on meetingthe instructor's incentive.
The Game's Most Difficult QuestionsTable 1 shows the number of monastery library
questions that successful teams answered and percen-tages of correct answers by library resource type. Of the228 monastery library questions that the game issued,successful teams answered 50.9% of them correctly.
Which Questions are the Easiest?Game players' accuracy rates were highest for web,
encyclopedia, and database questions at 67%, 62%, and62%, respectively. These percentages were a little over 2timeswhatwouldbeexpectedbychance. Becauseplayersdid not have to leave their personal computers to do the
TableCorrect Answers to Monastery Ques
Team # Questions # Correct b% Correct
Web a
InfoHunters 44 22 58
Heroes 35 21 86
Victors 44 18 100
Maize 32 19 80
Authorities 32 18 60
Valiant 41 18 43
# Total 228 116 –
Average % – – 67
a Key: Web=Web questions, Ency=Encyclopedia questions, Bk=Book questionCI=Citation-Index questions.
b Questions sometimes exceed 18 (the quota for number of scrolls) because teams laearn it back by answering another monastery library question.
research to answer these questions, they probably did theonline research at their computers to answer them.
When asked to identify the game's easiest questions,students immediately replied “web questions.” In fact,students agreed that any question that kept them onlineat their computers was an easy question.
• “Web questions. All you had to do was copy the termif they say Google and usually it was transparent inthe first link and that was it.”
• “Going to a web site.”• “Anything that usually had to do with going to awebsite.”
Why are Questions Requiring Use of CitationIndexes Difficult?
Despite the convenience of doing research in onlinecitation indexes on their personal computers, gameplayers scored lower than average accuracy rates (42%)on their answers to these questions. Only one inter-viewee specifically cited citation indexes, and shedescribed difficulties understanding the ISI Web ofScience's interface:
“I had more trouble with the citing questions especiallybecause like when you would search the database which wasthe ISI Web of Science or something, it would come up with aperson's name and there would be 19 of the one topic and thenthere would be 1 under the same topic name and like I wouldalways choose all of the ones that were under the same topicand I would get the answer wrong and I think that would bewhy. I just felt like maybe the answers were like weird likethey were not clear.”
The project team speculated on why game playersfared poorly on citation-index questions. Students maybe unfamiliar searching online citation indexes. Accus-tomed to search engines intowhich they type keywordsand scan lists of retrievals for ones on their topics ofinterest, students type author names into citationindexes and scan lists of citations that cite the author.The ISI Web of Science complicates the scanningprocess because it often lists titles for the same item
1tions Given by Successful Teams
Ency Bk EW Db CI % Total
75 75 33 75 27 50.0
75 33 43 75 75 60.0
50 33 38 75 21 40.9
100 50 30 75 75 59.4
75 75 50 50 43 56.3
38 30 50 43 100 43.9
– – – – – –
62 43 39 62 42 50.9
s, EW=Edited Works questions, Db=(journal-article) Database questions
nded on Oracle or Garrison spaces that required them to surrender a scroll and
July 2009 307
,
on separate lines due to differences in abbreviations,page numbers, and page ranges, requiring searchers toexert effort scrutinizing intermediary results. In fact, theplayer's comment above mentions the difficulty ofselecting from results-lists in the ISI Web of Science.The bottom line is that despite the online nature ofcitation indexes, they are tools that deviate fromtraditional database searching and require searchersto exert patience, attention, and effort to learn and usethem effectively.
The complicated nature of the game's citation-indexquestions could have had an adverse effect on gameplayers. These questions were lengthy due to thecomplexity of citation index search tasks, and due tothe inclusion of hints and possible answers. Gameplayers might have been tempted to give educatedguesses to lengthy citation-index questions rather thanexert the patience and perseverance needed to followthem through from beginning to end.
Why are Questions About Books and EditedWorks Difficult?
During focus groups, game players agreed that thegame's most difficult questions were the ones thatdisrupted online game play. For example, they had tosign off the computer to go to the U-M Library to checkfor answers in books and edited works questions.Because their accuracy rate for books (43%) and editedworks (39%) questions were below average, it is likelythat they guessed at answers to these questions. Gameplayers said as much:
• “Books are one of the questions I guessed on. I got itright all the time too.”
• “You kind of like had to go to the library to get asource and it's like, oh, I'm not going to the library,and it kind of puts a lag on it because you just kind ofhope somebody on your team does it but it goes onfor a couple days and then occasionally somebodywill figure it out and then you'll get another one to gothe library. It just kind of spreads it out and it takesmore time to get it done.”
• “We're kind of losing the art of finding the book andusing the library in that sense but that's just the wayit seems it's going.”
• “It's not that I personally have an aversion to books. Iquite like books. It's just when doing research …online it's somuchmore efficient… for instance evena book that's online like the [digitization project] thatGoogle is trying to do to add all the books onlinebecause then you can go … you're looking at thishuge book [online] and you want to do a specifictopic, you can search specific words or series ofwords and find it in the book.”
Despite the guaranteed availability of the books andedited works on reserve at the University Library,students confessed to us that they did not go to thelibrary to examine these items. Reserve staff confirmedthis telling us that there were only 5 circulations of the51 books and edited works on reserve.
308 The Journal of Academic Librarianship
Monastery libraries specializing in books and editedworks featured a handful of questions that did notrequire game players to go to the U-M Library to findanswers. Instead, students could search online toanswer them. Examples of these questions are:
• For which of the following edited works does theBook Review Index Online have reviews?
• Platt's book “King Death” focuses on the Black Deathand its aftermath in medieval England. What is thequickest and most efficient way to find additionalbooks in the U-M Library on this subject?
• Which of these edited works are reviewed in theJSTOR database?
We hypothesized that players would regard thesequestions as requiring less effort, and, thus, be morelikely to conduct research at their computers to answerthem correctly. The results confirmed our suspicions.Players were much more likely to give incorrectanswers to questions that required them to go to thelibrary to examine a particular book (28%) compared tosearching online for an answer (55%).
“Players were much more likely to giveincorrect answers to questions thatrequired them to go to the library to
examine a particular book (28%) comparedto searching online for an answer (55%).”
Game players always fared badly with edited worksquestions, answering a few points better (39%) thanthey would have done by chance (about 30%). Follow-up interviews would have to be conducted to learn whyplayers had difficulty with edited works questionsbecause except for this one player's comment, interviewremarks that targeted edited works were rare:
• The edited work … I really don't know why [it wasthe most difficult]. I know that those were the last[questions] that my team finished up. Just becausewith me not being familiar completely with Mirlynand didn't learn that before I started the game, I nowam [familiar with Mirlyn] but starting [the game], Ireally didn't understand how to look up the differentwriters.
What did Students Learn as a Result of Playingthe Game?
When interviewers asked students about what theylearned as a result of playing the game, they singled outthe game's ability to teach them about the tools ofresearch. Here is what they had to say in this regard:
• “I actually learned a lot about Search Tools [theUniversity Library's database selection system] andhow to use it because previously I had no idea how touse any of that stuff like the little hints that you gavewere really helpful and you can see how it applies toany other type of research you had to do.”
• “Doing the research for the game allowed us to doreally helped me. I'm in the middle of two differentpapers and I'm using Mirlyn for it, I'm usingProQuest, and I'm using other things for it, so I wasable to learn a lot that will be able to help me.”
• “I never searched databases before so I learned somesearching… so the questions made you do it so… I'mmore familiarwith searching thedatabases forpapers.”
• “[I learned] how to research and actually get resultsthat will help your paper or whatever it is you'redoing.”
• “I think another big part of it was the database part. Itkind of showed you how many databases are reallyavailable to us as students. We have so muchdifferent stuff—about a ton of different information.I feel like it also kind of told you how to do thesearches, like how to find a specific database like inthere you could do all research or like subjects or itjust kind of show you all the actions needed.”
• “The [databases] all had different interfaces, theyweren't all exactly the same.”
• “So if you look at one [database], you would befamiliar with it. Like after I answered like twoquestions on one database, I already knew how touse it like I didn't have to think about it.”
• “I feel like … the best part about the game is becauseyou're going to be learning how to use the databasesyou're going to be using in real life.”
Some interviewees remarked that they benefitedfrom game play because it made them do research tasksconnected with various online tools over and over againand confront and solve important problems during theresearch process. Here are their comments:
• “I actually took a mini-course on how to better[search] … JSTOR and different things like that …Someone else mentioned that this [game] is sotedious and so repetitive like doing an assignment …But [the game] is actually like a real life circumstanceand you're actually having to actually execute aquestion and put in actual research and stuff likethat. And yes, it was repetitive but repetitive in thesense that it actually kind of nails the topic into yourhead and then you know the topic better.”
• “We had a guest lecturer come in from the library,and he taught us how to use ProQuest and thingslike that. I wrote stuff down but I didn't reallyremember it so when I went to use ProQuest, someof the things I got confused but then when I had todo it for the game and it was more like step by stepwith the hints and what to do, what to type in, whatto click, for me that helped me a lot more to learn it,and now I know how to do it just because we wereable to copy and paste and go directly back and forthwhereas in a lecture form. I don't know if this is justme, I don't necessarily gain everything I can out oflectures but [the game] is more like visuallydesigned.”
Two students qualified their positive learning experi-ences saying that the game took longer than traditionalapproaches to learning about library research:
• “I did learn [about searching databases] but it tookfive times as much time.”
• “So basically the game makes me figure it out on myown, with your instructions, but I feel like if someoneteachesme like a librarian showsme on a PowerPointlike how to do this and then I do it myself once, I learnin like fiveminutes. [The game] is effective but I thinkit's inefficient.”
Upperclassmen were somewhat negative aboutlearning from game play telling us that they alreadyknew how to conduct library research. They remarkedas follows:
• “[I learned] nothing. Because I knew how to searchthe database already. Maybe because I'm a junior, Iknew how to use the databases. Maybe I got familiarwith like one or two [databases] better. But about theBlack Death it was like a lot of the answers you couldfind by using the find feature in Firefox or whateverso I would just go to the task and search and if it wasthere and then I would say yes it was there and click. Iwouldn't even read the title.”
• “I just feel like most of us already knew most of thetools that we used during the game to like search tosee which article is reliable and stuff like that.”
• “I think that it would be a lot more efficient … to justwrite a paper and have your teacher telling you, okay,you have to use three sources from a database, threesources from an encyclopedia or whatever, andputting those restrictions would force students tolearn, and they would do it, and they would learn iton their own.”
Students did not explicitly say that the game taughtthem how to think about what they were doing or givethem opportunities to do so. One student suggestedhowmonastery libraries should be ordered on the gameboard:
• “I wonder if you could order the [monastery libraries]like how good their resources were. Web resourcesmight not be as reliable as the last ones were I thinkdatabases or encyclopedias. I'm not sure … Encyclo-pedias seem like more reliable than websites … Theinformation is accurate, and it has relevance, it hasmore relevance because it's more … more has to betrue than Wikipedia, like you can just go toWikipedia, and it might not be true. Kind of likeMonopoly—like the first ones are always really cheapand then at the end it's really expensive.”
The first libraries on the board were the least reliableones, the “cheap” ones to which this student refers;however, this student did not make this observation.Yet, the reason why these first libraries were given thisplacement (i.e., the general-to-specific nature ofordered sources in the GenSpec Model) was differentfrom the reason this student had in mind (i.e., reliabilityof the information contained in web resources, books,journals, etc.). That she and her fellow game-playingclassmates failed to notice the general-to-specificnature of the sources they used to answer questions atthe game's monastery libraries is worth emphasizing.
July 2009 309
Students even failed to make a connection between thegame and the term papers they were researching andwriting while playing the game.
At the end of focused group interviews, project teammembers debriefed students on the GenSpec Model.They told students how this model starts them at theweb, advised them to build on this good start bysearching online encyclopedias, and encouraged themto achieve depth by searching the online catalog forbooks and online databases for journal articles. Here arestudent responses about this debriefing:
• “Tell us exactly what you just said. Tell us why it'suseful, don't say go and look up something … Sayingexactly what you just said made [the task of learninghow to do library research] more useful and more ofa desire in that 5 seconds so that I might nowwant to[play the game].”
• “Everything [Karen] just said about research and howyou should go about it, I would love to have on asheet … and put it at my desk and tape it to the wall.Then when I'm writing a paper, I would refer to thatand then it would be real world. I would be using it towrite papers and you wouldn't have to change it nexttime.”
We stand by our decision to keep mum about whatwe wanted students to learn from game play becausewe did not want to predispose students to act or thinkin a particular way. Instead, we were able to learn fromtheir observations about how to make games a part ofthe ongoing academic learning experience. That gameplayers did not come to a realization about the order ofmonastery libraries was troublesome to us researchers.It demonstrates our need to rethink the design of thegame. It also calls for instructors to be deliberate aboutintegrating information literacy games into theircourse curriculum, telling students learning objectivesbefore play begins, challenging them to think aboutwhat they are learning during game play, and debrief-ing them on these objectives upon the conclusion ofgame play.
Students' Preferences for Games VersusTraditional Approaches
Interviewers asked students whether they preferredlearning about library research from playing a gamesuch as the Defense of Hidgeon or more traditionalapproaches such as lecture-demonstrations, shortcourses, web tutorials, etc. Student remarks werevaried. Some preferred traditional approaches:
• “I don't think that it needs to be in the format of agame. Like I don't really see what benefit you'regetting of having that format of a game. You couldbenefit just as much from maybe offering like a freeseminar on how to use the tools that the universityoffers like a seminar on how to use ProQuest andWeb of Science and things like that. That waysomeone is physically showing you instead of youkind of digging through all the programs.”
• “Maybe if you can take a course or something likethat and people know what they're getting into.”
310 The Journal of Academic Librarianship
Others saw merit in games:
• “Make it like a responsibility of first-year introduc-tory level instructors—making their students likeactually write these papers like Bob [SI 110'sinstructor] does and helping us figuring out whereto find these sources because in order to really learnit you need to use it in action. The game is helping tokind of perpetuate that and actually using it in apaper will really get to there. I think a lot of …teachers who teach first year courses kind of forgetthat and maybe assume that we know how to do itwhere really the university as a whole ought to lookat how students do research.”
• “[The game] is a really good resource. It's reassuringbefore you're thrown into your first research projectand you're like, oh, I've got to use these resources, I'mnot entirely sure how.”
Alternatively, students could choose between thetwo approaches or enroll in a seminar that wouldcombine game play and traditional approaches:
• “I think it depends on the person because you knowsome people might think if they go to a lecture, I'mdone with it, let's go, for all the people would belike lecture, boring, I'd rather try this game and gofor it.”
• “Maybe you could have a seminar where you teachhow to use this by using the game and maybe eachperson can come up and do one part of the game orthen people can work on it. Kind of a long seminarwith using the game as the tool to teach it.”
Configuring Games into the Course CurriculumThe game's monetary prizes were an incentive to
only a handful of teams to play the game. Only after theinstructor offered the half-grade increase did mostteams play the game. Students told us directly that theinstructor's incentive motivated them to play thegame:
• “The only reason I was [playing the game was]because there is no opportunity for extra credit inthis college. That's the only reason I was doing it.”
• “For me at least the money is not the thing… thegrade is … because people care about their grade.Money is not…money is great but what I'm saying iswould I rather have $500 or would I rather have abetter grade in the class? And ultimately I wouldrather have a better grade in my class.”
• “I know just the monetary reward didn't get myteam like stirred like you weren't really doinganything. Like, yeah, it would be great to havemoney in our pocket but it was a lot of work to getthat money, and there was a chance you weren'tgoing to win like somebody [else] was going to comeout ahead of you. But the grade, once the grade gotput in, like that night we met and just started doingit just because again not because we have bad gradesbut just in case you get a bad grade or the paperwasn't as good or something. It helps and it's stupidnot to do it.”
“The game's monetary prizes were anincentive to only a handful of teams
to play the game. Only after theinstructor offered the half-grade increase
did most teams play the game.”
Instead of monetary awards, students want evidencethat game play will advance their knowledge oracademic standing, and they want rewards for gameplay that affect their final grade in the class. Studentswere adamant about incorporating the game into thecourse syllabus, activities, and requirements. Whenthey take a course, students want to know right fromthe start what activities will figure into grading.
• “Like to me, I thought if you would just have given itto us as an assignment for a grade, we probablywould have worked harder.”
• “It's just motivating people to actually see that this[game] is something that's going to benefit them andsomething useful for them to be learning from.”
• “I think rather than money, if you just make it a classrequirement you'd get a lot better turn out. I know Iwas just turned off to [the game] because the timingand not knowing about it in advance.”
DISCUSSION
To assess the potential of game-playing in collegeclassrooms, the project team neither gave the game abig build-up nor connected it to course requirements,preferring instead to gauge student enthusiasm solelyon playing the game. With few exceptions, the studentresponse was to engage in game play at a minimumlevel that was focused exclusively on meeting theinstructor's incentive for extra credit or, after testingthe waters, to opt out entirely. Such a responsedemonstrates that game play cannot take place as anadjunct or extra-credit exercise. It must be an integralcomponent of the course curriculum. In the future, wewould advise educators who want to incorporateHidgeon game play into their curriculum to cite gameplay in the syllabus and give attention to in-classdiscussion before, during, and after game play thatmakes students think about what they are learning.
Despite our efforts to put questions into the gamethat would encourage game players to think about whyHidgeon was structured in a particular way, mostplayers were intent on meeting the instructor's incen-tive, not on thinking about what they were learning. Inthe future, game play must be accompanied by in-classdiscussion which, in the case of Hidgeon, could involvediscussions of the GenSpec Model, how the modelorders sources from general to specific, the rationale forsuch an ordering, the disciplinary nature of databases,database selectionwithin one or more disciplines that isin keeping with the model, and how students couldenlist the GenSpec Model to complete a related courseassignment.
Asked about what they learned as a result of playingthe Defense of Hidgeon, most players focused on the“how-to” connected with library research such aslearning names of databases, becoming familiar with aparticular database's content and interface, choosingdatabases using Search Tools, and retrieving usefulresults. They did not explicitly say that the game taughtthem how to think about what they were doing or gavethem opportunities to do so nor did game playersrealize the GenSpec Model from the experience of gameplay. These findings demonstrate that informationliteracy games cannot stand on their own. Instructorsmust make an investment in game-related activitiesthat challenge students to think about how game playsupports other course requirements. Instructors wouldalso benefit from games that generate reports abouteach team's progress achieving game objectives, use ofbasic and advanced functionality, game-play accuracy,time spent playing the game, etc., so that they can leadclass discussions, help students overcome game-playobstacles, and help them make connections betweengame play tasks and related class activities.
The project team could improve on the administra-tion of the Defense of Hidgeon game, finding instructorswho would be willing to deploy the game alongsidecomparable subject matter, possibly in a history orhistory of science course, give students a library-research assignment, and challenge them with in-classdiscussions about game play that connects it to othercourse assignments. Unfortunately, the game would belimited to history classes. To accommodate otherdisciplines, instructors would have to replace Hidgeon'sexisting Black Death question-and-answer database,backstory, and imagery with ones in keeping with theirchosen topics. By the way, question-and-answer data-base development would be a considerable undertakingrequiring the development of a minimum of 160questions, 480 correct and incorrect answers, andfeedback explaining all 160 correct answers.
Rather than improve on Hidgeon, the project teamwould be more inclined to develop a new game that is apervasive and unobtrusive presence beside the onlinelibrary tools and digital collections that students nowuse to complete classroom assignments. Instead ofbeing comprehensive like Hidgeon, the new gamewould be one game made out of a collection ofnarrowly-focused mini-games that sense active playersand challenge them to play these mini-games. Themini-game format would enable students to discrimi-nate between the many tasks that make up libraryresearch, and, possibly, by becoming specialists atcertain research tasks, make it easy for them to applytheir task expertise to related class assignments.
Many instructors charge students with term-paperresearch and writing assignments. A new library-research game could be configured alongside the searchengines, library portals, online citation managers, orspecialized databases they use. The game's design couldencourage students to engage fellow classmates ingame play that furthers everyone's progress completingthe assignment. For example, students could inviteclassmates to build a shared bibliography on a broad-
July 2009 311
based topic. A group of like-minded students wouldsearch online, contribute citations to shared folders,then play mini-games in which they enhance contrib-uted citations with summaries, reviews, and ratings. Forexample, a relevance-rating mini-game would extractcitations from the shared bibliography, present citationsto active players, and give them a limited amount oftime to rate the citation according to its relevance to theparticular broad-based topic. The game would rewardstudents who are closest to the average relevance ratingof their peers.
Game play could give students opportunities to seeadvanced researchers at work so they could connectwhat they do to what the experts do. The Defense ofHidgeonwas limited in this regard. To some degree, thegame's advanced challenge functionality introducedplayers to concepts such as discipline, audience level,and credibility, but few teams challenged their oppo-nents, and when they did, teams did not completechallenges. A relevance-rating mini-game could beconceived as a venue for seeing other researchers atwork. The game would reward students who are closestto the relevance rating of a domain expert. Instructorscould supplement game play with in-class discussionsthat explore why peer-to-peer ratings differ and,presumably, why there is even greater disparitybetween peer-to-expert ratings. Additionally, futuregame designs should support instructors with anadministrative interface that issues daily reports tellingthem the extent to which their students are playing thegame, the nature of their game play, and individualstatistics about contributions, ratings, and rankings, sothat instructors can study these data in preparation forsuch discussions and use them to grade students fortheir game play.
Other rating mini-games could involve students inmaking judgments about audience level, credibility,objectivity, and currency. For example, an audiencelevel rating mini-game would extract citations from theshared bibliography, present them to active players, andgive them a limited amount of time to assign anaudience-level tag from a pulldown menu that bestdescribes the least amount of education that a readerwould have to read and understand the item. Peer-to-peer and peer-to-expert games would reward studentswho are closest to average relevance ratings. Instructorswho do in-game updates and post-game debriefingshould articulate why various ratings are likely to differso that players learn how to think critically about theresults of their online searches instead of acceptingthem without question.
SUMMARY
This paper describes a research effort to determinewhether undergraduate students will play games tolearn how to conduct library research. The project teamdeveloped the web-based board game Defense ofHidgeon that was designed to give game playersexperience with library-research content during gameplay. To avoid biasing research results, the teamdeployed the game in an undergraduate history oftechnology course with a minimum of introduction and
312 The Journal of Academic Librarianship
fanfare. The project team analyzed game activity logsand post-game interview comments to find outwhether students played the game, what they learnedabout library research, whether they would prefergames over traditional methods for learning informa-tion literacy concepts and skills, and how to incorporategames like Hidgeon into college classrooms.
Asked about what they learned as a result of playingthe Defense of Hidgeon, most players focused on the“how-to” connected with library research. That gameplayers did not come to a realization about the General-to-Specific Model that served as the game's under-pinnings was troublesome to the authors. It demon-strates the need to rethink the design of informationliteracy games and the ways inwhich instructors shouldintegrate games into their classroom proceedings.
“Asked about what they learned as a resultof playing the [online game], most playersfocused on the "how-to" connected with
library research.”
Given the option of enhancing the Defense of Hidgeonor designing a new information literacy game, theproject teamwould choose the latter, designing a gamethat is a pervasive and unobtrusive presence beside theonline tools and collections students use to research,write and document a writing assignment. The newgame's design should encourage students to engagetheir fellow classmates in game play that furtherseveryone's progress completing research assignments.Instead of being one comprehensive game-play experi-ence, a new game should be made out of a collection ofnarrowly-focused mini-games that sense active playersand challenge them to play various mini-games. Themini-game format would enable students to discrimi-nate between the many tasks that make up libraryresearch, and, possibly, by becoming specialists atcertain research tasks, make it easy for them to applytheir task expertise to related class assignments. Thegame could also give students opportunities to seeother researchers at work so they can connect whatthey do to what others do. Game design should includeinstructors, giving them an administrative interface thatreports the nature of and extent to which students playthe game so that they can engage students in classdiscussions to reflect on what they learn as a result ofplaying the game and grade students on their participa-tion in the game.
NOTES AND REFERENCES
1. Christopher N. Cox & Elizabeth Blakesley Lindsay, “Infor-mation Literacy Instruction Handbook,” Association ofCollege and Research Libraries, Chicago, 2008.
2. Larry L. Hardesty, “Faculty Culture and BibliographicInstruction: An Exploratory Analysis,” Library Trends 44(1995), pp. 39–67; Patricia Breivik, Student Learning in theInformation Age (Phoenix: Oryx Press, 1998); Paul Hrycaj
and Michael Russo, “Reflections on Surveys of FacultyAttitudes toward Collaboration with Librarians,” Journal ofAcademic Librarianship 33 (2007): 692–696.
3. Lesley Mutinta Moyo & Ellysa Stern Cahoy, “Library use in thee-learning environment,” Journal of Library Administration 45(2006): 339–359.
4. James Paul Gee, “What Video Games Have to Teach UsAbout Learning and Literacy,” Palgrave Macmillan, NewYork, 2003, pp. 51–71.
5.Mark Prensky, “Digital Game-based Learning,” New York,McGraw-Hill, 2001, p. 3.
6. Stephen Johnson, “Everything Bad is Good for You: HowToday's Culture is Actually Making Us Smarter,” RiverheadBooks, New York, 2006, p. 16.
7. James Paul Gee, What Video Games Have to Teach Us,pp. 90–91.
8. Kurt Squire & Constance Steinkuehler, “Meet the Gamers,”Library Journal 130 (2005): 38–41; Leslie Hitch and JimDuncan, “Games in Higher Ed: When Halo 2, Civilization IV,and Xbox 360 Come to Campus,” (2005) Available: http://net.educause.edu/ir/library/pdf/DEC0503.pdf (December3, 2008); Tom Storey, “The Big Bang,” OCLC Newsletter(March 2005): 7–12.
9. Jenny Levine, “Gaming and Libraries: Intersection ofServices,” Library Technology Reports 42 (2006): 31–59;Jenny Levine, “Gaming and Libraries: Broadening theIntersections,” Library Technology Reports 44 (2008); EliNieburger, Gamers—in the Library?!: The Why, What, andHow of Videogame Tournaments for All Ages (Chicago:American Library Association, 2007).
10. Scott Nicholson, “Library Game Lab of Syracuse,” (2008)Available: http://gamelab.syr.edu (December 3, 2008).
11. Lynn Van Leer, “Interactive Gaming vs. Library Tutorials forInformation Literacy: A Resource Guide,” Indiana Libraries24 (2006): 52–53.
12. Christy Branston, “From Game Studies to BibliographicGaming: Libraries Tap into the Video Game Culture,”Bulletin of the American Society for Information Scienceand Technology 32 (2006): 24–26, 29; Ameet Doshi, “How
Gaming could Improve Information Literacy,” Computersin Libraries 26 (2006): 14–17; G. J. Leach and T. S.Sugarman, “Play to Win! Using Games in Library Instruc-tion to Enhance Student Learning,” Research Strategies 20(2005): 191–203.
13. Levine, “Broadening the Intersections,”56;Donald T.Hawkinsand Barbara Brynko, “Gaming: The Next Hot Technology forLibraries?,” Information Today 23 (2006): 1, 51.
14. Bee Gallegos & Tammy Allgood, “Quarantined: Axl Wiseand the Information Outbreak: Creating an Online Gameto Teach Information Skills.” (2007) http://gaming.techsource.ala.org/index.php/Quarantined:_Axl_Wise_and_the_Information_Outbreak:_Creating_an_Online_Game_to_Teach_Information_Skills (December 3, 2008).
15. “The Info Game.” (2006) Available: http://library.austincc.edu/help/infogame/starthere.htm (December 3, 2008).
16. University of North Carolina-Greensboro, “The InformationLiteracy Game.” (2007) Available: http://library.uncg.edu/game/index.asp (December 3, 2008).
17. Thomas Kirk, “Problems in Library Instruction in Four-yearColleges,” in: John Lubans (Ed.), “Educating the LibraryUser,” R. R. Bowker, New York, 1974, pp. 83–103.
18. Karl V. Fast & D. Grant Campbell, “‘I still like Google’:University student perceptions of searching OPACs and theweb,” Proceedings of the ASIS annual meeting 2004,(Information Today, Medford, N.J., 2004), pp. 138–146;Allison J. Head, “Beyond Google: How do students conductacademic research?” First Monday, 12 (August 2007)Available: http://firstmonday.org/issues/issue12_8/head/index.html (December 3, 2008).
19. “Navigating the Defense of Hidgeon: The Plague Years.”(2008) Available: http://www.youtube.com/watch?v=u76tW-ne-yY (December 3, 2008).
20. Karen Markey & others, Engaging Undergraduates inResearch through a Storytelling and Gaming Strategy:Final Report to the Delmas Foundation, (Ann Arbor,Mich.: School of Information, University of Michigan,2008) Available: http://hdl.handle.net/2027.42/58630(December 3, 2008).
July 2009 313
