Gamifying requirement elicitation: Practical implications and outcomes in improving stakeholders collaboration

  • Published on

  • View

  • Download


  • Entertainment Computing 5 (2014) 335345Contents lists available at ScienceDirect

    Entertainment Computing

    journal homepage: ees .e lsevier .com/entcomGamifying requirement elicitation: Practical implications and outcomesin improving stakeholders collaborationq 2014 Elsevier B.V. All rights reserved.

    q This paper has been recommended for acceptance by Matthias Rauterberg. Corresponding author.

    E-mail address: (C. Ribeiro).Claudia Ribeiro a,, Carla Farinha b, Joo Pereira a, Miguel Mira da Silva aa INESC-ID, Department of Information Systems and Computer Science, Technical University of Lisbon, Rua Alves Redol 9, Lisbon, PortugalbOpensoft, Lisbon, Portugal

    a r t i c l e i n f o a b s t r a c tArticle history:Received 30 May 2013Revised 11 February 2014Accepted 1 April 2014Available online 11 July 2014

    Keywords:Requirements elicitationCooperationCollaborationCommunicationSix Thinking HatsGamificationThe requirements engineering process is a key phase of the Information System development since itdetermines its functionalities and its operation. Before requirements can be analyzed, modeled, orspecified they must be gathered through an elicitation process. Requirements elicitation is non-trivialbecause you can never be sure you get all requirements from the user or stakeholder by just asking themwhat the system should do. Requirements elicitation practices include interviews, questionnaires, userobservation, workshops, brainstorming, use cases, role playing and prototyping. However, these commonprocedures are still prone to be ambiguous or incorrect which can lead the Information Systems to failure.It is consensual that one of the major problem of this activity relates to the communication andcollaboration between different and distant stakeholders. Thus, recent studies have been proposingweb collaborative tools to gather these stakeholders in order to elicit requirements. The paper aims toevaluate the effectiveness and acceptance of such a collaborative tool which was developed by using agamification approach and the Six Thinking Hats method. The document also makes a discussion ofthe implication and outcomes of improving stakeholders collaboration.

    2014 Elsevier B.V. All rights reserved.1. Introduction

    Today we live in an Information Age where people rely on com-puters and technology to work, socialize or live [1,2]. This technol-ogy quite often comes to us through Information Systems. Buildingsuch systems is usually a complex and difficult task, demanding asignificant effort on planning and managing their developmentprocess. Therefore, system designers and developers use the Sys-tem Development Life Cycle (SDLC) framework which breaks downthe development process into a pipeline of activities. Several SDLCmodels have been created (waterfall, fountain, rapid prototyping,incremental, etc.) but all of them have the requirements elicitationactivity as the earliest stage in the pipeline. Before requirementscan be analyzed, modeled, or specified they must be gatheredthrough an elicitation process. The aim here is to understand anddefine how the system will operate [4]. Requirements elicitationis based on an intense communication between stakeholders andbetween stakeholders and analysts. Therefore, cooperation andcollaboration are vital in this process [5]. Requirements elicitationis non-trivial because you can never be sure you get all require-ments from the user or stakeholder by just asking them what thesystem should do. Several studies have been conducted with thegoal of edifying common limitations in this process, mainly aimingat understanding the role of communication, collaboration andcooperation between stakeholders. Nevertheless, despite of theresearch efforts, it still remains unclear how to overcome limita-tions that can account for 6070% of projects that fail to deliveron time, on cost and with the scope originally promised [3], costingaround 80100 times more if discovered at the implementationstage and are very hard to fix [4].

    Since communication is critical, requirement elicitation toolsmust ease this communication between stakeholders in order toarticulate their needs collaboratively, allowing their meetings evenat a different time and place to discuss those needs. In this context,game-based tools can bring numerous benefits to this processsince they typically provide immediate feedback, active participa-tion and the high motivation promoted by the competitive envi-ronment [57].

    Recent research as proved the benefits of adding game mechan-ics to common tasks outside the traditional video games environ-ments [8], including motivational benefits to participate in onlinecommunities. This approach is commonly referred in the literature

  • 336 C. Ribeiro et al. / Entertainment Computing 5 (2014) 335345as gamification, a concept that is already used in numerousapplications ranging across productivity, finance, health, educa-tion, sustainability, as well as news and entertainment media [9].

    The paper aims to evaluate the effectiveness and acceptance ofiThink system [13], a RE tool, which was developed by using agamification approach and the Six Thinking Hats method. The doc-ument also makes a discussion of the implication and outcomes ofimproving stakeholders collaboration. The evaluation was basedon Action Research in real world organizations: Action Researchallowed us contributing with practical actions on the organizationand generating knowledge about its context on the real world sit-uations. We performed two Action Research cycles: we studied theproblematic situation of the first environment, applied an action,evaluated the results and extracted lessons learnt. In the secondcycle we also studied the situation taking into account the lessonslearnt from the first cycle, applied an action with iThink, evaluatedthe results and extracted other lessons [10,11].2. Requirements elicitation

    As stated by Avison and Fitzgerald [4], the definition ofrequirements can be problematic, but in relation to informationsystems, it can be said to be everything that the set of relevantstakeholders want from a system. Requirements are, indeed, thekey information in Information Systems Development: they trans-late stakeholders needs, determining what and how the Informa-tion System will operate [12,13].

    Despite many years of computing and research efforts in therequirements elicitation field, this activity is still not well under-stood. Errors still happen on the requirements elicitation activityand still represent major causes for the failure or even the suspen-sion of the entire information system project [12,13].2.1. Ineffectiveness

    Many authors have been studying the reasons of the ineffective-ness of requirements elicitation activity. For example, Avison andFitzgerald [4] stated that analysts may not identify all the relevantstakeholders and just capture requirements from a small set ofusers, raising costly fixes when the time comes to identify forgot-ten requirements. They also state that stakeholders time con-straints to participate in the elicitation activity promote missedrequirements. Finally, they refer that analysts misinterpretrequirements because of the culture gap or may miss require-ments leaving the specification incomplete.

    Zowghi and Coulin [13] categorized issues and pitfalls in therequirements elicitation activity based on their revision of the lit-erature and their empirical experience. Their categories of issueswere particularity and uniqueness of process and project; complexcommunication between stakeholders and analysts; quality ofidentified requirements; conflicts of interests; and experience ofthe analyst.

    Davey and Cope [12] enumerated quite a few problems withrequirements, including incomplete, ambiguous, incorrect, exces-sive and inconsistent requirements. Also, they suggested otherproblems such as poor users collaboration, unnecessary designconsiderations, different views of different users or continuousacceptance of additional requirements.

    Resuming, practitioners consider requirements as main reasonsfor project failures. Within this field of study, there are numerousauthors that believe that problems begin with the complex andintense communication between disparate communities involvedin the requirements elicitation activity [14,15]. On the one hand,stakeholders do not always know what they want or how toarticulate their needs. On the other hand, analysts may not entirelyunderstand business concepts, misinterpreting required needs [4].

    2.2. Trends from social sciences

    Requirements elicitation is based on communication. As such,the social nature of this activity is undeniable [16]. Previous works,have tried to address the ineffectiveness of requirements elicita-tion activity that result form this social nature as described inthe previous section. Namely, recent trends have been studyingand using a range of methods derived from social sciences in orderto increase chances of success of requirements elicitation. Thesemethods include: ethnography, interviews and domain group work[13]. Ethnography focuses the observation of people in their natu-ral environment, translating stakeholders activities and interac-tions. Some researchers claim that ethnography may havesatisfactory results eliciting requirements [17]. Nevertheless, sev-eral limitations were also recognized, such as risk of incorrectinterpretations, impossibility of identifying new requirements ordifficulty of generalizing results [18,17,13]. Interviewing is aninformal interaction where analysts explore needs asking stake-holders about the system in use and the system to be [13]. Well-known limitations of interviewing are the limited stimulusresponse interaction and the need of participants to share basicconcepts and methods [16].

    Group work gathers stakeholders to collaborate reaching solu-tions about an identified problematic situation. Groups are particu-larly effective because they involve and commit the stakeholdersdirectly and promote cooperation [13]. Examples of such methodsare JAD, Creativity workshops or Focus Groups. JAD (Joint Applica-tion Development) aims to quickly determine system requirementsfor an Information System. Stakeholders elicit these requirementsthrough structured and focused discussion sessions about businessneeds [19]. Nevertheless, Coughlan [15] presented two studiesabout the practical usage of JAD and criticized the need for auseranalyst interaction that is excessively rigid or the importantrole of the moderator to keep the session focused on the final prod-uct solution. Creativity workshops, based on the Creative ProblemSolving of Alex Osborn and Sidney Parnes, encourage creative think-ing to discover and invent system requirements [20]. Pennell andMaiden [21] report results and lessons learned from two experi-ences with creativity workshops. They concluded that this creativethinking must be prepared and incubated in order to truly suc-ceeded. Focus Group is a group-based discussion to obtain feedbackfrom participants on a particular topic. In order to be effective onthe discussion, the group has special characteristics: homogeneousregarding key topics, focused on key topics but open to communi-cate freely [22]. Farinha and Mira da Silva [23,24] applied regularand web-based Focus Groups on real environments to evaluatethe success of this method eliciting requirements for the develop-ment of Information Systems projects. The results confirmed thatstakeholders effectively discussed different perspectives about thedesired system and cooperated in order to formalize requirements.However, some limitations were also pointed out such as the dom-inance of particular users, or the complexity of analysis or time-con-suming on the regular Focus Groups, or the lack of stakeholdersparticipation on the web-based Focus Groups.

    In sum, methods derived from social sciences have addresssome of inefficiencies inherent to requirement elicitation activity.Nevertheless, every method has its own strengths and weakness,as described previously. Typical weaknesses include dominant par-ticipants, biased opinions, high logistic costs and difficulties ongathering stakeholders at the same time and place [13]. Collabora-tion tools have tried to address these weaknesses as explainednext.

  • C. Ribeiro et al. / Entertainment Computing 5 (2014) 335345 3372.3. Collaboration tools

    Recognizing the importance of collaborative work and the hugedifficulty of gathering stakeholders at the same time and place,researches have been proposing web collaborative tools to elicitrequirements [25]. Such tools include variations of the WinWinspiral model, Athena, variations of wikis, iRequire, AnnotateProor Stakesource [26]. For example, the CoREAmethod (CollaborativeRequirements Elicitation and Analysis), based on the winwinspiral model, is a geographically distributed environment. Itincludes decision support for analysing and selecting require-ments. Athena is a collaborative approach supported by a tool toelicit requirements based on group storytelling. The stories aremerged in a single story, transformed into scenarios and translatedinto use cases. Wikis were also widely studied to deal with distrib-uted stakeholders, easing communication and increasing the par-ticipation of all stakeholders. This collaborative tool allowsspatially distributed stakeholders to add, remove, and amend con-tent on a common platform. There are several proposals based onwikis, such as WikiWinWin [27], SoftWiki [28], SmartWiki [29]or ShyWiki [30]. AnnotatePro [31] obtains requirements by draw-ing annotations directly on the users screen and using snapshots incombination with ordinary picture editing functionality. The snap-shots may easily be sent to the software engineers by email. iRe-quire [32] is a tool for mobile phones and enables users to blogtheir requirements whenever their need is triggered. The main fea-tures of iRequire are the possibility to take a picture of the environ-ment, document a user need, describe the main task and provide arationale, and check the summary of a need. Stakesource 2.0 [26] isa web application using standard technologies that uses social net-works and collaborative filtering, a crowd sourcing approach, toidentify and prioritize stakeholders and their requirements. Stake-holders can invite other stakeholders to participate, suggest andrate requirements.

    In Table 1 the weaknesses of the previously listed collaborationtools are summarized. Although several research efforts have beenmade towards methods and tools to better support requirementselicitation this activity still generate...


View more >