This article was downloaded by: [Florida State University]On: 21 December 2014, At: 21:02Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Interactive Learning EnvironmentsPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/nile20

Enhancing learning outcomes with aninteractive knowledge-based learningenvironment providing narrativefeedbackAndrew Stranieri a & John Yearwood aa School of Information Technology and Mathematical Sciences,University of Ballarat , Victoria, AustraliaPublished online: 02 Dec 2008.

To cite this article: Andrew Stranieri & John Yearwood (2008) Enhancing learning outcomes withan interactive knowledge-based learning environment providing narrative feedback, InteractiveLearning Environments, 16:3, 265-281, DOI: 10.1080/10494820802114176

To link to this article: http://dx.doi.org/10.1080/10494820802114176

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Enhancing learning outcomes with an interactive knowledge-based learning

environment providing narrative feedback

Andrew Stranieri* and John Yearwood

School of Information Technology and Mathematical Sciences, University of Ballarat, Victoria,Australia

(Received 25 July 2006; final version received 2 March 2008)

This paper describes a narrative-based interactive learning environment which aims toelucidate reasoning using interactive scenarios that may be used in training novices indecision-making. Its design is based on an approach to generating narrative fromknowledge that has been modelled in specific decision/reasoning domains. Theapproach uses a narrative model that is guided partially by inference and contextualinformation contained in the particular knowledge representation used, the generic/actual argument model of structured reasoning. The approach is described withexamples in the area of critical care nursing training. A study of the effectiveness of thisapproach on learning outcomes was conducted with final year nursing students andprovides evidence of improved learning outcomes.

Keywords: narrative; interactive learning environment; case-based learning; decision-making

1. Introduction

Expert level reasoning in fields such as medicine and law has been represented usingnumerous approaches, including logic, rules, decision trees, concept maps, argumentationschemes, and decision graphs. Setting out reasoning in a structured way may improvetransparency but does not in itself add to the understanding or absorption by apractitioner or student. Presenting reasoning as scenarios provides a means forpractitioners to assimilate the reasoning above abstract rules and has the potential toconnect with human understanding at the story level.

Narrative reasoning can provide a valuable approach to complex reasoning involved inproblem-solving and decision-making (Murray, 2000). Stories are effective as educationaltools because they are believable, easily rememberable, and entertaining (Bruner 1986;Neuhauser, 1993; Polkinghorne, 1988). The believability derives from their dealing withhuman experience that is perceived as authentic. They aid remembering because theyinvolve an audience in the actions and intentions of the characters. As audience we areengaged with the story on both an action level and a consciousness level, and it is throughthis dual engagement that we become involved with the minds of the characters andunderstand the story.

*Corresponding author. Email: a.stranieri@ballarat.edu.au

Interactive Learning Environments

Vol. 16, No. 3, December 2008, 265–281

ISSN 1049-4820 print/ISSN 1744-5191 online

� 2008 Taylor & Francis

DOI: 10.1080/10494820802114176

http://www.informaworld.com

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

Narrative-based interactive learning environments have been advanced by Peinado,Gervas, and Moreno-Ger (2005), Iuppa, Weltman, and Gordon (2004), Riedl andYoung (2004) and Cavazza, Charles, and Mead (2002). Narrative-based environmentsdeploy similar ideas to those described by Schank (1996) and anchored instruction(Cognition and Technology Group at Vanderbilt, 1990). In the majority of narrative-based interactive learning environments multiple storylines are stored in theenvironment in one way or another. For example, the branching decision treestructure used by Iuppa et al. (2004) encodes multiple pre-authored plans into a singledecision tree. A learner interacts with the system by selecting events that branch thestoryline in different ways. The control a learner has to shape the direction of the storyenhances his or her engagement with the situation and leads to a deeper engagementwith the material to be learnt.

Peinado et al. (2005) devised an interactive learning environment (ILE) thatmatches a learner’s actions against set storylines. If the learner deviates from a setstoryline the system generates a new storyline that will, as far as possible, realise theobjectives of the original learning plan. This is achieved with a case-based reasoningparadigm. Cases are encoded as story plots and case adaptation is used to generatenew storylines.

Weber and Brusilovsky (2001) represented concepts and rules associated withcomputer programming as cases in their adaptive and intelligent web-based educationalsystem ELM-ART. Students enter an interactive example, write a fragment of LISP code,and receive feedback on its quality. Incorrect code is diagnosed using case-based reasoningmechanisms. In addition to the use of cases to represent and reason with domainknowledge, Weber and Brusilovsky (2001) used cases to represent student experiencesabout learning. The ability of a student to view and edit their student model that an ILEmaintains is particularly important given the emerging trend towards life-long learning.

The pedagogy that is appropriate for learning expert level reasoning tends to bedifferent from the pedagogy for understanding basic knowledge in the discipline, or thatfor studying clinical problems in the abstract. Evidence indicates that this is achievedthrough induction from cases. Frize and Frasson (2000) distinguished three levels ofcognitive learning styles that are evident in medical schools. Figure 1 indicates the differentpedagogies that are appropriate for different learning contexts.

Figure 1. Frize and Frasson’s pedagogical approaches for different learning levels.

266 A. Stranieri and J. Yearwood

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

At the beginning of their curriculum students learn basic knowledge to build a base offundamental knowledge, as illustrated at the first level in Figure 1. In progressing toclinical problems, the second level shown in Figure 1, students acquire some proceduraland contextual knowledge. They make hypotheses and start to establish strategies forselection and rejection of these. The top layer of Figure 1 is concerned with acquiringknowledge through exposure and interaction with real situations or cases. The learner canaccumulate experience with a more complete base of cases and may reach expert status bybeing able to induce, from a set of cases, new rules or new cases. The transformation fromthe second level to the upper, expert level is achieved through experience with complex realcases. This fits within the framework of transformationalism (Mezirow, 2000) and in mostsituations the transformation from novice to expert will be incremental. Mezirow alsosuggested that transformation can be triggered by narrative that relates to the learner’sown experience.

Exposure to real-life problems in an interactive simulation of the real situation istargeted at facilitating the transition from novice to expert. However, given that narrativeplays such an important role in human learning, an ILE based on narrative structures islikely to enhance the learning.

There are three important aspects to a narrative-based ILE that aims to engage users tolearn expert level reasoning:

. its intelligence is based on a model of expert knowledge;

. this knowledge can be used to generate narrative descriptions that express practicalreasoning situations;

. its interaction is based on the learner interrogating or acting in the environment andthe system providing response and narrative.

According to Geoffrey (2005) the critical element in learning to reason as an expert isdeliberate practice with multiple examples which, on the one hand, facilitates theavailability of concepts and conceptual knowledge (i.e. transfer) and, on the other hand,adds to a storehouse of already solved problems. It is not surprising that case-basedlearning has been employed in law schools for many decades. In general students arepresented with a story or narrative of events and this is either read or acted out bystudents, leading them to a correct response or to understanding the effects of theirdecisions. Cases have been prominent in teaching about roles in which decisions have to bemade and are less likely to be used in school situations (Merseth, 1991). The benefits ofusing cases and stories for instruction have been demonstrated in many studies (Bransford,Sherwood, Hesselbring, Kinzer, & Williams, 1990; Bransford & Vye, 1989). Hung, Tan,Cheung and Hu (2004) discussed possible frameworks and design principles of good casestories and narratives. According to Herreid (1998) a good case:

. tells a story – has an interesting plot that relates to the experiences of the audienceand has a beginning, a middle, and an end;

. focuses on an interest-arousing issue – there should be drama and a case must havean issue;

. should create empathy with the central characters;

. should have pedagogic utility;

. is conflict provoking;

. is decision forcing;

. has generality.

Interactive Learning Environments 267

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

In this paper we describe an ILE that is underpinned by two pillars.

. A strong model of expert reasoning in the subject matter domain. A strong model ofexpert reasoning is provided by a knowledge representation model, called the genericactual argument model, advanced by Yearwood and Stranieri (2006), deployed innumerous knowledge-based systems to date, including Nurse, a system that depictssteps of best practice for critical care nurses to take when responding to a low oxygenalarm in a ventilated patient.

. A connection of the reasoning model to a narrative structure in a way that allowsemergent narrative from learner interaction. The narrative structure is based on themodel described by Bennett and Feldman (1981). The ILE reflects the emergentnarrative as a ‘story so far’ feature that acts as the voice of a third person narratorarticulating the emergent narrative as feedback to the learner. The narrativestructure also captures a sense of drama so that the ILE can guide the narrativetowards dramatic climaxes.

Rather than hard coding multiple storylines, the ILE presented here begins a scenario andthen enables a learner to perform actions on the patient. The ILE uses a strong domainmodel to infer whether the actions are appropriate and, if not, to identify the consequencesof the error. This approach draws on the ‘‘recovery boiler tutor’’ (RBT) approachadvanced by Woolf, Blegen, Jansen, and Verloop (1987), which deals with problems withthe recovery boiler, used in the production of pulp, which need to be diagnosed quicklyand accurately by plant controllers to avoid explosions and serious injuries. The RBTenables plant controllers to literally play with problems and refine their own problem-solving behaviours. As a user performs actions he/she is provided with feedback about theappropriateness of the actions and the consequences of mistaken actions in the form of astudent–system dialogue.

The ILE presented here differs from the RBT approach in that the underlying domainmodel derives from a generic model of knowledge representation described in the nextsection (Yearwood & Stranieri, 2006) and the student–system interface focuses on theinteractive construction of a case study. The narrative that emerges from the interaction ofthe learner and the ILE is responsive to the learner’s actions. The consequences are usedby the ILE to set events that will propel the emergent narrative towards critical outcomes,such as patient death, and important narrative elements, such as setting, concern, andresolution, underpin the emerging narrative.

The narrative theory of Bennett and Feldman (1981) described the structure of anarrative as consisting generally of a setting, concern, resolution sequence. The settingusually includes the time, place, and some of the characters. The concern is an action that,given the setting, creates a climactic (eventful, ironical, suspenseful) situation. Forexample, if someone is rock climbing and slips and falls, slipping and falling are theconcern. If the story ended at this point the audience would be left wondering, whathappened to the climber, was he hurt or killed? A complete narrative will provide ananswer to these questions. This stage is the resolution. The central action is the structuralelement that creates the central question the story must resolve. The resolution normallyresolves both the predicament created by the problem and the questions listeners mighthave had about the outcome. In the rock climbing narrative the resolution consisted oftelling the audience that the climber was taken to hospital for treatment.

The setting, concern, resolution sequence of Bennett and Feldman (1981) is well-suitedto the ILE in this work. The rich domain model deployed in the ILE is sufficiently

268 A. Stranieri and J. Yearwood

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

expressive to obviate the need to embed structures to represent causal relations betweenelements found in more complex story grammars. A relatively simple story grammar issufficient because the domain model is so expressive. The domain model deployed is basedon argument structures and is described in the next section. The ILE is presented afterthat, and an empirical evaluation follows in section 4.

2. Representation of the domain reasoning

An approach to representing knowledge called the generic actual/argument model(GAAM) has been advanced by Yearwood and Stranieri (2006). The GAAM model hasbeen applied to the development of numerous decision support systems in law, including:Split up, predicting the percentage split of assets a Family Court judge will awardsdivorcees (Stranieri, Zeleznikow, Gawler, & Lewis 1999); Embrace, assessing the strengthof claims for refugee status (Yearwood & Stranieri 1999); GetAid, determining eligibilityfor legal aid in Victoria (Stranieri, Zeleznikow, Gawler, & Yearwood, 2001); witnessselection in Scotland (Bromby & Hall, 2002). An engine developed to implement theGAAM (justReason, available online at www.justsys.com.au) automatically generates aweb-based decision support system accessible with any web browser.

The GAAM represents reasoning to a decision at two levels of abstraction, the genericargument level and the actual argument level. A generic tree that captures reasoningregarding risks associated with rooflight design is illustrated in Figure 2. The ‘root’ of thetree is the Occupational Health and Safety (OHS) risk rating associated with a particularrooflight element in a building. The linguistic variables ‘‘extreme,’’ ‘‘high,’’ ‘‘moderate,’’and ‘‘low’’ represent acceptable terminology for denoting the magnitude of risk in that field.

Every variable in a generic argument tree has a reason depicting its relevance. Thefactors ‘‘likelihood that an injury or illness will occur’’ and the ‘‘likely severity of theconsequence of that injury or illness should it occur’’ are relevant because riskmanagement theory and Australian legislation dictates that these two factors are relevantfor determining risk. Argument trees are intended to capture a shared understanding ofrelevant factors in the determination of a value (in this case the level of OHS risk).Irrelevant factors are not included in an argument tree. Thus, the rooflight colour is notconsidered relevant by designers or safety experts, so t is not represented as a node in thetree, although one can imagine circumstances where a colour is indeed relevant to OHS,such as in the specification of emergency lighting or signage.

An actual argument is an instantiation of variables in the generic tree by setting leafnode values and inferring up the tree. A linguistic variable value on a parent node isinferred from values on children nodes with the use of inference procedures. An inferenceprocedure is essentially a mapping of child variable values to parent variable values. InFigure 2 the inference procedures are denoted by A, B, C, D, and R. Thus, for example,the inference procedure R could take the form of a commonly used risk matrix whereassessments of likelihood and consequence combine to determine the level of riskpresented by a hazard. Thus a hazard for which the likelihood of occurrence is ratedmoderate but the consequence is major would be considered ‘‘Extreme.’’

For example, the risk rating is inferred using an inference procedure, R, from valuesinstantiated on two factors: In Australia the inference derives from a risk matrix formulaset by a government-based standards organisation. The height and location of a rooflightare factors that lead to an inference describing the consequence of a fall (i.e. the severity ofthe injury). The trolley system and protection for external works are used to infer anoverall level of protection and, therefore, the likelihood that a fall will occur. The existing

Interactive Learning Environments 269

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

Figure

2.

Thestructure

ofanargumenttree

inoccupationalhealthandsafety.

270 A. Stranieri and J. Yearwood

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

protection is also coupled with the frequency with which the rooflight will be maintainedto infer the likelihood of a fall.

In argumentation-based knowledge based systems (KBS) different inference mechan-isms can be used according to the nature of the knowledge being modelled. For example,in the ‘Split up’system (Stranieri et al., 1999) neural networks trained on data drawn fromdivorce property judgements were used to infer about half of the 35 nodes. In a differentsystem, known as ‘‘Embrace,’’ which supports the determination of someone’s refugeestatus, inferences were always left to the discretion of the decision-maker (Yearwood &Stranieri, 1999). In another system called GetAid Stranieri, Zeleznikow, and Yearwood(2001) assigned weights to each linguistic variable and then summed these weights andcompared the result with a pre-determined threshold to infer eligibility for legal aid.

Argument trees, such as that depicted in Figure 2, represent a template for reasoning incomplex situations. Thus, in a discussion about the level of risk posed by a particularrooflight, two designers might disagree at the root node level in that one designer perceivesthe risk to be high while the other perceives it to be moderate. This difference in perceptionmay derive from the different values assigned by each designer to subordinate nodes in theargument tree. For example, when one designer believes that existing protection iscertainly adequate whereas the other does not. The difference may also derive fromalternative inference procedures; one uses inference A, while the other uses a differentmapping mechanism. However, although the two designers disagree, they can bothreasonably accept the argument tree structure as a valid template for the expression oftheir beliefs.

Two designers may disagree on the tree itself in three ways: a node’s values may not beagreed upon; the relevance of the node may be questioned; the placement of the node maybe questioned. Dryzak and Neimeyer (2006) labelled agreement on relevant factors withindeliberative group reasoning a meta-consensus. In their work with citizen panelsdiscussing controversial government policy issues meta-consensus occurred relativelyeasily. Similarly, Yearwood and Stranieri (1999) noted that decision-makers in thediscretionary fields of refugee law and family law in Australia frequently arrived at varieddecisions yet readily agreed on an argument tree structure.

The generic argument level is very useful in determining the mapping to the narrativemodel because it is rich in contextual information, as well as providing information onreasons for relevance of premises (which ensures coherence) and reasons for inferenceprocedures (which captures values and principles behind the reasoning), as well assequencing information for events when their order is critical to the reasoning.

Table 1 illustrates the elements of the GAAM and the corresponding story elementsthat comprise the structured reasoning to narrative mapping central to the ILE.

The mapping of the GAAM to story elements provides the framework for the ILE.While some story elements are not represented in the GAAM, for example narrative voice,others have a limited representation. For instance, a character element corresponds to arole context variable value in the GAAM model. This does not represent all of theattributes and intentions that might be found in the story model of the character andpresents a minimalist character model. Character-based approaches exemplified byCavazza et al. (2002) and Peinado et al. (2005) involved the inclusion of an extensivemodel of character behaviour into non-player characters to make them respond to userchoices in flexible ways, thereby overcoming the need to hard code storyline branches.Instead, the ways in which an ideal expert would behave derives from the knowledgeencoded in the domain model. In the next section an example from the domain of criticalcare nursing is provided that illustrates the mechanisms in some detail.

Interactive Learning Environments 271

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

3. Extended example: interactive learning for intensive care nursing

Advances in critical care technologies and practices over recent decades have led todecision-making settings that are complex and demand extensive nursing training.Monitoring and responding to ventilated patients’ gaseous exchange is a central role forintensive care unit (ICU) nurses. Van Horn (1986) argued that there are too many factorsor possible solutions for a human to remember at once, even when the problem is brokendown into smaller pieces. Decisions must be made under pressure, and missing even asingle factor could be disastrous.

The decision-making involved in determining the actions a nurse should perform whena low oxygen alarm sounds is typically taught informally, ‘‘on the job,’’ in conjunctionwith some classroom exercises. In practice, nurse educators aim to instil knowledge ofthree aspects of practical reasoning to novices.

. Action – what action to perform next. For example, an action an ICU nurse has tolearn to perform is to check the shape of the pleth wave. This is a wave displayed ona monitor that is derived from an infrared finger probe detecting the level of oxygenin the bloodstream. The action that is to be performed has two contexts; one wherethe nurse has previously executed a wrong action or omitted an action, or where allprevious actions were correct.

. Incorrect action consequence – this is the consequence of performing the incorrectaction. For example, changing the finger probe is the correct action when the oxygenalarm is sounding and the pleth wave is inaccurate. An inaccurate pleth wave oftenindicates that the probe is not reading accurately. Increasing the oxygen levelwithout first checking the pleth wave is inappropriate, because the patient may notbe short of oxygen.

. Omission consequence – this is the immediate consequence of failing to perform anaction when it is should be performed. Failing to administer pure oxygen to thepatient when the alarm has sounded and the pleth wave is accurate results in apossible state of insufficient oxygen.

Reasoning involving the action and consequences following a low oxygen alarm in anAustralian hospital has been modelled using decision trees, described in Stranieri et al.

Table 1. Mapping of GAAM to Story elements.

Reasoning: GAAM element Story element

Context variable values SettingRole context variable RoleRole context variable value CharacterSet of child factor values,e.g. the likelihood of injuryis certain, the consequenceof injury is catastrophic

Concern. Our patient will be exposedto certain catastrophic injury

Inference procedure reason,e.g. the risk matrix, R in Figure 2

Point. Protagonists exposed tocertain catastrophic injury is at extreme risk

Parent factor value, e.g. thelevel of risk is low

Resolution

Not represented Narrative voiceReason for relevance ofchild factors

Coherence

272 A. Stranieri and J. Yearwood

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

(2004). In that study reasoning was modelled using a decision tree in order to implement adecision support system that represented best practice in critical care nursing. The decisiontree structure has been converted to an argument tree representation, shown in Figure 3,for the ILE.

The data items (extreme left) represent possible events or causal factors in ICUsituations. The claim variable (extreme right) represents the action an ICU nurse shouldperform at a point in time in a given situation. Inference procedures (arrows) representinference procedures that will be invoked to infer a value on each claim variable for any setof input data items. In the sample tree of Figure 3 there are many thousands of possibleunique inferences for the correct action from the data items, including unknown values.The inference procedure in this study was implemented as rules organised in a preferencehierarchy as follows.

(1) Rules with the highest priority represent actions that derive directly from a bestpractice protocol. A best practice protocol, often represented as decision trees,captures a sequence of correct actions. For example, the ICU protocol elicited inan earlier study by Stranieri et al. (2004) guidelines indicated that the oxygen levelshould be increased immediately after the SpO2 dropped and the pleth waveappeared accurate.

(2) Rules that derive from well-defined situations outside the protocol. For example, ifall of the data item values are unknown then the correct action is to check theSpO2.

(3) Rules that derive from expert heuristics that depict correct actions for situationswell outside the protocol. For example, checking the breath sounds immediately

Figure 3. ICU argument tree.

Interactive Learning Environments 273

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

after the SpO2 drops is incorrect. However, an expert nurse can specify what actionought to occur next given the error.

The ILE functions used the Narrate–Set–Infer cycle illustrated in Figure 4. The Narratephase generates a story depicting actions and feedback to date. This acts as the narrativevoice making the emergent narrative explicit. In the Set phase the learner guides thenarrative by selecting an action to perform and setting data item values to do withthe patient or equipment. The Infer phase involves the ILE executing an inferenceprocedure stored in the argument tree to infer what the correct next action should be.

The learner has initial input into the story by setting context variable values such as thename and gender of the patient and nurse. Figure 5 illustrates the main screen for the ILE.On the left is a list of all actions available to the nurse. The top pane on the right providesthe narration to date. Beneath that the learner is prompted to set data item values for the‘‘Check the signal indicator or pleth wave’’ action that was selected prior to the display ofthe screen in the Set phase. Once an action is selected and a data item value set the systeminvokes the inference procedure in the argument tree to determine what the correct nextaction should be (Infer). If the next action the learner selects is not correct two segments oftext are generated for the Narrate phase; a segment explaining why the action wasincorrect and another explaining the consequences associated with the non-performance ofthe correct action.

Table 2 depicts text segments associated with four actions. Table 3 depicts segmentsassociated with the omission of four actions.

Figure 6 illustrates a screen that re-presents the narrators voice back to the learner followingthe setting of the pleth wave to accurate. The learner is about to select the next action to infer,which is to check the leads.However, behind the scenes, the Infer phase has determined that thecorrect action to perform next is to increase oxygen. Figure 7 depicts the Narrate phase screenthat displays the incorrect action text explaining why checking the leads was not appropriateand the omission text explaining why increasing the oxygenwasmore important. TheNarrate,Set, Infer cycle continuesuntil a predefinedend state is reached.These end states depict recoveryor escalation of the concern to a point where a doctor is called.

The extended example illustrates the interactive learning environment with intensivecare nursing. The same ILE can also be used to automatically generate many different case

Figure 4. Narrate, Set, Infer cycle.

274 A. Stranieri and J. Yearwood

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

studies. In problem-based learning and learning through cases considerable resources areexpended for the construction of problems/cases and the provision of related resources.The construction of cases depicting past or hypothetical scenarios in a non-interactiveformat is important for the early stages of the transformation from novice to expert, asillustrated in Figure 1. The automated generation of case scenarios from a strong domainmodel is a useful function of a ILE in a non-interactive mode of control. In automatedcase study generation mode a ILE can randomly select actions and set data item values.The case study is generated as the output of the Narrate phase.

No attempt to include a student model was made in the ILE presented here. Thismeans that the system does not record past case studies generated by the student, or theirpast errors or concepts understood or misunderstood. This places additional cognitiveload on the learner to remember their own history, but simplifies the system considerably.One ramification of the lack of a student model is that the system cannot identify whetherthe student repeatedly makes the same mistake.

The next section describes an empirical study performed to determine whether the ILEfacilitates learning compared with other approaches.

4. A comparative study of learning outcomes with the ILE

Eighty-five students enrolled in a third year Bachelor of Nursing unit ‘‘Foundations ofmedical/surgical nursing’’ at the University of Ballarat participated in the study. Studentshad allocated themselves to tutorial groups at commencement of the semester. One

Figure 5. SET phase screen prompting the learner to set pleth data item values.

Interactive Learning Environments 275

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

tutorial comprising 20 students was randomly nominated as the narrative group. Anothertutorial with 27 students was randomly nominated as the decision graph tree group, andthe remaining tutorial group, comprising 38 students, was randomly nominated as acontrol group.

Two expert critical care nurses conducted one two-hour lecture for all third yearstudents together and three two-hour tutorials to each of the three tutorial groupsseparately. There was no indication from the subject lecturer that students in one tutorialgroup were more proficient, experienced, or engaged with ICU concepts and proceduresthan those in the other groups. Knowledge of ICU troubleshooting reasoning is highlyspecialised and rare outside a critical care unit. Consequently, specialist knowledge wasassumed to be so low in all groups prior to the lectures and tutorials that a pre-test wasconsidered unnecessary. A standard introduction, common to all the tutorials, wasdelivered by the two nurses. The tutorial for group 1 followed the tutorial introductionwith the decision graphs, which were a graphical representation of the knowledge elicitedfrom the expert ICU nurses. Tutorial group 2 received the common introductory materialbut were then briefly shown how to use the ILE and left to explore scenarios using the ILE.The remaining tutorial was conducted using clinical case studies without decision graphs

Table 2. Incorrect action text.

Action selected Action selected was incorrect

Check the SpO2 level [Nurse:name] needs to continuously check the SpO2but it was more important to take a more critical actionright now. The SpO2 was

Check the pleth signal [Nurse:name] needs to continuously check the pleth but itwas more important to take a more critical action rightnow. As it happens the pleth signal was

Administer bronchodilators Administering bronchodilators right now was not theright thing to do. Taken inappropriately bronchodilatorscan compromise the patient and should only beadministered after chest auscultation indicates a wheeze.

Obtain a CT pulmonary angiogram Preparing the patient for a CT pulmonary angiogram wasincorrect. It is invasive and was unnecessary.

Table 3. Action not performed text.

Action Omission

Check the SpO2 level Without checking the SpO2 [Nurse:name] has no way of knowing[Patient:name]’ s oxygenation status: an important part of arespiratory assessment.

Check the pleth signal Because [Nurse:name] did not check the pleth [Nurse:gender[m:he][f:she]] does not know whether the SpO2 is accurate.

Administer bronchodilators Without bronchodilators [Patient:name]’ s bronchospasm hasworsened. [Patient:gender [m:he][f:she]] is quite respiratorilydistressed. [Nurse:name] is extremely concerned for[Patient:name] and calls a MET response.

Administer diuretics Without administering diuretics the fluid overload exacerbated.The cause of the fluid overload could not be ascertained and[Patient:name] died within minutes. [Nurse:name] is stunnedit all happened so quickly.

276 A. Stranieri and J. Yearwood

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

and computer programs. This is the usual presentation for the unit content and was usedas a control group.

All three groups were conducted in similar, but not identical, standard tutorial roomswith the two nurses and the tutor in charge. Following the tutorial consenting students inall three groups were asked to complete the same 20 minute questionnaire before leavingthe tutorial. Completion of the questionnaire was not part of any assessable task as part ofthe unit and students who did not wish to participate were free to leave.

The questionnairewas coded either as 1, 2, or 3 for type of teaching strategy used, andwasanonymous. The questionnaire comprised a plausible case study and questions related to thestudents’ perception of whether the reasoning displayed by a nurse in the scenarios wasappropriate. The questionnaire responses were coded and scored on a depth of under-standing scale ranging from 0 (no apparent understanding of appropriate problem-solvingreasoning) to 17 (very deep understanding of concepts and their application). A short scalefor measuring satisfaction with the teaching strategy was also developed, with nine itemsscored on a Likert-type scale with five responses ranging from strongly agree to stronglydisagree. In addition to this, the two expert critical care nurses recorded their qualitativeassessments of the extent to which students seemed to be engaged by the tutorial.

4.1. Results of the study

Table 4 shows the number of subjects in each group that contributed responses on the‘test’ questions and the number that responded to the questions on satisfaction. Statistical

Figure 6. NARRATE phase screen after pleth set to accurate.

Interactive Learning Environments 277

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

tests of normality and tests for equal group variances provided evidence that the between-group variances were sufficiently small to use equal variances and the distributions weresufficiently close to normal to use a t-test for independent samples. From the fourthcolumn in Table 4 it can be seen that the mean learning results, as measured by the testquestions, were best for the group using the decision graph and then the narrative ILE.The t-statistics in column six show that both the narrative ILE group and the decisiongraph group performed significantly better than the group that underwent the standardcase study approach. The results are best interpreted as indicative only, despite thestatistical significance, because the three groups had access to different resources during

Table 4. Mean test and satisfaction scores and t-scores comparing group means.

Group Test n Mean + SD t df Significancea

Standard Learning 38 8.53 + 2.748Satisfaction 37 35.03 + 2.733

Decision graph Learning 27 11.52 + 2.359 4.581 63 0.001Satisfaction 25 36.12 + 2.166 1.674 60 0.099

Narrative ILE Learning 20 10.10 + 2.404 2.161 56 0.035Satisfaction 20 34.05 + 2.837 71.271 55 0.209

a2-tailed.

Figure 7. NARRATE phase screen after an incorrect action.

278 A. Stranieri and J. Yearwood

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

testing. The decision graph group was allowed to have access to the decision graph as theyanswered the test questions. The narrative ILE group did not have any resources otherthan the common material of lectures and their learning from the ILE interactions.

Satisfaction scores were lowest for the narrative ILE group. Responses to an open-ended question indicated some frustration with relatively minor software problems duringthe trial. The critical care expert’s qualitative assessment of engagement indicated that thenarrative ILE group were clearly far more engaged than the other groups. Theirdescriptions referred to a great deal of interaction amongst students and between studentsand themselves. The expert’s descriptions also noted that students in the narrative groupengaged with many scenarios in a short time. Each scenario was relatively short, so thestudents quickly commenced a new scenario exploring different narratives each time.Many individuals in the narrative group, once they had become accustomed to theenvironment, humorously tried to make the virtual patient die. This certainly contributedto engagement as perceived by the experts, although the differences did not attainstatistical significance in satisfaction as perceived by students.

5. Conclusion

We have described the design of a narrative-based interactive intelligent learningenvironment which aims to elucidate reasoning by generating scenarios that capture thereasoning and provide feedback as narrative from a narrator. The approach relies on astrong domain model of reasoning. It is expected that as knowledge-based systems becomemore prevalent, models will be more readily accessible, although their existence alone willnot enhance the transformation from novice to expert. An interactive learningenvironment that embeds domain knowledge into a narrative scenario has the potentialto aid the transformation. One of the key features of this approach is a mapping from thereasoning represented using the GAAM to a narrative model.

The basis of the system and its functionality has been illustrated using an extendedexample. The ILE allows the user to interactively interrogate or select actions within asetting and receive feedback in the form of a narrative. The learner has far more control inthe generation of the narrative in this approach than branching storyline approaches.

Future research involves the incorporation of a dramatic model, such as that advancedby Polti (1922) to guide the ILE in making events occur for dramatic impact. For instance,if the learner forgets to increase the oxygen the ILE will immediately make the patiententer a hypoxic state. Future work also involves transposing the ILE into a three-dimensional environment, such as a game engine, to enhance interaction and engagement.

An empirical study of the learning benefits of using this approach over traditionalapproaches with final year nursing students has shown an improvement in learning overstandard case study teaching. The combination of such an environment with traditionalteaching would provide added exposure to practical decision-making that is close to realdecision-making and enhance learning outcomes.

Notes on contributors

Dr Andrew Stranieri is Deputy Director of the Centre for Informatics and Applied Optimisation atthe University of Ballarat. His research interests include artificial intelligence, narrative,argumentation and data mining. He has recently published a monograph on data mining in lawand has published over eighty conference and journal articles over the last ten years. The knowledgeengineering technology he helped to pioneer has been used to develop numerous decision supportsystems including the on-line determination of eligibility for legal aid, the training of intensive care

Interactive Learning Environments 279

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

nurses, the prediction of property splits awarded by judges, witness selection, the prediction ofjudicial sentences and the provision of web based career guidance.

John Yearwood is Professor of Informatics in the School of Information Technology andMathematical Sciences and Director of the Centre for Informatics and Applied Optimization. Hisresearch spans areas of pattern recognition, argumentation, reasoning and decision support and itsapplications in health and law. He has been chief investigator on a number of ARC projects in theseareas. His work has involved the development of new approaches to classification based on modernnon-smooth optimization techniques, new frameworks for structured reasoning and their applicationin decision support and knowledge modelling. Some important outcomes relate to the use of textcategorization techniques for detecting drugs responsible for adverse reactions and the developmentof new algorithms. He is an Associate Editor for the Journal of Research and Practice in InformationTechnology. He has over 140 refereed journal and conference publications.

Acknowledgement

This work was supported by the Australian Research Council.

References

Bennett, W.L., & Feldman, M.S. (1981). Reconstructing reality in the courtroom: Justice andjudgment in American culture. Brunswick, NJ: Rutgers University Press.

Bransford, J.D., Sherwood, R.S., Hesselbring, T.S., Kinzer, C.K., & Williams, S.M. (1990).Anchored instruction: Why we need it and how technology can help. In D. Nix & R. Spiro(Eds.), Cognition, education, and multi-media: Exploration ideas in high technology (pp. 115–141).Hillsdale, NJ: Lawrence Erlbaum.

Bransford, J.D., & Vye, N.J. (1989). Cognitive research and its implications for instruction. InL.B. Resnick & L.E. Klopfer (Eds.), Toward the thinking curriculum: Current cognitive research(pp. 173–205). Alexandria, VA: Association for Supervision and Curriculum Development.

Bromby, M., & Hall, M.J. (2002). The development and rapid evaluation of the knowledge model ofADVOKATE: An advisory system to assess the credibility of eyewitness testimony. In T. Bench-Capon, A. Daskalopulu & R.G.F. Winkels (Eds.), Fifteenth Annual Conference on LegalKnowledge and Information Systems, JURIX 2002 (pp. 143–152). Amsterdam: IOS Press.

Bruner, J. (1986). Actual minds, possible worlds. Cambridge, MA: Harvard University Press.Cavazza, M., Charles, F., & Mead, S.J. (2002). Character-based interactive storytelling. IEEE

Intelligent Systems, 17(4), 17–24.Cognition Technology Group at Vanderbilt (1990). Anchored instruction and its relationship to

situated cognition. Educational Researcher, 19(6), 2–10.Dryzak, J., & Niemeyer, S. (2006). Reconciling pluralism and consensus as political ideals. American

Journal of Political Science, 50(3), 634–649.Frize, M., & Frasson, C. (2000). Decision support and intelligent tutoring systems in medical

education. Clinical and Investigative Medicine, 23(4), 266–269.Geoffrey, N. (2005). Research in clinical reasoning: Past history and current trends. Medical

Education, 39(4), 418–427.Herreid, C.F. (1998). What makes a good case? Some basic rules of good storytelling help teachers

generate student excitement in the classroom. Journal of College Science Teaching (January),163–165.

Hung, D., Tan, S. Cheung, W.S., & Hu, C. (2004). Supporting problem solving with case-storieslearning scenario and video-based collaborative learning technology. Educational Technologyand Society, 7(2), 120–128.

Iuppa, N., Weltman, G., & Gordon, A. (2004). Bringing Hollywood storytelling techniques tobranching storylines for training applications. Proceedings of narrative and interactive learningenvironments (NILE 2004) (pp. 1–8). Edinburgh: Center for Computer Games Research.

Merseth, K. (1991). The early history of case-based instruction: Insights for teacher education today.Journal of Teacher Education, 42(4), 243–249.

Mezirow, J. (2000). Learning to think like an adult: Core concepts of transformation theory.Learning as Transformation, 4(4), 335–361.

Murray, J. (2000). Hamlet on the holodeck. The future of narrative in cyberspace. Cambridge, MA:MIT Press.

280 A. Stranieri and J. Yearwood

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014

Neuhauser, P.C. (1993). Corporate legends and lore: The power of storytelling as a management tool.New York: McGraw-Hill.

Peinado, F., Gervas, P., & Moreno-Ger, P. (2005). Interactive storytelling in educational enviroments.In D. Nix & R. Spiro (Eds.), Proceedings of the 3rd international conference on multimedia andICT in education: Recent research developments in learning technologies, III (pp 1345–1349).Extremadura, Spain: Formatex Research Centre.

Polkinghorne, D.E. (1988). Narrative knowing and the human sciences. Albany, NY: SUNY Press.Polti, G. (1922). The art of inventing characters. New York: Century Bookbindery.Riedl, M., & Young, R. (2004). An intent-driven planner for multi-agent story generation.

Proceedings of the 3rd international joint conference on autonomous agents and multiagent systems(AAMAS 2004) (pp. 186–193). New York: IEEE Computer Society.

Schank, R.C. (1996). Goal-based scenarios: Case-based reasoning meets learning by doing. InD. Leake (Ed.), Case-based reasoning: Experiences, lessons and future directions (pp. 295–347). Menlo Park, CA: AAAI Press/MIT Press.

Stranieri, A., Yearwood, J., Gervasoni, S., Garner, S., Deans, C., & Johnstone, A. (2004). Web-based decision support for structured reasoning in health. Proceedings of the twelfth nationalhealth informatics conference (HIC 2004), Brisbane, Australia. Melbourne: Health InformaticsSociety of Australia.

Stranieri, A., Zeleznikow, J., Gawler, M., & Lewis, B. (1999). A hybrid rule-neural approach for theautomation of legal reasoning in the discretionary domain of family law in Australia. ArtificialIntelligence and Law, 7(2/3), 153–183.

Stranieri, A., Zeleznikow, J., & Yearwood, J. (2001). Argumentation structures that integratedialectical and non-dialectical reasoning. The Knowledge Engineering Review, 16(4), 331–358.

Van Horn, M. (1986). A guide to expert systems. New York: Bantam Books.Weber, G., & Brusilovsky, P. (2001). ELM-ART: An adaptive versatile system for web-based

instruction. International Journal of Artificial Intelligence in Education, 12(4), 351–384.Woolf, B., Blegen, D., Jansen, J.H., & Verloop, A. (1987). Teaching a complex industrial process. In

R. Lawler & M. Yazdani (Eds.), Artificial intelligence and education: Learning environments andtutoring systems (pp. 413–427). Norwood, NJ: Ablex Publishing Corp.

Yearwood, J., & Stranieri, A. (1999). The integration of retrieval, reasoning and drafting for refugeelaw: A third generation legal knowledge based system. Proceedings of the seventh internationalconference on artificial ntelligence and law (ICAIL’990) (pp. 117–137). New York: ACM Press.

Yearwood, J., & Stranieri, A. (2006). The generic actual argument model of practical reasoning.Decision Support Systems, 41, 358–379.

Interactive Learning Environments 281

Dow

nloa

ded

by [

Flor

ida

Stat

e U

nive

rsity

] at

21:

02 2

1 D

ecem

ber

2014