Towards an Intelligent Tutoring System Architecture that Supports Remedial Tutoring

Artificial Intelligence Review12: 469–511, 1998. 469c 1998Kluwer Academic Publishers. Printed in the Netherlands.

Towards an Intelligent Tutoring System Architecture thatSupports Remedial Tutoring

JULIKA SIEMERSchool of Computer Studies, University of Leeds, Leeds LS2 9JT, U.K. (Ph.: +44(0)113 2335430, Fax: +44(0)113 233 5468; E-mail: [email protected])

MARIOS C. ANGELIDESCentre for Multimedia, School of Computing, Information Systems and Mathematics, SouthBank University, 103 Borough Road, London SE1 0AA, U.K. (Ph./Fax: +44(0)171-815 7482;E-mail: [email protected])

Abstract. For successful teaching to take place an intelligent tutoring system has to be able tocope with any student errors that may occur during a tutoring interaction. Remedial tutoring isincreasingly viewed as a central part of the overall tutoring process, and recent research callsfor adaptive remedial tutoring. This paper discusses the issues of remedial tutoring that havebeen proposed or implemented to support efficient remedial tutoring. These issues serve touncover any underlying principles of remediation that govern remedial tutoring with intelligenttutoring systems. In order to incorporate these principles of remediation into intelligent tutoringsystems development processes this paper continues with the development of a model that canbe employed in the development of an intelligent tutoring system that is capable of offeringremedial tutoring according to these principles. This model is a formalisation of remedialinterventions with intelligent tutoring systems. To demonstrate how the model can be employedin developing an intelligent tutoring system, INTUITION, the implementation of an existingbusiness simulation game, has been developed. This paper concludes with an illustration ofhow the model for remedial operations provides for remedial tutoring within INTUITION. Theevaluation of INTUITION shows that the model for remedial operations is a useful methodfor providing efficient remedial tutoring.

Key words: business management gaming-simulation, education, intelligent tutoring systems,remedial tutoring

1. Introduction

Intelligent tutoring systems are designed to provide individualised learning.They provide helpful guidance and make the teaching process more adaptableto the student by exploring and understanding the student’s special needs andinterests, and by responding to these as a human teacher does. In order toprovide this adaptability to the student an intelligent tutoring system makesuse of its three knowledge models, i.e. the domain model, the tutoring modeland the student model (Winkels 1992).

VICTORY PIPS: 145612 LAWKAPaire248.tex; 23/10/1998; 23:49; v.6; p.1

470 JULIKA SIEMER AND MARIOS C. ANGELIDES

The domain modelincludes an explicit representation of the domain-specific knowledge and the problem solving knowledge of the topic, whichthe intelligent tutoring system intends to teach the learner. The domain model,therefore, provides the ‘expert knowledge’ against which the behaviour ofthe user can be compared. Intelligent tutoring systems are also equipped withteaching expertise which is contained in theirtutoring model. An intelligenttutoring system has the ability to perform diagnosis of the user’s currentknowledge about the subject area being taught. This is achieved by collectingfeedback from the student during the course of interaction and by analysingthis feedback against a wide range of predefined student behaviours. Thisinformation about the student is stored in thestudent model. The systemuses this information to tailor its instruction according to the needs of theindividual student.

An intelligent tutoring system uses the information from its three knowl-edge models to guide the student’s interaction with the system. Within theinteraction the intelligent tutoring system has to recognise and correct anystudent errors that might occur (Silverman 1992). Experience and recentresearch increasingly call for adaptive remediation of student errors, andremediation is increasingly viewed as a central part of the overall tutoringprocess (Alpert et al. 1995). The research presented in this paper focuses onthe specific aspect of remedial tutoring with intelligent tutoring systems.

The objective of this paper is to uncover the underlying principles of reme-diation which embody the characteristics that an intelligent tutoring system,that offers remedial tutoring, should exhibit, and develop a model that canbe deployed in developing an intelligent tutoring system capable of offeringremedial tutoring according to the underlying principles of remediation.

For this purpose, this paper first stresses the importance of remedial tutoringwithin the overall teaching process. It then commences with an investigationof practices with existing intelligent tutoring systems and proposed ideas toreveal aspects relevant to remedial tutoring. These aspects serve to uncoverthe basic underlying principles of remediation that govern remedial tutoringwith intelligent tutoring systems.

In order to incorporate the principles of remediation into intelligent tutoringsystems development processes this paper continues with the development ofa model that can be employed in the development of an intelligent tutoringsystem that is capable of offering remedial tutoring according to these princi-ples. This model is a formalisation of remedial interventions with intelligenttutoring systems. In this model the principles serve as a basis for providingremediation in an intelligent tutoring system. They are not a panacea becausethe exact nature of remediation depends on the nature and structure of thedomain of discourse as some principles suggest.

aire248.tex; 23/10/1998; 23:49; v.6; p.2

TOWARDS AN INTELLIGENT TUTORING SYSTEM ARCHITECTURE 471

To demonstrate how the proposed model can be employed in developing anintelligent tutoring system, INTUITION, the implementation of an existingbusiness simulation game that has been developed using the proposed model,is presented. The evaluation of INTUITION shows that the model for remedialoperations is a useful method for providing efficient remedial tutoring.

2. The Need for Remedial Tutoring

The purpose of this section is to present the role and significance of remedialtutoring within the overall teaching process. Both diagnosis and remedialtutoring are introduced as integral tasks of an intelligent tutoring system.The section then illustrates how much research in the field of intelligenttutoring systems has concentrated on student and error modelling techniquesin order to improve diagnosis to provide better remediation. However, morerecent suggestions for alternative approaches to tutoring may effect the wayremediation can be viewed and implemented.

The early teaching theory views educational processes as the communi-cation of the domain knowledge to the student and has dominantly beenapplied in systems, such as Computer Aided Instruction systems, whichsimply present the teaching material to the student in a sequential order.Wenger (1987) initially defines this form ofknowledge communicationas‘the ability to cause and/or support the acquisition of one’s knowledge bysomeone else, via a restricted set of communication operations’.

Within this early work little attention was paid to the implementation ofprocedures that correct student errors as they occur during learning. However,humans are well known for being susceptible to errors, and empirical studiesof the behaviour of human teachers have shown that diagnosis and remediationform a substantial part of the overall tutoring interaction (Winkels 1992).Causes of errors may include the complexity of the situation in which adecision has to be taken, time pressure or other stress factors, uncertaintyabout a problem or situation, or decay of knowledge over time (Silverman1992). A teaching process cannot be designed to eliminate all causes of humanerror completely (Clancey and Soloway 1990).

Furthermore, students learn from their mistakes and, in particular, from theprocess of correcting these mistakes (Hasslberger 1994; Fox 1991). Therefore,remediation has increasingly been viewed as an integral part of the overalltutoring process giving the student support where he is weakest.

This recognition of the importance of remedial tutoring has led to an evolu-tion of the overall structure of the teaching process on which intelligenttutoring systems research has been based (Winkels and Breuker 1992). The

aire248.tex; 23/10/1998; 23:49; v.6; p.3


teaching process of an intelligent tutoring system can be divided into fourseparate functions:

� the planning of a series of teaching actions� the monitoring of the execution of these actions with the student, i.e. the

student behaviour is being compared against the expected outcome inorder to detect any errors

� the diagnosis of any detected errors in order to determine the cause of anerror

� the remediation of the error.

Comparing this structure with the earlier view of the teaching process, whichhas been described above as the communication of the domain knowledge,results in the observation that the more recent view of the teaching processfeatures diagnosis and remediation as central aspects. The majority of intelli-gent tutoring systems which have been developed are based on this structureof the teaching process (Ohlsson 1991).

Most of these intelligent tutoring systems were developed based on theassumption that students’ thinking processes can be modelled, traced andcorrected within a problem solving context using computers (Derry and Lajoie1993). This concern for student modelling approaches was a major focus ofintelligent tutoring system research in the 1980s. As a result many successfulsystems have been built using the model-tracing method.

However, diagnostic processes are still very domain specific (Sack 1990).The systems developed, for example, largely fall into procedural domains,such as geometry, programming, physics and algebra. Furthermore, it isargued that research on complete student and error models has hardly shownany progress and serious doubts exist whether such progress can be expectedin the near future or if it is at all possible (Bierman et al. 1992). The doubt thatit is feasible to construct adequate cognitive models has led some researchersto reject the student modelling approach and to investigate better or morecost-effective alternatives.

Whilst both these streams of researchers have contributed to the researchin the field of instructional technology there is a strong belief that the futurelies in merging the findings of the two streams described here. Such a mergemight lead to the reconciliation of the theoretical differences between the twostreams. More importantly, these two streams might complement each other,thereby defining a new mainstream approach to intelligent tutoring systemresearch (Derry and Lajoie 1993).

Both, the late recognition of the importance of adaptive remediation andthe existence of two largely separate research streams, coincide with a lack offormalisation and refinement of the architecture of intelligent tutoring systemsin order to incorporate remedial tutoring. Remedial tutoring requires a good

aire248.tex; 23/10/1998; 23:49; v.6; p.4


foundation to promote successful error elimination in intelligent tutoringinteractions (Winkels and Breuker 1992). Research in the field of remedialtutoring is urgently required (Woolf and Hall 1995; Anderson et al. 1990;Lesgold 1988). The following section, therefore, attempts to merge researchof the two streams outlined above within the area of remedial tutoring. It inves-tigates the aspects that constitute satisfactory remedial tutoring by exploringideas of both these research streams in order to uncover any underlying princi-ples of remedial tutoring which will serve as the basis for the development ofamodel for remedial operationsthat can be employed in the implementationof remedial interventions with intelligent tutoring systems.

3. Towards a Set of Remedial Principles

This section discusses remedial issues that have emerged from system devel-opments and theoretical research approaches in the field of intelligent tutoringsystems. For this purpose this section explores the state of the art of remedia-tion through a discussion of suggestions which have been made and throughan investigation of existing intelligent tutoring systems, placing specialemphasis on those ideas which have been implemented with automated reme-dial tutoring.

3.1. Direct and indirect remedial tutoring

As outlined above student modelling was the major concern of the intelligenttutoring system movement in the 1980s and the use of the model-tracingapproach for diagnosis was a typical feature of the systems that were devel-oped. Within this approach remediation was generally carried out in formof pre-defined error messages which were linked to the result of diagnosis.However, much research is still required to improve diagnostic approaches(Sleeman et al. 1989). The faulty behaviour of the student can currentlynot always be traced to an underlying misconception or missing conception(Payne and Squibb 1990). In order to account for this problem, this paperdistinguishes between two different results of a diagnostic process.� A diagnostic process has a positive result if the process is able to deter-

mine why the student behaved in the way he did, i.e. if it is able todetermine the student’s misconception or missing conception underlyingthe error.

� A diagnostic process has a negative result if the process is unable todetermine why the student behaved in the way he did, i.e. if it is unableto determine the exact cause for the error.

aire248.tex; 23/10/1998; 23:49; v.6; p.5


The level of success of diagnosis depends on how comprehensive the library ofmisconceptions is, and on the complexity of the mechanisms for determiningwhich misconception(s) gave rise to a detected error. In some cases multiplemisconceptions are the cause of the error. The lack of universal sophisticateddiagnostic methods and the consequent possibility of negative diagnosticresults have led to the neglect of remediation. The elimination of a detectederror is a crucial task within a tutorial process (Laurillard 1990), and gooderror diagnosis is not necessarily a prerequisite for good error remediation(Sleeman et al. 1991). Consequently, remedial tutoring does not only haveto be based on positive diagnostic results (De Corte et al. 1991). Remedialaction can be provided for any detected error (Silverman 1992).

At this point it might be adequate to reiterate the difference between errordetection and error diagnosis. The literature frequently views error detectionas part of error diagnosis. However, within this paper the following distinc-tion between error diagnosis and detection is made: Error diagnosis is theprocess of analysing an error in order to understand its cause (Wenger 1987;Self 1992). Diagnosis is usually preceded by error detection usingdifferen-tial modelling(Moyse and Elsom-Cook 1992) where the intelligent tutoringsystem detects an error by discovering a deviation of the student’s perfor-mance from the performance the system expects. Once a deviation has beendetected diagnosis takes place, i.e. the intelligent tutoring system attempts todetermine the cause for the deviation.

The aim of remediation has to be the elimination of a detected error throughthe correction of the cause of the error if the cause has been determined. If thecause of the error can not be determined then the aim of remediation couldonly be the elimination of the external manifestation of the cause of the error.It is currently impossible to guarantee positive diagnosis of any detectederror. As a consequence remediation may either have to treat a diagnosedmisconception or missing conception, or, if diagnosis has been negative, itmay have to treat the external indication of an error in human behaviour,i.e. the erroneous behaviour displayed by the student (Silverman 1992). Thefirst principle of remediation establishes the diversity of the target scope oferrors which remedial tutoring may have to address. It recognises the needfor the elimination of both positively and negatively diagnosed errors throughremedial treatment:

Principle 1: In the case of successful diagnosis remediation can aimdirectly at a diagnosed misconception or missing conception. Alterna-tively, when diagnosis has been negative, remediation can address amisconception or missing conception indirectly through the correction ofthe external manifestation of an error in human behaviour.

aire248.tex; 23/10/1998; 23:49; v.6; p.6


In order to account for both positively and negatively diagnosed errors, thispaper discusses aspects that contribute towards remediation that go beyondthe information that is given by any diagnosed underlying misconceptionsor missing conceptions and includes factors, such as information about thestudent who committed the error and the situation in which the error occurred.

3.2. The timing of the remedial intervention

A remedial intervention may be triggered as soon as an error has been detected.Alternatively, an intelligent tutoring system may accumulate occurring errorsover a certain period and then carry out all necessary remedial tutoring collec-tively at a designated point within the tutoring process. Thus remediation canbe immediate or delayed.

Work on the timing of remedial interventions in intelligent tutoring systemshas so far been largely intuitive. The GIL system, for example, waits forthe student to complete his solution and then provides an explanation oferrors (Van Merrienboer et al. 1992). Anderson’s LISP tutor (Anderson et al.1990), on the other hand, is an example of a system that provides immediateremediation. However, form the empirical evidence from the systems thathave been implemented the following guidelines are beginning to emerge(Silverman 1992).

Immediate remediation has the advantage that it is easier for the student toanalyse the mental state that led to the error and therefore make appropriatecorrections. Furthermore, immediate remediation can give immediate feed-back before the student has to commit himself to an incorrect solution therebyavoiding the frustration that might build up whilst he struggles unsuccess-fully in an error state. However, particularly in less structured domains, suchas business management, a possible danger is that the student might form alocal solution according to the focused remediation given, without viewingor rethinking the solution within the more global context of the overall task.

Delayed remediation on the other hand has the advantage that the errorbecomes visible to the student. Also, students may find immediate remediationannoying. With the LISP tutor, for example, more experienced programmersdid not always like the immediate remediation.

If, however, delayed remediation is chosen as the sole approach withina system there are two inherent dangers. The student might anticipate theremediation and await its arrival without putting maximum effort into thetask. The second danger arises from the experience that judgement biasesare generally hard to remove. Once the student has committed himself to asolution he might be reluctant to accept its inappropriateness when – at a laterstage – remediation takes place.

aire248.tex; 23/10/1998; 23:49; v.6; p.7


Finally, delayed remediation is often more suitable for less structureddomains since it might not always be possible to detect an error within atask before the entire task has been completed.

The above discussion gives rise to the following principle of remediation:

Principle 2: Remediation should be carried out with consideration forappropriate timing. Whilst immediate remediation is more appropriatewithin a structured subject domain, delayed remediation is generallymore suitable within less structured domains.

3.3. Student-invoked versus system-invoked remediation

Both immediate and delayed remediation may either be invoked by the studentor by the system (Silverman 1992). An advanced student may invoke aremedial intervention on an as-needed basis. Program debuggers and spellcheckers, for example, can be considered as providing some form of passiveremediation. The student decides whether he wants to activate remediation.Accordingly the receipt of remediation is voluntary (Alpert et al. 1995). Thiskind of remediation may be suitable for the more advanced student withina narrow teaching domain who can judge whether he can afford to skip aremediating interruption. When the student is a novice, or when the teachingdomain is broader, system-invoked remediation seems more appropriate. Thestudent may require intervention when the student makes a mistake withoutrealising it or when the student does not know what to do next. A studentwho has problems with the English language, for example, may appreciatea system-activated spell-checker or grammar-checker that assists him incre-mentally during the task as each difficulty occurs.

Also, research has led to the observation that student-invoked remediationin which the student can decide when he requires feedback is often moreeffective (Corbett et al. 1990). The PAT (Pump Algebra Tutor) system (Barker1995), for example, offers student-invoked remediation. The student has toask for remedial intervention when he feels that remediation is required forthe mathematical problem he has to solve.

Attempts have been made to bring the two modes together by establishingthe right balance between student-invoked and system-invoked remediationaccording to the student’s individual preference (Moyse and Elsom-Cook1992; Cumming and Self 1991; Milheim and Martin 1991). The Mole-Hill system, for example, which teaches Smalltalk programming offers bothstudent-invoked and system-invoked remediation the its users (Singley et al.1993).

These arguments give rise to the following principle of remediation:

aire248.tex; 23/10/1998; 23:49; v.6; p.8


Principle 3: An intelligent tutoring system may have to provide for student-invoked or system-invoked remediation. The student may want to call upremediation when he becomes aware of an error. Alternatively, he mayneed the system to point out his error to him and invoke remedial tutoring.

3.4. Remedial teaching strategies

After deciding about the dimensions above one is still left with many questionson how to convey the remedial information to the student. These are questionsof remedial teaching style. This section proposes that remedial informationshould be conveyed by a variety of methods, i.e. through the use of differentstrategies. A remedial strategy determines the style of material delivery that isemployed in order to lead the student through the remedial intervention. Manyintelligent tutoring systems apply different teaching strategies in differentteaching situations. Such a variety of teaching strategies may also be usedwithin remedial tutoring. Choosing a remedial strategy is part of planninghow the remedial material should be communicated to the student.

The DOMINIE system (Spensley et al. 1990), for example, is a tutoringsystem which embodies a variety of typical overall teaching strategies tochoose from (Elsom-Cook 1991):� Cognitive Apprenticeship. This strategy is based on the idea that cogni-

tive skills can be learnt in the same way as an apprentice in the craftslearns, by watching an expert in action and asking questions. The appren-tice starts with the performance of small separate tasks which are gradu-ally increased in size or linked to other tasks until the apprentice is ableto perform the entire task by himself.

� Successive Refinement. This strategy is based on the principle that thematerial to be taught should be explained to the student in steps withgradually increasing levels of detail. This way the student is providedwith an initial framework into which subsequent teaching of the domaincan be fitted.

� Practice. The student is presented with a problem on the screen and isasked to carry out a task.

� Demonstration. Presenting the student with an example-demonstrationmay force a student to go through the correct reasoning process. Thismay lead the student to become aware of his error and establish a correctunderstanding of a concept (Alpert et al. 1995).

Intelligent tutoring systems may use these general teaching strategies in reme-dial situations to clarify a particular concept or provide extra exercises to thestudent in order to consolidate his knowledge. However, in addition to theseregular strategies intelligent tutoring systems may also have to employ purely

aire248.tex; 23/10/1998; 23:49; v.6; p.9


remedial strategies that specifically handle errors. Each error that may occurduring a tutorial intervention must be approached with a suitable strategy, andthis knowledge must be part of the tutoring model (Wenger 1987). Severalpurely remedial strategies have been proposed and implemented:

� Socratic Hinting. Socratic hinting attempts to place the user in a specificframe of mind (Silverman 1992). For this purpose the system providesthe student with short reminders or questions which force him to reasonabout what he does and does not know. In this way the student modifieshis concept and debugs his own error (Kearsley 1987). WHY (Stevensand Collins 1977) is an intelligent tutoring system for tutoring on rainfallprocesses which applies Socratic hinting. The Socratic method leadsthe student to find errors or contradictions by entrapping him in theconsequences of his own conclusions.

� Analogue reasoning. Analogue reasoning is a remedial strategy that maybe used to present the student with a situation that portrays his problemfrom a different viewpoint. Alternatively, it may be used to remind thestudent of problem solutions or decisions that were successful for similarproblems in earlier teaching interactions.

� Reteaching. Reteaching is a remedial strategy which attempts to correcterrors by reminding the student of previously learnt concepts or facts(Wenger 1987). Referring to the concept of reteaching as a remedialstrategy may seem unjustified since the recalled teaching process mayuse a regular teaching strategy. However, the remedial aspect of thisstrategy lies in its repetitive character, i.e. in the idea of reminding thestudent by recalling exactly the same learning process.

Both kinds of strategies, i.e. regular teaching strategies and purely reme-dial teaching strategies, may be used for remedial tutoring. The remedialintervention may, for example, employ regular teaching strategies to presentpreviously learnt material to the student in a way different from that of theearlier teaching process. The student may be taken through the entire orthrough part of a teaching process again in order to ‘relearn’ the erroneousconcept.

These ideas suggest that an intelligent tutoring system should ideallyinclude a selection of different strategies for its remedial interventions tochoose from. The issue of selecting an appropriate remedial strategy is a deli-cate one and much further research is required to arrive at a commonly agreedselection mechanism. However, the success and failure of various ideas thathave been implemented and empirical studies that have been carried out haveresulted in some guidelines for successful strategy selection. Although thisselection mechanism has matured as a mechanism for regular teaching strate-

aire248.tex; 23/10/1998; 23:49; v.6; p.10


gies, the underlying ideas and theories can directly be related to remedialtutoring as described below.

In the first instance the range of regular teaching strategies that might beapplicable within a certain remedial situation is restricted by the degree ofstructure of the subject area in which the error occurs. The remedial strate-gies span the scope of subject areas form structured to unstructured (Silver-man 1992). Strategies such as successive refinement and demonstration, forexample, tend to be suitable for more structured problems where a positivediagnostic result might be available providing the details required to providestep-by-step refinement or a demonstration. Strategies such as Socratic hint-ing and analogue reasoning, on the other hand, are more frequently applicablewithin less structured areas.

Once the choice of strategy is restricted by the subject area strategy selec-tion can be directed by following the concept of guided discovery learning(Elsom-Cook 1990). As a result of experience with, and research in, intelli-gent tutoring systems guided discovery learning has recently gained attentionas a promising intelligent tutoring system teaching style. Within this teach-ing style the tutor attempts to reduce intervention as the teaching processcommences. The selection of a teaching strategy takes place within theseglobal constraints and is based on the personal preference of the student fora particular strategy and the past success rates of the student with differentstrategies.

The discussion above demonstrates that attempts have been made to createa strategy selection mechanism. Although further research and experience isrequired to refine this mechanism the following principle of remediation canbe elevated at this point.

Principle 4: An intelligent tutoring system has to be able to apply differentremedial strategies in order to adapt to different remedial situations. Thechoice of strategy depends on the degree of structure of the subject area,and the preference and experience of the student for/with a particularstrategy.

3.5. Strategy delivery

Remedial tutoring has to offer effective feedback and advice through a varietyof different remedial strategies. This will assure that different students receiveremediation in a way they can best understand. However, apart from choosingan appropriate strategy in order to provide adaptability to the student, reme-diation has to be concerned with the way the remedial strategy is presentedto the student (Woolf and Hall 1995).

aire248.tex; 23/10/1998; 23:49; v.6; p.11


The use of different kinds of remedial strategies for the correction of thesame error requires multiple representations of the remedial domain (Canfieldet al. 1992; Cumming and Self 1991). Applying Socratic hinting as a reme-dial strategy, for example, may require a screen that allows for a question-answering dialogue whilst cognitive apprenticeship might call for a pictorialrepresentation of the issues being addressed.

Also, the same remedial strategy may involve the use of different visualpresentations. Analogue reasoning, for example, may require the intelligenttutoring system to provide the same information in different pictorial presen-tations. A primary mathematics student, for example, may be taught theconcept of addition or subtraction by either giving him change when payingfor his shopping, or by asking him to count some sweets.

The requirement for these different kinds of presentation suggests thefollowing principle of remediation:

Principle 5: The use of different remedial strategies requires multiplerepresentations of the remedial domain in formats such as graphics, textand animation.

3.6. Student-centred remediation

Intelligent tutoring systems should be able to provide for adaptive remediationby applying suitable remedial strategies and presentations of each subjectmatter unit as needed, choosing the form that is most beneficial to the studentfor a particular instructional situation (Gegg-Harrison 1992). In order toprovide for this adaptability to a student through aspects, such as the selectionof an appropriate remedial strategy and its presentation, knowledge about thestudent’s needs is required (Cumming and Self 1991).

The student model may provide information about the student’s experi-ence with particular remedial strategies and different ways of presentation.Accordingly, it provides information on approaches that have proven to besuccessful or unsuccessful within earlier remedial interventions, and remedi-ation can adapt accordingly.

A further common discriminating factor to consider in order to distinguishbetween students with different needs, at a stage where the creation of afull blown cognitive model has not yet been achieved, is the differentiationbetween student advancement stages.

Much research has been carried out on the differences between novices andmore advanced learners (Dreyfus and Dreyfus 1986; Silverman 1992). Thefollowing analogy of downhill skiing illustrates why remedial tutoring mayhave to be sensitive to different student advancement stages.

aire248.tex; 23/10/1998; 23:49; v.6; p.12


The novice skier learns to ski with his skis constantly in the� position ofthe ‘snow plough’ allowing him to stop with ease at any point when he feelsfrightened or about to lose control. The intermediate skier learns to use thesnow plough only when turning. Turns are the dangerous points at which heis forced to face directly downhill. At all other times the skis are kept parallel.Finally, the advanced skier keeps the skis parallel at all times. Although thesnow plough behaviour is an ‘incorrect’ skiing mode, newcomers need itfor ease and safety reasons. The advanced skier replaces it with the propertechnique.

This analogy illustrates why remedial tutoring may have to be sensitive todifferent student advancement stages.

Firstly, remediation for students at different advancement stages mayrequire the presentation of different kinds or detail of knowledge. The noviceskier, for example, may require an explanation on how to ski in the snowplough position, whilst the advanced skier requires explanations on how tokeep his skies parallel.

Secondly, the same error may have to be addressed in different ways forstudents at different advancement stages. If, for example, a novice skiermakes a mistake in his snow plough skiing, remediation might have to includedetailed explanations or demonstrations on how to ski properly in the snowplough position. If, however, an intermediate skier makes an error withinhis snow plough turn, he might only need a subtle reminder, because he hasalready been taught about the snow plough.

Accordingly, a novice requires different remediation from an advancedstudent. Whilst the competent student may be able to appreciate and integrateshallow and subtle remediation, the novice student might require the correc-tion or explanation of intermediate ideas. A system, therefore, has to adjustits remedial tutoring to the advancement stage of a student.

These suggestions give rise to the following principle of remediation:

Principle 6: Remediation has to be adapted to the needs of the student. Theneeds relevant to a remedial intervention are manifested in the student’spreference for a remedial strategy or the remedial environment in whichthe error is corrected, and in the student’s advancement stage.

3.7. Student participation in strategy selection and delivery

Although some of the current student models may be able to provide someinformation on the student’s advancement stage and his experience with cer-tain tutoring methods, current student models remain very basic, and forstudent models to attain the desired benefits a lot of further research isrequired (Gegg-Harrison 1992). More complete student models are a long

aire248.tex; 23/10/1998; 23:49; v.6; p.13


term solution. However, we cannot wait for, not even expect, perfect studentmodels.

Suggestions, therefore, have been made to include the student directly inthe selection of tutoring aspects such as the teaching strategy used, the waythe teaching material is presented, and the level of advancement the studentfeels is currently appropriate to his understanding (Lepper and Chabay 1988;Fox 1991). For the overall teaching process Moyse (1992) suggests thatan intelligent tutoring system could simply ask the student for informationabout his experience or preferences and that this form of adaptation and thestudent’s engagement would make the tutoring process more meaningful tothe student. Similarly, Cumming and Self (1991) argue that using multipleremedial strategies implies that the learner may take his share in choosing thestrategy that suits him best. Several systems have implemented the conceptof consulting the student into their overall teaching process.

White and Frederikson (1990), for example, present a system that teacheselectronic troubleshooting. The system attempts to involve the student in theteaching process. Apart from allowing the student to decide for himself whatsort of problem he wants to solve, it also encourages the student to decide whatkind of teaching strategy he wants to use. The Explanation Planner (Woolfand Hall 1995) consults the student about his preferred choice of presentationof the teaching material. The system provides the student with a choice ofalternative presentations allowing him to change between graphics, text andanimation.

Similarly, the idea of student involvement may be used to acquire additionalinformation about the needs and preferences of the student. An intelligenttutoring system may ask the student for an estimate of his advancement stage.In a similar way the student may participate in the selection of remedialstrategy by expressing his needs and preferences for a particular remedialstrategy (Jones et al. 1992; Milheim and Martin 1991; Cumming and Self1991). For this purpose the system may offer a choice of remedial strategiesto the individual student from which he can make his personal choice insituations where alternative strategies are available and the system itself isunable to determine the best strategy.

This suggests the following principle of remediation:

Principle 7: An intelligent tutoring system should consult the studentabout his needs and preferences to complement the information retrievedfrom the student model for the purpose of planning the remedial interven-tion.

Incorporating the student’s preferences into a remedial intervention throughstudent involvement in situations where the system can offer alternative reme-

aire248.tex; 23/10/1998; 23:49; v.6; p.14


dial approaches is of great benefit for teaching and remediation. Studentparticipation increases adaptability to the student’s needs and motivation(Moyse and Elsom-Cook 1992; Clancey 1988). However, there are furtherforms of human participation which have been proposed in order to supportadaptability to the student: the participation of a learning companion and thesupervisor.

3.8. Student collaboration

Although there is evidence that people learn communally, ever since the firstyears of computer-based instruction teaching with computers has largely beendirected towards individualised learning. However, more recent research haspropagated the advantages of student collaboration and their considerationwithin the development of intelligent tutoring systems (Reusser 1993; Kaplanand Rock 1995; Reinhardt 1995).

One typical collaborative learning activity is the situation in which twostudents engage in a dialogue in which they critique each other’s solutionsof a task (Katz and Lesgold 1993). It is claimed that the collaboration ofthe students reveals the students’ different learning approaches. When thestudents bring their different viewpoints to the task, they can achieve greaterinsights from trying to reconcile the two positions or at least to understandhow they could both be valid.

Therefore, it has been argued that the learning companion can supportremedial tutoring by offering his approach to problem solving. If a studentmakes an error within a task a companion student may come in and providethe student with his understanding, i.e. his viewpoint, of the same task (Katzand Legold 1993; Teasley and Roschelle 1993).

This discussion gives rise to the following principle of remediation:

Principle 8: A companion student can provide external remedial support.

3.9. The human supervisor

An intelligent tutoring system offers individualised tuition to the student andis capable of reducing the workload of the teacher significantly. However,the intelligent tutoring system is a teaching tool which cannot replace theteacher completely. Vivet (1992) views the assumption that the intelligenttutoring system can be used without any human help for the student andtherefore act as an ‘automatic teacher’ as a dream. Although the challenge ofrealising this dream may encourage progress in the field, progress may also bepossible by including the human supervisor in the tutoring process. Without

aire248.tex; 23/10/1998; 23:49; v.6; p.15


the continual presence of human tutors, the new educational and pedagog-ical technologies will weaken the quality and efficiency that computer-basedinstructional programs can attain. The human supervisor may support theteaching process in situations where the system comes to a hold or where thestudent does not feel that the system provides an appropriate service (Chanand Baskin 1990). Intelligent tutoring systems have to be developed in such away that they provide for the integration of the human supervisor. Includingthe human supervisor into the tutoring process and thereby into the process ofremediation to provide additional guidance to the student has therefore beensuggested by many researchers (Reinhardt 1995).

These ideas are postulated in the following principle of remediation:

Principle 9: An intelligent tutoring system should seek support from thesupervisor when the remedial intervention appears to be unsuccessful.

This section proposes the principles of remediation which are summarizedin Table 1 and which constitute the basis for comprehensive and satisfactoryremedial tutoring with intelligent tutoring systems.

This paper is aimed at the formalisation of the process of remediationwhich may be triggered during a teaching interaction with an intelligenttutoring system. For this purpose the following section constructs a model forthe implementation of remedial interventions in intelligent tutoring systems.

4. Towards a Model for Remedial Operations in Intelligent TutoringSystems

This section integrates the principles of remediation into amodel for remedialoperations. The model provides a general and flexible basis for the imple-mentation of remedial tutoring in an intelligent tutoring system.

The remedial operation is one possible, but not necessarily unique, wayfor providing remedial tutoring according to the principles of remediation.The model is independent from the domain of discourse and ensures thedevelopment of remedial operations which address any detected error asrequested by principle 1. According to Figure 1 the model for remedialoperations includes the following aspects:� the remedial goal. The remedial goal represents the misconception or

missing conception the remedial intervention aims to correct.� the target level. The remedial target level determines whether the reme-

dial intervention needs to correct an error through the correction of thestudent’s displayed behaviour, i.e. at the behavioural level, or through thecorrection of the error that caused the incorrect behaviour of the student,

aire248.tex; 23/10/1998; 23:49; v.6; p.16


Table 1. The principles of remediation

Principle 1 In the case of successful diagnosis remediation can aim directly at a diag-nosed misconception or missing conception. Alternatively, when diagnosishas been negative, remediation can address a misconception or missingconception indirectly through the correction of the external manifestationof an error in human behaviour.

Principle 2 Remediation should be carried out with consideration for appropriate tim-ing. Whilst immediate remediation is more appropriate within a structuredsubject domain, delayed remediation is generally more suitable within lessstructured domains.

Principle 3 An intelligent tutoring system may have to provide for student-invoked orsystem-invoked remediation. The student may want to call up remediationwhen he becomes aware of an error. Alternatively, he may need the systemto point out his error to him and invoke remedial tutoring.

Principle 4 An intelligent tutoring system has to be able to apply different remedialstrategies in order to adapt to different remedial situations. The choice ofstrategy depends on the degree of structure of the subject area, and thepreference and experience of the student for/with a particular strategy.

Principle 5 The use of different remedial strategies requires multiple representationsof the remedial domain in formats such as graphics, text and animation.

Principle 6 Remediation has to be adapted to the needs of the student. The needs rele-vant to a remedial intervention are manifested in the student’s preferencefor a remedial strategy or the remedial environment in which the error iscorrected, and in the student’s advancement stage.

Principle 7 An intelligent tutoring system should consult the student about his needsand preferences to complement the information retrieved from the studentmodel for the purpose of planning the remedial intervention.

Principle 8 A companion student can provide external remedial support.

Principle 9 An intelligent tutoring system should seek support from the supervisorwhen the remedial intervention appears to be unsuccessful.

i.e. at the epistemic level. Both target levels may be addressed for noviceor advanced students.

� the temporal context. The temporal context defines the most suitablepoint of time within the teaching interaction at which the student shouldbe interrupted in order to receive remedial support.

� the repair plan. The repair plan defines the steps that may form part ofa remedial intervention. It also incorporates the remedial strategy whichdescribes the way in which the remedial material is delivered to thestudent.

� theremedial environment. The remedial environment defines the mediain which remediation will take place. This media maintains the situationsand activities in which the remedial intervention is carried out.

aire248.tex; 23/10/1998; 23:49; v.6; p.17


Figure 1. The model for remedial operations.

The aspects of the model are described in greater detail below.

4.1. The remedial goal

According to principle 1 of remediation, a remedial operation pursues thegoal of correcting any detected error. Error correction may involve both thecorrection of a misconception or the presentation of a missing conception.The remedial goalrepresents the misconception or missing conception thatis addressed by the remedial operation and thereby guides the planning of theremedial operation.

The remedial operation may achieve its remedial goal by aiming for acommon teaching goal in the domain. In this case the remedial goal is equiv-alent to a teaching goal and remediation may be carried out by re-teachinga concept which has been presented earlier in the teaching process, possiblyusing a different teaching strategy. Alternatively, the remedial operation mayhave to pursue a remedial goal which forms part of the remedial domainknowledge in the domain model.

4.2. The target level of a remedial operation

The second consideration of principle 1 is that remediation may either addressa diagnosed misconception or missing conception directly, or indirectly bycorrecting the external manifestation of an error in the student’s behaviour.

aire248.tex; 23/10/1998; 23:49; v.6; p.18


Direct error correction may either involve the direct rectification of a diag-nosed misconception, i.e. the student’s erroneous knowledge state, or thepresentation of a concept the student may be lacking.

In the case of indirect error correction the remedial operation has to addressthe erroneous behaviour of a student without ‘knowing’ the underlyingmisconception or missing conception.

Accordingly, the use of direct and indirect remediation involves modifica-tions at two different kinds of levels of the student’s knowledge state. Themodel for remedial operations distinguishes between the behavioural andthe epistemic target levels which may be addressed by a remedial operation(Silverman 1992; Wenger 1987).

At the behavioural levelthe procedural performance of the student isaddressed. Factual knowledge is not addressed at this level. At this levelthe remedial operation generally employs remedial strategies, such as cogni-tive apprenticeship, Socratic hinting, analogue reasoning and demonstrations.These remedial strategies generally are restricted to exposing correct behav-iour to the student without articulating any justification for the behaviour.Remedial operations that address the behavioural level require some inter-pretation by the student in order to be converted into useful knowledge.

At the epistemic levelthe remedial operation modifies the student’s erro-neous knowledge state via direct communication or through practice thatmanifests the deeper understanding of a concept, like it is done in the SOPHIEsystem where students are expected to run their own mini-experiments topractice and understand electronic troubleshooting (Brown and Burton 1987).When addressing the epistemic level, explanations have to supply the inter-pretations of the concept under consideration. Strategies, such as cognitiveapprenticeship, successive refinement and practice, support the provision ofexplanations of a concept and may therefore be used to address the epistemiclevel.

Principle 6 demands a remedial operation to adapt to students at differentadvancement stages. To keep things simple the model for remedial operationsonly considers two advancement stages: novice and advanced. Each of the twotarget levels above may have to be addressed at different advancement stages.At the same time, different target levels may be addressed in order to adaptremediation to students at different advancement stages. For an advancedstudent the remediation may address the behavioural level, whilst a novicestudent may require remediation that addresses the epistemic level.

4.3. The temporal context of a remedial operation

According to Principle 2 remediation may be carried out immediately afteran error has been detected, or at a later stage within the tutoring process.

aire248.tex; 23/10/1998; 23:49; v.6; p.19


These considerations are summed up under the aspect of the temporal contextin which a remedial intervention may take place. The model for remedialoperations distinguishes between two kinds of temporal contexts in whichremediation may be carried out: immediate and delayed.

According to Principle 3, in both temporal contexts remedial tutoring caneither be student-invoked or system-invoked. In the student-invoked modethe student may then decide whether he wants to receive immediate remedi-ation or whether he wants to see the effect of his error before he is providedwith remedial support. In the system-invoked mode it is left to the intelli-gent tutoring system to determine the most appropriate point of time for theremedial intervention to take place.

The consideration for student-invoked remediation directly implies theneed for student involvement. It implies that an intelligent tutoring systemmay have to provide the student with the option of deciding for himselfwhether he wants to make use of remedial support or whether he intends tocarry on without error correction.

4.4. The repair plan

It is argued that an intelligent tutoring system dynamically adapts its teach-ing processes, including any remedial interventions, through instructionalplanning (Gegg-Harrison 1992). The need for the adaptation of remedia-tion is depicted in principle 6. Planning has long since been recognised as asignificant basis for the overall tutoring process. Analytical investigations ofteaching (Leinhardt and Greeno 1991) have resulted in a characterization ofteaching as a complex cognitive skill which requires the construction of plansfor the various different teaching tasks.

In order to provide for the adaptation of remedial tutoring to the needsof the student as depicted in principle 6, every remedial operation shouldtherefore be structured according to arepair plan. The repair plan describesthe order and the function of the steps that need to be carried out in orderto provide for adaptive remediation of any anticipated student errors (VanMarcke 1992; Winkels 1992). At the same time the concept of a plan allowsfor the incorporation of the remedial strategies proposed in principle 4 whichdetermine the way in which the material is delivered to the student. A repairplan can be viewed as a sequence of steps similar to Wenger’s didacticepisodes within a regular teaching process (Winkels 1992; Wenger 1987):� Introduction: A remedial operation generally commences with an intro-

duction of the remedial intervention to the student. In this first step thestudent is made aware of the interruption.

� Justification of the interruption: The student is then provided with thereason and an explanation for the interruption. This stage is of consid-

aire248.tex; 23/10/1998; 23:49; v.6; p.20


erable importance if the student is not aware of the fact that somethingwent wrong and that an error has been detected by the system.

� Repair: After the stages of preparation the actual repair can take place.The repair may involve linking new knowledge to old knowledge inorder to eliminate a missing conception, or it may involve the correctionof a misconception.

� Consolidation: The remedial intervention may conclude with the consoli-dation of the new knowledge state of the student.

A repair plan may be created under different circumstances or conditions,and based on the suggestions about human involvement made in principles7, 8 and 9, planning may be supported by the system, by the student him-self, by a learning companion or by the supervisor. The model for remedialoperations therefore distinguishes between four different types of planning:pre-programmed planning, student-led planning, companion-led planning andsupervisor-led planning.

Pre-programmed planningA repair plan may be established by the intelligent tutoring system itself.In this case ofpre-programmed planning, the system has overall control ofthe remedial operation. In a pre-programmed remedial operation the systemdetermines the steps of actions the student is led through and the remedialstrategy used to present the selected actions. This selection may be based onfactors such as the misconception or missing conception to be treated, thestudent’s advancement stage and earlier experience with different remedialstrategies which may reveal a preference.

Although intelligent tutoring systems have to provide pre-programmedremedial operations to handle anticipated errors, pre-programmed remedi-ation may not always be the most appropriate way of planning. Principles7, 8 and 9 of remediation suggest the involvement of the student himself, acompanion of the student and the human supervisor in certain situations andunder different conditions. Nevertheless, the co-ordination of the involvementof these three parties has to be controlled by the intelligent tutoring systemitself.

The remainder of this section pursues further the concept of human involve-ment and presents remedial planning activities for a remedial operation whichinvolve the student, a learning companion or the supervisor as suggested inprinciples 7, 8 and 9. Student-led planning, companion-led planning andsupervisor-led planning are discussed in turn.

Student-led planningThe student himself may be involved in the planning of the remedial inter-vention. Principle 7 suggests that an intelligent tutoring system may consult

aire248.tex; 23/10/1998; 23:49; v.6; p.21


the student in order to determine his preference for a particular remedialstrategy. Accordingly, student-led planning may involve the determination ofthe remedial strategy. Also, the student may have the opportunity to select ordeselect certain steps within the repair plan. The student, for example, maybe left with the option of skipping the step of consolidation after the repairhas taken place.

In order to involve the student in the selection of a suitable remedial strategy,the intelligent tutoring system has to provide expertise that can relate remedialgoals to the remedial strategies and which are suitable for the remedial goalto be achieved. The system has to communicate the available choices ofremedial strategies to the student and provide data to the student to enablehim to make informed decisions about his choice.

However, the student cannot always provide all the information and supportthe system requirements, and the system can never know everything it needsto know about the cause of an error (Van Marcke 1992) in order to providesatisfactory remediation. In certain situations the student might unexpectedlymake a mistake which, according to his student model, he should not havemade. In this case, the system should eventually seekexternal support by alearning companion or the supervisor.

Companion-led planningPrinciple 8 suggests that a knowledgeable companion of the student maysupport remedial planning. By keeping track of two (or more) different studentmodels in parallel, the system should be able to incorporate a second student,the learning companion, into the teaching process. In such a collaborativelearning environment, two students interact with the system or with eachother (Katz and Legold 1993; Teasley and Roschelle 1993). Consequently,a learning companion may support the process of remediation when thesystem’s remediation attempts have been unsuccessful or are not available. Inthis case the learning companion may use any information or feedback he hasreceived from the system at an earlier stage during the teaching interaction.Based on this experience and his own views and knowledge about the domainof discourse in which the error occurred, the learning companion can supportthe remedial operation of the system.

Supervisor-led planningPrinciple 9 suggests the involvement of the human supervisor in the remedialintervention. Accordingly, planning may be supervisor-led. This approachmay be applied when several attempts of pre-programmed or companion-led remediation have failed or when the system has come to a halt. Thesystem may then advise the student to consult his supervisor. Alternatively,

aire248.tex; 23/10/1998; 23:49; v.6; p.22


the student may want to seek help from his supervisor when he feels that heis not receiving the remediation he requires (Chan and Baskin 1990).

In order to maintain control over the teaching interaction, the system has todetermine the kind of help the supervisor may be able to provide. The super-visor may decide on the remedial strategy the repair plan will incorporate.

In order to support the remedial operation the supervisor should have theopportunity to familiarise himself with the current knowledge state of thestudent by gaining insight into the student model created by the system.This should include some kind of record of any previous errors and the waythey have been treated. Such a history of remediation of student errors isessential information. Based on his own experience and knowledge about thestudent and the way previous remedial interventions have been attempted,the supervisor may then provide the student with the remedial support herequires.

4.5. The remedial environment: Presenting remedial material to the student

Principle 5 suggests that employing different remedial strategies requiresmultiple representations of the remedial domain. Theremedial environment,therefore, constitutes a further component of the model for remedial opera-tions.

The remedial environment provides the student with the situation or activityin which the remedial domain knowledge is presented and the remedial inter-vention may take place (Canfield et al. 1992; Burton 1988), such as graphicalexample displays as attempted in the SOPHIE system (Brown and Burton1987) and Steamer system (Hollan et al. 1984); dialogue windows as used inthe WHY system (Stevens and Collins 1977), the SCHOLAR system (Collinset al. 1975) and the Explanation Planner (Woolf and Hall 1995); or maybemultimedia as explored in the CLORIS system (Parkes and Self 1990) andthe Mu.P.P.E.T. system (Agius and Angelides 1996).

Apart from providing a means of presentation the remedial environmentmay provide a controlled setting to support companion-led and supervisor-led remediation. In situations where evidence is available that a learningcompanion has knowledge about the remedial goal being pursued, the systemmay ask the learning companion to present an explanation, within a particularenvironment. Similarly, the supervisor may support remediation and carry outthe remediation within a particular situation or activity given by the systemor chosen by the supervisor.

The characteristic aspects of a remedial operation are illustrated in themodel for remedial operations with intelligent tutoring systems in Figure1. Although no existing system encompasses all these aspects, the modelconstitutes a tool for thinking about remediation in computational terms both

aire248.tex; 23/10/1998; 23:49; v.6; p.23


Figure 2. An architecture for an intelligent tutoring system that includes the model for remedialoperations.

in current and future intelligent tutoring systems. The model for remedialoperations serves as a foundation for the development of intelligent tutoringsystems that provide remedial tutoring according to the requirements depictedin the principles of remediation. The model is generic in the sense that it isboth domain-independent, i.e. it is independent of whether a system teachesmathematics, geography or a foreign language, and implementation-free, i.e.it is a development method for intelligent tutoring systems that are capableof providing remedial tutoring.

5. Towards An Architecture for an Intelligent Tutoring System thatIncludes the Model for Remedial Operations

The previous section has discussed the characteristic aspects that constitutea remedial operation. The remedial operation requires remedial knowledgeresources as building material within the domain, student and tutoring modelsof an intelligent tutoring system (Clancey and Soloway 1990). This sectionproposes a revised intelligent tutoring system architecture which integratesthe remedial knowledge required by the remedial operation as illustrated inFigure 2.

aire248.tex; 23/10/1998; 23:49; v.6; p.24


5.1. The domain model

The domain modelhas to be extended by the specific domain knowledgerequired for remedial interactions. Certain errors may be caused by the lackof knowledge that does not directly fall into the domain of discourse, suchas prerequisite knowledge. This knowledge may have to be included in thesystem for the purpose of error detection and correction. The domain modelmay have to support a remedial operation by structuring and representing itscontent in different ways in order to provide for the use of different remedialstrategies and environments.

5.2. The tutoring model

In addition to providing the necessary remedial domain knowledge and itsdifferent kinds of representations, an intelligent tutoring system has to providethe remedial tutoring knowledge required for the correction of errors propa-gated by students. Accordingly, the tutoring model has to contain knowledgeabout the remedial goals that may have to be pursued. The remedial goalsknowledge may include information about remedial strategies that can beused, the target levels at which they may be addressed, at what point of timethey may be pursued, and what steps of action may be appropriate.

Also, the tutoring model incorporates the remedial strategies required toconvey remedial material to the student. Such a remedial strategy can be aregular teaching strategy with the only difference that the goal it pursues is aremedial one, i.e. it addresses the correction of a diagnosed error. However,the tutor model may have to include exclusiveremedial teaching strategiesfor use in remedial operations.

In order to determine whether the remedial process should be carried outat a behavioural or at an epistemic level, the remedial process may requireinformation from the tutoring model about the remedial goal to be pursued.The nature of the remedial goal may determine the target level to be addressed.If, for example, presentation of a missing conception has been selected as theremedial goal due to negative error diagnosis, remediation may implicitlyaddress the behavioural level. On the other hand the same remedial goal maybe addressed at an epistemic or behavioural level depending on the student’sadvancement stage. The intelligent tutoring system may therefore requireinformation from the student model about the student’s advancement stage.

5.3. The student model

In order to provide relevant information about the student, the model forremedial operations suggests the construction of ahistory of remediation

aire248.tex; 23/10/1998; 23:49; v.6; p.25


during a tutorial interaction. The history of remediation is a chronologicalrecord of the student’s detected errors and the ways in which these errorshave been corrected or attempted to be corrected. It can therefore be referredto in order to support the choice of remedial goal, the remedial target level,the remedial strategy, the remedial environment or an appropriate kind ofplanning.

The history of remediation, therefore, forms an additional part of thestudent model. Goodyear (1991) argues that an intelligent tutoring systemmay improve its teaching by using the experience collected during the teach-ing process. A chronological record of treated errors may give informationabout the success-rate of a particular remedial intervention or a specific aspectof the remedial operation, such as a remedial strategy or a remedial environ-ment used with earlier errors.

The student model can also store information on the student’s advancementstage. The intelligent tutoring system may then consider the advancementstage of the student during its remedial intervention.

Furthermore, in order to support companion-led planning, the system has toassure that the student model includes the information required to determinewhether the learning companion has the knowledge required to participate inthe remedial planning process.

6. Developing INTUITION Using the Intelligent Tutoring SystemArchitecture that Includes the Model for Remedial Operations

INTUITION (INtelligent TUITION) is an intelligent tutoring system whichhas been developed using the model for remedial operations as discussedabove. INTUITION is the implementation of the Metal Box Business Simu-lation Game (CRAC 1978) which was developed to give students an insightinto the work of business managers and thereby acquire an understanding ofbusiness management.

The player (or group of players) starts the game as one of the managers ofthe Metal Box Company having to solve financial, production or marketingproblems. The business must be run efficiently to be able to pay for salaries,materials and services and to cover the costs of the development of newproduction resources, such as an expansion of the size of the factory. Addi-tionally, the business has to yield a surplus to make a reasonable profit. Thecompany has up to three players who are appointed to the following roles andwho are responsible for the following tasks: theproduction directorhas tomake decisions on the amount of boilers to produce and has to determine theselling price, thesales directormakes decisions on any market research orresearch and development to be undertaken, the number of sales persons to be

aire248.tex; 23/10/1998; 23:49; v.6; p.26


recruited and which customers the sales staff should call on, and thefinancialdirector has to master all calculations and is responsible for completing theCompany’s accounts.

6.1. The architecture of INTUITION

INTUITION has been implemented on an Apple Macintosh using the object-based hypertext package HyperCard 2.1. The implementation of INTUITIONis based on the standard three-models architecture of an intelligent tutor-ing system and integrates the remedial knowledge required by its remedialprocesses. Accordingly, INTUITION consists of the domain model, thestudent model and the tutoring model as illustrated in Figure 3. As an initialdevelopment stage the knowledge in the domain, student and tutoring modelsis specified and organised into stacks of hypercards. The factual informationof the domain and student models is stored on cards of the domain model andthe student model stacks respectively. The tutoring model which controls howthe student moves through the game is implemented in the scripts of vari-ous Hypertext buttons which are combined on the hypercards of the tutoringmodel stack. All stacks and hypercards are then linked to other stacks andhypercards to construct the required network that represents the INTUITIONgame. The stacks that constitute INTUITION are briefly explained below. Amore detailed explanation can be found in (Siemer 1995).

The domain model

INTUITION contains eight stacks of hypercards which jointly constitute thedomain model: Rules, playing domain knowledge, simulated competitors’domain knowledge, working memory, examples, prompting cards, miscon-ceptions and missing conceptions, and consolidation.

Rules stackThe Rules Stack contains the rules governing the business game. In additionto using its rules hypercards for running the game, the system may use thisstack to explain the rules of the game should the system detect a deviationfrom them, to offer some help in applying them correctly and to correct anymisconception or fill any missing conceptions which occur.

Playing domain knowledge stackThis stack contains the factual data that is required to run the game, such asthe market demand for the products, the distribution of the market demand onthe customers in the current game, and the current selling prices of all threecompetitors.

aire248.tex; 23/10/1998; 23:49; v.6; p.27


Figure 3. The Architecture of INTUITION.

Simulated competitors’ domain knowledge stackINTUITION uses two separate stacks to store the information about thesimulated roles of two competing companies. INTUITION simulates thisinformation about the two competing companies and stores it in these stacksas the game progresses.

aire248.tex; 23/10/1998; 23:49; v.6; p.28


Working memory stackAll correct task solutions and decisions are generated by the system duringgame play according to the given market conditions and progression of thecurrent game. These results are stored in the working memory stack. Inthis way an ‘ideal’ student model emerges against which the solutions anddecisions of the gamed director roles can be evaluated to detect any errors.

Examples stackThe examples stack includes remedial knowledge in the form of exampleswhich may be displayed to the players during a remedial intervention.

Prompting cards stackThe prompting cards stack contains the remedial knowledge in differenttextual formats which the remedial process may request when using spe-cific remedial teaching strategies, such as successive refinement, cognitiveapprenticeship or Socratic hinting.

Misconception and missing conceptions stackTo be able to correct misconceptions or fill missing conceptions the domainmodel has access to a library of all common misconceptions and missingconceptions which might occur during an interaction.

Consolidation stackThe consolidation stack contains the knowledge and procedures which areused to lead a player through a process in which he can consolidate newlyacquired knowledge after the correction of a misconception or missingconception has taken place.

The tutoring model

The tutoring model includes the teaching strategy and the teaching goalknowledge required to run the game. The tutoring model is implemented asprogram scripts on the hypercards of the following three stacks: Tutoringknowledge, planner and student consult.

Tutoring knowledge stackThe tutoring knowledge stack contains the central operating mechanism forrunning and controlling the game. It controls the market situation, the allo-cation of tasks to a particular player, the execution of a simulated roles, thedetection, diagnosis and remediation of errors.

aire248.tex; 23/10/1998; 23:49; v.6; p.29


Planner stackThe planner stack is accessed when a remedial process is carried out either bythe student or by the supervisor. For this purpose the planner stack may offerseveral options (depending on the kind of error to be treated) to the studentto choose from to express a preference for a particular remedial teachingstrategy.

Student consultThe student consult stack provides an interface to the student through whichINTUITION consults the student about the knowledge-aspect(s) of a widerconcept with which the student would like some help following the strategyof cognitive apprenticeship. The wider concept corresponds to the miscon-ception or missing conception which has to be repaired. The student is thenprovided with the relevant knowledge about the selected knowledge-aspects.

The student model

The student model includes the current knowledge of the player about thegame, the role he plays in the current game and the roles he played inprevious games, his performance during the various steps of the current andprevious games and how well he managed the resources he was allocated toby the system. The student model of INTUITION contains six stacks: Initialhypercards, student history of remediation, student overlay model, students’misconceptions or missing conceptions, student decisions, and final report.

Initial hypercards stackINTUITION creates certain stacks during the actual game-play. The initialhypercards stack delivers the first card onto which these stacks can be built.These initial cards contain an overall description of the stack to be built. Everynew player, for example, is assigned his own individual ‘student history ofremediation’ stack as described below.

Student history of remediation stackFor every occurrence of a diagnosed misconception or missing conception thestudent history of remediation stack records all the details about the remedialprocess that has been carried out to correct the misconception or fill themissing conception. In this way a chronological record of all the remedialtutoring is built up during the course of interaction for each individual player.

Student overlay model stackThe student overlay model stack contains the emerging knowledge of theplayer and can be viewed as a subset of the ‘complete’ domain knowledgethe player is expected to acquire during the game.

aire248.tex; 23/10/1998; 23:49; v.6; p.30


Students’ misconceptions and missing conceptions stackThe students’ misconceptions and missing conceptions stack contains a sepa-rate hypercard for every player (previous or current) that records all miscon-ceptions and missing conceptions that have been diagnosed for that player.

Student decisions stackThe student decisions stack records all the decisions made by the managersof the Metal Box Company, and it contains a central interaction card for eachdirector which INTUITION uses during interaction with the relevant director.

Final reportOnce the game has come to an end INTUITION creates a final game reportwhich it presents to the student.

The stacks within the domain, tutoring and student model as described aboveconstitute the architecture of INTUITION. They integrate the remedial knowl-edge required by the remedial operation. The following section describeshow INTUITION’s remedial operation uses its knowledge models to provideremedial tutoring.

6.2. The model for remedial operations within INTUITION

INTUITION’s remedial processes use the system’s knowledge models toprovide adaptability to the needs and preferences of the players. INTUI-TION’s domain model provides the remedial domain knowledge required forthe common errors that may occur within the game. INTUITION’s tutoringmodel incorporates a range of different remedial strategies, and its repair planprovides a tool for the structure and organisation of any remedial process tobe carried out. Planning, i.e. the determination of the steps of actions and theremedial strategy used to carry them out, may be supported by the system,the affected player, a companion player or the supervisor in order to provideadaptability to the player’s preferences and needs. INTUITION’s studentmodels provide knowledge about these needs and preferences, and aboutthe knowledge acquired by the student so far. Furthermore, INTUITION’sstudent models include the students’ history of remediation, i.e. a record ofthe student’s experience with previous remedial processes. After remediationINTUITION takes the student back into the game where he is given thechance to correct his decision.

INTUITION’s remedial capabilities are described below in order to demon-strate how the model for remedial operations can be used for the developmentof a real application. INTUITION provides for remediation according to the

aire248.tex; 23/10/1998; 23:49; v.6; p.31


principles of remediation by employing the model for remedial operations asfollows.

The remedial goal.Error detection and diagnosis take place once the playerhas completed the decisions which form part of a particular task. The player’sdecisions and solutions are compared against the calculated correct solutionsgenerated by the system to detect any errors. INTUITION then attempts todiagnose the detected error and determines a remedial goal, i.e. the miscon-ception to be repaired or the missing conception to be filled.

The target level.A misconception or missing conception may be treated atdifferent target levels. The remedial operations within INTUITION addressboth the behavioural and the epistemic target levels. Wenger (1987) suggeststhat different teaching strategies may be used to address different target levels.INTUITION uses this approach within its remedial tutoring. It uses differentremedial strategies with different target levels. Successive refinement, forexample, is a strategy used in remedial situations to address the epistemictarget level, whilst Socratic hinting is used to address the behavioural level.

The temporal context.Remediation only takes place once all the possibleerrors within a particular gamed step of play have been diagnosed and theassociated remedial goals have been determined, i.e. remediation is invokedby the system and is carried out delayed. The delayed system-invoked reme-dial context is determined by the nature of INTUITION’s domain of businessmanagement. The semi-structured character of this domain suggests a delayedsystem-invoked remedial context, because the decisions within a specific stepof play may be interrelated and influence each other. The system, therefore,waits until all decisions within a task have been finalised before remedia-tion is carried out. For every misconception or missing conception that hasbeen triggered after a gamed step of play, the system determines a particularremedial operation adapted to the individual needs of the player.

The repair plan.The remedial operation follows a repair plan which consistsof steps of action and uses remedial strategies to lead the players through thesesteps of action. Any remedial operation may include the steps of initial inter-ruption, introduction, the actual repair and consolidation in which the studentmay apply his new understanding of the concept involved. The tutoring modelincludes successive refinement, cognitive apprenticeship, reteaching, Socratichinting and demonstration as its remedial strategies. Every remedial goal isassociated with one or more of these remedial strategies.

INTUITION incorporates a mechanism which determines whether theinitial repair plan is pre-programmed, student-led, companion-led or

aire248.tex; 23/10/1998; 23:49; v.6; p.32


supervisor-led. This control mechanism follows certain principles and is influ-enced by certain controlling factors, such as the remedial goal to be pursued,the point of time at which remediation takes place, the history of the student’sremediation and the number of occurrences of the same misconception ormissing conception. Within the different planning approaches, the steps ofaction and the remedial strategy used to take the student through the plan ofactions, are determined as discussed below.

Steps of action.INTUITION follows a repair plan which generally includesthe steps of initial interruption, introduction and the actual repair. The consoli-dation step may be optional depending on the planning approach. Withina repair plan in which the selection of the steps of action is student-led,companion-led or supervisor-led, INTUITION leaves the choice of goingthrough the consolidation step to the student, the companion or the super-visor respectively. In the case where the steps of action are pre-programmedthe novice player is normally taken through an appropriate consolidation stepwhilst the advanced player may be asked to correct his error directly after therepair.

Remedial strategies.A minimum requirement for any selection process is theavailability of alternative options. The same concept applies to strategy selec-tion. Before the system can decide whether planning is to be supported by ahuman, it has to check whether alternative remedial strategies are available forthe misconception to be repaired or missing conception to be filled. If a partic-ular misconception or missing conception can be repaired using one specificremedial strategy only, this remedial strategy will be pre-programmed.

However, INTUITION offers a choice of remedial strategies for the reme-diation of most misconceptions or missing conceptions, and according tothe model for remedial operations the selection of an appropriate remedialstrategy may be carried out pre-programmed, student-led, companion-ledor supervisor-led. INTUITION applies the following algorithm, based onsuggestions made in the relevant literature (Winkels 1992; Silverman 1992),in determining its planning approaches. This research does not claim that thealgorithm is the only suitable approach. Proposing a mechanism that deter-mines the most suitable planning approach for a specific remedial operationgoes beyond the scope of this paper.

For an advanced student who has not committed the error before, the reme-dial operation is pre-programmed. In this case INTUITION uses the assump-tion that for advanced students remediation at the behavioural level may besufficient (Silverman 1992). An appropriate strategy is Socratic hinting wherethe student is provided with a quick reminder to correct his mistake. Accord-

aire248.tex; 23/10/1998; 23:49; v.6; p.33


ingly, INTUITION uses Socratic hinting in its initial attempt to correct theerror if, according to INTUITION’s tutoring model, Socratic hinting is a usefulremedial strategy for the misconception or missing conception under consid-eration. If the advanced student fails to correct his error he is down-graded tothe novice advancement stage for that particular remedial intervention. Forthe novice student the algorithm is as explained below.

If the misconception to be repaired or the missing conception to be filledallows the application of more than one remedial strategy INTUITION incor-porates all four planning approaches in the following order. INTUITION initi-ally uses the student-led planning approach. For this purpose INTUITIONpresents the student with a range of remedial strategies to choose from. Oncea remedial operation has been completed, the player has to go back into thegame to correct his mistake. However, the student’s strategy selection maynot necessarily result in an optimal choice (Moyse 1992; Romiszowski 1990).If, as a consequence, the approach chosen by the student has not led to thecorrection of his misconception or acquisition of the missing conception,INTUITION uses the pre-programmed approach in a new attempt to correctthe misconception or fill the missing conception. In this case the systemitself decides on the remedial strategy to be used next based on the previousexperience the student has had with remedial strategies in earlier remedialinterventions according to a selection mechanism proposed by Angelides andTong (1995). INTUITION collects relevant data from the student’s historyof remediation and infers the success rate of the various remedial strate-gies which have been applied to a misconception or missing conception ofa player during game play. The results of this investigation are then used toselect a remedial strategy for the misconception or missing conception undertreatment.

If pre-programmed and student-led strategy selection have been unsuc-cessful the system explores the companion-led approach. For this purposeINTUITION accesses the student models of all the other players and checksfor any players who may have experience with the concept which the studenthas been found to have a problem with. If such a knowledgeable player isfound, he is provided with the source of the error and is asked to explain theerroneous concept to his colleague.

The supervisor-led approach is used when all other planning approacheshave failed, i.e. the supervisor only gets involved with the remedial operationif the system and all suitable companion players, if any, have failed to correcta misconception or fill a missing conception (Chan and Baskin 1990). In thisway the involvement of the supervisor is kept to a minimum (Vivet 1992).

In order to gain an insight into the current knowledge state of the studentthe supervisor is allowed access to the student model. INTUITION translates

aire248.tex; 23/10/1998; 23:49; v.6; p.34


the system data from the student model into a format which the supervisor canunderstand. Once the supervisor has familiarised himself with the knowledgeand skills the student has acquired and the student’s history of remediation,he is then provided with a choice of remedial environments in which he maycarry out the actual repair. He may, for example, recall the game situationin which the error was committed or demand the system to display a relatedexample. The supervisor may then use the chosen environment to explain therelevant concept to the student.

The remedial environment.INTUITION presents its remedial material indifferent remedial environments. These remedial environments used for aspecific remedial operation may be determined by the remedial strategy used.A demonstration, for example, may typically take place within an exampleenvironment, whilst cognitive apprenticeship as a remedial strategy is carriedout using a textual presentation.

7. The Evaluation of INTUITION

In order to investigate whether employing the model for remedial operationsin the development of intelligent tutoring systems provides efficient remedialtutoring, INTUITION has been evaluated against the principles of remediationwith a group of 16 postgraduate students who interacted with the system over anumber of games. The students were provided with a questionnaire (appendixA) which was designed to gain feedback from the students about the waythey perceived the various issues of remedial tutoring that are depicted in theprinciples of remediation.

The results of the evaluation show that INTUITION offers efficient reme-diation where necessary. INTUITION’s remedial processes use the system’sknowledge models to provide adaptability to the needs and preferences of theplayers. INTUITION’s repair plan incorporates a range of different remedialstrategies, and it provides a tool for the structure and organisation of any reme-dial process to be carried out. Planning, i.e. the determination of the steps ofactions and the remedial strategy used to carry them out, can be supported bythe system, the affected player, a companion player or the supervisor in orderto provide adaptability to the player’s preferences and needs. INTUITION’sstudent models provide knowledge about these needs and preferences, andINTUITION manages to provide the supervisor with a student’s experiencewith previous remedial processes and the knowledge acquired by the studentso far. After remediation INTUITION takes the student back into the gamewhere he is given the chance to correct his decision.

aire248.tex; 23/10/1998; 23:49; v.6; p.35


The evaluation of the learning achievement after the tutorial interventionexamines whether the player understands the skills and knowledge he wassupposed to acquire. In order to assess whether the remedial operation led thestudent to understand the aspects of the game underlying his error, a gamesituation had to be designed that requires the student to use those aspects in adifferent context. INTUITION provides this game situation through its inher-ent quarterly structure, i.e. its recurring playing cycles. Every new quarterwithin a game provides the student with a game scenario in which the playerhas the opportunity to apply his remediated knowledge. The players’ studentmodels record whether the player is able to apply his knowledge successfully.If the student can solve the problem or correct the errors within a problem,i.e. if he can apply the knowledge and skills in the scenario presented to him,the remedial operation can be considered as having contributed towards thelearning achievement of the student.

The results from the observation of the student’s performance during theexperiment and the investigation of the student models afterwards correspondwith the feedback from the players. Players were asked whether INTUITIONprovided useful remediation to which the majority of players claimed thatINTUITION provided useful remediation “most of the time”. More signifi-cantly, the answer to the question of whether players were able to apply theirremediated knowledge successfully in a subsequent quarter were restricted to“always” (50% of the subjects), “most of the time” and “sometimes”. Conse-quently, overall remediation with INTUITION can largely be considered assuccessful.

The majority of students who used INTUITION regarded the time of reme-diation as appropriate. None of the students questioned ever felt that theywere interrupted unnecessarily or that INTUITION’s remediation was notrequired. However, more immediate remediation would have been helpful insituations in which the committed error did not relate to other decisions withinthe same step of play (e.g. a novice financial director would have preferred theremediation of any independent errors on the revenue and expenditure state-ment before moving on to the calculations on other operating statements). Inrelation to this, the ability to trigger a student-invoked remedial interventionand thus not having to refer to the rules of the game all the time as an alter-native, or at least to refer the student directly to the rule that is relevant to thestudent’s error was raised.

The delayed approach to treating errors is largely justified, because delayedremediation has the advantage that the error may be detected and pointed outto the student. At the same time delayed remediation forces the studentto rethink the overall step of action instead of creating local sub-optimalsolutions (Silverman 1992).

aire248.tex; 23/10/1998; 23:49; v.6; p.36


An issue which arose from the evaluation and which provides a suggestionfor the possible refinement of the principles of remediation is the aspect ofremediation before an error occurs. According to Principle 2 remediationshould be carried out immediately or delayed. However, students’ commentssuggested the introduction of error prevention. A remedial operation maybe carried out before the error can be committed in order to warn studentsabout common errors. If students are made aware of errors in advance theymight not make the mistake. However, this is error avoidance rather thanerror correction. The use of preventive methods might be an issue for furtherresearch.

It can be concluded that the evaluation of INTUITION provides a goodindication for the usefulness of the model for remedial operations as a firststep towards a formal basis for remediation with Intelligent Tutoring Systems.At the same time it offers suggestions for the improvement of both remedialtutoring and the formalisation of remediation in the model for remedial oper-ations.

Although the evaluation method developed for the evaluation ofINTUITION provides a tool for the examination of remedial tutoring withIntelligent Tutoring Systems, alternative methods might be applicable andmore advanced evaluation methods in the future may eventually providebetter suggestions for the improvement of remedial tutoring with Intelli-gent Tutoring Systems. Only good evaluation tools will provide us with theconstructive feedback required for the successful advancement of work in thefield of Intelligent Tutoring Systems.

8. Conclusion

A good intelligent tutoring system should be able to provide helpful guidanceand adapt the teaching process to the student by exploring and understandingthe individual player and his special needs and interests (Swartz and Yazdani1992). For successful teaching to take place an intelligent tutoring systemalso has to be able to cope with any student errors that may occur during aconsultation.

This paper has stressed the importance of remedial tutoring within theoverall teaching process. Recent investigations of the adaptation of tutoringapproaches for the purpose of providing remedial tutoring have determineda number of remedial issues that should be considered in order to provideefficient remedial tutoring. The result of these investigations are summarisedin theprinciples of remediation. The principles of remediation constitute thebasis for comprehensive and satisfactory remediation with intelligent tutoringsystems.

aire248.tex; 23/10/1998; 23:49; v.6; p.37


In order to formalise the process of remediation which may be triggeredduring a teaching interaction with an intelligent tutoring system, this paperthen constructs a model for the implementation of remedial operations inintelligent tutoring systems. The principles of remediation are integrated intoa functionalmodel for remedial operations.

To demonstrate that the model for remedial operations, when incorporatedinto the general intelligent tutoring system architecture, provides for remedialtutoring INTUITION an intelligent tutoring system for business managementgaming-simulation which was developed as part of this research is presented.The evaluation of INTUITION has given evidence that the model for remedialoperations proves to be a useful formal basis for the provision of satisfactoryremedial tutoring with intelligent tutoring systems.

Appendix A: Questionnaire

I. General

1) What director roles were allocated to you?At what advancement level did you play: novice or advanced?

Game No Director Role Played Advancement Level

1

2

3

4

5

II. Learning Achievement

2) Did the system provide useful instruction?

� never� seldom� sometimes� most of the time� always

aire248.tex; 23/10/1998; 23:49; v.6; p.38


3) Were you able to apply successfully the knowledge you obtained withina tutorial interaction in a subsequent quarter of the game?

� never� seldom� sometimes� most of the time� always

Domain Model4) Can the system answer arbitrary questions about the subject?5) Does the system teach prerequisite skills?6) Can the system give an explanation of a problem solution (including one

of a problem posed by the user)?7) Can the systems give alternative explanations, perhaps using analogy?8) Were you able to initiate some new area of investigation?

Student Model9) Are the problems presented by the system adapted to the users’ needs?

Tutoring Model10) Does the system offer a flexible style of tutoring?11) Do the systems provide hints, pieces of advice, corrections, remedial

demonstrations, traces of reasoning, interpretations, explanations, simu-lations, motivation?

12) Was the system able to provide tutoring using alternative methods? Pleaselist the methods.

13) Did the system use different methods of remediation when you made amistake? Please give example(s).

Overall System Control

Strategy Selection14) At any point, did the system allow you to determine the way you were

tutored, or did you feel the system took over leaving you with no options?What were you allowed to do? Would you have liked to express any otherpreferences? Please give example(s).

Remediation15) Do the systems intervene if the user appears to be having difficulty?

aire248.tex; 23/10/1998; 23:49; v.6; p.39


16) Did the system explain the cause of your mistakes or did it simply correctthe mistakes? Please give example(s).

Needs and Preferences17) Are the system’s explanations tailored to the user?18) Is the system’s tutroing sensitive to the individual student needs and

preferences?19) Does the system provide informative feedback?20) Is tutoring tailored to your level of advancement?Please give example(s).21) If you played at different advancement levels did you notice a difference

in the tutoring methods used? Did you find the differences useful? Pleasegive example(s).

Proactive/Reactive22) Did the system intervene at a time at which you considered it useful or

would you have preferred intervention at a different point in time? If thetiming was inappropriate, when should intervention have taken place?Please give example(s).

23) Did you ever feel that the system interrupted you unnecessarily? When?Please give example(s).

24) Do the systems enable the student to communicate his plans (i.e. inten-tions) prior to executing them?

25) Did the system provide you with a facility to call for help on issues thatyou did not understand? How? Please give example(s).

PLEASE FEEL FREE TO MAKE ANY ADDITIONAL COMMENTS:

References

Agius, H. W. & Angelides, M. C. (1996). Integrating Intelligent Tutoring into Multimedia:The Mu.P.P.E.T. System, submitted for publication inJournal of Information Technology.

Alpert, S. R., Singley, M. K. & Carroll J. M. (1995). Multiple Multimodal Mentors: DeliveringComputer-Based Instruction via Specialized Anthropomorphic Advisors.Behaviour andInformation Technology14(2): 67–79.

Anderson, J. R., Boyle, C. F., Corbett, A. T. & Lewis, M. W. (1990). Cognitive Modellingand Intelligent Tutoring. In Clancey, W. J. & Soloway, E. (eds.)Artificial Intelligence andLearning Environments, 7–50. MIT Press.

Angelides, M. C. & Tong, A. K. Y. (1995). Implementing Multiple Tutoring Strategies in anIntelligent Tutoring System for Music Learning.Journal of Information Technology10:52–62.

Barker, D. (1995). Seven New Ways to Learn – Carnegie Mellon University is Changing theWay in Which Teachers Teach and Students Learn.Byte20(3): 54–55.

Bierman, D. J. (1992). Kamsteeg, P. A. and Sandberg, J. A. C., Student Models, Scratch-Pads,and Simulation. In Costa, E. (ed.)New Directions for Intelligent Tutoring Systems, NATOASI Series, Vol. 91, 135–145. Springer-Verlag: Berlin, Heidelberg.

aire248.tex; 23/10/1998; 23:49; v.6; p.40


Brown, J. S. & Burton, R. R. (1987). Reactive Learning Environments for Teaching ElectronicTroubleshooting. In Pouse, W. B. (ed.)Advances in Man-Machine Systems Research3.JAI Press: Greenwich.

Burton, R. R. (1988). The Environment Module of Intelligent Tutoring Systems. In Polson,M. C. & Richardson, J. J. (eds.)Foundations of Intelligent Tutoring Systems, 109–142.Lawrence Erlbaum Associates: Hillsdale, New Jersey.

Canfield, A. M., Schwab, S., Merrill, M. D., Li, Z. & Jones, M. K. (1992). InstructionalTransaction Theory: Resource Mediation. In Giardina, M. (ed.)Interactive MultimediaLearning Environments – Human Factors and Technical Considerations on Design Issues,75–81. Springer-Verlag: Berlin, Heidelberg.

Careers Research and Advisory Centre (CRAC) (1978).‘Stelrad Limited’ The Metal BoxBusiness Game. Hobsons Press: Cambridge.

Chan, T.-W. & Baskin, A. B. (1990). Learning Companion Systems. In Frasson, C. & Gauthier,G. (eds.)Intelligent Tutoring Systems, 6–33. Ablex Publishing Corporation: Norwood, NewJersey.

Clancey, W. J. & Soloway, E. (1990). Artificial Intelligence and Learning Environments:Preface. In Clancey, W. J. & Soloway, E. (eds.)Artificial Intelligence and Learning Envi-ronments, 1–6. MIT Press: Cambridge, Massachusetts.

Clancey, W. J. (1988). The Role of Qualitative Models in Instruction. In Self, J. (ed.)Artifi-cial Intelligence and Human Learning – Intelligent Computer-Aided Instruction, 49–68.Chapham and Hall: London.

Collins, A., Warnock, E. H. & Passafiume, J. (1975). Analysis and Synthesis of TutorialDialogues. In Bower, G. (ed.)The Psychology of Learning and Motivation9, 49–87.Academic Press: New York.

Corbett, A. T., Anderson, J. A. & Fincham, J. M. (1991). Menu Selection vs. Typing:Effects on Learning in an Intelligent Programming Tutor. In Birnbaum, L. (ed.)TheInternational Conference on the Learning Sciences, 107–112. Charlottesville, Associationfor the Advancement of Computing in Education.

Cumming, G. & Self, J. (1991). Learner Modeling in Collaborative Intelligent EducationalSystems. In Goodyear, P. (ed.)Teaching Knowledge and Intelligent Tutoring, 85–106.Ablex Publishing Corporation: New Jersey.

De Corte, E., Verschaffel, L. & Schrooten, H. (1991). Computer Simulation as a Tool in Study-ing Teachers’ Cognitive Activities during Error Diagnosis in Arithmetic. In Goodyear, P.(ed.)Teaching Knowledge and Intelligent Tutoring, 367–378. Ablex Publishing Corpora-tion: New Jersey.

Derry, S. J. & Lajoie, S. P. (1993). A Middle Camp for (Un)Intelligent Instructional Computing:An Introduction. In Lajoie, S. P. & Derry, S. J. (eds.)Computers as Cognitive Tools, 1–11.Lawrence Erlbaum Associates: New Jersey.

Dreyfus, H. L. & Dreyfus, S. E. (1986).Mind over Machine, The Power of Human Intuitionand Expertise in the Era of the Computer. Basil Blackwell: Glasgow.

Elsom-Cook, M. (1990). Guided Discovery Tutoring. In Elsom-Cook M. (ed.)GuidedDiscovery Tutoring, 3–23. Paul Chapman Publishing: London.

Elsom-Cook, M. T. (1991). Dialogue and Teaching Styles. In Goodyear, P. (ed.)TeachingKnowledge and Intelligent Tutoring, 61–84. Ablex Publishing Corporation: New Jersey.

Fox, B. A. (1991). Cognitive and Interactional Aspects of Correction in Tutoring. In Goodyear,P. (ed.)Teaching Knowledge and Intelligent Tutoring, 149–172. Ablex Publishing Corpo-ration: New Jersey.

Gegg-Harrison, T. S. (1992). Adapting Instruction to the Student’s Capabilities.Journal ofArtificial Intellgence in Education3: 169–181.

Goodyear, P. & Stone, C. (1992). Domain Knowledge, Epistemology and Intelligent Tutoringin Social Science. In Moyse, R. & Elsom-Cook, M. T. (eds.)Knowledge Negotiation,69–96. Academic Press: London.

Hasslberger, J. (1994). Aus Fehlern lernen – Analyse der Fehler beim Arbeiten am Computer.VB akzente8–9: 15–19.

aire248.tex; 23/10/1998; 23:49; v.6; p.41


Hollan, J. D., Hutchins, E. L. & Weitzman, L. (1984). STEAMER: An Interactive InspectableSimulation-Based Training System.AI Magazine2: 15–27.

Jones, M. K., Li, Z. & Merrill, M. D. (1992). Implementing Learner Control in an Auto-mated Instructional System. In Dijkstra, S., Krammer, H. P. M. & van Merrienboer, J.J. G. (eds.)Instructional Models in Computer-Based Learning Environments, 487–498.Springer-Verlag: Berlin, Heidelberg.

Kaplan, R. & Rock, D. (1995). New Directions for Intelligent Tutoring.AI Expert 10(2):30–40.

Katz, S. & Lesgold, A. (1993). The Role of the Tutor in Computer-Based CollaborativeLearning Situations. In Lajoie, S. P. & Derry, S. J. (eds.)Computers as Cognitive Tools,289–318. Lawrence Erlbaum Associates: New Jersey.

Kearsley, G. P. (1987). Overview. In Kearsley, G. P. (ed.)Artificial Intelligence and Instruction:Applications and Methods, 3–10. Addison-Wesley.

Laurillard D. (1990). Generative Student Models: The Limits of Diagnosis and Remediation.In Elsom-Cook M. (ed.)Guided Discovery Tutoring, 42–53. Paul Chapman Publishing:London.

Leinhardt, G. & Greeno, J. G. (1991). The Cognitive Skill of Teaching. In Goodyear, P. (ed.)Teaching Knowledge and Intelligent Tutoring, 233–268. Ablex Publishing Corporation:New Jersey.

Lepper, M. R. & Chabay, R. W. (1988). Socializing the Intelligent Tutor: Bringing Empathyto Computer Tutors. In Mandl, H. & Lesgold, A. (eds.)Learning Issues fir IntelligentTutoring Systems, 242–257. Springer-Verlag: New York.

Lesgold, A. (1988). Toward a Theory of Curriculum for Use in Designing Intelligent Instruc-tional Systems. In Mandl, H. & Lesgold, A. (eds.) Learning Issues for Intelligent TutoringSystems, 114–137. Springer-Verlag: New York.

Milheim, W. D. & Martin B. L. (1991). Theoretical Bases for the Use of Learner Control:Three Different Perspectives.Journal of Computer-Based Instruction18(3): 99–105.

Moyse, R. & Elsom-Cook, M. (1992). Knowledge Negotiation: An Introduction. In Moyse,R. & Elsom-Cook, M. T. (eds.),Knowledge Negotiation, 1–20. Academic Press: London.

Moyse, R. (1992). VIPER: The Design and Implementation of Multiple Viewpoints for Tutor-ing Systems. In Moyse, R. & Elsom-Cook, M. T. (eds.)Knowledge Negotiation. AcademicPress: London.

Ohlsson, S. (1991). Knowledge Requirements for Teaching: The Case of Fractions. InGoodyear, P. (ed.)Teaching Knowledge and Intelligent Tutoring, 25–60. Ablex PublishingCorporation: New Jersey.

Parkes, A. P. & Self, J. A. (1990). Towards ‘Interactive Video’: A Video-Based IntelligentTutoring Environment. In Frasson, C. & Gauthier, G. (eds.)Intelligent Tutoring Systems,56–82. Ablex Publishing Corporation: Norwood, New Jersey.

Payne, S. J. & Squibb, H. R. (1990). Algebra Mal-Rules and Cognitive Accounts of Error.Cognitive Science14: 445–481.

Reinhardt, A. (1995). New Ways to Learn.Byte. 20(3): 50–72.Reusser, K. (1993). Tutoring Systems and Pedagogical Theory: Representational Tools for

Understanding, Planning and Reflection in Problame Solving. In Lajoie, S. P. & Derry,S. J. (eds.)Computers as Cognitive Tools, 143–178. Lawrence Erlbaum Associates: NewJersey.

Romiszowski, A. J. (1990). The Hypertext/Hypermedia Solution – But What Exactly is theProblem? In Jonassen, D. H. & Mandl, H. (eds.)Designing Hypermedia for Learning,NATO ASI Series, Vol. 67, 321–354. Springer-Verlag: Berlin, Heidelberg.

Sack, W. (1990). Finding Errors by Overlooking Them, In Frasson, C. & Gauthier, G. (eds.)Intelligent Tutoring Systems, 206–233. Ablex Publishing Corporation: Norwood, NewJersey.

Self, J. A. (1992). Computational Mathetics: The Missing Link in Intelligent Tutoring SystemsResearch? In Costa, E. (ed.)New Directions for Intelligent Tutoring Systems, NATO ASISeries, Vol. 91, 38–56. Springer-Verlag: Berlin, Heidelberg.

aire248.tex; 23/10/1998; 23:49; v.6; p.42


Siemer, J. (1995). INTUITION – Applying Intelligent Tutoring to Gaming-Simulation.Journalof Computing and Information Technology3(1): 35–43.

Silverman, B. G. (1992).Critiquing Human Error – A Knowledge Based Human-ComputerCollaboration Approach. Academic Press: London.

Singley, M. K., Carroll, J. M. & Alpert, S. R. (1993). Incidental Reification of Goals in anIntelligent Tutor for Smalltalk. In Lemut, E., du Boulay, B. & Dettori, G. (eds.)Cogni-tive Models and Intelligent Environment for Learning Programming, 145–155. Springer-Verlag: Berlin.

Sleeman, D., Kelly, A. E., Martinak, R., Ward, R. D. & Moore, J. L. (1989). Studies ofDiagnosis and Remediation with High School Algebra Students.Cognitive Science13:551–568.

Sleeman, D., Ward, R. D., Kelly, E., Martinak, R. & Moore, J. (1991). An Overview of recentStudies with Pixie. In Goodyear, P. (ed.)Teaching Knowledge and Intelligent Tutoring,173–186. Ablex Publishing Corporation: New Jersey.

Spensley, F., Elsom-Cook, M., Byerley, P., Brooks, P., Federici, M. & Scaroni, C. (1990).Using Multiple Teaching Srategies in an Intelligent Tutoring System. In Frasson, C. &Gauthier, G. (eds.)Intelligent Tutoring Systems, 188–205. Ablex Publishing Corporation:Norwood, New Jersey.

Stevens, A. L. & Collins, A. M. (1977). The Goal Structure of a Socratic Tutor. InProceedingsof the National ACM Conference, 256–263. Seattle, Washington, USA.

Swartz, M. L. & Yazdani, M. (1992).Intelligent Tutoring Systems for Foreign LanguageLearning: The Bridge to International Communication. Springer-Verlag: Berlin, Heidel-berg.

Teasley, S. D. & Roschelle, J. (1993). Constructing a Joint Problem Space: The Computer as aTool for Sharing Knowledge. In Lajoie, S. P. & Derry, S. J. (eds.)Computers as CognitiveTools, 229–260. Lawrence Erlbaum Associates: New Jersey.

Van Marcke, K. (1992). A Genertic Task Model for instruction. In Dijkstra, S., Krammer, H. P.M. & van Merrienboer, J. J. G. (eds.)Instructional Models in Computer-Based LearningEnvironments, 171–194. Springer-Verlag: Berlin, Heidelberg.

Van Merrienboer, J. J. G. & Krammer, H. P. M. (1992). A Descriptive Model of InstructionalProcesses in Interactive Learning Environments for Elementary Computer Programming.In Dijkstra, S., Krammer, H. P. M. & van Merrienboer, J. J. G. (eds.)Instructional Models inComputer-Based Learning Environments, 213–228. Springer-Verlag: Berlin, Heidelberg.

Vivet, M. (1992). Uses of ITS: Which Role for the Teacher? In Costa, E. (ed.)New Directionsfor Intelligent Tutoring Systems, NATO ASI Series, Vol. 91, 171–182. Springer-Verlag:Berlin, Heidelberg.

Wenger, E. (1987).Artificial Intelligence and Tutoring Systems – Computational and CognitiveApproaches to the Communication of Knowledge. Morgan Kaufman Publishers: Los Altos,California.

White, B. Y. & Frederiksen, J. R. (1990). Causal Model Progressions as a Foundation for Intel-ligent Learning Environments. In Clancey, W. J. & Soloway, E. (eds.)Artificial Intelligenceand Learning Environments, 99–157. MIT Press: USA.

Winkels, R. (1992).Explorations in Intelligent Tutoring and Help. IOS Press: Amsterdam.Winkels, R. & Breuker, J. (1992). What’s in an ITS? A Functional Decomposition. In Costa,

E. (ed.) New Directions for Intelligent Tutoring Systems, NATO ASI Series, Vol. 91.Springer-Verlag: Berlin, Heidelberg.

Woolf, B. P. & Hall, W. (1995). Multimedia Pedagogues – Interactive Systems for Teachingand Learning.IEEE Computer(May): 74–80.

aire248.tex; 23/10/1998; 23:49; v.6; p.43

Documents

Towards an Intelligent Tutoring System Architecture that Supports Remedial Tutoring