Using Automated Tests and Restructuring Methodsfor an Agile Development of Diagnostic Knowledge Systems

Joachim Baumeister and Dietmar Seipel, and Frank PuppeDepartment of Computer Science, University of Wuerzburg, Germany

email:{baumeister, seipel, puppe}@informatik.uni-wuerzburg.de

Abstract

We introduce an agile process model for the develop-ment of diagnostic knowledge systems. As key prac-tices of this evolutionary process model we identifystructured modifications of the knowledge by restruc-turing methods, and the continuous validation of theknowledge using automated testing methods. Finally,we report promising experiences gained by the evalua-tion of a development project implementing a medicaldocumentation and consultation system.

IntroductionClassical process models are often not appropriate for de-veloping diagnostic knowledge systems, if, e.g., a full spec-ification is not known beforehand, and the project team issmall. In our context, we investigated the development ofmedical knowledge systems, that were typically constructedand maintained by 1-2 physicians. In that case, the knowl-edge is usually formulated by the domain specialists them-selves. If we consider classical process models like, e.g.,CommonKADS (Schreiberet al. 2001), then we see thatthey are facing some problems, when applied under the cir-cumstances described above:• Technical feasibility is not always known in advance. A

small prototype would provide a good basis for decidingabout the overall project.

• Full specification is often not known in advance. Dur-ing the development of a knowledge system new require-ments arise and previously defined requirements becomeless important.

Moreover, for small-size or mid-size development projectsdocument-centred approaches seem to be cumbersome andcostly for domain experts. Verbose specifications and nec-essary decisions about the project design often deter expertsfrom starting or continuing a knowledge system project. Infact, experts were motivated, if early results would be takenout from small specifications, and, if systems could growincrementally from a small pilot.

In this paper, we briefly introduce a novel process model,which allows for an agile construction and maintenance ofdiagnostic knowledge systems. We identify the automated

Copyright c© 2004, American Association for Artificial Intelli-gence (www.aaai.org). All rights reserved.

validation and restructuring of knowledge as the key prac-tices for a successful application of the agile process model.

In knowledge engineering research, validation techniqueshave been undergoing fruitful research for the last decades.Classical work by (Coenen & Bench-Capon 1993) was con-tinued, e.g., by (Preece 1998). Especially, for the rule-basedimplementation of knowledge systems an extensive frame-work was defined by (Knauf 2000). Interestingly, this frame-work does not only describe the evaluation of test knowledge(represented as test cases), but especially its suitable gener-ation. In the context of our projects, test cases were eitheralready available or manually defined by the experts, whichon one hand raises the question for completeness, and on theother hand is a complex and time consuming task. For thisreason, the integration of a formal method generating suit-able test cases will be a promising extension of the presentedwork.

The introduction of restructuring methods for a step-wiseand algorithmic modification of the knowledge base was in-spired by refactoring methods introduced for software engi-neering (Opdyke 1992; Fowler 1999). As an advantage togeneral software engineering, where test case adaptation ismainly done manually, the implementation of restructuringmethods for knowledge systems often can propagate theirchanges to the attached test knowledge, e.g., by modifyingthe corresponding objects in test cases. However, the re-finement of the knowledge base performed by restructuringmethods differs from refinement techniques, e.g., describedby (Boswell & Craw 1999; Knaufet al. 2002), since re-structuring is mainlynotapplied for improving the accuracyof the system, but for improving the design of the knowl-edge base, i.e., the structure of the implemented knowledge.Especially for this reason a restructuring method is currentlyinitiated manually, though supported by automated adapta-tions of attached knowledge.

An interesting issue remaining partially solved is themaintenance of test knowledge. This problem was for-mulated by Menzies as therecursive maintenance prob-lem(Menzies 1999). In his article, he argued that test knowl-edge was introduced to simplify the maintenance of knowl-edge, but the maintenance of test knowledge also needsto be considered sufficiently, possibly by meta-test knowl-edge. A partial solution for this problem will be presentedby the automated propagation of the restructuring methods

also adapting test cases according to the performed changes.Modifications of the knowledge base and the test knowledgewith respect to a changing world, i.e., domain expertise, canbe resolved by refinement techniques mentioned in the fol-lowing.

The paper is organized as follows: The following sectionbriefly introduces the agile process model and its steps, re-spectively. Based on this process model we describe auto-mated tests and restructuring methods in the next sections.Thereafter, we report promising experiences gained by theevaluation of a development project implementing a med-ical documentation and consultation system. We concludethe paper with a summary of the presented work, and showpossible directions for future research.

An Overview of the Agile Process ModelThe presented process model was inspired by the agile pro-cess model eXtreme programming (XP). In software engi-neering research and practice XP (Beck 2000) has attractedgreat attention, and showed its significance in numerousprojects. An agile process model has the following prop-erties:• an early, concrete and continuing feedback• an incremental planning approach• a flexible schedule of the development process• the design process lasts as long as the system lastsIn contrast to XP the presented process model does not con-sider the implementation of general software using an allpurpose programming language (e.g., C or Java), but fo-cusses on the development of knowledge systems, that areconstructed by the insertion, the change, and the review ofknowledge represented in a declarative language.

The Steps of the Agile Process ModelThe agile process model (cf. Figure 1) is a light-weight pro-cess model and consists of the following cycle: analysisof the system metaphor, design of the planning game, im-plementation (plan execution: including tests, restructuringand maintenance), and integration. We briefly discuss these4 steps in the following.

System Metaphor

Planning Game

ImplementationIntegration

Figure 1: The agile process model for developing knowledgesystems.

The System Metaphor. The system metaphor describesthe basic idea and the designated goals of the knowledgesystem to be implemented, and it is used to facilitate a bet-ter communication between the developer (i.e., the domain

specialist) and the user of the system. Thus, the metaphorstands for a common system of names and a common systemdescription. Using a common system metaphor can greatlysimplify the development and the communication betweenuser and developer.

Thus, thelocal system metaphordefines names and se-mantics of the basic entities used during the developmentof the system, e.g., diagnoses (solutions), questions (input),and cases (solved problems). Theglobal system metaphordescribes the overall idea of the system to be implemented.Typical classes are theconsultation system(focussing on theinference of solutions), thedocumentation system(focussingon the standardized and correct acquisition of input data, andtheembedded system(focussing on the technical integrationof the system into an existing machine).The Planning Game. The planning game is the startingpoint of the development cycle: During the planning gamethe developer and the user decides about the scope and thepriority of future development, i.e., extensions or modifica-tions of the current system. For each extension/modificationa plan is defined, which is documented bystory cards. Be-sides the desired functionality the costs and priority of eachplan are estimated. Based on the estimated values of thesefactors the developer and the user define the next release byselecting story cards. They define the scope of the release byordering the collected cards and defining a release deadlineaccording to the risk estimations made before. It is worthnoticing, that these factors provide a benchmark for feed-back in order to enable adaptation of plan estimation in thefuture.The planning game provides a flexible method for guidingthe development process of knowledge systems. On oneside, plans are documented in story cards and deliver a struc-tured sequence of the development process, in which theuser as well as the developer are integrated. On the otherside, during the definition of stories the user and the devel-oper specify the expected behavior of the planned knowl-edge extension, and thus prepare useful validation knowl-edge for the subsequent implementation phase. Further-more, by providing methods for estimating and document-ing implementation costs (derived from the implementationvelocity), an accurate feedback can be given to assess thewhole development process.The Implementation. The implementation phase consid-ers the realization of the story cards specified in the planninggame phase. In the context of the agile process model, theimplementation phase follows a test-first approach: Any im-plementation of the functionality of a story is preceded bythe implementation of appropriate tests. Therefore, we dis-tinguish between atest-implementation phaseand acode-implementation phase.In the test-implementation phase the developer defines testknowledge describing the expected behavior of the newstory to be implemented. The kind of test knowledgedepends on the representational language of the code-implementation. It is easy to see that, e.g., the test knowl-edge for a rule-based knowledge representation can differfrom the test knowledge of a model-based representation.

Test knowledge needs to be automatically executable, i.e.,the results of the test can be evaluated automatically by thesystem. As a main idea of the process model, the continuousapplication of the cyclic process yields a suite of tests, whichcan be executed as a whole. We discuss the importance ofautomated tests and test suites in the following sections inmore detail.The code-implementation phase considers the actual real-ization of the story, e.g., by acquiring and formulating newknowledge, or by restructuring existing knowledge. In thecontext of this paper we omit a detailed description of thissub-phase.The Integration. If the newly implemented functionalitypasses the corresponding tests and the test suite, respec-tively, then this knowledge is committed to the knowledgebase. Since the integration is done continuously, we alwayscan access a running system enclosing the currently imple-mented knowledge. For a reasonable integration additionaltests need to be available, which are too time consuming tobe included into the working test suite, but which are appliedduring integration to check more aspects of the functionalbehavior of the knowledge system. We call these testsinte-gration tests. Integration tests often contain a larger num-ber of previously solved cases, which can be run against theknowledge system. These previously solved cases typicallycontain a set of question-answer pairs and a set of expecteddiagnoses for these pairs, but sometimes also knowledgeabout dialog behavior is available. Running thousands ofcases can take several minutes or hours. Therefore, it is notpractical to include them into the working test suite, sincethe suite usually is applied many times during the imple-mentation of a story. Nevertheless, before the integration ofa new version these integration tests are an essential indica-tor for the correct behavior of the knowledge system.

Automated Tests

One of the key properties of the agile process model is theapplication of automated tests. For an automated applica-tion of tests the expected result of a test needs to be knownbeforehand. The most prominent example for automatedtests is the use of empirical testing, i.e., running previouslysolved test cases. But there exist further validation tech-niques, which can be simply adopted for automation. Thereexist tests that could be executed without any test knowl-edge, e.g., anomaly testing. However, some tests necessar-ily require test knowledge, which has to be defined by theexpert, e.g., empirical testing requires the presence of ap-propriate test cases.

Significance of Tests

At first sight the construction of tests is an additional andhuge effort during the implementation phase. Nevertheless,implementing tests besides the actual functionality is goodfor the following reasons:• Validation of the code:Tests are primarily defined to vali-

date the subsequent implementation. If the system passesthe tests, then the developer and the user feel confident,

that the newly implemented functionality has the expectedbehavior.

• Removing communication errors:Tests are often imple-mented as examples of typical system runs. Defining suchexamples in cooperation with the user (which has to knowthe typical system behavior) will clarify story definitions.It is worth noticing, that often ambiguous definitions aretimely exposed due to the test-implementation phase.

• Detecting side effects:Since all tests are collected in acommon test suite, all available tests will be executed be-fore completing the implementation of a story. Thus, sideeffects can easily be discovered, i.e., a new functionalityhas accidentally changed the behavior of a previously im-plemented functionality.

For this reason, we identify testing as one of the key prac-tices of the agile process model. In the following, we clas-sify tests methods with respect to their validated target andwe describe the application of automated tests.

Classification of Tests

In general, we can classify test methods according to theproperties of the knowledge that should be validated. Thus,we identify methods for validating thecorrectnessof theknowledge system, for findinganomaliescontained in thesystem, for testing therobustnessof the system, and for test-ing the understandabilityof the implemented knowledge.Methods for these types are briefly discussed in the follow-ing.Correctness. In the past, testing has been focussed on val-idating the correctness of a system. For example, empiricaltesting is the most popular method for correctness testing.Here, the implemented knowledge system runs previouslysolved test cases and infers solutions for each case. The in-ferred solutions are compared with the stored solutions ofthe case and differences are presented to the user. If a sounddialog behavior of the system is also of interest, thense-quentialized test casescan be used, i.e., ordinary test casesaugmented with the correct sequence of asked questions.Anomalies. The detection of anomalies is also an impor-tant issue in validation research. An anomaly is defined as acertain part of the knowledge base, which a priori is not in-correct butcancause the system to behave irregularly. Ex-amples for anomalies are redundant, cyclic, or ambivalentknowledge. For rule-based systems (Preece, Shinghal, &Batarekh 1992) have presented a classification of anomaliesand methods for detecting them. In principle, the classifica-tion of anomalies can be seamlessly transferred to other rep-resentations, e.g., case-based reasoning and Bayesian net-works.Robustness. If the knowledge system is intended to be usedin stressful environments, then the robustness of the systemis an important issue to consider. Methods for testing therobustness have been presented by (Groot, van Harmelen, &ten Teije 2000). The robustness of a rule-based system ismeasured by degrading the quality of system input, i.e., ap-plying noise to test cases, and by reducing the quality of theimplemented structural knowledge, i.e., removing or slightly

modifying implemented rules.Understandability. The understandability of the imple-mented knowledge was only studied a little in the past. How-ever, for the agile development of knowledge systems, theunderstandability of the working knowledge base is veryimportant. Understandability of knowledge can be approxi-mately measured by metrics, that take the (relative) size andcomplexity of knowledge into account. For example, thecomplexity of a rule base can be determined by the followingfactors: The overall number of rules, the average complexityof rules with respect to their conditions, the average numberof rules per diagnosis, the average number of findings perdiagnosis, that are applied in the corresponding rules. Anextensive framework for determining rule base complexitiesare presented in (Atzmueller, Baumeister, & Puppe 2004).

For some of these methods additional test knowledge isrequired, e.g., empirical testing and robustness testing re-quire appropriate test cases. Other methods can be exe-cuted even without any supplementary test knowledge, e.g.,anomaly testing. It is worth noticing, that test knowledge ismostly acquired manually by the domain specialists duringthe implementation phase.

Automated Application of Tests

For the agile development of knowledge systems test meth-ods need to be adapted to be executed automatically. Thus,the expected result of the particular tests need to be knownbeforehand. If any of the tests fail, then an error is reported.Otherwise, the successful pass of the tests is reported with ashort success message.

In its simplest form, empirical testing can be automatedwithout any adaptation, since previously solved cases areused for which the correct solution is already known. How-ever, if the cases were gathered by a real life application,then we cannot expect all cases to be solved correctly. Then,an expectation value is required, which specifies the mini-mum percentage of cases to be solved correctly.

For finding anomalies the developer has to decide aboutthe kind of anomalies, for which an error should be reported,e.g., ambivalency and circularity may indicate serious defi-ciencies of the knowledge base.

An expectation value also needs to be specified, if the ro-bustness of a system should be tested. Thus, a thresholdvalueξ has to be set, which defines the maximum noisinessfor which the knowledge system has to behave correctly. Forexample, if thresholdξ = 0.1, then the knowledge systemshould derive the correct solution for each case even if atleast 10% of the original input data is noisy.

Methods for testing the understandability of the imple-mented knowledge heavily depend on expectation values.For example, diagnoses with exceptionally large derivationrules can point to parts in the knowledge base, that may re-quire restructuring in the future. A smaller size of derivationknowledge for single diagnoses mostly increases the under-standability of the rule base. For this reason, an expectationvalue needs to be defined, which specifies the upper boundof the term ”exceptionally large”.

Restructuring MethodsThe agile process model proposes the evolutionary develop-ment of knowledge systems. Thus, a system grows by incre-mental extensions of new knowledge or by structured mod-ifications of already implemented knowledge. We call suchmodifications of knowledgerestructuring methods, and theyare commonly used for refining the design of the knowledgebase, which for example can be measured by understand-ability metrics. Since an evolutionary development processimplies the application of modifications from time to time,we identify restructuring methods as the second key practiceof the agile process model.

Restructuring methods for knowledge bases were inspiredby refactoring methods known in software engineering re-search, e.g., (Fowler 1999). A restructuring method is de-scribed by five parts: A meaningful and uniquename, asummarygiving a brief description of the method, amoti-vation for using the method, a list ofconsequencesreport-ing conflicts and restrictions of the method, and themechan-ics, an algorithmic description of the accomplishment of themethod.

Usually, a restructuring method is initiated by the decisionof the developer. Common restructurings are the modifica-tion of the question type, e.g., a multiple-choice questionincluded in the knowledge base should be changed to a setof semantically equal yes/no questions. We call this methodTRANSFORMMC2YN. For example, in a medical domainthe multiple choice question”examination doppler” withpossible values{stenosis, insufficiency} can be translatedinto the two yes/no questions depicted in Figure 2.

q1 : examination doppler - stenosis - insufficiency

q1_1 : examination doppler=stenosis - yes - no

q1_2 : examination doppler=insufficiency - yes - no

Figure 2: Restructuring of a multiple-choice question.

It is easy to see, that such modifications imply a subse-quent change of already implemented knowledge, e.g., forTRANSFORMMC2YN we need to consider the change ofall rules containing the original multiple-choice question.Another important issue to be considered is the coherentadaptation of test knowledge. For example, test cases needto be adapted according to the transformation, i.e., by ex-changing the multiple-choice values with the correspondingyes/no answers.

Restructuring methods differ from ordinary knowledgemodifications by specifying a structured procedure for im-plementing the particular modification. Thus, for each re-structuring the mechanics and consequences of this modifi-cation are identified and described, e.g., the required adap-tation of rules or test knowledge. An important implicationcan be drawn from this property: If the explicit procedure isdefined for each restructuring, then the execution of such a

method can be supported by interactive tools, that performthe required adaptations. This automatization of knowledgemodifications enable developers to manage even complexchanges of a knowledge base, that were very difficult to per-form manually in the past. For example, the restructuringdescribed above can imply the change of hundreds of rulesand thousands of test cases, if applied to a real world knowl-edge base.

However, performing a restructuring can cause a knowl-edge base to become invalid. For example, a restructuringmethod decreasing the value range of an one-choice questioncan produce contradictory rules, if two rules have the same(transformed) one-choice answer in their rule conditions butcontradictory rule actions. Therefore, any execution of a re-structuring method is preceded by afeasibility test, whichdetermines if the method will produce conflicts when exe-cuted. If this feasibility test fails, then the developer eitheruses default values to resolve the conflict or has to manuallymodify the knowledge base in order to remove the detectedconflicts. Basically, the execution of a restructuring methodpasses the following tasks:

1 Testing the actual state:Before a restructuring method isexecuted the test suite is applied. A restructuring shouldonly be considered, if the knowledge base is in a validstate.

2 Feasibility test:Checks, if the restructuring causes unre-solvable conflicts, if executed on the existing knowledgebase.

3 Method application:For all included knowledge (e.g., on-tological objects, rules, models, cases) the following sub-tasks are performed:3.1 Conflict resolution: If the execution of the method

causes conflicts, which are not resolvable by defaultvalues, then the conflicts are solved with interaction ofthe user. Otherwise, default values are used for conflictresolution.

3.2 Method execution:If no conflicts are remaining, thenthe method is executed according to the specified re-structuring mechanics.

4 Testing the resulting state:After the restructuring methodhas been applied, the knowledge base again is tested usingthe test suite. A restructuring method only is successful,if the restructured knowledge base is in a valid state.

A collection of18 typical restructurings, faced in real-worldknowledge system development, is extensively describedin (Baumeister 2004). In summary, we apply restructur-ing methods to improve knowledge base design. Typicalrestructurings are the modification of question types or theextraction of rule conditions. Restructuring methods differfrom ordinary knowledge modifications by their explicitlydescribed procedure. Thus, the methods can be automatedand tool support can be offered.

ExperiencesThe presented agile process model was successfully ap-plied in the ECHODOC1 project (Lorenzet al. 2003), a

1former QUALI TEE

medical documentation and consultation system supportingthe anaesthesiologist when performing a transesophagealechocardiography examination (TEE). The system is guid-ing the physician during the TEE examination, i.e., present-ing appropriate questions according to the current examina-tion phase, and suggesting diagnoses derived from the ex-amination process.

Figure 3: The knowledge modeling environmentd3web.KnowME (1) with editors for defining rules(2), executing tests (3), and restructuring methods (4).

Currently, the knowledge base contains 252 questionsgrouped into 113 question sets, and 77 diagnoses. The rulebase contains 254 rules implementing strategic knowledgedefining the dialog behavior and structural knowledge forderiving solutions.

The domain specialists implemented the knowl-edge base using the knowledge modeling environmentd3web.KnowME, which offers visual editors for the de-velopment of diagnostic knowledge systems. Furthermore,integrated editors for defining and executing automated testsand restructuring methods are included. Figure 3 depictsthe main window of d3web.KnowME together with editorsfor editing rules, performing restructurings, and executingautomated tests. Currently, the workbench provides editorsfor seven different test approaches (e.g., empirical testingfor structural knowledge, sequentialized cases for testingthe dialog behavior), and supports the automated executionof restructuring methods, that consider type transformationsand hierarchical modifications.

First evaluations show, that the phases of the agile processmodel appeared to be very useful. Since the domain expertswere only working part-time on the project, the use of sto-ries in conjunction with the planning game was very advan-tageous. Due to the evolutionary nature of the development

project the application of tests and restructuring methods ap-peared to be very significant. In most cases, modificationsof the already implemented knowledge were performed suc-cessfully by using the restructuring methods; the methodscommonly considered the change of question types or therearrangement of the question hierarchy. It is worth notic-ing, that the rearrangement of the question hierarchy oftenimplied the adaptation of the corresponding test cases rep-resenting the desired dialog behavior. Unfortunately, the of-fered restructuring methods only consider the correspondinginferential knowledge, but not the attached test knowledge,which has to be modified manually. However, it is plannedto provide refined methods in the future providing an im-proved, automated behavior.

In summary, the domain specialists were more encour-aged to perform (even bigger) changes on the knowledgebase, because they were supported by appropriate restructur-ing methods, which are executing reproduceable changes onthe knowledge. Additionally, test methods always ensured avalid state of the knowledge system, i.e., an expected behav-ior defined by the tests. A preliminary study of the practicalapplication of ECHODOC has been undertaken, and a real-life evaluation of the system at the University of WuerzburgHospitals is currently planned and scheduled.

Conclusion

In this paper, we introduced an agile process model for de-veloping (diagnostic) knowledge systems. We motivatedthat testing and restructuring are the key practices for asuccessful application of the evolutionary development ofknowledge systems. Thus, we described the significance oftests for the development process in conjunction with im-portant properties of test knowledge, e.g., their automatedapplication. Further, we defined a scheme for classifyingthe different types of test methods. As the second key prac-tice we described restructuring methods as a structured andalgorithmic approach for modifying an existing knowledgebase. Due to their explicit procedural definition restructur-ing methods can be automated, and tool support becomespossible.

The presented process model was evaluated on a runningproject developing a documentation and consultation systemin the medical domain. During the development the sig-nificance of automated tests and restructuring methods wasshown. First experiences are very promising, though there isstill room for improvement, e.g., not all restructuring meth-ods offer a propagation to the corresponding test knowledge.Finally, we think that also the inclusion of methods for auto-matic test case generation will be a very promising directionfor future work.

References

Atzmueller, M.; Baumeister, J.; and Puppe, F. 2004. Qual-ity Measures for Semi-Automatic Learning of Simple Di-agnostic Rule Bases. InProceedings of the 15th Inter-national Conference on Applications of Declarative Pro-gramming and Knowledge Management (INAP 2004).

Baumeister, J. 2004.Agile Development of DiagnosticKnowledge Systems (submitted). Ph.D. Dissertation, Uni-versity Wurzburg, Germany.Beck, K. 2000. Extreme Programming Explained: Em-brace Change. Addison-Wesley.Boswell, R., and Craw, S. 1999.Organising KnowledgeRefinement Operators, In: Validation and Verification ofKnowledge Based Systems. Oslo, Norway: Kluwer. 149–161.Coenen, F., and Bench-Capon, T. 1993.Maintenance ofKnowledge-Based Systems. Academic Press.Fowler, M. 1999. Refactoring. Improving the Design ofExisting Code. Addison-Wesley.Groot, P.; van Harmelen, F.; and ten Teije, A. 2000. Tor-ture Tests: A Quantitative Analysis for the Robustness ofKnowledge-Based Systems. InKnowledge Acquisition,Modeling and Management, LNAI 1319, 403–418. Berlin:Springer Verlag.Knauf, R.; Philippow, I.; Gonzalez, A. J.; Jantke, K. P.; andSalecker, D. 2002. System Refinement in Practice – Us-ing a Formal Method to Modify Real-Life Knowledge. InProceedings of 15th International Conference - Florida Ar-tificial Intelligence Research Society (FLAIRS-2002), 216–220.Knauf, R. 2000.Validating Rule-Based Systems: A Com-plete Methodology. Aachen, Germany: Shaker.Lorenz, K.-W.; Baumeister, J.; Greim, C.; Roewer, N.; andPuppe, F. 2003. QualiTEE - An Intelligent Guidanceand Diagnosis System for the Documentation of Trans-esophageal Echocardiography Examinations. InProceed-ings of the 14th Annual Meeting of the European Societyfor Computing and Technology in Anaesthesia and Inten-sive Care (ESCTAIC).Menzies, T. 1999. Knowledge Maintenance: The State ofthe Art. The Knowledge Engineering Review14(1):1–46.Opdyke, W. F. 1992.Refactoring Object-Oriented Frame-works. Ph.D. Dissertation, University of Illinois, Urbana-Champaign, IL, USA.Preece, A.; Shinghal, R.; and Batarekh, A. 1992. Verify-ing Expert Systems. A Logical Framework and a PracticalTool. Expert Systems with Applications5(3/4):421–436.Preece, A. 1998. Building the Right System Right. InProceedings of KAW’98 Eleventh Workshop on KnowledgeAcquisition, Modeling and Management.Schreiber, G.; Akkermans, H.; Anjewierden, A.; de Hoog,R.; Shadbolt, N.; de Velde, W. V.; and Wielinga, B.2001. Knowledge Engineering and Management - TheCommonKADS Methodology. MIT Press, 2 edition.