"AN EVALUATION AND SELECTION METHODOLOGY - Insead

* *

"AN EVALUATION AND SELECTIONMETHODOLOGY FOR EXPERT SYSTEM SHELLS"

by

Louis LE BLANCand

Tawfik JELASSI**

N° 90/39/TM

Associate Professor of Policy and Administration, School ofPublic and Environmental Affairs, Indiana University,Bloomington, Indiana 47405 U.S.A.

Associate Professor of Information Systems, INSEAD, Boulevardde Constance, Fontainebleau 77305 Cedex, France

Printed at INSEAD,Fontainebleau, France

AN EVALUATION AND SELECTION METHODOLOGY

FOR EXPERT SYSTEM SHELLS *

Louis A. Le Blanc

Policy and Administration FacultySchool of Public and Environmental Affairs

Indiana UniversityBloomington, Indiana 47405

U.S.A.

and

Tawfik Jelassi

Technology Management AreaINSEAD

Boulevard de Constance77305 Fontainebleau

France

May, 1990

* Forthcoming in Expert Systems with Applications: The InternationalJournal, 1991.


FOR EXPERT SYSTEM SHELLS

ABSTRACT

This paper illustrates an evaluation and selection methodology for

expert system (ES) shells. The methodology incorporates three stages:

1) ES shell screening; 2) shell evaluation; and, 3) assurance of final

ES shell selection. Initially, developing a short list through

screening of commercial shell products determines whether appropriate

software exists and narrows the field of available expert system

software products for detailed consideration. The second stage

determines which of the remaining ES shells (the finalists) best meets

the needs of the organization, from both functional and technical

perspectives. The final stage compares user requirements with the

features of the selected ES software by defining how these requirements

will be satisfied by building expert system applications with the

selected product. The methodology also considers the possibility that,

at any stage of the process, no expert system shell is suitable and

that a system must be developed with programming languages such as

LISP, PROLOG, or some conventional programming language.



TABLE OF CONTENTS

1.0 Introduction

1.1 Overview of Expert System Shells1.2 ES Shell Definition1.3 The ES Software Selection Problem1.4 Structure of the Paper

2.0 A Decision Methodology for ES Software Selection

2.1 ES Shell Screening

2.1.1 Identify Candidate Software2.1.2 Screening Criteria2.1.3 Pick Finalists

2.2 ES Shell Evaluation

2.2.1 Expand Evaluation Criteria2.2.2 Obtain Package Information2.2.3 Evaluate ES Shells2.2.4 Additional Selection Requirements

2.3 Specific ES Design

2.3.1 Alter Functional Requirements2.3.2 ES Software Modifications and Supporting Programs2.3.3 Finalize ES Shell Selection

3.0 Summary and Conclusion

Page 1



1.0 INTRODUCTION

1.1 Overview of Expert System Shells

An expert system (ES) is a computer program for capturing human

knowledge of a specific problem domain and delivering for decision

making purposes. This expertise is often represented in the form of

if-then rules, though it may take other forms of knowledge

representation, such as "semantic networks" or "frames" (see, e.g.,

Harmon and King, 1985). Many organizations are developing ES

applications with only minimal assistance from their information

systems department by using the relatively inexpensive and widely

available "off-the-shelf" ES shells.

Historically, rather than develop a custom-built ES application,

it was possible at times to borrow extensively from a previously

constructed ES. This practice has resulted in the development of

software that is known as ES shells or skeletal systems. The shells

are ES with their knowledge component (which contains the facts and

rules of a specific problem domain) removed and leaving only the

explanation, inference, and user interface components.

Currently, these commercial software packages include a ready-

built inference engine, user interface, and framework for the knowledge

base. All the system developer has to do is fill in the rules and

facts of the knowledge base, state the goals of the system, and make

the right connections with the user interface module. This module

can provide a variety of techniques to interact with an end user.

Page 2

Increasingly, it also includes the ability to integrate with existing

databases and transaction processing systems of an organization.

The knowledge base and inference engine are the heart of the ES

shell. The inference engine, which is the reasoning component of the

ES, examines the knowledge base and determines what to do in order to

achieve the goal which it has been instructed to perform.

1.2 SS Shell Definition

An ra Shell is "the dialog structure and inference engine which,when linked to a knowledge base, functions as a fully operational

expert system" (Liebowitz, 1988). It offers several advantages over

building ES from scratch. In addition to saving significant resources,

using an ES shell allows the knowledge engineer to concentrate on the

development of the knowledge base, from which the expert system derives

its power.

An ES shell constitutes one of the three technological levels of

software that comprise the ES development environment (Henderson, 1987;

Sprague, 1980; and Turban, 1990). Shells are essentially integrated

packages of software that provide a set of capabilities to build

specific ES quickly, easily, and at low cost. The other two levels

are: Specific Ea, which are operational systems that actually support

and/or advise managers/end-users on a specific issue; and Ea tools,

which are software elements that provide developers with the facility

to construct both specific ES and ES shells. Examples of such ES tools

include procedural programming languages, such as C and PASCAL, and

artificial intelligence languages like LISP, PROLOG, and FORTH.

(Insert Figure 1 About Here)

Page 3

Figure 1 (adapted from Sprague, 1980) illustrates the three

ES technological levels defined by Turban (1990) and the relationships

between them. Specific ES applications can be developed either

directly from ES tools or by adapting the ES shell to satisfy the

application requirements. In the latter situation, the ES builder

could employ the iterative design approach to rapidly select

capabilities from the ones available in the ES shell (or delete

unnecessary features) as required by the specific ES (represented by

the iterative cycling between the ES shell and the specific ES in

Figure 1).

A "prototyping" methodology could be employed between the "low"

technological level of ES tools and a specific ES application. Given

the ES development capabilities of a shell, ES applications of all

dimensions can be rapidly prototyped.

Expert system shells are available for a number of hardware

platforms, including mainframes, mini and microcomputers, as well as

special-purpose machines such as parallel processors. According to

Newquist (1988), a new generation of ES vendors is emerging and placing

ES software (i.e., ES shells) into the information systems mainstream.

As companies like Apple Computer, IBM, Cullinet Software, and McCormack

& Dodge become involved with ES technology, the choices for organi-

zations will multiply. The options will range from the choice of

hardware platform to the choice of computer language and software

interfaces.

Page 4

1.3 The ES Software Selection Problem

Publications which have addressed the issue of evaluating and

selecting ES software (Waterman and Hayes-Roth, 1982; Hayes-Roth,

Waterman and Lenat, 1983; Liebowitz, 1985; Harmon and King, 1985;

Forman and Nagy, 1985 and 1987; Harmon, et al., 1989; and Vetter, 1989)

identified criteria, especially user-related ones, which are critical

in selecting a suitable ES shell. However, these authors did not

suggest how to incorporate multiple user criteria, as well as technical

attributes, into a complete and thorough evaluation and selection

process. Furthermore, Lynch (1984 and 1985) and Meador and Mezger

(1984) suggest that inadequate examination of prospective software

packages leads to serious difficulties if not failures when

implementing information systems.

Although a number of approaches to selecting application software

for transaction processing systems (TPS) and management information

systems (MIS) have been proposed (Breslin, 1986; Curry and Bonner,

1983; Gray, 1987; Martin and McClure, 1983), some very critical factors

were omitted. These factors include assuring that the selected

software package is superior to a custom alternative, or that a

screening process is provided to reduce the number of packages

subjected to detailed evaluation.

To advance the importance of evaluation and selection of ES

software (as compared to that of other information systems), it should

be noted that ES shells are used to develop multiple management support

applications. From the same ES shell, an organization may develop

specific ES applications ranging from consultative and training to task

execution in functional areas as diverse as marketing, manufacturing

and finance. However, other types of software is usually employed only

Page 5

for a single application, for example a general ledger or a material

requirements planning system which respectively only support a single

functional area.

To efficiently develop specific ES applications using the

iterative design approach (i.e., prototyping), an ES shell need be

available (Naumann and Jenkins, 1982; Leary, 1987). The basic

objectives of the ES shell are: 1) to permit quick and easy development

of a wide variety of specific ES; and, 2) facilitate the iterative

development process by which specific ES can respond quickly to

changes. In a top-down fashion in order to satisfy these overall

objectives, an ES shell must satisfy a number of general criteria or

requirements. These criteria can be categorized according to ES

components, such as user interface, inference engine and knowledge

base. Once the ES shell is established, an enterprise can implement

specific ES rapidly. Therefore, the very critical software evaluation

and selection process for ES shells should take place prior to any

systems analysis and design (iterative development) efforts for

specific ES applications (Turban, 1990).

1.4 Structure of the Paper

This paper illustrates a method to select the most appropriate ES

shell where multiple criteria exist not only from functional

requirements but also from technical and vendor-support perspectives.

As an essential part of the methodology, an initial stage determines

whether an ES software product is even suitable for a particular

application, or should an ES be developed from programming languages or

other ES tools.

Page 6

[Insert Figure 2 About Here]

The proposed selection process in this paper also ensures that, at

each successive stage of the methodology, an ES shell is superior to a

custom-built ES application (see Figure 2). It continually reduces the

number of ES software products under consideration until a final

selection of a shell is made or constructing an ES application from

tools is chosen as the best alternative.

This paper is primarily addressed to academics interested in

software selection methodologies as well as practitioners faced with

ES-related problems. Section Two suggests a multiple criteria method-

ology for ES shell selection. The three stages of this process - ES

software screening, ES shell evaluation, and ES software assurance -

are described. Section Three concludes the paper with some final

remarks.

2.0 A DECISION METHODOLOGY FOR ES SOFTWARE SELECTION

There are three principal stages in the proposed ES shell

evaluation and selection methodology: 1) screening of prospective

candidates and development of a short list of ES software packages;

2) selecting an ES shell, if any, which best suits the application

requirements; and, 3) matching user requirements to the features of the

selected shell and describing how these requirements will be satisfied

through the building of prototypes for specific ES. The detailed

procedures involved in each stage of the selection process are

described in the following sections.

Page 7

The complete methodology provides a project outline for the

evaluation and selection of ES software. At the end of each stage, the

methodology produces a deliverable - a well-defined output or result

(i.e., the short list of software packages at the conclusion of the

initial stage). Given the project nature of this software evaluation

process, scheduling (i.e., determination of start and finish dates) for

each stage, as well as the entire project, is possible. (A Gantt chart

or other similar device can be used for this purpose.) Staffing

requirements can also be determined, including both the number of staff

members and their required skills. To evaluate and select ES software,

a project team should be assembled with personnel from user departments

as well as information systems professionals.

2.1 ES Software Screening

During this first stage of the evaluation and selection

methodology, three key issues must be addressed: 1) Is there an ES

shell that can be used or should a specific ES be developed from

tools?; 2) What ES shells are available?; and, 3) Which ES software

packages should be seriously considered and evaluated in detail?

Examples of commercially available mainframe and microcomputer-based ES

shells are given in Table 1. For more examples, see Computerworld

(1988).

[Insert Table 1 About Here]

Page 8

The purpose of developing a short list of shell products is to

narrow the field of available ES software for consideration during

Shell Evaluation. A short list of candidate ES software (two or three)

eliminates any unnecessary effort or confusion which might result

because too many alternative ES shells are evaluated.

2.1.1 Identify Candidate ES Software

The project team must first identify available ES shells that

operate within the enterprise's specific computer hardware and are

compatible with its operating system. An enterprise may have dedicated

hardware for running ES applications, such as a LISP machine. This

would certainly be a technical constraint for ES shells under

consideration. To initially identify appropriate ES shells for a

particular application, there are several publishers (e.g., catalogs

published from Datapro and ICP, as well as Expert Systems Strategies by

Harmon and Associates) who provide profiles of ES software vendors and

the products they offer.

2.1.2 Screening Criteria

The list of screening criteria will contain relatively few items

and should concentrate on functional requirements not commonly provided

by ES shells and which are very specific to the organization evaluating

ES software. Most commercial software packages share many standard

functions and capabilities. At the same time, organizations have many

identical requirements for software. However, software packages have

unique capabilities and features which can readily distinguish one

package from another. Similarly, organizations have unique

requirements for any software package to satisfy.

Page 9

[Insert Figure 3 About Here]

The screening process matches the unique capabilities of

commercial software packages with the unique requirements of the

organization. Figure 3 illustrates the overlap between unique

capabilities of software with the unique requirements of an organi-

zation. The commercial software packages (e.g., ES shell) with the

greatest overlap of unique capabilities with unique organizational

requirements would be retained for more detailed scrutiny in the next

stage of the evaluation and selection methodology (i.e., ES Shell

Evaluation).

At this point in the process, the concern is not how well the

respective software packages meet the screening criteria, but only if

the screening criteria (i.e. unique capabilities) are offered by the

software. In the next stage of the methodology, all criteria

(including both standard and unique capabilities) are evaluated as to

how well they are performed by each respective software package on the

short list.

The screening criteria can be categorized into four major types:

1) technical requirements; 2) functional requirements; 3) documentation

and training; and 4) vendor information.

Technical Requirements - An organization's hardware and software

strategy will likely dictate the screening criteria in the technical

area. To be considered, a shell must fit the framework of the proposed

system; it must be compatible with the hardware and software direction

already identified. For example, the Social Security Administration

required ES shell software to operate on their current hardware and

operating system platform (Leary, 1989). The operating system is

Page 10

clearly a strict technical requirement. Others could include

programming languages, peripherals, memory needs, or data communication

capabilities.

Functional Requirements - The functional requirements of an ES

shell can be classified according to the following system components:

1) User Interface; 2) Inference Engine; and 3) Knowledge Base. The

functional requirements associated with each of these three components

readily distinguish ES shell evaluation and selection from other

software appraisal efforts (Harmon and King, 1985; Harmon, et al.,

1989).

The user interface component of an ES provides the dialog

structure through which the user can access the ES. Normally, the user

interacts with the ES via a consultative mode; and many ES interface

components include an explanation module. The inference engine

component allows hypotheses to be generated from information contained

in the knowledge base; while the latter is comprised of facts and rules

of thumb based upon experience.


Table 2 lists several examples of functional criteria to conduct

the first-cut screening according to the ES components. (Forman and

Nagy (1987) offer a more exhaustive list of ES shell criteria.)

Possible screening criteria for the user interface component would be

that the ES shell offers certain necessary features for the knowledge

engineer (e.g., knowledge base editor and rule trace) as well as needed

features for the user (e.g., prompted-menu display and explanations and

justifications). The inference engine component could necessitate

generating new facts by modus ponens and decision tress, while control

Page 11

strategies should include forward chaining and procedural control. The

knowledge base criteria might require representing facts by object-

attribute-value (O-A-V) triplets and frames, handle uncertainty, and

model relationships with variable rules.

Documentation and Training - ES software packages normally include

the documentation required to install and support the ES shell. It

should be detailed, complete, and easy to understand. The availability

of vendor-developed training sessions and materials may be very

important, especially when the organization's personnel are

inexperienced in implementing software.

Vendor Information - A vendor's ability to support its package

through training, consultation, installation, and maintenance

assistance is an important consideration in evaluating ES software

packages. The vendor should also be able to refer an evaluation team

to a user who is willing to talk to them about the ES package and the

accompanying support.

Some organizations may wish to acquire a run-time version of an ES

shell. This shell is modified to incorporate a specific knowledge base

and to deactivate certain programming features. Not every vendor may

offer such a product.

The financial stability of a vendor can also be an important

consideration. Financially successful vendors that have been in

existence for more than a few years are more likely to adequately

support their packages initially and in the future because such vendors

attract and retain competent personnel.

Page 12

But financial success alone does not ensure adequate and continued

support. Vendor image, package reputation, the unit price, and the

number of installations are also important considerations. Either the

vendors themselves or the users to whom they directed the prospective

buyer should be able to provide the needed information in these areas.

2.1.3 Pick Finalists

The matching of the screening criteria against the list of ES

software and their capabilities will cause the elimination of many (but

hopefully not all) shells. The following are typical reasons to

eliminate potential ES software candidates: 1) a vendor has only a few

employees and has been in business less than a year; 2) operating

systems software and hardware is not supported by a vendor; and,

3) system documentation is inadequate.

By reducing the number of ES software packages under consider-

ation from as many as twenty to two or three, a project team can more

effectively devote its attention to the critical details that can make

the difference between selecting an adequate ES shell and selecting a

superior one. Moreover, by determining which ES software packages are

available for the application, the screening process also determines

whether an ES shell can be used or if a specific expert system should

be constructed from ES tools (see Figure 2).

Page 13

2.2 ES Shell Evaluation

This second stage focuses on the two or three ES shells that were

identified in the screening of ES software. The objective is to

evaluate in detail the ES shell finalists and select the one software

product that best meets the needs of the organization. The primary

tasks of ES software evaluation are: 1) to further define the detailed

evaluation criteria; 2) obtain shell product information; and,

3) evaluate the ES software finalists and pick one as the best

alternative.

2.2.1 Expand Evaluation Criteria

The screening criteria are expanded in more detail and fall into

the same four categories: 1) technical requirements; 2) functional

requirements; 3) documentation and training; and 4) vendor information.

Although all categories are expanded during shell evaluation, the

functional requirements receive the main attention and are related to

the interface, inference and knowledge components of the ES shell. The

purpose of this task is to develop a rather comprehensive functional

view of the proposed system and to summarize the requirements that must

be satisfied by the ES. As the project team defines the functional

criteria, they should also document the levels of importance and need

to the user.

The following functional requirements for ES software, identified

in Harmon and King (1985) are those for a hypothetical enterprise:

1) user interface features, including those for knowledge engineering

and data extraction from external sources; 2) inference engine

features, including generating new facts and control strategies; and

3) knowledge base features for handling facts, relationships, and

Page 14

uncertainty. Table 3 exhibits an expansion of the above summary list

of functional shell requirements.


2.2.2 Obtain Package Information

Once the system requirements have been established and the

criteria reviewed, the capability of each ES shell to satisfy the

requirements must be measured. Several techniques may be used to

gather enough information to determine how well each package meets the

requirements.

In many cases, the project team can meet directly with the vendor

sales and support personnel and discuss each requirement. But if

requirements are so comprehensive and detailed that a more formal

procedure should be followed, a request for proposal (RFP) can be

submitted to vendors. It should be noted that preparing an RFP can be

time-consuming and costly. In situations where the requirements are

not so detailed and complex, it can be replaced by a less formal and

more direct procedure such as a basic letter of request.

2.2.3 Evaluate ES Shells

Once the vendors' responses to requirements have been received,

the actual evaluation process can begin. The review is very detailed

at this point, since the project team is looking for specific strengths

and weaknesses of each package.

Page 15

The project team is searching for deciding factors - not only what

ES software packages have and how well they provide it, but also what

they don't have. Detailed information is desired on the functions of

the ES software and its related processing, including if and how

functions that are not included in the ES shell could be implemented.

Shovel and Lugasi (1987) compared three models for evaluating and

selecting a computer system from four alternatives in a manufacturing

case study. The three scoring models were: 1) the additive weight

model; 2) the eigen-vector model; and, 3) the multi-attribute utility

model. The authors reported that these models provided identical

rankings of combinations of computer hardware, software availability

and system support.

Given a short list of software alternatives, any scoring model

will likely yield identical rankings. In evaluating ES shells, Forman

and Nagy (1987) applied the Analytic Hierarchy Process (Saaty, 1980) to

select a shell from among just three alternative software packages.

However, this study did not indicate how the short list was developed

nor were any comparisons made with different scoring or ranking

techniques.

Generating the short list is more critical than the method of

ranking the software packages. Criticism levied against weighting

schemes for software selection decisions (Klein and Beck, 1987) can be

minimized through the screening process and development of a short

list. Naumann and Pelvis (1982) successfully applied weighting and

scoring measures to select a systems development tool from a relatively

short list (four) of candidate techniques. By weighting criteria for

only two or three packages rather than for a dozen (in which case the

aforementioned criticism is probably valid), the proposed evaluation

Page 16

process allows for a very detailed and focused inspection of just the

few best alternative ES software products.

2.2.4 Additional Selection Requirements

The ranking scores are not necessarily the determining factor for

selecting a particular ES shell. They should be used as a decision

tool - a means for organizing and summarizing the significant quantity

of information that the project team has collected. The highest score

may not always indicate the best ES shell. The scores may not

accurately reflect certain intangible factors such as the cosmetic

appearance of reports and screens, how easy it will be to use the ES

software, etc.

Tailoring - The scores may not indicate how much time or the level

of technical expertise needed to "tailor" the ES shell. Tailoring can

be either costly if it is relatively extensive, or difficult if the

internal structure of the software is complex. The importance of the

technical processing architecture will depend on how much tailoring is

anticipated. Furthermore, the architecture of the ES shell also

determines how much modification is even possible.

Documentation - A decision to use a particular ES shell should not

be made on the basis of functional requirements alone. The ES

software's documentation is a very important non-functional factor.

Its accuracy and level of detail can affect the time it will take to

evaluate and modify the package.

Comparing documentation is sometimes very difficult at this stage

of the evaluation process due to differences in format, style, etc.

Still, it is important to review the vendors' documentation and to

Page 17

reconfirm that the information collected on maintenance and support,

for instance, is accurate and correct.

The availability of and easy access to vendor-developed training

sessions is very important. Training materials, such as user tutorials

or video-based learning aids, may be especially critical when company

personnel are inexperienced in developing ES applications.

At this stage, the vendor should be able to refer the project team

to current users of his software. The comments of these customers

should prove invaluable. Site visits and demonstrations of the ES

software in operation may be helpful.

2.3 Specific ES Design

Assuming that ES software which is anticipated to provide

satisfactory performance has been selected (see Figure 2), the project

team is ready to confirm the selection by developing some specific ES

prototypes based on the chosen shell. The primary reasons for this

stage are to ensure that the ES package can be used effectively and to

provide one last chance to reconsider the ES software decision.

It is often difficult to determine the degree of user satisfaction

until the design process has begun for specific applications utilizing

the selected ES software. Therefore, this stage involves the design of

demonstration prototypes of specific ES built from the ES shell. Such

prototypes can provide significant benefits before finalizing the

selection decision (Alavi, 1984; Meador and Mezger, 1984; and Janson,

1986).

Page 18

2.3.1 Alter Functional Requirements

Based on the capabilities of the selected shell as experienced in

the prototyping exercise of specific ES, the definition of user

requirements might be altered to include package features not

previously considered, or to change or eliminate others. The modified

requirements should be reviewed with the users. The effect of ES

software deficiencies perhaps can be minimized by altering user

procedures or postponing the implementation of some requirements until

shell enhancements could be made.

2.3.2 ES Software Modifications and Supporting Programs

Typically, the specific ES being developed requires certain

functions and/or interfaces not provided by the software. If an ES

shell does not meet all the functional requirements of a system, the

following alternatives should be considered: 1) persuade the vendor to

include additional features; 2) develop supplemental software; and,

3) modify the vendor's software. The chosen alternative will depend on

the extent of the ES shell's deficiencies, the potential costs and

benefits of altering the software, and the size and technical skills

of the programming staff.

Vendor-Supplied Enhancements - If possible, the vendor should be

persuaded to do the modification for the purchaser. This is often the

best alternative, since the vendor will usually update and maintain the

software on a routine basis.

Page 19

Supporting Programs - Developing software to supplement the

vendor's ES package is often the most practical alternative. The

vendor will normally continue to service the ES shell; but if this

alternative is selected, the supplemental software should conform to

the standards used by the vendor in developing the ES shell.

Alter Code - Modifying an ES shell is usually not recommended. If

the software is modified, the vendor may be reluctant or may even

refuse to service the package. Updates to the software may not be

compatible with the modifications effected. In some cases, this may

not even be an option, since the purchaser of the ES shell does not

have (or cannot get at any price) a copy of the source code. In this

instance, all that the purchaser can do is to build a front-end or

back-end to the software package.

2.3.3 Finalize ES Shell Selection

It is not unusual for an organization to complete the last stage

of the evaluation and selection process for ES software, only to

realize that the selected shell is not the best choice. Perhaps too

many compromises have been made and users are no longer satisfied.

Possibly, the tailoring effort has become so extensive that a custom ES

(i.e., specific ES application built from tools) would be a better

choice (see Figure 2). Therefore, a final commitment to using a

particular shell should be avoided until the design of specific ES

using the potential software package has progressed to the point where

user satisfaction is ensured.

Page 20

3.0 SUMMARY AND CONCLUSION

Using ES shells for the development of specific ES will reduce

personnel requirements and development costs. Conducting the

evaluation of ES software increases the effort necessary for developing

specific ES, but this undertaking is offset by the aforementioned

advantages of using a shell package. Despite the promises offered by

ES software, the performance of some ES shells is much less than

expected. Weak or non-existent selection procedures may explain much

of this poor implementation record. The methodology proposed in this

paper will hopefully reduce the risks associated with ES software and

facilitate success in developing specific ES from shells (Alavi, 1984;

Janson, 1986).

The most critical phases of the methodology are determining the

criteria for an ES shell as incorporated in the first phase (the

development of a short list) and the confirmation element of the third

phase (design of specific ES with the selected shell). Initially, the

screening process determines whether a shell is feasible and reduces

the number of ES software packages to be evaluated in detail. Finally,

the development of specific ES with the selected shell ensures that the

ES software can be used effectively and provides a last chance to

consider building specific ES from tools.

Page 21

REFERENCES

Alavi, M., "An Assessment of the Prototyping Approach to InformationSystems Development," Communications gf the gO, 1984, 27 (6),556-563.

Breslin, J. Selecting and Installing Software packages. Westport, CN:Quorom Books, 1986.

Domputerworld, "Expert System Shells," October 17, 1988, 90-96.

Curry, J. W., and Bonner, D. M. Mow tg Find and Bu y good Software: AGuide for Business Anal Professional people. Englewood Cliffs, NJ:Prentice-Hall, Inc., 1983.

Forman, E. H., and Nagy, T. J. "A Multicriteria Model to Select anExpert System Generator." In proceedings g the Expert Systems inGovernment Conference, Washington, DC: IEEE/MITRE, 1985.

Forman, E. H., and Nagy, T. J. "EXSYS vs. TOPSI/OPS5 vs. MICRO-PS:A Multicriteria Model to Select an Expert System Generator."Telematics And Informatics, 1987, 4 (1), 37-54.

Gray, C. D. The bight Choice: A Complete Guide IQ Evaluating,Selecting, and Installing MEEII Software. Essex Junction, VT:Oliver Wight Limited Publications, Inc., 1987.

Harmon, P., and King, D. Expert Systems: Artificial Intelligence inbusiness. New York, NY: John Wiley & Sons, Inc., 1985.

Harmon, P., Maus, R., and Morrissey, W. Expert Systems: Tools &Applications. New York, NY: John Wiley & Sons, Inc., 1985.

Hayes-Roth, F., Waterman, D. A., and Lenat, D. B. Building ExpertSystems. Reading, MA: Addison-Wesley, 1983.

Henderson, J. C., "Finding Synergy Between Decision Support Systems andExpert Systems Research," Decision Sciences, 1987, 18 (3), 333-349.

Janson, M., "Applying a Pilot System and Prototyping Approach toSystems Development and Implementation," Information And Management,1986, 10 (4), 209-216.

Klein, G., and Beck, P. O., "A Decision Aid for Selecting amongInformation System Alternatives," MIS Quarterl y , 1987, 11 (2)177-185.

Leary, E., "Collecting Shells for Rapid Prototyping," Computerworla,November 23, 1987, p. S2.

Leary, E., "Expert Systems at the Social Security Administration,"Journal QL policy Analysis and Management, 1989, 8 (2), 200-203.

Page 22

Liebowitz, J., "Determining Functional Requirements for NASA Goddard'sCommand Management System Software Design Using Expert Systems,"D. Sc. dissertation, George Washington University, Washington, DC,1985.

Liebowitz, J. introduction t2 Expert Systems. Watsonville, CA:Mitchell Publishing, 1988.

Lynch, R. K., "Implementing Packaged Application Software: Hidden Costsand New Challenges," Systems, Objectives, Solutions, 1984, 4 (4),227-234.

Lynch, R. K., "Nine Pitfalls in Implementing Packaged ApplicationsSoftware," Journal gf Information Systems Management, 1985, 2 (2),88-92.

Martin, J., and McClure, C., "Buying Software off the Rack," HarvardBusiness Review, 1983, 61 (6), 32-47.

Meador, G. L., and Mezger, R. A., "Selecting An End User ProgrammingLanguage For DSS Development," ma Quarterly, 1984, 8 (4), 267-281.

Naumann, J. D., and Jenkins, A. M., "Prototyping: The New Paradigm ForSystems Development," MIS Ouarterly, 1982, 6 (3), 29-44.

Naumann, J. D., and Palvia, S., "A Selection Model for SystemsDevelopment Tools," MI$ Ouarterly, 1982, 6 (1), 39-48.

Newquist, H., "AI Adapts to Use of the Vernacular," Computerwor14,October 17, 1988, pp. 82-83.

Saaty, T. L., The Analytic Hierarchy Process, York, NY; McGraw-Hill,1980.

Shoval, P. and Y. Lugasi, "Models for Computer System Evaluation andSelection," Information k Management, 1987, 12 (3), 117-129.

Sprague, R. H., Jr., "A Framework for the Development of DecisionSupport Systems," MIS Quarterly, 1980, 4 (4), 1-26.

Turban, E. Decision Support And Expert Systems (second Edition).New York, NY: Macmillan Company, 1990.

Vedder, R. G., "PC-Based Expert System Shells: Some Desirable and LessDesirable Characteristics," Expert Systems, 1989, 6 (1), 28-42.

Waterman, D. A. A Guide tga £pert Systems. Reading, MA:Addison-Wesley, 1986.

Waterman, D. A., and Hayes-Roth, F. An investigation gf Tools forBuilding Expert Systems. Santa Monica, CA: Rand Corporation, 1982.

FIGURE 1. LEVELS OF ES TECHNOLOGY

Sisecific ES Applicanons

MoralIv•Deveicio►ent

IS Soo*

Noguild

Swine EShorn Tools

EvaluateU Shells

Yes

ConfirmU Shell

Selection

FIGURE 2. A MULTIPLE CRITERIA DECISION METHODOLOGY

FOR ES SHELL SELECTION

DevelopMon list

IS Software

PurchaseES Software

Standard

R•quinments

Standard N

CapablIttios I

U

U'N

U/

/

FIGURE 3. MATCHING OF SOFTWARE SCREENING CRITERIA

Software Organization

*TABLE 1. REPRESENTATIVE ES SHELL PRODUCTS

Large Machine Shells**

Product

KEEARTESEEMYCINS.1Level 5KESKnowledge CraftIBM KnowledgetoolRulemaster 2KBMSADS

Microcomputer Shells

Product

1st-ClassEXSYSM.1KES-PCGURUPersonal ConsultantVP-ExpertKnowledgeproNexpert

Vendor

IntellicorpInference Corp.IBMStanford UniversityTeknowledge, Inc.Information BuildersSoftware A & ECarnegie Group, Inc.IBMRadian Corp.AI CorporationAION Corporation

Vendor

lst-Class Expert SystemsEXSYS, Inc.Teknowledge, Inc.Software A & EMicro Database SystemsTexas InstrumentsPaperback SoftwareKnowledge Garden, Inc.Neuron Data

*Numerous shells are available in both large machine andmicrocomputer versions.

**Includes minicomputers, mainframes and specialized LISP machines.

TABLE 2. POTENTIAL SCREENING CRITERIA OF ES COMPONENTS

User Inference KnowledgeInterface Engine Base

Knowledge Base Modus PonensEditor

Decision TreePrompted-Menu Algorithm

DisplayForward

Explanation Chaining

Justification

BackwardChaining

GraphicsMixed

Chaining

ProceduralControl

Database/Spreadsheet

Links

Hypertext

O-A-V Triplets

Frames

Variable Rules

CertaintyFactors

TABLE 3. DETAILED FUNCTIONAL CRITERIA FOR ES SOFTWARE

Interface Component

Knowledge Engineering FacilitiesKnowledge Base EditorTraces and ProbesGraphic Display (Windows)

End-User ElementsExplanations and JustificationsLine-Oriented DisplayPrompted-Menu Display

Sources of DataInstrumentsData BasesOther Languages and Procedures

Inference Component

Generating New FactsModus PonensResolutionDecision Tree Algorithm

Control StrategiesBackward ChainingForward ChainingDepth-First SearchBreadth-First SearchProcedural Control

Knowledge Component

Representing FactsA-V PairsO-A-V TripletsFrames

RelationshipsIf-Then RulesVariable RulesExamples

UncertaintyInheritanceCertainty FactorsProbabilities

1986

86/01 Arnoud DE MEYER

86/02 Philippe A. NAERTMarcel VEVERBERGNand Guido VERSVIJVEL

86/03 Michael BRIMN

86/04 Spyros MAKRIDAKISand Michele HIBON

86/05 Charles A. VYPLOSZ

86/06 Francesco GIAVAllI,Jeff R. SHEEN andCharles A. VYPLOSZ

86/07 Douglas L. MacLACHLANand Spyros MAKRIDAKIS

86/08 Jose de la TORRE andDavid H. NECKAR

86/09 Philippe C. HASPESLACH

86/10 R. MOENART,Arnoud DE MEYER,J. BARBE andD. DESCHOOLMEESTER.

86/11 Philippe A. NAERTand Alain BULTEZ

86/12 Roger BETANCOURTand David GAUTSCHI

86/13 S.P. ANDERSONand Damien J. NEVEN

"The R i D/Production interface".

"Subjective estimation in integratingcommunication budget and allocationdecisions: a case study", January 1986.

"Sponsorship and the diffusion oforganizational innovation: a preliminary view".

"Confidence intervals: an empiricalinvestigation for the series in the M-Competition" .

"A note on the reduction of the workweek",July 1985.

"The real exchange rate and the fiscalaspects of a natural resource discovery",Revised version: February 1986.

"Judgmental biases in sales forecasting",February 1986.

"Forecasting political risks forinternational operations", Second Draft:March 3, 1986.

"Conceptualizing the strategic process indiversified firms: the role and nature of thecorporate influence process", February 1986.

"Analysing the issues concerningtechnological de-maturity".

"From "Lydiametry" to "Pinkhamization":misspecifying advertising dynamics rarelyaffects profitability".

"The economics of retail firms", RevisedApril 1986.

"Spatial competition a la Cournot".

86/16 B. Espen ECKBO andHervig M. LANGOHR

86/17 David B. JEMISON

86/18 James TEBOULand V. MALLERET

86/19 Rob R. VEITZ

86/20 Albert CORHAY,Gabriel HAVAVINIand Pierre A. MICHEL

86/21 Albert CORHAY,Gabriel A. HAVAVINIand Pierre A. MICHEL

86/22 Albert CORNAY,Gabriel A. HAVAVINIand Pierre A. MICHEL


86/24 David GAUTSCNIand Vithala R. RAO

86/25 H. Peter GRAYand Ingo WALTER

86/26 Barry EICHENGREENand Charles VYPLOSZ

86/27 Karel COOLand Ingemar DIERICKX

86/28 Manfred KEYS DEVRIES and Danny MILLER

86/29 Manfred KETS DE VRIES



"Les primes des offres publiques, la noted'information et le marche des transferts decontr8le des societie.

"Strategic capability transfer in acquisitionintegration", May 1986.

"Towards an operational definition ofservices", 1986.

"Nostradamus: a knowledge-based forecastingadvisor".

"The pricing of equity on the London stockexchange: seasonality and size premium",June 1986.

"Risk-premia seasonality in U.S. and Europeanequity markets", February 1986.

"Seasonality in the risk-return relationshipssome international evidence", July 1986.

"An exploratory study on the integration ofinformation systems in manufacturing",July 1986.

"A methodology for specification andaggregation in product concept testing",July 1986.

"Protection", August 1986.

"The economic consequences of the FrancPoincare", September 1986.

"Negative risk-return relationships inbusiness strategy: paradox or truism?",October 1986.

"Interpreting organizational texts.

"Why follow the leader?".

"The succession game: the real story.

"Flexibility: the next competitive battle",October 1986.

INSEAD WORKING PAPERS SERIES

"Comparaison Internationale des marges brutesdu commerce", June 1985.

"Nov the managerial attitudes of firms vithFMS differ from other manufacturing firms:survey results", June 1986.

86/31 Arnoud DE MEYER,Jinichiro NAKANE,Jeffrey G. MILLERand Kasra FERDOVS

86/32 Karel COOLand Dan SCHENDEL

"Flexibility: the next competitive battle",Revised Version: March 1987

Performance differences among strategic groupmembers", October 1986.

86/14 Charles WALDMAN

86/15 Mihkel TOMBAK andArhoud DE MEYER

87/06 Arun K. JAIN, "Customer loyalty as a construct in theChristian PINSON and marketing of banking services", July 1986.Naresh K. MALHOTRA

86/33 Ernst BALTENSPERGERand Jean DERMINE

86/34 Philippe HASPESLAGHand David JEMISON

86/35 Jean DERMINE

86/36 Albert CORHAY andGabriel HAVAVINI

86/37 David GAUTSCHI andRoger BETANCOURT

86/38 Gabriel HAVAVINI

86/39 Gabriel HAVAVINIPierre MICHELand Albert CORHAY

86/40 Charles VYPLOSZ

86/41 Kasra FERDOWSand Wickham SKINNER

86/42 Kasra FERDOWSand Per LINDBERG

86/43 Damien NEVEN

86/44 Ingemar DIERICKXCarmen MATUTESand Damien NEVEN

1987


87/02 Claude VIALLET

87/03 David GAUTSCHIand Vithala RAO

87/04 Sumantra CHOSHAL andChristopher BARTLETT

87/05 Arnoud DE MEYERand Kasra FERDOWS

"The role of public policy in insuringfinancial stability: a cross-country,comparative perspective", August 1986, RevisedNovember 1986.

"Acquisitions: myths and reality",July 1986.

"Measuring the market value of a bank, aprimer", November 1986.

"Seasonality in the risk-return relationship:some international evidence", July 1986.

"The evolution of retailing: a suggestedeconomic interpretation".

"Financial innovation and recent developmentsin the French capital markets", Updated:September 1986.

"The pricing of common stocks on the Brusselsstock exchange: a re-examination of theevidence", November 1986.

"Capital flovs liberalization and the EMS, aFrench perspective", December 1986.

"Manufacturing in a new perspective",July 1986.

"FMS as indicator of manufacturing strategy",December 1986.

"On the existence of equilibrium in hotelling'smodel", November 1986.

"Value added tax and competition",December 1986.

"Prisoners of leadership".

"An empirical investigation of internationalasset pricing", November 1986.

"A methodology for specification andaggregation in product concept testing",Revised Version: January 1987.

"Organizing for innovations: case of themultinational corporation", February 1987.

"Managerial focal points in manufacturingstrategy", February 1987.

"Equity pricing and stock market anomalies",February 1987.

"Leaders who can't manage", February 1987.

"Entrepreneurial activities of European MBAs",March 1987.

"A cultural view of organizational change",March 1987

"Forecasting and loss functions", March 1987.

"The Janus Bead: learning frost the superiorand subordinate faces of the manager's job",April 1987.

"Multinational corporations as differentiatednetworks", April 1987.

"Product Standards and Competitive Strategy: AnAnalysis of the Principles", May 1987.

"KETAFORECASTING: Vmys of improvingForecasting. Accuracy and Usefulness",May 1987.

"Takeover attempts: what does the language tellus?, June 1987.

"Managers' cognitive maps for upward anddovnvard relationships", June 1987.

"Patents and the European biotechnology lag: astudy of large European pharmaceutical firms",June 1987.

"Why the EMS? Dynamic games and the equilibriumpolicy regime, May 1987.

"A new approach to statistical forecasting",June 1987.

"Strategy formulation: the impact of nationalculture", Revised: July 1987.

"Conflicting ideologies: structural andmotivational consequences", August 1987.

"The demand for retail products and thehousehold production model: nev views oncomplementarity and substitutability".

87/07 Rolf BANZ andGabriel HAVAVINI


87/09 Lister VICKERY,Mark PILKINGTONand Paul READ

87/10 Andre LAURENT

87/11 Robert FILDES andSpyros MAKRIDAKIS

87/12 Fernando BARTOLOMEand Andre LAURENT

87/13 Sumantra GHOSHALand Nitin NOHRIA

87/14 Landis GABEL

87/15 Spyros MAKRIDAKIS

87/16 Susan SCHNEIDERand Roger DUNBAR

87/17 Andre LAURENT andFernando BARTOLOME

87/18 Reinhard ANGELMAR andChristoph LIEBSCHER

87/19 David BEGG andCharles VYPLOSZ


87/21 Susan SCHNEIDER


87/23 Roger BETANCOURTDavid GAUTSCHI

87/29 Susan SCHNEIDER andPaul SHRIVASTAVA

"The internal and external careers: atheoretical and cross-cultural perspective",Spring 1987.

"The robustness of MDS configurations in theface of incomplete data", March 1987, Revised:July 1987.

"Demand complementarities, household productionand retail assortments", July 1987.

"Is there a capital shortage in Europe?",August 1987.

"Controlling the interest-rate risk of bonds:an introduction to duration analysis andimmunization strategies", September 1987.

"Interpreting strategic behavior: basicassumptions themes in organizations", September1987

87/41 Cavriel HAVAVINI andClaude VIALLET

87/42 Damien NEVEN andJacques-F. THISSE

87/43 Jean CABSZEVICZ andJacques-F. THISSE

87/44 Jonathan HAMILTON,Jacques-F. THISSEand Anita WESKAMP

87/45 Karel COOL,David JEMISON andIngemar DIERICKX

87/46 Ingemar DIERICKXand Karel COOL

"Seasonality, size premium and the relationshipbetween the risk and the return of Frenchcommon stocks", November 1987

'Combining horizontal and verticaldifferentiation: the principle of max-mindifferentiation", December 1987

"Location", December 1987

"Spatial discrimination: Bertrand vs. Cournotin a model of location choice", December 1987

"Business strategy, market structure and risk-return relationships: a causal interpretation",December 1987.

"Asset stock accumulation and sustainabilityof competitive advantage", December 1987.

87/24 C.B. DERR andAndre LAURENT

87/25 A. K. JAIN,N. K. MALHOTRA andChristian PINSON

87/26 Roger BETANCOURTand David GAUTSCHI

87/27 Michael BURDA


87/30 Jonathan HAMILTON

"Spatial competition and the Core", August

W. Bentley MACLEOD

1987. 1988and J. F. THISSE

87/31 Martine OUINZII andJ. F. THISSE


87/33 Yves DOZ andAmy SHUEN

87/34 Kasra FERDOWS andArnoud DE MEYER

87/35 P. J. LEDERER andJ. F. THISSE


87/37 Landis LABEL


87/39 Manfred KETS DE VRIES1987

87/40 Carmen MATUTES andPierre REGIBEAU

"On the optimality of central places",September 1987.

"German, French and British manufacturingstrategies less different than one thinks",September 1987.

"A process framework for analyzing cooperationbetween firms", September 1987.

"European manufacturers: the dangers ofcomplacency. Insights from the 1987 Europeanmanufacturing futures survey, October 1987.

"Competitive location on networks underdiscriminatory pricing", September 1987.

"Prisoners of leadership", Revised versionOctober 1987.

"Privatization: its motives and likelyconsequences", October 1987.

"Strategy formulation: the impact of nationalculture", October 1987.

'The dark side of CEO succession", November

"Product compatibility and the scope of entry",November 1987

88/01 Michael LAWRENCE andSpyros MAKRIDAKIS


88/03 James TEBOUL


88/05 Charles WYPLOSZ

88/06 Reinhard ANGELMAR

88/07 Ingemar DIERICKXand Karel COOL

88/08 Reinhard ANGELMARand Susan SCHNEIDER

88/09 Bernard SINCLAIR-DESGAGNe

88/10 Bernard SINCLAIR-DESGAGN4

88/11 Bernard SINCLAIR-DESGAGNe

"Factors affecting judgemental forecasts andconfidence intervals", January 1988.

"Predicting recessions and other turningpoints", January 1988.

"De-industrialize service for quality", January1988.

"National vs. corporate culture: implicationsfor human resource management", January 1988.

"The svinging dollar: is Europe out of step?",January 1988.

"Les conflits dans les canaux de distribution",January 1988.

"Competitive advantage: a resource basedperspective", January 1988.

"Issues in the study of organizationalcognition", February 1988.

"Price formation and product design throughbidding", February 1988.

"The robustness of some standard auction gameforms", February 1988.

"When stationary strategies are equilibriumbidding strategy: The single-crossingproperty", February 1988.


"Business firms and managers in the 21stcentury", February 1988

88/13 Manfred KETS DE VRIES "Alezithymia in organizational life: theorganization man revisited", February 1988.

88/14 Alain NOEL "The interpretation of strategies: a study ofthe impact of CEOs on the corporation",March 1988.

88/15 Anil DEOLALIKAR and

Lars-Hendrik ROLLER

88/16 Gabriel HAWAWINI

88/17 Michael BURDA

88/18 Michael BURDA

88/19 M.J. LAWRENCE andSpyros MAKRIDAKIS

88/20 Jean DERMINE,Damien NEVEN andJ.F. THISSE

88/21 James TEBOUL

88/22 Lars-Hendrik ROLLER

88/23 Sjur Didrik FLAMand Georges ZACCOUR

88/24 B. Espen ECKBO andHervig LANGOHR

88/25 Everette S. GARDNERand Spyros MAKRIDAKIS

"The production of and returns from industrialinnovation: an econometric analysis for adeveloping country", December 1987.

"Market efficiency and equity pricing:international evidence and implications forglobal investing", March 1988.

"Monopolistic competition, costs of adjustmentand the behavior of European employment",September 1987.

"Reflections on "Wait Unemployment" inEurope", November 1987, revised February 1988.

"Individual bias in judgements of confidence",March 1988.

"Portfolio selection by mutual funds, anequilibrium model", March 1988.

"De-industrialize service for quality",March 1988 (88/03 Revised).

"Proper Quadratic Functions with an Applicationto AT&T", May 1987 (Revised March 1988).

"Equilibres de Nash-Cournot dans le marcheeuropeen du gaz: un cas oil les solutions enboucle ouverte et en feedback coincident",Mars 1988

"Information disclosure, means of payment, andtakeover premia. Public and Private tenderoffers in France", July 1985, Sixth revision,April 1988.

"The future of forecasting", April 1988.

88/26 Sjur Didrik FLAMand Georges ZACCOUR

88/27 Murugappa KRISHNAN

Lars-Hendrik ROLLER

88/28 Sumantra GHOSHAL andC. A. BARTLETT

"Semi-competitive Cournot equilibrium inmultistage oligopolies", April 1988.

"Entry game with resalable capacity",April 1988.

"The multinational corporation as a network:perspectives from interorganizational theory"May 1988.

88/29 Naresh K. MALHOTRA,Christian PINSON andArun K. JAIN

88/30 Catherine C. ECKELand Theo VERMAELEN

88/31 Sumantra CHOSHAL andChristopher BARTLETT

88/32 Kasra FERDOVS and

David SACKRIDER

88/33 Mihkel M. TOMBAK



88/36 Vikas TIBREVALA andBruce BUCHANAN

88/37 Murugappa KRISHNANLars-Hendrik ROLLER



88/40 Josef LAKONISHOK andTheo VERMAELEN

88/41 Charles WYPLOSZ

88/42 Paul EVANS

88/43 B. SINCLAIR-DESGAGNE

88/44 Essam MAHMOUD andSpyros MAKRIDAKIS

88/45 Robert KORAJCZYKand Claude VIALLET

88/46 Yves DOZ andAmy SHUEN

"Consumer cognitive complexity and thedimensionality of multidimensional scalingconfigurations", May 1988.

"The financial fallout from Chernobyl: riskperceptions and regulatory response", May 1988.

"Creation, adoption, and diffusion ofinnovations by subsidiaries of multinationalcorporations", June 1988.

"International manufacturing: positioningplants for success", June'1988.

"The importance of flexibility inmanufacturing", June 1988.

"Flexibility: an important dimension inmanufacturing", June 1988.

"A strategic analysis of investment in flexiblemanufacturing systems", July 1988.

"A Predictive Test of the NBD Model thatControls for Non-stationarity", June 1988.

"Regulating Price-Liability Competition ToImprove Welfare", July 1988.

"The Motivating Netale of Envy : A ForgottenFactor in Management, April 88.

"The Leader as Mirror : Clinical Reflections",July 1988.

"Anomalous price behavior around repurchasetender offers", August 1988.

"Assymetry in the EMS: intentional orsystemic?", August 1988.

"Organizational development in thetransnational enterprise", June 1988.

"Croup decision support systems implementBayesian rationality", September 1988.

"The state of the art and future directionsin combining forecasts", September 1988.

"An empirical investigation of internationalasset pricing", November 1986, revised August1988.

"From intent to outcome: a process frameworkfor partnerships", August 1988.

88/47 Alain BULTEZ,Els GIJSBRECHTS,Philippe NAERT andPiet VANDEN ABEELE

88/48 Michael BURDA

88/49 Nathalie DIERKENS

88/50 Rob WEITZ andArnoud DE MEYER

88/51 Rob VEITZ

88/52 Susan SCHNEIDER andReinhard ANCELMAR


88/54

Lars-Hendrik ROLLERand Mihkel M. TOMBAK

88/55 Peter BOSSAERTSand Pierre HILLION

88/56 Pierre HILLION

88/57 Wilfricd VANHONACKERand Lydia PRICE

88/58 B. SINCLAIR-DESGAGNEand Mihkel M. TOMBAK

"Asymmetric cannibalism between substituteitems listed by retailers", September 1988.

"Reflections on 'Wait unemployment' inEurope, II", April 1988 revised September 1988.

"Information asymmetry and equity issues",September 1988.

"Managing expert systems: from inceptionthrough updating", October 1987.

"Technology, work, and the organization: theimpact of expert systems", July 1988.

"Cognition and organizational analysis: vho'sminding the store?", September 1988.

"Whatever happened to the philosopher king: theleader's addiction to power, September 1988.

"Strategic choice of flexible productiontechnologies and welfare implications",October 1988

"Method of moments tests of contingent claimsasset pricing models", October 1988.

"Size-sorted portfolios and the violation ofthe random walk hypothesis: Additionalempirical evidence and implication for testsof asset pricing models", June 1988.

"Data transferability: estimating the responseeffect of future events based on historicalanalogy", October 1988.

"Assessing economic inequality", November 1988.

88/63 Fernando NASCIHENTOand Vilfried R.VANHONACKER

88/64 Kasra FERDOWS

88/65 Arnoud DE MEYERand Kasra FERDOVS

88/66 Nathalie DIERKENS

88/67 Paul S. ADLER andKasra FERDOWS

1989

89/01 Joyce K. BYRER andTavfik JELASSI

89/02 Louis A. LE BLANCand Tavfik JELASSI

89/03 Beth H. JONES andTavfik JELASSI

89/04 Kasra FERDOWS andArnoud DE MEYER

89/05 Martin KILDUFF andReinhard ANCELHAR

89/06 Mihkel M. TOMBAK andB. SINCLAIR-DESGACNE

"Strategic pricing of differentiated consumerdurables in a dynamic duopoly: a numericalanalysis", October 1988.

"Charting strategic roles for internationalfactories", December 1988.

"Quality up, technology down", October 1988.

"A discussion of exact measures of informationassymetry: the example of Myers and Majlufmodel or the importance of the asset structureof the firm", December 1988.

"The chief technology officer", December 1988.

"The impact of language theories on DSSdialog", January 1989.

"DSS software selection: a multiple criteriadecision methodology", January 1989.

"Negotiation support: the effects of computerintervention and conflict level on bargainingoutcome", January 1989."Lasting improvement in manufacturingperformance: In search of a new theory",January 1989.

"Shared history or shared culture? The effectsof time, culture, and performance oninstitutionalization in simulatedorganizations", January 1989.

"Coordinating manufacturing and businessstrategies: I", February 1989.

88/59 Martin KILDUFF

"The interpersonal structure of decision

89/07 Damien J. NEVEN

"Structural adjustment in European retailmaking: a social comparison approach to banking. Some view from industrialorganizational choice", November 1988. organisation", January 1989.

88/60 Michael BURDA

88/61 Lars-Hendrik ROLLER

88/62 Cynthia VAN HULLE,Theo VERMAELEN andPaul DE WOUTERS

"Is mismatch really the problem? Some estimatesof the Chelvood Cate I/ model vith US data",September 1988.

"Modelling cost structure: the Bell Systemrevisited", November 1988.

"Regulation, taxes and the market for corporatecontrol in Belgium", September 1988.

89/08 Arnoud DE MEYER andHellmut SCHUTTE

89/09 Damien NEVEN,Carmen MATUTES andMarcel CORSTJENS

89/10 Nathalie DIERKENS,Bruno GERARD andPierre MILLION

"Trends in the development of technology andtheir effects on the production structure inthe European Community", January 1989.

"Brand proliferation and entry deterrence",February 1989.

"A market based approach to the valuation ofthe assets in place and the growthopportunities of the firm", December 1988.

89/11 Manfred KETS DE VRIESand Alain NOEL

89/12 Vilfried VANHONACKER




89/16 Vilfried VANHONACKER,Donald LEHMANN andFareena SULTAN

89/17 Gilles AMADO,Claude FAUCHEUX andAndre LAURENT

89/18 Srinivasan BALAK-RISHNAN andMitchell KOZA

89/19 Vilfried VANHONACKER,Donald LEHMANN andFareena SULTAN

89/20 Vilfried VANHONACKERand Russell VINER

89/21 Arnoud de MEYER andKasra FERDOWS

89/22 Manfred KETS DE VRIESand Sydney PERZOW

89/23 Robert KORAJCZYK andClaude VIALLET

89/24 Martin KILDUFF andMitchel ABOLAFIA

89/25 Roger BETANCOURT andDavid GAUTSCHI

89/26 Charles BEAN,Edmond MALINVAUD,Peter BERNHOLZ,Francesco GIAVAllIand Charles VYPLOSZ

"Understanding the leader-strategy interface:application of the strategic relationshipinterview method", February 1989.

"Estimating dynamic response models when thedata are subject to different temporalaggregation", January 1989.

"The impostor syndrome: a disquietingphenomenon in organizational life", February1989.

"Product innovation: a tool for competitiveadvantage", March 1989.

"Evaluating a firm's product innovationperformance", March 1989.

"Combining related and sparse data in linearregression models", February 1989.

"Changement organisationnel et rialitisculturelles: contrastes franco-americains",March 1989.

"Information asymmetry, market failure andjoint-ventures: theory and evidence",March 1989

"Combining related and sparse data in linearregression models",Revised March 1989

"A rational random behavior model of choice",Revised March 1989

"Influence of manufacturing improvementprogrammes on performance", April 1989

"What is the role of character inpsychoanalysis? April 1989

"Equity risk premia and the pricing of foreignexchange risk" April 1989

"The social destruction of reality:Organisational conflict as social drama"April 1989

"Two essential characteristics of retailmarkets and their economic consequences"March 1989

"Macroeconomic policies for 1992: thetransition and after", April 1989

89/27 David KRACKHARDT andMartin KILDUFF


89/29 Robert GOGEL andJean-Claude LARRECHE

89/30 Lars-Hendrik ROLLERand Mihkel M. TOMBAK

89/31 Michael C. BURDA andStefan GERLACH

89/32 Peter HAUG andTavfik JELASSI

89/33 Bernard SINCLAIR-DESGAGNE

89/34 Sumantra CHOSHAL andNittin NOHRIA

89/35 Jean DERMINE andPierre BILLION



89/38 Manfrd KETS DE VRIES

89/39 Robert KORAJCZYK andClaude VIALLET

89/40 Balaji CHAKRAVARTHY

89/41 B. SINCLAIR-DESGAGNEand Nathalie DIERKENS

89/42 Robert ANSON andTavfik JELASSI

89/43 Michael BURDA

89/44 Balaji CHAKRAVARTHYand Peter LORANGF.

89/45 Rob WEITZ andArnoud DE MEYER

"Friendship patterns and cultural attributions:the control of organizational diversity",April 1989

"The interpersonal structure of decisionmaking: a social comparison approach toorganizational choice", Revised April 1989

"The battlefield for 1992: product strengthand geographic coverage", May 1989

"Competition and Investment in FlexibleTechnologies", May 1989

"Intertemporal prices and the US trade balancein durable goods", July 1989

"Application and evaluation of a multi-criteriadecision support system for the dynamicselection of U.S. manufacturing locations",May 1989

"Design flexibility in monopsonisticindustries", May 1989

"Requisite variety versus shared values:managing corporate-division relationships inthe it-Form organisation", May 1989

"Deposit rate ceiliess and the market value ofbanks: The case of France 1971-1981", May 1989

"A dispositional approach to social networks:the case of organizational choice", May 1989

"The organisational fool: balancing a leader'shubris", May 1989

"The CEO blues", June 1989

"An. empirical investigation of internationalasset pricing", (Revised June 1989)

"Management systems for innovation andproductivity", June 1989

"The strategic supply of precisions", June 1989

"A development fraaevork for computer supportedconflict resolution", July 1989

"A note on firing costs and severance benefitsin equilibrium unemployment", June 1989

"Strategic adaptation in multi-business firms",June 1989

"Managing expert systems: a framevork and casestudy", June 1989

89/46 Marcel CORSTJENS,Carmen MATUTES andDamien NEVEN

89/47 Manfred KETS DE VRIESand Christine MEAD

"Entry Encouragement", July 1989

"The global dimension in leadership andorganization: issues and controversies",April 1989

89/64 Enver YUCESAN and "Complexity of simulation models: A graph(TM) Lee SCHRUBEN theoretic approach", November 1989

89/65 Soumitra DUTTA and "MARS: A mergers and acquisitions reasoning(TM, Piero BONISSONE system", November 1989AC, PIN)

89/48 Damien NEVEN andLars-Hendrik ROLLER

89/49 Jean DERMINE

89/50 Jean DERMINE




89/54 S. BALAKRISHNANand Mitchell KOZA

89/55 H. SCHUTTE

89/56 Vilfried VANHONACKERand Lydia PRICE

89/57 Taekvon KIM,Lars-Hendrik ROLLERand Mihkel TOMBAK

89/58 Lars-Hendrik ROLLER(EP,TM) and Mihkel TOMBAK

89/59 Manfred KETS DE VRIES,(08) Daphna ZEVADI,

Alain NOEL andMihkel TOMBAK

89/60 Enver YUCESAN and(TM) Lee SCHRUBEN

89/61 Susan SCHNEIDER and(All) Arnoud DE MEYER

"European integration and trade flows",August 1989

"Home country control and mutual recognition",July 1989

"The specialization of financial institutions,the EEC model", August 1989

"Sliding simulation: a new approach to timeseries forecasting", July 1989

"Shortening development cycle times: amanufacturer's perspective", August 1989

"Why combining works?", July 1989

"Organisation costs and a theory of jointventures", September 1989

"Buro-Japanese cooperation in informationtechnology", September 1989

"On the practical usefulness of meta-analysisresults", September 1989

"Market growth and the diffusion ofmultiproduct technologies", September 1989

"Strategic aspects of flexible productiontechnologies", October 1989

"Locus of control and entrepreneurship: athree-country comparative study", October 1989

"Simulation graphs for design and analysis ofdiscrete event simulation models", October 1989

"Interpreting and responding to strategicissues: The impact of national culture",October 1989

"On the regulation of procurement bids",November 1989

"Market microstructure effects of governmentintervention in the foreign exchange market",December 1989

89/66 B. SINCLAIR-DESGAGNE(TM,RP)

89/67 Peter BOSSAERTS and(PIN) Pierre HILLION

89/62 Arnoud DE MEYER "Technology strategy and international B 6 D(TM) operations", October 1989

89/63 Enver YUCESAN and

"Equivalence of simulations: A graph theoretic(TM) Lee SCHRUBEN

approach", November 1989

90/16 Richard LEVICH andFIN Ingo WALTER

90/17 Nathalie DIERKENSPIN

90/18 Wilfried VANHONACKERMKT

90/19 Beth JONES andTM Tavfik JELASSI

90/20 Tavfik JELASSI,TM Gregory KERSTEN and

Stanley ZIONTS

90/21

Roy SMITH andFIN

Ingo WALTER

90/22 Ingo WALTERFIN

90/23 Damien NEVEN

90/24 Lars Tyge NIELSEN

90/25 Lars Tyge NIELSEN

90/26 Charles KADUSHIN andOB/BP Michael BRIBE

90/27 Abbas FOROUGHI andTM Tavfik JELASSI

90/28 Arnoud DE MEYERTM

90/29 Nathalie DIERKENSFIN/AC

90/30 Lars Tyge NIELSENFIN/EP

90/31 David GAUTSCHI andMKT/EP Roger BETANCOURT

90/32 Srinivasan BALAK-SM RISHNAN and

Mitchell KOZA

"Tax-Driven Regulatory Drags EuropeanFinancial Centers in the 1990's", January 1990

"Information Asymmetry and Equity Issues",Revised January 1990

"Managerial Decision Rules and the Estimationof Dynamic Sales Response Models", RevisedJanuary 1990

"The Effect of Computer Intervention and TaskStructure on Bargaining Outcome", February1990

"An Introduction to Group Decision andNegotiation Support", February 1990

"Reconfiguration of the Global SecuritiesIndustry in the 1990's", February 1990

"European Financial Integration and Its

Implications for the United States", February1990

"EEC Integration towards 1992: SomeDistributional Aspects", Revised December 1989

"Positive Prices in CAPE", January 1990

"Existence of Equilibrium in CAPM", January1990

"Why netvorking Fails: Double Binds and theLimitations of Shadow Networks", February 1990

"NSS Solutions to Major Negotiation StumblingBlocks", February 1990 •

"The Manufacturing Contribution toInnovation", February 1990

"A Discussion of Correct Measures ofInformation Asymmetry", January 1990

"The Expected Utility of Portfolios ofAssets", March 1990

"What Determines U.S. Retail Margins?",February 1990

"Information Asymmetry, Adverse Selection andJoint-Ventures: Theory and Evidence",Revised, January 1990

1990

90/01 B. SINCLAIR-DESGAGNE "Unavoidable Mechanisms", January 1990TM/EP/AC

90/02 Michael BURDA

"Monopolistic Competition, Costs of

EP

Adjustment, and the Behaviour of EuropeanManufacturing Employment", January 1990


"Management of Communication in International

TM

Research and Development", January 1990

90/04 Gabriel HAVAVINI and "The Transformation of the European Financial

FIN/EP Eric RAJENDRA

Services Industry: From Fragmentation toIntegration", January 1990

90/05 Gabriel HAVAVINI and, "European Equity Markets: Toward 1992 and

FIN/EP Bertrand JACOUILLAT

Beyond", January 1990

90/06 Gabriel HAVAVINI and "Integration of European Equity Markets:FIN/EP Eric RAJENDRA

Implications of Structural Change for KeyMarket Participants to and Beyond 1992",January 1990


"Stock Market Anomalies and the Pricing ofFIN/EP

Equity on the Tokyo Stock Exchange", January

1990

90/08 Tavfik JELASSI and

"Modelling vith MCDSS: What about Ethics'',

TM/EP B. SINCLAIR-DESGACNE January 1990

90/09 Alberto CIOVANNINI

"Capital Controls and International Trade

EP/FIN and Jae VON PARK

Finance", January 1990

90/10 Joyce BRYER and

"The Impact of Language Theories on 055TM Tavfik JELASSI

Dialog", January 1990

90/11 Enver YUCESAN

"An Overview of Frequency Domain MethodologyTM

for Simulation Sensitivity Analysis',January 1990

90/12 Michael BURDA

"Structural Change, Unemployment Benefits andEl

nigh Unemployment: A U.S.-EuropeanConoarison", January 1990

90/13 Soumitra DUTTA and

"Approximate Reasoning about TemporalTM Shashi SHEKHAR

Constraints in Real Time Planning and Search",January 1990

90/14 Albert ANGEHRN and

"Visual Interactive Modelling and IntelligentTM Bans-Jakob LOTHI

DSS: Putting Theory Into Practice",

January 1990

90/15 Arnoud DE MEYER, "The Internal Technological Renewal of aTM Dirk DESCHOOLMEESTER, Business Unit with a Mature Technology",

Rudy MOENAERT and January 1990Jan BARBE

EP/SM

FIN/EP

FIN/EP

90/33 Caren SIEHL,08 David BOWEN and

Christine PEARSON

"The Role of Rites of Integration in ServiceDelivery", March 1990

90/34 Jean DERMINE "The Gains fro. European Banking Integration,PIN/EP a Call for a Pro-Active Competition Policy",

April 1990

90/35 Jae Von PARK "Changing Uncertainty and the Time-VaryingEP Risk Premia in the Term Structure of Nominal

Interest Rates", December 1988, RevisedMarch 1990

90/36 Arnoud DE MEYER "An Empirical Investigation of ManufacturingTM Strategies in European Industry", April 1990

90/37 William CATS-BARIL "Executive Information Systems: Developing anTM/08/SM Approach to Open the Possibles", April 1990

90/38 Wilfried VANHONACKER "Managerial Decision Behaviour and theMKT Estimation of Dynamic Sales Response Models",

(Revised February 1990)

Documents

"AN EVALUATION AND SELECTION METHODOLOGY - Insead