MIS planning: A methodology for systems prioritization

Information & Management 27 (1994) 261-274

Research

MIS Planning: A methodology for systems prioritization

Ritu Agarwal a, Linda Roberge b, Mohan R. Tanniru b,*

a Department of MIS and Decision Sciences, The Unic~ersiry of Dayton, Dayton, OH 45469-2130, USA b School of Management, Syracuse Unicersity, Syracuse, NY 13244.2130, USA

Abstract

The literature proposes a number of approaches that allow for the consideration of corporate goals and objectives in prioritizing information systems using both financial and non-financial criteria. Research in cognitive psychology suggests that an individual confronted with a simultaneous consideration of both qualitative and quantitative factors tends to assign greater salience to concrete factors than to more abstract criteria. This paper proposes a multi-dimensional methodology that allows for the prioritization of systems proposals based on attributes that are mostly qualitative as the first step in the resource allocation phase of MIS planning. This initial prioritization is used in conjunction with other quantitative factors to arrive at a final system portfolio. The methodology is illustrated with the aid of a case study conducted at a non-profit organization.

1. Introduction

The relationship that an organization wishes to maintain with information technology is defined by the MIS planning exercise. Successful planning for its utilization is rapidly becoming a key deter- minant in an organization’s long-term ability to survive and compete [Rackoff et al., 1985; Led- erer and Mendelow, 19861. Specific tasks performed in MIS planning include both the identification of information systems that will support strategic and operational activities and the effective allocation of limited resources to developing them. Seveal methodologies for supporting MIS

* Corresponding author.

planning tasks have been described in the literature. For example, Porter [1980] identifies five forces of competition (buyers, sellers, competitors, substitutes, and new entrants) that affect the competitiveness of a firm, and indicates that information technology (IT) can play a strategic role in influencing these forces. Specifically, the influence of IT on the value chain [Porter and Miller, 19851 can form the basis for generating alternative strategies in order to gain a competitive advantage. Strategies and objectives devel- oped by an organization through such methods can be related to alternate information systems proposals using techniques such as strategic set transformation.

Criteria suggested in prior research as useful for evaluating competing systems proposals have

037%7206/94/$07.00 0 1994 Elsevier Science B.V. All rights reserved

SSDI 0378-7206(94)00032-S

262 R. Agarwal et 01. /Information & Management 27 (1994) 261-274

ranged from strictly quantitative, measurable considerations (such as net present value, return on investment and payback period) to more qualitative, intangible factors (including project risk and the project’s relationship with strategic needs and organizational culture) [Davis and Wetherbe, 1983; Klein and Beck, 19871. Among the qualitative considerations, particular importance has been accorded to the risk dimension, which has been related to various system characteristics such as complexity, budget, development time, and newness of technology [Drury, 1985; Fazollalahi, 19841. Risk is also inherent if a particular system has a negative impact on one objective and a positive impact on another. In addition, the significance of the political dimension in system prioritization has been emphasized [Davis and Olson, 19851. While earlier research has discussed many multi-attribute decision models to address the political dimensions in resource allocation decisions, their use in systems prioritization is not so prevalent.

Most of the methodologies do recognize a need to consider the role of the qualitative dimensions along with quantitative dimensions (tangible costs and benefits) for system prioritization. However, a questionable assumption underlies many of these procedurres - that decision makers are rational agents who are able to assess and weigh the importance of various data in decision making. The methodologies do not provide adequate means to account for errors of judgement that frequently lead to biased choice processes and an associated non-optimal allocation of resources [Slavic et al., 19881. Baises are particularly perva- sive in multiattribute decision making situations, where several disparate and incompatible aspects of each choice must be considered simultaneously.

Studies in cognitive psychology indicate that the human brain is an inadequate information processor, subject to a variety of biases and errors. Cognitive limitations compel decision makers to develop strategies and heuristics for deal- ing with large quantities of information and these heuristics are often less than optimal [Simon, 1965; Slavic, 1972; Hogarth, 19801. The processes of information acquisition, information process-

ing, output, and feedback in judgemental tasks are all prone to certain kinds of errors of judgement, such as anchoring and adjustment (where prediction is made by anchoring on a clue or value and then adjusting to allow for the unique circumstances under review), availability (where the chance availability of particular cues in the immediate environment affects judgement), hind- sight bias (constructing plausible explanations for what happened in the past that do not necessarily follow from events), etc. Of particular salience to the problem of system prioritization is the consideration of both the quantitative and qualitative aspects of a project. Typically, in a situation where all aspects of a system are considered simultaneously, there can be a tendency to weigh the quantitative or concrete criteria (such as payback period, net present value, etc.) more heavily than qualitative and abstract criteria (such as congru- ence with strategic objectives, effect on productiv- ity, etc.).

However, negative consequences can result when disproportionate influence is wielded by quantitative criteria: under such circumstances, projects that are financially attractive will be assigned a higher priority than projects that are strategically important. If IT is to be aligned with the long-term strategic needs of an organization and if the critical nature of systems is to be evaluated in an unbiased manner, then systems should be viewed first on a strategic dimension [Clemens, 1991a, 1991bl. The result of this exercise is a systems portfolio that rank orders projects on mostly qualitative dimensions. This initial portfolio can then be utilized, with quantitative factors, to arrive at a final systems portfolio.

Such a two-step approach allows an organization to view the MIS planning activity as a resource assignment decision. Because the initial prioritization is based on the strategic needs of the organization, it can be made independent of resource consumption: if needs are strategic, the selection of systems to meet these needs should not be resource bound, because new resources may be raised and assigned to them. Those that rank low on a strategic scale may then compete with other maintenance/enhancement/on-going types of projects for allocating the remaining

R. Aganval et al. /Information & Management 27 (1994) 261-274 263

resource, resulting in a final system portfolio. McFarlan [1984] evocatively describes such a strategy as a movement away from the “efficiency oriented” approach to resource allocation in systems development.

2. Systems prioritization on multiple dimensions

The methodology described here builds upon and synthesizes several techniques identified in prior literature. The inputs to it are a set of strategic needs that have been identified either directly by the users based on their goals/ objectives or extracted from long term corporate goals identified as critical for corporate survival. This is typically the first activity performed in MIS planning. Deriving the system prioritization subsequently consists of three primary steps (Fig. 1):

STEP 1: Identify the Relationship between Needs and Systems (N --, S):

The needs (N) defined by various user (U> groups or derived from corporate goals/strategies are

mapped to business subsystems 6) to determine which subsystems should be altered using IT in order to satisfy these needs.

STEP 2: Establish Ranking of the Needs (Prioritize

N)

The literature emphasises the need to link organizational strategies to MIS plans, the importance of technological risk in evaluating projects [McFarlan, 19811 and the impact of organizational culture in gaining commitment to system development efforts. Further, these approaches suggest that political factors must be incorporated into any system prioritization scheme. The proposed methodology scores the user needs on these four qualitative dimensions: strategic, technical, philosophical or political, prior to evaluating them on quantitative attributes.

The strategic dimension relates user needs to the strategic goals of the organization, explicitly or implicitly expressed, while the technical dimension evaluates the complexity of the technology required to satisfy a particular need. The philosophical dimension depends on the compati- bility of system architectures, proposed to meet a

TD

I r(N,TW

r(N,SD) r(N,PhDJ SD- U-N-S A PhD

f

r(N,PD)

PD

SD: Strategic Dimension TD: Technical Dimension F’D: Political Dimension PhD: Philosophical Dimension

N: Needs S: Subsystems U: Users r: rank

Fig. 1. Multi-dimensional methodology for systems prioritization.

264 R. Agurwal et al. /Information & Management 27 (1994) 261-274

user need, with the culture of the organization and its perceptions about its position in the technology arena. The political dimension associates a ‘weight’ that is a function of the user’s political strength in the organization. Each proposed need is scored on a scale of 1 to 5.

STEP 3: Initial Prioritization of System Proposals (Prioritize S using N --) S and Prioritized N)

Once the scores have been decided, they are transferred to subsystems using the mapping relationship of step 1 to obtain an initial system prioritization.

2.1. Implementation of STEP 1 - Establish User Need / System Relationship

2.1.1. Identification of User groups C(i) If needs are extracted directly from corporate

goals/objectives, then user groups are defined as those who will directly benefit from the satisfac- tion of these needs to some degree. If however, needs are extracted from users directly, then the planning group must identify all groups who may have an interest in using IT to fulfill their objectives. Following King [1978], users are defined as individuals that either have various claims on the organization (e.g. creditors, stock holders, regula- tors, etch or that have a stake in the way the organization is operated (e.g. managers, employees. vendors, customers, etc). Although the relative importance of each of these groups will not be equal, each group is considered, to some degree, a user of information technology in the organizational infra-structure.

2.1.2. Identification of Needs (N) Each user group may identify certain needs

and these can be extracted directly (e.g. the needs of creditors with regard to reporting, of regula- tors with regard to monitoring certain activities, etc) or indirectly (e.g. the needs of customers via the sales or marketing department, of vendors via purchasing, of stock holders via the finance department, etc>. Of course, the same need may be identified by many users and a user may identify several needs.

2.1.3. Identification of Subsystems (S) Any documentation technique may be used to

identify the process and data flows that help to accomplish the user needs. Flow diagrams assist in grouping processes into relevant, functionally cohesive subsystems. In the methodology proposed here, IBM’s Business Systems Planning technique [ 19811 is used to determine organizational processes that are critical to these informa- tional needs. These processes are then clustered into subsystems based on their data sharing and control characteristics.

2.1.4. Relationship between Needs and Systems (N

+ S) Since systems are formed by combining pro-

cesses, a user’s need may be satisfied by changes made to one of many systems and a change made to a system may impact many user needs. In addition, the alteration of a system may only partially satisfy a user need (e.g., increased efficiency in processing within a given system may reduce processing delays) or it may require that all relevant systems that impact a need be addressed simultaneously. The impact of automation on a user group objective is assessed subjec- tively (i.e., using the input provided by the MIS department). Also, a particular degree (i.e., which process will be automated) and type of automation (i.e., batch or on-line) is assumed to reduce the number of alternatives that are feasible and make the problem more tractable.

2.2. Implementation of STEP 2 - Prioritize Needs

on Multiple Dimensions

2.2.1. Strategic Dimension (N + SD) User needs are evaluated in terms of their

effect on the strategic and operational plans. This requires a determination of how well a user need meets strategic and operational objectives. The strategic grid technique [Cash et al., 19881 may be used to derive the role of technology on organizational strategies and to relate user needs to them. If an organization views the role of information system as strategic and a user need satisfies a strategic objective, then it is assigned a larger score. Thus, a need is assigned a score of five if it

R. Aganval et al. /Information 8s Management 27 (1994) 261-274 265

is congruent to the organizational view of the role of IT and a score of one if it is in conflict.

2.2.2. Technical Dimension (N + TD) Nolan [1979] hypothesizes that there are cer-

tain definable and recognizable stages of growth in the organizational utilization of IT. If the technology required to meet a user need is far more sophisticated than the current stage of the organization, then the risk associated with that project may be significant. One approach that can be employed to identify the level of sophistication required is to characterize information support in terms of such factors as number of systems to be interfaced (complexity), number of other needs and systems that have to be successfully com- pleted (dependency and complexity), and the type of technology proposed (degree of control that should be exercised on the resources). A score on the technical dimension is thus obtained by juxta- posing the level of sophistication required by a system with the organization’s current standing in the growth stage model. Needs for highly depen- dent information support and/or new technology are assigned a score of one, while needs with low dependency and needing established technology are scored five.

2.2.3. Political Dimension (N + P) The relative importance of user groups within

the organization’s formal or informal hierarchy is used to establish the significance of the needs. Since a user’s political strength can influence the support for systems required to meet the needs, users may be rank ordered based on their position on the management ladder and their control over organizational resources (if they are internal to the organization), or their claim on organizational resources (if they are external to the organization, e.g., shareholders).

2.2.4. Philosophical Dimension (N --, PhD) An organization’s philosophy must be compati-

ble with its proposed system characteristics in order to generate corporate commitment. This step requires identification of the organizational attributes and idiosyncrasies that may impact the organization. Some of these features are based

on the organization’s experience with technology (high or low), leadership (leader or follower), and risk profile (risk taker or averse). If user needs are in conflict with these features, there is less organizational commitment. For example, if the corporation is risk averse, a user need for new technology may find little support. Similarly, if the company is a leader in technological innovation, then such a user need is likely to receive strong support. A score of five on the philosophical dimension reflects a need that is closely aligned with the organizational philosophy, while one indicates conflict.

2.3. Implementation of STEP 3: Prioritize Systems

A simple weighted scoring scheme is used for prioritization. Each scored need from the other dimensions is multiplied by the need/system matrix. This results in a weighted score for each and they are then rank ordered based on their total weight. Alternately, the group involved in making the decision may assign a weight to each qualitative dimension to generate a single rank that reflects the overall importance of each system.

This procedure is highly subjective and has to be approved by the group involved in the planning decision. Though conceptually simple, it is also robust because the planning group may per- form sensitivity analysis; i.e., determine how the rankings may change with respect to a particular mapping procedure. Such a what-if analysis capa- bility makes this multi-dimensional model for system prioritization a decision support tool for resource assignment.

3. A case study: System prioritization at a Bureau of Collections

The Bureau of Collections (BOC) is located within the Department of Health and Social Ser- vices in one mid-western state of the USA. The BOC is responsible for preparing and collecting bills for care and services provided at three state mental health institutions and three centers for the developmentally disabled. Further, the Bu- reau collects for care at juvenile correctional in-

266 R. Agawal et al. /Information Q Management 27 (1994) 261-274

Table 1

List of user groups and their political ranking.

1. State Management Services I

2. BOC Investigators 6

3. Accounting Department 7

4. Auditors 7

5. Facilities Administrator 4

6. State Treasurer 8

7. Department of Economic Assistance 8

8. Medicare/Medicaid 5 9. County Boards 4

10. Responsible Party 3

11. Private Insurance Companies 3

12. BOC Management 6

13. Department of Health and Social Services 8

14. State Legislature 9

15. BOC Employees 2

16. Systems Development Personnel 1

stitutions from parents who are able to pay and pursues delinquent accounts that are referred from counties within the state. The BOC seeks payment from insurance companies, from individuals or their parents on an ability-to-pay basis, and from Medicare and Medicaid. There are several policies governing how payments are ap- plied to care provided and this process can often be quite complex. Fig. 2 summarizes some statis- tics relevant to BOC activities.

3.1. STEP 1: Identify the Relationship between Needs and Systems

Users (I/j: The activities of the billing system are influenced by various external agents, such as state level Management Services, State Trea-

surer, Department of Economic Assistance, County Mental Hygiene Boards, Department of Health and Social Services, private insurance companies, and the parties responsible for payment. In addition, other agents, such as auditors, BOC investigators, management, users, and employees also interact with and affect the system. The billing system communicates directly with some of these external agents and is influenced by policies set by others. The internal and external agents constitute user groups with a stake in the billing system (see Table 1).

Subsystems (S): The data flow diagram associated with BOC operations is used to group processes into the subsystems, as shown in Table 2. The clustering procedure used to derive these subsystems is based on the data set sharing concepts defined as a part of the BSP methodology.

Needs (NJ: Each BOC claimant identified a set of objectives (or needs) that they consider critical for improving their operations and these were gathered and tabulated by the analyst in Table 3. Any overlap in user needs is captured in the rows of Table 4.

Needs to Subsystems (N + S): The relationship between the organizational systems and the objectives is a function of the effectiveness of a change to a given system (in this case, the automation of each subsystem) in accomplishing each objective. Note that the mapping between the type of system chosen for automation and the degree to which it meets a particular objective must be addressed jointly by both developers and the relevant user group. The assessment has to be

* Total Collections

* Annual Budget * Annual Payroll * Number of Employees

* Annual Number of Patient

Accounts Handled

50 million dollars

650,000 dollars 500,000 dollars 30 (15 clerks, 7 investigators, 8 staff)

5,000-6,000

Fig. 2. The bureau of collections

R. Agarwal et al. /Information & Management 27 (1994) 261-274 267

made based on the measures that users select to evaluate the degree to which their objective is met. If the measure can be separately and partially influenced by multiple systems, then the developer establishes the automation that is needed to address the measures within each system while the user establishes the proportional improvement. In all other cases, where the impact of a measure is local (within one system) or joint (all related systems have to be modified to effect the measure), this assignment process is not an issue. Two assumptions are made. First, in deriving the mapping (shown in Table 5) only one version of system automation is considered. In fact, more must be considered if the versions differ significantly in their impact. Secondly, it is assumed that the version defined meets all related objectives; i.e., if a small degree of automation of the system is needed to meet objective k and a higher level of automation is required for objective m, then we must consider a version that satisfies both objectives. This constraint can be relaxed in the second step of the evaluation phase after identifying the set of critical systems.

Once the degree of system change needed to meet the appropriate objectives is determined, it is necessary to determine the extent to which each subysystem will meet the user objectives; this takes into consideration the fact that a single

Table 2

System configuration at BOC.

Subsystem

SSl

Main task

Process payments

Individual processes

Prepare receipts and deposits

Process bad checks

subsystem alteration may not completely meet an objective.

There are four ways in which a subsystem can impact a single objective. First, it is possible that each related system may contribute to the accom- plishment of a given objective ‘equiproporti- onately’ as is the case with objectives 9 and 29. Objective 9 requires an increase in the auditability of the system, with audit trails incorporated into all subsystems. However, this objective is still partially satisfied by a subset of these systems. The same is true for objective 29, which calls for an increase in the effectiveness and efficiency of operations.

Second, the impact of a subsystem on a given objective may be disproportionate, as is the case with objective 18: to institute standardization of forms and, since a majority of the claims pro- cessed are in Medicaid and Medicare operations, the impact of such standardization may be higher for operations processing Medicare/ Medicaid claims.

Third, maybe an objective can be achieved only if two or more subsystems are revised to- gether; i.e. no single system alone can meet the objective. For example, objective 25 desires a decrease in the total number of employees, while objective 32 calls for an increase in staffing levels. Accomplishing both these objectives may require

ss2

ss3

ss4

ss5

SS6

SW

SS8

Process rejected claims

Update facility ledger

Process Medicaid claims

Process Medicare claims

Update patient information

and ledger

Process delinquent accounts

Prepare billing

Process rejected claims

Prepare statements for facilities and boards

Update ledger

Process Medicaid claims

Process Medicare claims

Set-up, close, and update patient files and ledger

Prepare refunds Process investigations and delinquent accounts

Prepare billing

268 R. Agatwal et al. /Information & Management 27 (1994) 261-274

that subsystems 3 and 6 be automated to reduce weights and the worth of each subsystem is then

manpower and to shift the displaced staff to computed as a weighted sum as opposed to a

assist in claims processing. simple sum. Finally, it is possible that some objectives can-

not be met by any of the subsystems. In the case of objectives 1.5 and 16, the modification of existing subsystems will have very little, if any, impact on private fund-raising activities or finding alternate sources of funds to provide mental health care. Note that subsystems 6 (update patient information) and 1 (process payments) appear to be critical in meeting multiple objectives, assuming that each is assigned equal weight. If deemed necessary, objectives can be assigned differential

3.2. STEP 2: Establish Ranking of the Needs

Strategic Dimension (SD): The strategic plans of the organization must consider the effect of the environment on both classes of user groups over the planning horizon (of five years here). A forecast of the environmental factors that could potentially affect BOC’s operations is provided in Table 6; certain environmental factors, such as technology forecasts, may also be useful when

Table 3 Needs provided by various claimants.

OBOI

0B02

0B03

0B04

OB05

0B06

OB07

OBOX

0B09

OBlO OBll

OB12

0B13

OBt4

OBlS OB16

OB17

OBlX

OB19

0B20

OB21

0B22

0B23

OB24

OB25

0B26

0827

0B28 0B29 OB30

0B3 I 0832 0B33

0B34

OB3S

reduce the number of exceptions handled by the billing system

increase the flexibility of the system to easily adapt to changes in legislation

consolidate patient information while preserving security and privacy concerns

discourage bad checks and reduce the effort in processing them

reduce the necessity for refunds

provide accurate, timely, and complete information

provide adequate means for acknowledgement of investigator accomplishments

reduce the cost of data entry for the accounting unit

improve the auditability of the system

reduce the institution’s effort in providing BOC with needed information

accelerate payments to facilities take effective action that will reduce the number of delinquent accounts

reduce the time delay between issue of receipt/remittance advice and its recording by the treasury

provide accurate total receipts report

increase private source contributions

reduce reliance on public sources

accelerate the revenue cycle

use standard forms and follow rules in completing them process claims as early as possible

reduce the current level of effort put into the certification and recertification process

speed up payment due to the county hospitals

positively identify responsible party before billing

reduce or eliminate old equipment

provide for appropriate management reports

reduce the total number of employees

increase labor efficiency

reduce work load variability

reduce errors in operations increase the effectiveness and efficiency of BOC operations cut costs of providing all public health related services

reduce monotony on the job hire new employees to reduce work load

provide job security in the face of automation threat replace old system with a modern system using the latest technology

speed up refund check processing


projects are evaluated on the technical dimen- strategic grid), each objective is scored to derive sion. However, in general, while the other three the strategic dimension in Table 7. For example, dimensions (political, philosophical and techni- the objective ‘consolidate patient information’ cal) try to characterize the ability and willingness (Need no. 3) is assigned a score of five since it of the organization to manage projects internally, contributes to the achievement of operational the strategic dimensions tries to map the organi- support goals, while ‘increase flexibility’ (Need zation’s relationship with information technology no. 2) is given a relatively low score of one. based on an external view. For this reason, envi- Similarly, the objective ‘reduce reliance on public ronmental factors play a bigger share in establish- sources’ (Need no. 16) although an extremely ing the project’s role on this dimension. useful activity, receives a low score of one.

Based on the projected role of IT (low strategic support and high operational support on the

Technical Dimension (TD): Using the concepts of Nolan’s growth stage model and the level of

Table 4 Mapping of users to needs (user/objective matrix).

Needs Users

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19

20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

1 1 1

1 1 1 1

1 1 1

1 1

1 1 1

1 1 1

1 1 1

1 1 1

1

1 1 1 1

1

270 R. Agarwal et al. /Information & Management 27 (1994) 261-274

technological sophistication that each system de- mands, coupled with the knowledge that the organization as a whole is not very skilled in the use of IT, each objective is scored on the technical dimension. Objectives that require alteration of a single subsystem are assigned a higher score. For example, objective 34 needs the alteration of systems 1, 6, and 8, while objective 14 needs changes in system 1 only, and hence the difference in their scores. A distinction may also be made

between objectives affected by single systems if they require varying levels of technical sophistication.

Political Dimension (PD): Each claimant group is next assigned a score based on its position in both the formal and informal hierarchy or power structure, with the highest score being given to the most powerful and influential. A score on the political dimension is then obtained by multiplying the matrix of the mapping between users and

Table 5 Mapping of user needs to systems (system/objective matrix).

Needs Subsystems

SSl ss2 ss3 ss4 ss5 SS6 ss7 sss SS3&SS6

1 2 3 4 1.0 5 6 7 8 0.2 9 0.125

10 0.25 11 12 13 1.0 14 1.0 15 16 17 0.1 18 19 20 21 22 23 0.25 24 0.3 25 26 0.125 27 0.3 28 29 0.125 30 31 0.2 32 33 34 0.3 35

5.275

0.125 0.25

0.1 0.125 0.125

1.0

0.05 0.05

1.0

0.2

0.125 0.125 0.1 0.1 0.1 0.125 0.125

0.75 3.075

0.125 0.125 0.125 0.125

0.3 0.5 0.5 1.0

0.3 0.3 0.5

0.2 0.2

0.125 0.125

0.1 0.1 0.125 0.125

0.2

3.3

0.2

2.1

1.0 0.125 0.5 1.0

1.0 1.0

1.0 0.5 0.1 0.1 0.125 0.125 0.125 0.5

1.0

0.05 0.05 0.1 0.2

1.0 0.1 0.3

0.125 0.5 0.3 0.125

0.2

0.4

6.35

0.25 0.2

1.0 0.125 0.125 0.1

0.3 0.125 0.125

0.2 1.0

0.3 1.0 4.825 3.325 2.0


needs with this vector and normalizing it on a scale of 5, based on the relative weight of each need in comparison to that need with the maxi- mum score.

Philosophical Dimension (PhD): Scores on the philosophical dimension are obtained by relating the organizational characteristics of BOC (service based, risk averse, technology follower, etc). to the type of technical philosophy required for at- taining the objectives. For example, objective 22, obtain positive identification of the responsible party before billing, received 5 (as it is in concert with the risk averse nature of the organization), whereas objective 34, introduce latest technology, received a 1 (the organization is primarily a follower in technology innovation). Scores assigned to objectives are shown in Table 7.

3.3. STEP 3: Initial Prioritization of System Pro- posals

A total point score for each system on each dimension is computed by multiplying the score associated with each need on multiple dimensions with the need/system relationship. These totals

Table 6 Projected environmental impact on BOC.

are then used to generate a ranking of subsystems, as shown in Table 8.

Here, subsystem 6 remains critical on all but one dimension. Subsystem 1 is high on the political dimension as it meets the needs of two addi- tional users, county boards and private insurers, both of which wield significant political influence. If the highest ranking three systems are identified as critical and selected for implementation, then the rest can be prioritized using factors such as costs/ benefits. The assumption that these three will be chosen and designed may affect the cost estimates of the others (due to certain overlaps). If, however, resources are not available for three systems, then a certain amount of group discussion will be required before arriving at a consensus on the selection.

Given the qualitative nature of this prioritization process, assumptions regarding the entries (need/subsystem impact) or the scores may be challenged. Each user is allowed to state his/her needs without any prioritization. If, however, the total number of these needs becomes unwieldy, prioritization may become necessary.

If assumptions change, then it is likely that the

Economic Revenue from taxes will not increase at a rate sufficient to support increased demand for public health and social services. Taxpayers will continue to push for legislation that places a ceiling on tax rates and oppose any attempts to raise revenue through

taxes. This should result in tighter development and operations budgets for government departments and agencies. Interest rates

will continue to be high, i.e. about 10%.

Human Resources The private sector will continue to attract more qualified employees than the public sector. Employment in the public sector will become less attractive in the future because of its inability to provide a competitive work environment for its work force.

Technological Resources Computer hardware will continue to decrease in cost at a rate at least equal to the present rate of decrease. Computer networking

will be available commercially at reasonable cost. Software will be more expensive to develop in-house, but many off-the-shelf packages will be available at low cost.

Legislation New legislation will aim to increase the effectiveness and efficiency of public services through tighter controls.

External Environment The systems interacting with the billing system will increase their utilization of the computer.

Work Load on the Billing System The number of patients is estimated to increase at an average rate of 3% per year.

Management Public service management will continue to be risk averse compared to their private sector counterparts.

272 R. Agawal et al. /Information & Management 27 (1994) 261-274

Table 7 Table 8 Ranking of needs on multiple dimensions.

Needs Dimensions

- 1

2

3

4

5 6

7

8 9

10

11

12

13

14

15

16

17

18 19

20

21

22 23 24

25

26

27

28

29

30 31

32

33

34

35

Strategic Technical Philosophical Political

5 5 5 2 1 3 3 2 5 5 5 2 3 5 5 5 5 5 5 2 5 5 5 3 5 5 5 2 5 2 5 2 5 1 5 2 5 3 5 1 5 5 5 1 4 5 5 2 5 5 5 2 5 5 5 2 2 _ 3 2 1 _ 1 5 5 1 5 2 5 4 5 2 5 4 5 1 5 5 5 1 5 5 5 1 5 5 5 1 1 2 1 2 5 3 5 2 1 _ 1 2 5 1 5 2 5 2 1 2 5 1 5 2 5 1 5 2 1 1 5 2 5 2 5 1 1 1 1 1 1 _ 1 1 1 3 1 1 5 5 5 1

Rank order of systems on multiple dimensions.

Strategic Technical Philosophical Political

SS6 SS6 SS6 SSl SSJ SSJ ss7 SS6 SSl SSI SSl SSJ ss4 ss3 ss4 SS8 ss3 ss4 ss3 ss4 SS8 SS8 SS8 ss3 ss5 ss5 ss5 ss5 ss2 ss2 ss2 ss2

Note: If “role of IT” assumption is changed from operations1

to strategic, then the STRATEGIC rank order changes to

SS6, SSl, SS4, SS8, SSJ, SS3, SS5, SS2

score on the strategic dimension, but high on the operational dimension.

3.4. Prioritization using Quantitatirle Factors

To provide a reference point for contrast, alternate systems proposed for the BOC are prioritized using financial criteria. A quantitative rank is assigned to these systems based on the net present value of the cash flows resulting from the automation of each subsystem to meet respective user needs. Based on this ranking, only subsystems 4, 1 and 5 have positive cash flows (see Table 9). Thus only subsystems 1 and 4 are close enough to be considered acceptable on both dimensions.

There is an anomaly in the evaluation of subsystem 6; what is its rank after implementing subsystems 1 and 4? Is there overlap between these systems that could significantly reduce its unattractive cost/benefit ratio? Given resource constraints, only subsystems 1 and 4 were actually

ranking of the systems will be affected. The current rank ordering on the strategic dimension is based on the assumption that the role of IT is primarily geared towards operational support. However, if this assumption is challenged and the role is assumed to be strategic, then the system prioritization is altered. Table 8 shows the new ranking on the strategic dimension based on this new assumption, where the rank of subsystem 7 has dropped considerably in relation to others (1, 6, 4 and 8). This is partly due to the fact that subsystem 7 affected many objectives with a low

Table Y

NPV of cash flows by automating each subsystem.

Independently

ss4 $ 16,295 SSI 13,773 ss5 8,045 ss2 - 7,576 ss3 - 9,265 SS8 - 16,887 ss7 - 29,796 SS6 - 56,592

If SSl and SS4 are chosen

ss5 $ 24,000 SS6 1,000 ss2 600 ss3 - 9,265 SS8 - 16,887 SSJ - 29,796

R. Agarwal et al. /Information 8s Management 27 (1994) 261-274 273

undertaken as a first step towards automation. The methodology did, however, allow the BOC to study the implications of each system’s projected impact on both quantitative and qualitative dimensions without giving undue emphasis to ei-

ther.

3.5. Discussion

The approach has identified four dimensions often considered critical in project prioritization and demonstrated their potential impact when project rankings on these dimensions are presented without explicit cost/benefit consideration. The procedure does not, however, dictate what dimensions ought to be chosen in order to reduce the potential for inherent bias in the evaluation process. In this particular case, given that the system was operational in nature, the dimensions chosen were mapped to specific features of the existing system. However, when one is confronted with strategic systems whose scope is not quite clear and requirements not well defined, one may choose to define a diifferent set of dimensions (e.g., impact on competition, impact on market share, etc) and relate them to an overview of the proposed systems. Ultimately, the rankings have to be discussed and consensus reached before any final decision can be made on the allocation of organizational resources. The major thrust of this paper is to encourage a separate ranking of systems to reduce human bias towards concrete data.

One could argue that assigning numerical scores to projects on the qualitative dimensions is a highly subjective task, and the use of concrete criteria (such as numerical scores> may offset the bias we are trying to reduce. While the procedure does force the group to rank projects using numerical measure so that they can be compared on a particular dimension, the number itself is not that important. One could have chosen ‘very appropriate’ for a rank of 5 and ‘not at all appropriate’ for a 1 and the procedure would in essence remain the same. In addition, even if numbers are used, all projects are ranked on the same numerical scale with no monetary relevance, which often is the primary source of bias. Lastly,

while rankings on dimensions are a reflection of an individual’s subjective bias, the procedure does provide a mechanism to externalize these biases so that they can be discussed and defended in a group setting.

The political dimension is just one way to operationalize a claimant’s relative importance and its potential impact on a project. If one wants to remove the stigma associated with the term “political”, but still wants to capture the essence of project rankings on this dimension, it can be called ‘influence’ or ‘user impact’ without alter- ing the basic definition or its role. Lastly, no attempt is made here to combine the project rankings of each distinct dimension into one single rank, even though such a combination is typically done in practice. The decision to combine rankings into a single one using a weighting scheme has to be an outcome of the group’s subjective trade-offs among these dimensions.

4. Conclusions

Although traditional planning methodologies recognize the importance of qualitative data in MIS planning, they generally provide no means for incorporating these factors into the decision making process. The methodology presented here allows for an initial system prioritization using only qualitative dimensions. This preliminary ranking of systems projects can then be used in concert with quantitative factors to arrive at a final system portfolio. Such a two-step approach to system prioritization can reduce the bias that exists when concrete data is presented to decision makers at the same time as abstract data. The ability to evaluate proposals on multiple qualitative dimensions is critical if IT is to contribute in significant ways to the long-term needs of an organization.

Before any generalization, two issues need further exploration. The first relates to the relative variation betwen rankings of systems under both the single- and two-step approach. Would management be willing to finance high ranking projects that are not cost effective, independent of how critical they are? i.e. would they change their

274 R. Agarwal et al. /Information & Management 27 (1994) 261-274

initial proritization after the quantitative data is presented? If there is a threshold of ‘criticality’ above which they will be willing to accept projects that are obviously not cost effective, it is possible to determine this threshold?

Second, it is necessary to determine the feasi- bility of extracting system rankings from all those concerned. Based on our experience at the BOC, it appears that ranking on the political dimension may be difficult. However, it may be possible to use group decision making techniques that maintain a certain degree of anonymity in generating such a ranking.

References

Cash, J.I.. McFarlan, F.W., and McKenney, J.L., Informtion

Systems Management -The Issues Facing Senior Executit,es,

Homewood, IL: Invin, 1988, 2nd. Ed.

Clemens. E.K., “Evaluating Strategic Investments in Informa-

Clemens, E.K., “Corporate Strategies for Information Tech-

nology: A Resource-Based Approach”, IEEE Computer,

November, 1991b, 23-32.

tion Technology”, Communications of the ACM, Vol. 34,

No. 1, January, 1991a, 22-36.

Davis G.B. and Wetherbe, J.C., “Developing a Long-Range

Information Architecture”, Proceedings of the National

Computer Conference, Anaheim, CA, May, 1983.

Davis G.B. and Olson, M., Management Information Systems,

New York: McGraw-Hill, 1985.

Drury D.H., “A Survey of Data Processing Steering Commit-

tees”, Information and Management, 9 (19851, pp. 1-7.

Fazolallahi, B., “A Framework for MIS Planning”, Ph.D.

Dissertation, School of Management, Syracuse University,

Syracuse, N.Y.. 1984.

Hogarth, R.. Judgement and Choice: The Psychology of Deci-

sion, New York: Wiley, 1980.

IBM, Business Information Systems - Information System

Planning Guide, GE20-0527-3, July, 1981, 3rd. Edition. King, W.R., “Strategic Planning of Management Information

Systems”, MIS Quarterly, March, 1978.

Klein G. and Beck, P.O., “A Decision Aid For Selecting

Among Information System Alternatives”, MIS Quarterly,

June 1987, pp. 177-185.

Lederer A.L. and Mendelow, A.L., “Issues in Information Systems Planning”, Information and Management, 10

(19861, pp. 245-254 McFarlan, F.W., “Portfolio Approach to Information Sys-

tems”, Harvard Business Review, Sept.-Oct., 1981. McFarlan, F.W., “Information Technology Changes the Way

You Compete”, Harvard Business Retiew, May-June, 1984

pp. 98-103.

Porter, M.E., Strategy: Techniques for Analyzing Industries and

Competitors, New York: The Free Press, 1980.

Rackoff, N., Wiseman, C., and Ulrich, W.A., “Information

Systems for Competitive Advantage: Implementation of a Planning Process,” MIS Quarter/y, December, 1985, pp.

285-294.

Nolan, R., “Managing the Crisis in Data Processing”, Har-

vard Business Reuiew, 1979, pp. 115-126. Porter, M.E. and Miller, V.E. “How Information Gives you

Competitive Advantage,” Hanbard Business Reiliew, July-

August 1985, pp. 149-160.

Simon, H.A., The Shape of Automation for Men and Manage-

ment, New York: Harper & Row, 1965. Slavic, P., “From Shakespeare to Simon: Speculations - and

some Evidence - about Man’s Ability to Process Informa-

tion”, Oregon Research Institute Monograph, Vol. 12, No.

12, April 1972.

Slavic, P., Fischhoff, B., and Lichtenstein, S., “Response

Mode, Framing, and Information-Processing Effects in

Risk Assessment”, in Decision Making, D.E. Bell, H. Raiffa, and A. Tversky (eds), New York: Cambridge Uni-

versity Press, 1988.

Vitale, M.R., “The Growing Risks of Information Systems

Success”, MIS Quarterly, December, 1986.

Documents

MIS planning: A methodology for systems prioritization