12
Information & Management 25 (1993) 217-228 North-Holland 217 SOS Measuring and aligning information systems with the organization A case study Jonathan Miller University of Cape Town, Rondebosch, South Africa Alusaf (Pty) Ltd, a major international producer of alu- minum, assessed the performance of its Information Systems (IS) function using a validated survey instrument. The results of the survey were interpreted by management and led to a series of focused actions. A subsequent survey provided objec- tive evidence that the targeted actions had significantly im- proved user perceptions of IS performance. Information sys- tems activity had also been brought into greater alignment with organizational priorities. The article discusses measure- ment of IS performance and the notion of aligning IS with organizational goals. It describes the measurement instru- ment, the project conducted by Alusaf and the outcomes. The results represent a step towards operationalizing the concept of alignment between business and IS goals and activities. Kqvwords: IS management; IS alignment; IS success. Jonathan Miller is Director of the Centre for Information Systems, Uni- versity of Cape Town, South Africa. He received his Ph.D. from that uni- versity. His publications have ap- peared in the MIS Quarterly, the Pro- ceedings of the International Confer- ence on Information Systems (ICIS) and several South African journals. Prior to entering academe, he was Information Systems Manager at Cal- tex Oil (SAX He is a Past President of the Operations Research Society of South Africa and a Vice President of the Computer Society of South Africa. He is on the Program Committee for ICIS 1993. His research interests lie in the measurement of IS effective- ness and alignment and the value of groupware in the deci- sionmaking process. Correspondence to: J. Miller, Graduate School of Business, University of Cape Town, Rondebosch 7700, South Africa. 1. Introduction Measurement of the contribution of Informa- tion Systems (IS) to organizational wellbeing has become increasingly more difficult. As IS per- vades the total fabric of the organization, radi- cally influencing communications and underpin- ning business process reengineering, the bound- aries between individual systems applications be- come blurred. Conventional cost and benefit as- sessments, always imperfect, are becoming less and less relevant. Yet measurement has also be- come more important. Current philosophies such as “Total Quality Management (TQM)” stress objective and reliable measures of performance of both organizational processes and customer satisfaction. A prerequisite in many international contracts is adherence to standards such as the IS09000 series which embody comprehensive programs to measure both product and process quality. Whether at the level of the individual system or the overall IS function, the difficulty of mea- suring the contribution of IS to organizational performance has been widely recognized. For in- stance an important book on the business value of computers commences with the assertion that “there is no relationship between expenses for computers and business profitability” [21]. The author presents extensive empirical evidence to support this claim and lays out the complex con- tributions of computers to business. He rejects traditional financial measures in favor of his trademarked notion of “Return-on-Manage- ment.” But this method in itself requires complex judgments to distinguish “mission critical” from “management information” systems and sophisti- 0378-7206/93/$06.00 0 1993 - El sevier Science Publishers B.V. All rights reserved

Measuring and aligning information systems with the organization: A case study

Embed Size (px)

Citation preview

Page 1: Measuring and aligning information systems with the organization: A case study

Information & Management 25 (1993) 217-228

North-Holland

217

SOS

Measuring and aligning information systems with the organization

A case study

Jonathan Miller University of Cape Town, Rondebosch, South Africa

Alusaf (Pty) Ltd, a major international producer of alu-

minum, assessed the performance of its Information Systems

(IS) function using a validated survey instrument. The results

of the survey were interpreted by management and led to a

series of focused actions. A subsequent survey provided objec-

tive evidence that the targeted actions had significantly im-

proved user perceptions of IS performance. Information sys-

tems activity had also been brought into greater alignment

with organizational priorities. The article discusses measure-

ment of IS performance and the notion of aligning IS with

organizational goals. It describes the measurement instru-

ment, the project conducted by Alusaf and the outcomes. The

results represent a step towards operationalizing the concept

of alignment between business and IS goals and activities.

Kqvwords: IS management; IS alignment; IS success.

Jonathan Miller is Director of the Centre for Information Systems, Uni- versity of Cape Town, South Africa. He received his Ph.D. from that uni- versity. His publications have ap- peared in the MIS Quarterly, the Pro- ceedings of the International Confer- ence on Information Systems (ICIS) and several South African journals. Prior to entering academe, he was Information Systems Manager at Cal- tex Oil (SAX He is a Past President of the Operations Research Society of

South Africa and a Vice President of the Computer Society of South Africa. He is on the Program Committee for ICIS 1993. His research interests lie in the measurement of IS effective- ness and alignment and the value of groupware in the deci- sionmaking process. Correspondence to: J. Miller, Graduate School of Business, University of Cape Town, Rondebosch 7700, South Africa.

1. Introduction

Measurement of the contribution of Informa-

tion Systems (IS) to organizational wellbeing has become increasingly more difficult. As IS per- vades the total fabric of the organization, radi- cally influencing communications and underpin- ning business process reengineering, the bound- aries between individual systems applications be- come blurred. Conventional cost and benefit as- sessments, always imperfect, are becoming less and less relevant. Yet measurement has also be- come more important. Current philosophies such as “Total Quality Management (TQM)” stress objective and reliable measures of performance of both organizational processes and customer satisfaction. A prerequisite in many international contracts is adherence to standards such as the IS09000 series which embody comprehensive programs to measure both product and process quality.

Whether at the level of the individual system or the overall IS function, the difficulty of mea- suring the contribution of IS to organizational performance has been widely recognized. For in- stance an important book on the business value of computers commences with the assertion that “there is no relationship between expenses for computers and business profitability” [21]. The author presents extensive empirical evidence to support this claim and lays out the complex con- tributions of computers to business. He rejects traditional financial measures in favor of his trademarked notion of “Return-on-Manage- ment.” But this method in itself requires complex judgments to distinguish “mission critical” from “management information” systems and sophisti-

0378-7206/93/$06.00 0 1993 - El sevier Science Publishers B.V. All rights reserved

Page 2: Measuring and aligning information systems with the organization: A case study

218 SOS Information & Management

cated analyses to separate operational from man- agement costs. In a comprehensive review of 180 articles, including 100 empirical studies, DeLone and McLean [6] classify the variety of measure- ment approaches that have been applied and the lack of consensus as to their efficacy for evaluat- ing the benefits of IS. They conclude that there is no one measure of IS success, but six major and interrelated dimensions.

Related to IS success is “alignment.” Indeed alignment of information systems with the organi- zation repeatedly features as a key concern in surveys of IS priorities (eg. North America [9], Europe [19], South Africa [18] and Australia [23]). Several authors consider this notion as funda- mental to IS success. King [ll] refers to coordina- tion of IS and business plans and proposes that the information systems strategy set should be derived from the organization’s strategy set. Us- ing the Miles and Snow typology [14], Das et al. [5] argue that the content of strategic MIS plan- ning and the planning process should match the organization’s strategic type. Henderson and Venkatraman [8] propose a comprehensive strate- gic alignment model linking business and IT at the strategy level and at the level of infrastruc- ture and processes. They argue for alignment across internal and external domains and func- tional integration across business and IT areas.

There are a few empirical studies of align- ment. These include Tavakolian [22], who surveys 52 large US organizations and finds a much greater degree of centralization of IT activities among Miles and Snow’s “defenders” than among “prospectors.” He concludes that IS administra- tors should shape their IT structures in light of the competitive strategy of their firms. Zviran [241 studies 131 Israeli companies and demonstrates that there is a clear association between IS objec- tives and organizational objectives. Lederer and Mendelow [12] interview 20 top IS officials and find a range of difficulty in achieving King’s ob- jective of coordination between business and IT plans. They support the need for coordination and identify top management mandating of the planning process as key to success.

As a step towards operationalizing the notion of business/IS alignment and relating it to per- ceived organizational benefits, the author and colleagues have compared the perceived perfor- mance of a range of IS activities with their rela-

tive importance to the organization [15,16,171. The statistical correlations between importance and performance cover a wide range and prove to be associated with overall IS performance ratings as perceived by both user and IS personnel. The instrument and method allow comparisons across firms and industries.

This article describes one firm’s attempt to improve the contribution of IS to the business. They measured IS performance using the instru- ment mentioned above, diagnosed deficiencies and took corrective action. While the firm’s ob- jectives were very pragmatic, their approach en- abled the author to investigate alignment of IS and the business as well, and to see whether shifts in perceived performance were accompa- nied by changes in alignment. The next section describes the measurement instrument and previ- ous empirical results that establish a connection between IS performance measures and business alignment. Subsequent sections describe the com- pany, the surveys conducted, actions taken and results achieved.

2. Characteristics of the instrument

Building on the work of Bailey and Pearson [2] and Alloway and Quillard [l], the author and colleagues developed and applied a new percep- tual instrument to evaluate the overall IS func- tion [151. The instrument, here termed the Miller-Doyle instrument, taps the perceptions of respondents regarding organizational importance and IS performance on a range of items. Ap- pendix 1 lists the items in abbreviated form. Re- spondents, who may include both users and IS staff, assess the same items twice on different scales (Appendix 2).

The nature and administration of the Miller- Doyle instrument can be distinguished from other efforts to measure IS constructs such as User Information Satisfaction (UIS) in several ways.

(i) A current framework for IS [7] was used as a basis for developing the set of items comprising the instrument. The 37 items shown in Appendix One thus reflect operational characteristics of facilities and systems, planning and control is- sues, and the roles and characteristics of execu- tives, users and implementors.

(ii) UIS is defined as “The extent to which

Page 3: Measuring and aligning information systems with the organization: A case study

Information & Management

users believe the information system available to them meets their information requirements” [lo]. The notion of UIS thus lies squarely in the infor- mation arena and the connection to organiza- tional activity is left implicit. Alignment, however, should make explicit the connection between or- ganizational needs and IS capabilities. It is the difference between “satisfying the customer’s ex- pressed requirements” and “meeting the latent needs of the customer” [20]. UIS instruments address the former and the current one the latter.

(iii) Early instruments (eg. [2]) used perfor- mance-related scales and an importance weight- ing for each item. The Miller-Doyle instrument separates and compares importance and perfor- mance scales. The importance scale is regarded as a measure of the organizational importance of the particular aspect of IS compared to the sec- ond scale which assesses performance of that aspect. This reflects the stated objective of opera- tionalizing alignment.

(iv) Both the current approach and UIS stud- ies capture people’s perceptions. However, the current instrument uses wording to tap cognitive perceptions of organizational priorities and IS performance. It does not encourage affective re- actions to personal IS experiences characteristic of UIS instruments.

(v) Surveys of IS effectiveness rarely treat IS people as more than simply providers of technical information. The studies reported here specifi- cally seek responses from both IS professionals and users.

3. Empirical findings

Two large-scale studies are mentioned here. In the first, responses to the instrument from about 800 user and IS managers in the manufacturing, retailing and financial services sectors were gath- ered and analyzed. The instrument yielded satis- factory levels of construct validity, predictive va- lidity, test-retest reliability and reliability in the face of measurement error. In particular factor analysis revealed six stable and intuitively mean- ingful constructs underlying the 37 items in the instrument (15,161. These factors are named:

1. Traditional Systems 2. End-User Computing

J. Miller / Measuring and aligning IS 219

1 3 ,,,,,,,,,,/,1,1,1, I, I r , * 1 7 17

.2 .3 .4 .5 .b .7 .a .9 Pearson’s r

Importance-Performance Alignment : IS Staff

Fig. 1. IS performance ratings and importance - performance alignment.

3. 4. 5. 6.

Strategic Issues Responsiveness to User Participation

Change

IS Staff Characteristics

An important finding from the first study was that high IS performance ratings associated with high levels of correlation between business im- portance and IS performance. The second study confirmed and deepened this finding [17]. In this latter study over a thousand IS and user man- agers in eleven organizations covering manufac- turing, retailing, finance and the public sector participated (response rate of 60%). Overall IS performance ratings and correlations between importance and performance varied widely across the organizations, but the association between the two dimensions emerged clearly. In particu- lar, high correlations associated with high perfor- mance ratings perceived by the user community. Figure I shows this relationship.

The results also showed that IS staff and users were generally in agreement on the business im- portance of the various elements of IS. However, counter-intuitively, the alignment in the views of IS staff and users on importance alone did not relate to IS performance ratings. It was the align- ment between importance and performance that is key. Figure 2 shows this finding. Against this background the application of the Miller-Doyle instrument in a particular organization is now discussed.

Page 4: Measuring and aligning information systems with the organization: A case study

220 SOS Information & Management

4. Case study of AIusaf (Pty) Ltd

4.1. Background

Alusaf (Pty) Ltd. is the major producer of primary aluminum in South Africa and ranks among the top twenty corporations in the coun- try. The company is owned by Alusuisse (Switzer- land) and South African conglomerates. At the time of the case Alusaf had a turnover of $400 million and employed 2800 people. It produced 170000 tonnes of Aluminum annually and ex- ported 50 percent.

A new chief executive took office in 1982. The focus he brought to bear strongly emphasized formal strategic planning initiatives and over the period 1985-1987 a series of conferences and workshops took place. Management set mission statements, goals, objectives and action plans for the company as a whole and at the operational level. These documents reflected the primary strategic thrusts of the company - cost reduction through production efficiencies, and quality con- trol. A cohesive top management team and a healthy climate of participative management evolved, reenforced by the relaxed, informal busi- ness and social environment characteristic of the surrounding community.

Alusaf s Information Systems department con- sisted of fourteen senior staff members, orga- nized into Systems, Technical Support and Oper- ations. DP operations were highly centralized. Two Hewlett Packard 3000/7Os handled all data processing and were connected to the plant, labo-

User Rating of IS Performance

5.51

I I 1 3~,,,,,,,,,,,,,,,,,,,,,,,,,““,, I”

2. .3 .4 .5 .b .7 .a .P Pearson’s r

IS Staff - User Alignment : Importance

Fig. 2. IS performance ratings and user-IS staff alignment on importance.

Table 1

Overall results: First survey.

Users IS Staff Difference

Importance 5.26 6.09 0.83 *** Performance 4.21 5.41 1.14 ***

* * * Significant at < 0.001.

ratories, finance and administration and the other organizational areas through 150 terminals. Op- erational data was held in a central data base and accessed via a series of purchased packages that handled payroll, stores, maintenance, quality con- trol, production control, sales and finance. A fourth generation language and business report writer were available and used to a small extent. Microcomputers had yet to appear.

The IS Manager reported to the Senior Man- ager Management Services, Peter Cowie, who in turn reported to the Director, Finance. Computer activity was overseen by a steering committee with the Technical Director as chairman and Finance, Management Services, IS and Technical Process managers as members. However this committee met only annually and was not re- garded as effective in providing business direc- tion. At the start of this case (1987), the Informa- tion Systems department had yet formally to ex- amine its own direction and strategy as already undertaken by other functional units. Therefore, the Management Services Manager decided to conduct a survey and lay the foundation for fu- ture IS planning and evaluation.

4.2. The first survey

, The idea of a perception survey was tested with the IS staff. Initial reactions were negative: “We don’t need criticism from the users . . . users don’t understand IS . . . perceptions are vague.” Nonetheless Cowie proceeded with the survey, distributing the Miller-Doyle instrument to sixty four company managers and senior IS staff. Forty managers and thirteen IS staff re- sponded, representing 83 percent of those sur- veyed. The importance and performance scales in Appendix 2 have a range of one (low) to seven (high). Table I shows the average results for each scale across all items in the questionnaire.

Users rate Information Systems as “very im- portant,” and IS staff rate it as approaching “crit-

Page 5: Measuring and aligning information systems with the organization: A case study

Information & Management

Table 2

Ratings by factor: First survey.

Factor Importance Performance

Users IS Staff Users IS Staff

1 Trad systems 5.55 6.51 4.47 5.57

6 IS Staff char. 5.35 6.14 4.34 5.31

3 Strategic issues 4.96 5.85 4.28 5.38

2 End user camp. 5.34 5.92 4.22 5.50

4 Responsiveness 5.02 5.77 4.02 5.36

5 User participat. 4.96 5.67 3.93 5.18

ical.” Users rate IS performance as slightly above average, whereas IS staff rate it as well above “good.” The differences between IS Staff and user ratings are statistically highly significant and the largest encountered in an extensive survey of eleven organizations [ 171.

The responses to individual items were grouped into the six factors underlying the ques- tionnaire. Table 2 shows the Importance and Per- formance ratings by factor for the two groups of respondents, in decreasing order of user perfor- mance rating.

The table reveals that the overall gap between user and IS perceptions applies to each area of IS activity as well. However users and IS agree that the traditional systems area, IS staff characteris- tics and end-user computing are most important and place them in the same order.

In line with section 3 and Figs. 1 and 2, IS Staff and User ratings of the importance and performance of all the individual items in the questionnaire were compared. The Pearson’s r correlation between IS and users for importance is 0.77. This is a statistically significant and quite strong association. There is also a significant, but not as strong association between IS and users ratings of performance. The r value is 0.63. The correlation coefficients linking importance and performance are 0.50 and 0.55 for Users and IS Staff respectively. These statistics will be referred to in subsequent sections.

4.3. Action taken

The size of the absolute difference between IS Staff and User performance ratings surprised the IS group and lead to much debate. Cowie and his staff concluded that they had been too involved

.I. Miller / Measuring and aligning IS 221

with meeting and overcoming the technical chal- lenges of implementing particular packages. While this focus was understandable at the time, they agreed that IS had lost touch with their users and had unwittingly become too product focused. The user assessment of IS performance was a “message” that IS needed to adopt a marketing focus. As Cowie wrote in a letter to the author:

“I am now busy analyzing the results in detail and

hope to build the results of this analysis into a strategic

information systems plan I am extremely excited

about the results so far and firmly believe that the next

year will see us focusing our energy in the right areas.”

[31

The plan put into effect indeed had a strong marketing orientation [4]. A Mission Statement for IS, in line with the company Mission State- ment, was designed (see Appendix Three). The IS group published a framework for IS that showed the integration of information systems with business processes. The Management Ser- vices area was reorganized to bring the IS group closer to colleagues in the Corporate Administra- tion and Audit Departments.

A customer analysis was undertaken and

“a program of customer visits was set in motion. The

MIS people went into the world of the user to see what they were doing and to learn something of the business

environment.” [4]

User groups were created for major systems and a systems support analyst assumed the role of mentor for each group. Analysts committed to staying in touch with their user groups, improving their understanding of the relevant business is- sues and focusing on customer needs.

A variety of promotional activities took place:

Cowie presented the IS Strategic Plan to the Board of Directors IS hosted Top Management at information conferences. IS Staff took user groups to computer exhibi- tions. an IS Newsletter was produced social events were held to celebrate implemen- tation milestones.

Page 6: Measuring and aligning information systems with the organization: A case study

222 SOS Information & Management

These management actions greatly increased the exposure of IS Staff to business needs and users to the capabilities of computing technology. In particular they served to address two aspects of IS, Responsiveness to Change and User Partic- ipation.

While no major new systems were introduced in the year following the survey, a variety of user-requested adaptations of existing systems and packages resulted from the user group and mentoring activity and were undertaken by IS.

After diagnosing the results in the important but poorly performed area of End-User Comput- ing, the IS Department took particular action. They purchased a hundred or so microcomputers with wordprocessing and analytical software and deployed them throughout the organization. The cost was absorbed into the IS budget. Further- more, they launched a financial scheme by which staff throughout Alusaf could obtain interest free loans to acquire their own PCs. Employees ac- quired 120 PCs in this way.

nine IS staff). Table 3 compares the overall rat- ings for importance and performance in the two surveys. The only statistically significant shifts are in the user ratings. The largest change is for the users’ rating of IS performance, which essentially shifts from “average” to “good.” The shift in user perceptions of importance is not very large and only marginally significant. In relation to the larger survey mentioned earlier, however, it serves to shift the user perception of IS importance from being the lowest in the set to about the middle [17]. The gaps between perceptions of users and IS staff remain statistically significant, but are much narrower than in the first survey.

4.4. The second survey

A year later the same population responded to a second survey. Sixty three questionnaires were issued and forty nine returned (forty users and

Figures 3 and 4 examine the responses of the users in more detail. Figure Three shows small but consistently positive increases in user impor- tance ratings. The largest increases are for User Participation and IS Staff Characteristics, the two “behavioral” factors in the Ein-Dor & Segev framework that underlies the instrument. Figure Four, drawn to the same scale as Figure Three, shows much greater across the board increases in user ratings of IS performance. The aspects specifically targeted for attention, User Participa- tion, Responsiveness to Change and End-User Computing all enjoyed large increases, as did IS Staff Quality.

I I Traditional

End User Computing

Strategic Issues

Response to Change

2nd Survey

User Participation IS Staff

Fig. 3. Importance ratings.

Page 7: Measuring and aligning information systems with the organization: A case study

Information & Management J. Miller / Measuring and aligning IS 223

Fig. 4. Performance ratings.

The intense focus on the total IS function and some aspects in particular resulted in a general improvement in perceptions of IS performance as well as particularly large improvements in the areas specifically tackled. One interpretation is that respondents considered the specific actions undertaken to be well targeted and beneficial. The positive results in those areas led to carry- over effects as well. ’

1 It was not possible to have a control group against which to

compare the results of this study. Therefore there is the

possibility that the observed results have alternative expla-

nations. One is that of a “Hawthorne” effect. However in

two other longitudinal studies by the author, IS made con-

siderable efforts and spent a lot of money to improve IS

effectiveness, but user ratings remained constant at 3.8 and

4.0 respectively. It is apparently not inevitable that close

attention to the IS area results in improved ratings. A

second alternative relates to extraneous changes that might

affect the results, but Alusaf did not go through other

organizational changes during the period that might explain

the shifts in user perceptions of IS. Thirdly, retesting the same group may influence the scores. It is believed that a year’s break is sufficient to reduce this possibility consider-

ably. Finally there is the question of the stability of percep-

tual observations. Other studies using the Miller-Doyle in- strument provide encouraging support for the stability of

user perceptions of IS performance when measured at widely spaced intervals [16].

4.5. Achieving alignment

Section 3 reported a positive association be- tween IS performance ratings and measures of alignment between IS importance and perfor- mance. There was no such association for align- ment between IS staff and user perceptions of importance or performance separately. Those studies were cross-sectional in nature - snapshots in time - but suggest that improvements in IS performance ratings over time should be accom- panied by changes in the relevant alignment coef- ficients. Table 4 shows the user ratings of IS performance and the various alignment coeffi- cients (Pearson’s r) for the two surveys.

The perceived alignment between business im- portance and IS performance increased notice-

Table 3

Overall ratings: Comparison of survey results.

First Second Difference Survey survey

Importance Users 5.26 5.42 +0.16 * IS Staff 6.09 5.98 -0.11

Performance Users 4.27 4.99 +0.72 *** IS Staff 5.41 5.50 + 0.09

* Significance - 0.05, ***Significance < 0.001.

Page 8: Measuring and aligning information systems with the organization: A case study

224 SOS Information & Management

4.5 5 5.5 6 6.5 I

Importance

Fig. 5. Importance-performance alignment for IS staff: First survey.

ably for both users and IS staff. There was also improvement in those r values was not as large as an increase in alignment between the views of for the increases in importance-performance users and IS staff on both the importance and alignments. performance of the various aspects of IS, but the The results are consistent with the proposition

6

8

25

G d

4

3

‘Pearson’s r = 0.78

L

3

m q

4 4.5 5 5.5 6 6.5 1

Importance

Fig. 6. Importance-performance alignment for IS staff: Second survey.

Page 9: Measuring and aligning information systems with the organization: A case study

Information & Management .I. Miller / Measuring and aligning IS 225

Table 4 Comparsion of survey results.

User Rating of IS Performance

Importance-Performance

Correlations (r)

Users

IS Staff

User-IS-Staff Correlations (r)

Importance

Performance

1st 2nd

Survey Survey

4.27 4.99

0.50 0.73

0.55 0.78

0.77 0.86

0.63 0.72

that user assessment of IS performance is influ- enced by the ability of IS staff to focus success- fully on those aspects of IS they (IS) regard as most important. To the extent that IS staff cor- rectly interpret users, they will then succeed in focusing on user priorities.

Figures 5 and 6 are scatter plots of the IS Staff importance-performance responses to all the items in the questionnaire in the first and second surveys respectively. The “tightening” of the band of responses from the first to second surveys (reflected in the increased r values) is evident. This, with an improvement in the already signifi- cant alignment between IS staff and user views on the relative importance of the different as- pects of IS, suggests that management action indeed focused IS better on user-related priori- ties and that this focus translated into effective action. ’

4.6. Summary

The Alusaf project is an example of one orga- nization’s attempt to measure IS effectiveness and make improvements. Initially the company had intended to identify areas of poor perfor- mance and concentrate on improving them. The large discrepancy between user and IS staff rat- ings that emerged from the first survey came as a surprise. Interpretation led to a shift from a product- to marketing-orientation and the spe- cific managerial actions described. It also led to

* The scatter plots can be divided into quadrants highlighting “acceptable” and “unacceptable” situations. The author

has found this useful for interpretation and focused action

planning by practitioners.

more focused efforts to meet user needs with existing systems and to make a direct investment in end user computing.

Alusaf had not explicitly attempted to increase alignment measures. In retrospect, however, their actions clearly should have had a measurable effect on alignment between IS staff and users. Furthermore, the exposure of IS staff to the business through heightened contact with users and users to IS through attendance at confer- ences etc., should have resulted in measurable improvements in the relevant alignment coeffi- cients.

It is a tribute to the managers and IS person- nel involved that they succeeded. They were able to make tangible changes, demonstrate measur- able improvements in perceptions, and bring about a closer alignment between perceived im- portance and performance.

5. Conclusions

This article has argued that, in an era of all- pervasive application of information systems and technology, traditional methods of measuring the contribution and effectiveness of the information systems function are inadequate. Effectiveness criteria must recognize the dynamic nature of organizations, adapt to continually changing orga- nizational effectiveness criteria and be influential in shaping management action. In particular such action should have as a key objective improving the alignment between IS and the business in respect of goals, infrastructure and processes.

The results of the development of the percep- tual instrument described here and analysis of the outcomes of large scale studies suggests that the popular approach of tapping user attitudes as a surrogate for IS effectiveness is not fully satis- factory (see also [13]). More fundamental is achievement of alignment between organizational priorities and IS performance.

The instrument described here makes the con- nection between IS capabilities and organiza- tional purposes explicit. It stresses the roles of both users and IS staff as “expert witnesses” in assessing the alignment between IS and the orga- nization. The case study demonstrates the value of such measurement. However the ability to measure is clearly only the first step. The case

Page 10: Measuring and aligning information systems with the organization: A case study

226 SOS Information & Management

also demonstrates that effective management and a willingness to make technological and behav- ioral changes are essential to effect improve- ments.

Overall cost-effectiveness Security of data

Postscript

Hardware and systems downtime Efficient running of current systems Direct user access to data and models Models to analyze business alternatives Data analysis to support decisionmaking Ease of access to computer

Since the time of this case, Alusaf has em- barked upon the construction of the largest smelter ever built at one time in the world. It will be capable of smelting 466000 tonnes of alu- minum a year. 1400 employees will be required to run the new plant. Five general managers will manage the operation - Production, Technical Services, Human Resources, Information Ser- vices, and Finance & Administration. The total production process will be automated and there will be a full integration of process and commer- cial computing. The information function has been elevated to divisional status along with pro- duction, finance and engineering. Alusaf has not attempted directly to assess the value of the changes effected by its IS project, but the signifi- cant repositioning of the IS function and associ- ated budgetary changes at the very least suggest a large positive shift in perceptions of the role of IS in the company.

system Top management involvement Application of modem technology Increased IS effort on new systems IS strategic planning and resource allocation Use of IS steering committee Priorities reflecting org. objectives IS providing competitive advantage Office communications and IS Degree of training in user

235 3 3 3 3 3 3 3 3

proficiency

Acknowledgments

The author gratefully acknowledges the assis- tance of Mr Peter Cowie of Alusaf in the prepa- ration of this paper. The anonymous reviewers also provided valuable suggestions incorporated in the final version.

Processing of change requests Time required for new development Flexibility of systems Responsiveness to changing user needs Improving of new systems development Users’ feeling of participation Users’ understanding of systems Users’ feeling of influence over services Quick and flexible access to computer data Attitude of EDP staff Communication with EDP staff Technical competence of EDP staff Quality of systems analysts User-oriented systems analysts IS support for users with IS proposals

3s 4 4 4 4 4 5 5 5 5 6 6 6 6 6 6

Appendix 2. Scales

Appendix 1. Items in Miller-Doyle instrument (Descriptions abbreviated)

(Abbreviated descriptions; numbers refer to fac- tors described in text. Two items load equally onto two factors)

Accuracy/ completeness of output 1 Relevance of products/ services provided 1 Timeliness of output information 1 Currency of output information 1 Users’ confidence in systems 1

Importance Scale (“assess the importance to your organization’s activities”)

Irrelevant Possibly Very Critical

Useful Important

1 2 3 4 5 6 I

Performance Scale (“assess your organization’s performance on this item”)

Very Poor Poor Good Excellent

1 2 3 4 5 6 7

Page 11: Measuring and aligning information systems with the organization: A case study

Information & Management J. Miller / Measuring and aligning IS 227

Appendix 3. Alusaf mission statement for infor- mation systems

Purpose

We are a service organization and provide the Shareholders and Management with a quality Management Information Service.

We are committed to relating information sys- tems to business needs in order to provide a competitive advantage to the Company.

We are committed to promoting the need for effective information and planning and control systems and a general awareness of the business systems to all decision makers within the Com-

pany. We are an integral part of this dynamic organi-

zation and make a significant contribution to the effective management of the business.

Values

Marketing orientation We believe that the success of our customers is

a measure of our own success. We seek to satisfy their needs and dedicate

our efforts to provide them with distinctive pro- fessional service and technical support.

Quality We believe the quality of our products and

services is paramount. We expect a dedication to quality from all our people as well as our suppli- ers of goods and services. We will concentrate on doing the job right first time and will not tolerate wastage.

Human relations We believe in the development of our people

and will encourage training to enable each em- ployee to progress within the Company.

We helieve that communication is the corner- stone of good people relations and we encourage sincere face-to-face communication at all levels.

We believe that our own results should be achieved by a team effort from all of our people.

Management style We believe in participating in the team build-

ing style of management based on clear objectives

and strong leadership. We associate ourselves with the sound business and generally accepted practices of this organization, and will maintain ethical and cordial relations with outside institu- tions.

References

[l] Alloway, R.M. and Quillard, J.A., “Top Priorities for the Information Systems Function,” Center for Information

Systems Research, MIT, Working Paper No. 79, Sept.

1981.

[2] Bailey, J.E. and Pearson, S.W., “Development of a Tool for Measuring and Analyzing Computer User Satisfac-

tion,” Management Science (2951, May 1983, pp. 530-545.

[3] Cowie, P.W., Private Communication, 1988.

[4] Cowie, P.W., “The Development of a Marketing Ori-

ented Strategy for the Corporate Information System.”

Proceedings of the Interex 1989 Conference for Hewlett-

Packard Users, San Francisco, September 11-15 1989.

[5] Das, S.R., Zahra, S.A. and Warkentin, M.E., “Integrat- ing the Content and Process of Strategic MIS Planning

with Competitive Strategy,” Decision Sciences, 22 (51,

November/December 1991, pp. 953-984.

[6] DeLone W.H. and McLean E.R., “Information Systems Success: The Quest for the Dependent Variable,” Infor-

mation Systems Research, 3 (11, March 1992, pp. 60-95.

[7] Ein-Dor, P. & Segev, E., A Paradigm for Management Znformation Systems. New York: Praeger, 1981.

[8] Henderson J.C. and Venkatraman N. “Strategic Align-

ment: A Model for Organizational Transformation Through Information Technology,” In Transforming Or- ganizations, Eds. Kochan T.A. and Useem M. Oxford

University Press, New York, 1992.

[9] Index Group. Critical Issues of Information Systems Man- agement for 1991: The Fourth Annual Survey of I/S Management Issues, Boston: Index Group, 1990.

[lo] Ives, B., Olson, M.H. & Baroudi, J.J., “The Measure- ment of User Information Satisfaction,” Communications of the ACM, (26:10), Oct. 1983, pp. 785-793.

[ll] King, W., “Strategic Planning for MIS,” MIS Quarterly, 2(l) 1978, pp. 27-37.

[12] Lederer A.L. and Mendelow A.L., “Coordination of In- formation Systems Plans with Business Plans,” Journal of Management Information Systems 6(2) Fall 1989, pp. 5-19.

[13] Melone, N.P., “A Theoretical Assessment of the User-

Satisfaction Construct in Information-Systems Research,” Management Science 36(l) pp. 76-91.

[14] Miles, R.E. and Snow, CC., Organizational Strategy, Structure and Process, New York: McGraw Hill 1978.

[15] Miller, J. & Doyle, B.A., “Measuring the Effectiveness of Computer-Based Information Systems in the Financial Services Sector,” MIS Quarterly (ll:lJ, March 1987, pp. 107-124.

1161 Miller, J., “Information Systems Effectiveness - An In- ter-Industry Comparison,” in: Information Systems in Practice and Theory, North-Holland: Elsevier Science

Publishers B.V., 1988, pp. 87-107.

Page 12: Measuring and aligning information systems with the organization: A case study

228 SOS Information & Management

[17] Miller, J., “Information Systems Effectiveness: The Fit

Between Business Needs and Systems Capabilities,” in

DeGross J.I., Henderson J.C., and Konsynski B.R. (eds.),

Proceedings of the Tenth International Conference on

Information Systems, December 4-6, 1989, Boston, Mas-

sachusetts.

(181 Miller J. and Pitt, L., “Top Priorities in Information

Systems,” South African Journal of Business Management (21, 4), 1990, pp. 120-128.

[19] Price Waterhouse, Information Technology Review 1990/ 1991. London. Price Waterhouse, 1990.

[20] Senge, P., “Building Learning Organizations,” Journal for Qualify and Participation, March 1992, pp. 30-38.

1211 Strassman, P.A., The Business Value of Computers, New Canaan: The Information Economics Press, 1990.

[22] Tavakolian, H., “Linking the Information Technology

Structure With Organizational Competitive Strategy,”

Mis Quarterly September 1989, pp. 309-317. [23] Watson, R.T., “Key issues in Information Systems Man-

agement: An Australian Perspective,” Australian Com- puter Journal August 1989.

[24] Zviran, M., “Relationships Between Organizational and

Information Systems Objectives: Some Empirical Evi-

dence,” Journal of Management Information Systems 7 cl), Summer 1990, pp. 65-84.