-

Faculty of Economics, Business Administration and Econometrics

Serie Research Memoranda

Factors affecting the success of management suppo~Analysis and meta-analysis

Maarten Celderman

Research Memorandum 1995-20

May 1995

Universiteit amsterdam

2 systems:

Factors affecting the success ofmanagement support systems:

Analysis and meta-analysis*

Maarten GeldermanVrije Universiteit Amsterdam

Department of Economics, Business Administration, andEconometrics

email: mgelderman@econ.vu.nlphone: + 31 20 444 6073

fax: + 31 20 444 6005

Mw 8, 1995

Abstract

This paper gives a traditional review and meta-analysis of the literature on management supportsystems (MSS) success. Based on an extensive sur-vey of published research in the problem domainfactors, affecting MSS success are presented. Botha theoretical examination and an overview of em-pirical research of each factor are provided.

Correlations above r = 0.3 are found for usermaturity of IS department, flexibility, reahsm ofuser expectations, quahty of user documentation,formal development, user training, managementsupport, and user expectations.

Thus far, user involvement is the most widelyinvestigated variable in empirical research. In thispaper, the author makes an attempt to distinguishobjective user involvement from user involvementas experienced by the user. Effect sizes for thelatter variable appeared to be larger than finclmgsfor the fik. Indicating that a ‘feeling’ of userinvolvement is more important than user involve-ment itself.

A further analysis of the data shows effect sizes

‘Werking paper presented at the AAA Is/MAs-forum1995. 01995, Maarten Gelderman. This work wil1 be sub-mitted for publication. Copyright may be transferred with-out further notice. This paper benefited from remarks ofCees van Halem, Edu Spoor, Esther IJskes and an anony-mous reviewer.

1

for laboratory studies to be lower, and more homo-geneous than findings of field research. Further-more, correlation between the contingency factorsand user information satisfaction appeared to behigher than findings which used usage as the inde-pendent variable. The relation between the con-tingency factors and usage appears to be dimin-ishing over time. This may be caused by the factthat MSS less often fall below the leve1 at whichmanagers cease to use the system.

Keywords: Management Support Systems,Management Support Systems Success, Meta-analysis, Evaluation, User Satisfaction

1 Introduction

This article aims at describing the current state ofknowledge concerning the question ‘which factorsinfluence MSS success.’ hr this analysis MSS axedefined as computer-based information systemswhich provide ‘information support for manage-ment activities and functions’ [48, p. 9101. Thusresearch treating ‘computer-based systems thatare used to support managers in their decisionmaking in planning, coorclmation, control, orga-nizing, forecasting, budgeting, administration andgeneral management’ [79, p. 1551 bas been incor-

nL

porated in this meta-analysislPrevious analyses of research in the area of MSS

success can be found in [2, 44, 49, 101, 1051. Ofthese studies only Alavi and Joachimsthaler [2]performed a meta-analysis. Our study bas somedistinguishing features. Firstly, it covers a widerrange of variables, secondly it covers more researchthat has been carried out in the field-33 studiesin this analysis versus 6 studies in the analysisof Alavi and Joachimsthaler-, further our studycovers more recent research. Finally we includedmeasures of heterogenity in our research. Thusrebuking a traditional criticism on meta-analysis.

In Section 2 the research design of the(meta)analysis is discussed. In Section 3 the fìnd-ings of previous research are discussed and ana-lyzed. Each subsection in this section starts witha theoretical assessment of the variable under con-sideration. If enough data are available a meta-analysis of the research findings is presented. Sec-tion 3 concludes with an analysis of differencesin effect sizes between different research settings,between different dependent variables, and overtime. The next section of the paper contains adiscussion with some suggestions for further research, and fìnally a summary is given.

2 Research design

2.1 Meta-analysis

Social research bas often been criticized for lack ofresults, as compared to the natural sciences. Anexplanation for this lack of results may be thatresearchers do not build on each others work [88].Meta-analysis enables the researcher to draw con-clusions over many smaller studies, using statis-tical techniques. Thus one can find evidente foreffect sizes that normally can’t be found due totoo smal1 samples. One can look for moderatingvariables explaining differences in research find-ings, and one can look for relationships betweenreported research outcomes [37, 45, 88, 1021. Alaboratory experiment by Cooper and Rosenthal

‘This definition implies that no differente is made be-Ween MIS, DSS, EIS, etc. A similar way of looking at %f.ss’hss been discus.& by Thiriez (951. Snitkin also proposesto revise the traditional DSS notion ‘on the basis of an “inuse” concept’ [91].

2. RESEARCHDESIGN

has shown that researchers using meta-analysis aremore likely to report objective results [17].

2.1.1 The selection of literature

To gather the literature necessary to perform oursurvey the ABI/INFORM CD-ROMS for the periods1985-1990 and 1989-1994 have been queried usingthe key words presented in Table 1. The list of ti-tles and abstracts thus obtained was scanned man-ually for articles to be incorporated in this meta-analysis, looking for empirical research treatingthe systems defined in the introduction of this pa-per.

References of al1 read articles were checked forother relevant literature, and literature not yet ob-tained but available was incorporated in the re-search. Relevant articles already in the posses-sion of the author and provided by colleague re-searchers were incorporated in the study as well.

Unpublished research results were not incorpo-rated into the research. While the traditionalargument says that published research tends toshow higher effect sizes than unpublished research,an analysis by Rosenthal lead to the conclusionthat ‘There is certainly no clear differente betweenmean effect sizes obtained from journals comparedto unpublished materials’ [88, p. 401. Alavi andJoachimsthaler [2] incorporated unpublished re-search in their meta-anaIysis, but do not reporta differente in effect size between published andunpublished 6ndings. Furthermore, not incorpo-rating unpublished studies-which are supposedto be of lower quality then published studies-implicitly assigns these studies a weight of zero,thus excluding from the meta-analysis results ofstudies which quality falls below the publicability-threshold.

2.1.2 Selection of an effect size estimator

To integrate statistics it is necessary to bring themunder a common denominator-the effect size esti-mator. Cohen’s d and Glass’ A are common effectsize estimators in meta-analysis. Although thesemeasures have the advantage of familiarity in ocontext of meta-analysis, the decision was madeto use the correlation coefficient r in this study, asthis effect size estimator is more familiar to ‘tra-ditional’ researchers. Furthermore, this measure

2 . 1 Meta-analysis

‘management Support SySteIn’ OR‘executive support system’ OR

‘management information system’ OR‘decision support system’ OR

‘expert system’

A N D

‘empirical’ OR‘research’ OR‘quahty’ OR‘SUCCeSS’ OR

‘assessment’ OR‘factors’

Table 1: Key words used to query the ABI-INFORM CD-ROM for potentially relevant literature.

can be applied to both independent and correlatedvariables, and can be calculated directly from com-monly reported statistics like t, z, and p, withoutthe need to make assumptions about the size ofexperimental and control groups [88].2

2.1.3 Other issues

Calculation of aggregate statistics Al-though T wil1 be used to present final results, cal-culations have been carried out using Fisher’s zr-transformation-z, = $ In E-which distribu-tion more closely approximates normahty than Titself [88, p. 211. The unweighted average resultsare obtained from the arithmic average of the z,‘s.Weighted average results are obtained by weight-ing the z?‘s by their degrees of freedom.

A correction of z, for sample size effect is madeby wlying ti. h which p is the unbiasedT of the whole population, and n is the samplesize. As p is not available, the correction factorbas been estimated in two iterations, both timessubstituting T for p.3

Treatment of dependent effect sizes Somestudies report effect sizes for more than one depen-dent variable. ‘IIeating those dependent results asindependent would result in weighting research bythe number of estimates it reports [88]. To avoidthis problem, the average effect size of studies re-porting multiple results is incorporated in the re-search. When results of more than one study were

2For those who prefer to use d instead of r it may sufficeto know that d can be converted into r essily by applying

d= &$=q’which for equal experimental and control

groups-p = q-equak d = ti.

3Due to strong convergente this limited number of iter-ations wil1 suffice.

presented in one paper, both were incorporated inthe meta-analysis as being independent studies.This does not imply over-weighing of this studies.When a study with a degrees of freedom and astudy with b degrees of freedom are presented inthe same paper both studies wil1 be weighted bytheir own degrees of freedom. If we would takethe paper as a unit of analysis the average resultof both studies would be weighted by a + b degreesof freedom.

Heterogeneity An often heard criticism onmeta-analysis is that integrating studies which usedifferent operationalizations of dependent and in-dependent variables and different sampling unitsis the same as comparing apples and orsnges.4 Aexample from medicine may shed some light onthis issue. Take the case when a researcher ontheoretical grounds bas found a medicine for sometill then incurable disease. In order to test thismedicine one group of researchers uses tablets, an-other injections and a third ointments. Thus wehave three different independent variables. In or-der to measure the success of the treatment someresearchers count the number of patients still ahveafter three years, another groups uses the percent-age of survivors after five years, stil1 others countthe number of patients who are no longer ill anda last group ask patient how happy they feel, thuswe have diierent dependent variables as well. Wemay use their results to calculate an average effectsize. When this effect size is greater than zero weknow that the medicine has some effect in somesituations. We have to assess the homogeneity ofthe effect size as well however. When we find that

4Glsss, cited in [88] eloquently states that apples andoranges are good things to mix when we wsnt to generalizeto fruit.

4 3. ANALYSIS

the differences between the studies are only causedby sampling we know the effect size. In fact-provided that the effect size if greater than zero-we can be very confident in the new medicine as itworks equally well no matter how it is applied or

how success is measured. In this situation, how-ever it is far more probable that the effect size isheterogeneous. When this is the case we wil1 haveto look for the cause of this heterogeneity. Whenenough research bas been carried out we may beable to find that the medicine only works whenapplied in the form of tablets, in that case the het-erogeneity is caused by the independent variables,or, when heterogeneity is caused by the dependentvariable, we may find that patients live longer butdo not get better.5 When we do not have enoughdata available to draw unambiguous conclusions,the heterogeneity may draw attention to areas inwhich more research is warranted [39].

Insuflkient information Some articles do notprovide al1 statistics necessary to derive effect sizesfor the research. In one case [98] the authorsexplicitly mentioned the availability of completeresults.6 Papers which only report the results of a(stepwise) regression are not incorporated in theresearch, as effect sizes for the individual indepen-dent variables cannot be derived.

In other, typically older, papers, the authorsonly indicate the significante of a relation. Suchfindmgs are incorporated in the meta-analysis nev-ertheless. Non-significant findings get assigned apvalue of 0.5. A pvalue equal to the reportedleve1 of significante is assigned to significant find-ings.

3 Analysis

This section starts with a discussion of the suc-cess measures applied in the empirical investiga-tion of MSS success. In the following subsectionsthe independent variables, supposed to influenceMSS success are discussed. To make this sectionsurvey-able the different independent variables are

50f course more complex interact ions between depen-dent and independent variables are possible.

6The authors were contacted and their results were in-corporated in the meta-analysis. 1 want to thank Mr Udoand Mr Davis for their kind and quick reaction.

arranged according to the research framework ofIves, Hamilton, and Davis [48]. They define fiveenvironments: the user, the external, the organi-zational, the IS development, and the IS operationsenvironment. Three processes-use, developmentand operation- are deemed to be relevant for MSSsuccess. Finally they incorporate characteristics ofthe MSS itself as idhenCing MSS success.

When enough research outcomes were avail-able to perform a meta-analysis, the results arepresented next to the paragraph. Each tablepresents outcomes for al1 studies (‘total’), for stud-ies carried out in a laboratory setting-typically abusiness simulation-( ‘laboratory’) , and for stud-ies carried out in the field by administering sur-veys (‘survey’). Furthermore, the results are an-alyzed by dependent variable. Three categoriesare distinguished: usage, satisfaction, and othermeasures. As one study may report results formore than one dependent variable, the number ofstudies reported under ‘usage’, ‘satisfaction’, and‘ether’, does not necessarily sum up to the numberof studies reported under ‘total’.

.i . . :~ . . .

3 . 1 Success measures-z 2 L ‘.iThe final aim of an MSS is to enhance goal attain-ment of the organization [20,27,50,55, 831, or at-tainment of the goals of the dominant coalition inthe organization [20]. This goal bas to be reachedby increasing efficiency and/or effectiveness of de-cisions and/or the decision making process [go].Such benefits however, often are merely qualita-tive in nature. It is difficult, if not impossible tomake these items operational [27, 32, 50, 55, 831,thus alternatives like usage of information sys-tems [l, 8, 9, 13, 20, 22, 28, 34, 36, 68, 84, 991,more or less formal measures of user (informa-tion) satisfaction (UIS) [l, 8, 9, 10, 25, 26, 36,46, 50, 51, 55, 61, 69, 78, 98, 991, and in labora-tory settings the impact of MSS on decision mak-ing [13, 20, 991, and the influence of MSS usage onresults [13, 69, 99, 1031 have been proposed andapplied in research.7

‘An exception is the assessment of DSS performance per-formed by Le Blanc [59, 581. He assesses the performanceof a DSS for vessel tral%c coordination. His success measureis the decrease in traiIic accidents. An assessment like thisis only possible when the benefits of MSS are supposed to bealmost infinitely high, as is the case with trafhc accidents,

3.1 Success measures

Ginzberg claims that task performance, usageand user satisfaction form a hierarchy. Task per-formance is the ultimate aim, which can only beachieved by usage. Users will only use the systemif they accept it, and they wil1 only accept the MSSif they are-at least to a certain extent-satisfiedwith it [35]. While usage and acceptance are nec-essary conditions for improved task performance,they are not sufficient conditions. A ‘bad’ systemmay satisfy the users, may be accepted by themand may be used. However, we can safely assumesystems success and satisfaction and acceptanceof the system to be related. This allows US to useacceptance, satisfaction and usage as indicators ofthe concept system success.

Although operationalizations of MSS succes&like the ones mentioned above, may have theirshortcomings [47, 701, a commonly accepted andapplied measure of MSS success is necessary [42,971. Barki states that usage and UIS ‘remain thebest surrogates of success’ [6, p. 901. As differ-ent methods of evaluation are available, it may bepreferable to use more than one method to evalu-ate MSS [8, 921.

Another feature of the success of MSS may bethe realization of secondary goals. Hockart re-ports the example of a CE0 who by using an EISwants to communicate to his organization that hewants more emphasis on quantitative analysis inthe planning and control process [87]. Further-more an MSS might be an instrument in restruc-turing of an organization (see for instance the re-marks in Section 3.2.3). These applications ofMSS have not been investigated in the empiricalresearch analyzed in this article. They are notdeemed to be strongly related to the quality ofMSS.

User information satisfaction User informa-tion satisfaction is the most commonly used de-pendent variable in MSS research (see Table 9 andMelone [70] who also discusses the evolution of UISinstruments). Low user information satisfactionhas been said to imply ‘a mismatch between theuser% requirements and perceptions of the systemcharacteristics.’ [47, p. 2081. The application ofuser information satisfaction (UIS) as a surrogatefor MSS quality presupposes that UIS wil1 be influ-

involving severe health exposure for the people concemed.

5

enced by MSS quality. Systems that better meetusers information requirements and are more easyto use wil1 have more satisfied users [27]. However,users are most likely to evaluate the system onthe basis of attainment of their own goals, whichwil1 not necessarily be congruent with organiza-tional goals. Nonetheless, Iivari and Ervasti pro-vide some evidente that UIS and systems effective-ness are associated with each other [47].

The most well-known instrument to assess UIS

is the Ives, Olson, and Baroudi [50] instrument.However, this instrument bas be criticized forlack of some desirable psychometrie properties-discriminant validity, test/retest reliability, facevalidity-and not including ease of use [27,33,97].However, Dol1 continues ‘These concerns are notwidely shared.’ [27, p. 2621. More problem-atic is the content of the instrument-generaluser satisfaction with IS staf? and services, userinvolvement/knowledge-which not only bas notbeen validated for assessing specific applications[27], but also may introduce bias in results con-cerning user involvement. When assessing the in-fluence on user involvement this variable is bothindependent, and part of the dependent variablethus introducing an ‘Easter egg fallacy’s (a similarargument is made by [27]).

Usage Usage is only applicable as a success mea-sure when it is voluntarily [6, 8, 9, 27, 50, 761.Given the definition of MSS used in this paperthis probably does not cause any problems. How-ever, Ives et UI. suggest that users may use anMSS known to provide erroneous information as amesns of self-protection for justifying poor deci-sions [50].

Satisfaction and usage Baroudi et al. investi-gated the relation between usage and satisfaction.Usage and satisfaction may just be correlated be-cause they are influenced by a common factor.One may also influence the other. By applyingpath analysis they conclude that satisfaction in-fluences usage [9]. Thus the measurement of usagemay merely be a proxy for the measurement of UIS.Zinkhan, estimated a more elaborate model usingEQS and found usage influencing satisfaction [104].Evans-quoted by [Ill-suggests that below some

aFinding what has been hidden to find.

6 3 . ANALYSE3

minimal leve1 of satisfaction the user wil1 cease touse the system. Srinivasan provides evidente thatusage and UIS are not necessarily positively corre-lated [93].

3.2 Environment al variables

3.2.1 User variables

User characteristics may influence MSS success intwo ways. Firstly, MSS may be more likely to besucces&1 with users who have some specific char-acteristics than with others. Secondly, it may benecessary to match the design of the MSS with cer-tain user characteristics. A general problem withresearch findings in the MSS arena is that findingsof the first kind (e.g. MSS is unlikely to be suc-cessful with type z users) in reality may be of thesecond kind (e.g. type y MSS is unlikely to be SUC-

cessful with type x users) [35].

Managerial experience of user Experiencedmanagers develop more abilities to gather and pro-cess information, thus information is more acces-sible to them. Hence they are more likely to useMSS [77, 98, 1041. On the other hand experiencedmanagers might have less need for an MSS to sat-isfy their information needs, as they have acquiredtheir own ways of information gathering or simplyneed less information, implying less expected MSSusage [62]. Alavi and Joachimsthaler aggregatedthe results of four studies investigating this influ-ence. Using DSS performance as independent vari-ables they found T to be 0.166. After correctionfor sampling and measurement errors they foundan T of 0.206. Hence it may be concluded thatmanagers with more work experience benefit morefrOm MSS.

Computer experience of user Managers withmore computer experience are more familiar withthe ways to use a computer, the way comput-ers function, and the limitations of informationtechnology. Thii will make them fee1 morecomfortable-having a more positive (or less nega-tive) attitude-with the MSS, hence enhancing theprobability of success [46, 73, 1041. Further, it islikely that users with more computer experiencehave more realistic expectations which wil1 leadto a higher probability of success. Furthermore,

more experienced users wil1 probably find the sys-tem more easy to use [77]. Alavi and Joachim-sthaler found 5 studies investigating the influenceof computer experience. In their case r after cor-rection for sampling and measurement error was0.233, before correction T was 0.185. These re-sults are quite comparable to our findings for lab-oratory research as reported in Table 2, which isnot surprising considering the fact that 82% of the33 studies incorporated in their analysis reportedfindings from laboratory settings.

The results reported in Table 2 do not pro-vide convincing evidente for the hypothesis thatcomputer experience influences MSS success. Thereported correlations are low and heterogeneous.The weighted correlations between computer ex-perience and UIS and ‘ether’ measures even is neg-ative. This negative correlation is caused by thestudy of Udo and Davis [98], which happens to bethe study with the largest sample size. Further-more, Gatian reports a smal1 correlation for indi-rect users-cases in which the MSS itself is used byan assistant of the final user [32]. When we leaveout both these studies a reasonably homogeneousweighted correlation of 0.2036 (z = 2.9772 (p =0.002), x2 = 0.67 (p = 0.41)) is found. However,we should be careful in interpreting this result, asno satisfactory explanation can be given for thecontradictory results of Udo and Davis.

Attitude towards computers MSS, as de-fined above, are computer-based. Thus we expectthat the user’s general attitude towards comput-ers influences the success of MSS. Both Igbaria andNachman and Udo and Davis investigated the in-3uence of this variable. Aggregated results arepresented in Table 2. The expected influence wasfound by both of them and was significant, whilebeing homogeneous.

Education of user Higher educated user willbe more likely to understand the MSS and it’s lii-itations and are less likely to resist change. Be-sides, users who have a higher education ‘wil1 bemore able to leam and therefore are more likely toadapt to [MSS]’ [99, p. 1431. Thus education wil1have a positive influence on MSS success. Mawhm-ney, on the other hand suggests an opposite, indi-rect effect which may attenuate the obtained cor-

3.2 Entionmental variables 7

n cif.Computer experience [13, 32, 46, 68:&?]

Tul Z P x2 P

total 7 540 0.1114laboratory 1 47 0.2218survey 6 493 0.0926usage 2 96 0.1361satisfaction 4 397 0.0010other 2 162 0.0518

Attitude towards computers [46, 981survey 2 216 0.2218satisfaction 2 216 0.2003other 1 115 0.2403

Education [46, 68, 981survey 4 315 0.0063usage 2 99 -0.0940satisfaction 2 216 0.0512other 1 115 0.0922

Age I46, 68, 981survey 4 315 -0.0396usage 2 99 0.0248satisfaction 2 216 -0.1707other 1 115 0.0339

Cognitive style [l, 6, 991total 3 128 -0.0783laboratory 2 89 -0.1097survey 1 39 -0.0151usage 2 86 -0.0507satisfaction 1 39 -0.1532

Organizational leve1 [46, 68, 981survey 3 200 -0.1612usage 2 99 -0.1915satisfaction 1 101 -0.0995

Expectations [22, 36, 86, 84, 981total 6 375 0.3305laboratory 1 135 0.1886survey 5 240 0.3574usage 4 252 0.3887satisfaction 3 202 0.2283other 3 147 0.2308

Realism of expectations [8, 361survey 2 69 0.3849usage 2 69 0.3330satisfaction 2 69 0.4002

0.0993 2.458 0.0070.2218 1.551 0.0610.0874 2.022 0.0210.1749 1.536 0.062

-0.0149 0.021 0.508-0.0210 0.182 0.428

0.2226 3.309 0.001 0.04 0.850.1997 2.990 0.001 0.02 0.890.2403 2.603 0.005 -

0.0286 0.201 0.421 2.01 0.57-0.0873 0.932 0.824 0.06 0.810.0493 0.751 0.226 0.18 0.680.0922 0.998 0.159 -

-0.0578 0.859 0.195 4.04 0.260.0261 0.260 0.603 0.00 0.96

-0.1675 2.524 0.006 0.54 0.460.0339 0.370 0.644 -

-0.0855 0.950 0.829-0.1161 1.094 0.863-0.0151 0.097 0.539-0.0669 0.547 0.708-0.1532 1.005 0.842

-0.1393 2.130 0.983-0.1795 1.888 0.971-0.0995 1.020 0.846

0.3033 4.953 0.0000.1886 1.589 0.0560.3339 4.715 0.0000.4244 4.962 0.0000.1770 2.780 0.0030.1939 2.140 0.016

0.4131 3.358 0.000 1.20 0.270.3556 2.955 0.002 0.70 0.400.4336 3.479 0.000 1.74 0.19

other 1 26 0.3881 0.3881 2.122 0.017

Table 2: Unweighted (r,J and weighted (rW) correlation for the influence of user characteristics on MSS

success. The signifìcance of the fklings is reported by z. The homogeneity is reported as ~2. Thestudies integrated in the research are mentioned after the label. The number of studies is reported undern. D.f. indicates the sum of the degrees of freedom of the integrated studies.

13.97 0.03

13.15 0.021.40 0.24

31.81 0.004.03 0.04

1.46 0.481.18 0.28

-

2.61 0.11

0.52 0.770.19 0.66

15.97 0.01

14.69 0.0123.89 0.003.35 0.193.51 0.17

8 3 . ANALYSIS

relations: users with higher education tend to havehigher position in the organization and hence wil1be less likely to use computers [SS].

As can be seen in Table 2 no significant influ-ence of education on MSS success is found. Thesmal1 amount of heterogeneity in the total statis-tics is almost entirely due to the different depen-dent variables.

Age of user Age is supposed to be positivelycorrelated with cognitive differentiation [104], neg-atively with computer experience [104], positivelywith computer anxiety-which has a negative in-fluence on computer usage-[68], negatively withpersonal computer usage in general [60], and pos-itively with managerial experience. Operatingthrough these variables, age wil1 have an indirectinfluence on MSS success. The direction of this ef-fect is not immediately clear from theory, but anegative correlation is most probable. F’rom Ta-ble 2 it can be concluded that age indeed mainlybas a negative influence on user satisfaction, butbas no significant influence on usage of the MSS.

Cognitive characteristics of user Cognitivestyle of the user represents the way users perceiveand process information [80, 99, 1051. The num-ber of dimensions necessary to describe cognitivestyle is not clear, but most studies use the di-mensions simple/complex, field-dependent/field-independent, and systematic/heuristic [105]. MSS

should support important managerial activitiesamong which are problem sensing, problem for-mulation, and problem solution [80]. The way inwhich these activities can be supported dependson the cognitive style of the user.

Traditionahy, laboratory research on the influ-ence of cognitive style on MSS success bas focusedon the congruente between reports generated bythe MSS and the cognitive style of the user (e.g.[5, 13, 631). As heuristics are supposed to preferqualitative data [99], to prefer graphics above ta-bles, to use less data [99] and to need more flexiblesystems [99], Bariff and Lusk suggest to use psy-chological tests as a basis for MSS and especiallyreport design [5]. In these traditional applicationsMSS success wil1 be determined by the match be-tween MSS content and the optimal MSS content asdetermined by individual characteristics [ll, 1051.

Another item to which considerable attentionhas been devoted is whether an MSS would be use-ful at ah, given certain characteristics of the user(e.g. [13, 11, 63, 99, 1041). Persons for whom anMSS is less suited may be less inclined to use it.Thus analytics may accept systems which heuris-tics reject [66]. On the other hand MSS are sup-posed to provide heuristics with capabilities theylack themselves, thus helping them more than ana-lytics, hence heuristics may benefit more from MSSand have a more favorable attitude towards MSS

than analytics [ll, 99, 1041.Purthermore, characteristics of the user have

been said to influence MSS success indirectly,g by(partly) determining resistance to change and dis-functional behaviour-aggression towards MSS ordesigner, projection of mistakes on the system,distrust of the system and avoidance of the sys-tem [5]. Vasarhelyi suggests a relation betweencognitive style and aflinity for computers [99].

Huber made an analysis of cognitive style re-search [44]. He found-deriving his conclusionspartly from conference presentations by Taylorand Benbasat-that theories about the influenceof cognitive style on MSS are inadequate, that mea-surement instruments are inadequately developedand that research design in empirical investiga-tions is often faulty. F’urther he doubts whethercognitive style is of practical signifìcance, as cogni-tive style in published research typically explainsonly 10% of total variante. According to Huber,this percentage will be lower in practice, due toreporting bias-‘studies where little variante wasexplained tend to go unreported”O [44, p. 569]-and the application of laboratory experiments”[44]. Huber discourages cognitive style researchfor MSS design, pointing to the fact that ‘devel-opment of empirical-based bodies of knowledge ispainfully slow. [. . .] [B]y the time a scientificallysatisfactory data base could be established, “DSSgenerators” may wel1 be so flexible and friendlyand data accessing technology may wel1 be so ad-vanced that the idea of a stable DSS design may beobsolete.’ [44, pp. 570-5711. Furthermore, Huber

gZmud argues that there is no direct influence of indi-vidual characteristics in genera1 on MSS success [los].

‘OFor a contradictory statement see [88].“Kerlinger, on the contrary, claims that correlations in

laboratory experiments tend to be lower than in reality [54].See Table 8 for some tentative evidente.

3.2 Environmentd variables

doubts whether MSS design should consider cogni-tive style at all.

Alavi and Joachimsthaler included cognitivestyle research in their meta-analysis of DSS imple-mentation research [l]. After correction for sampleand measurement error, they found rsatisfxtion =0.209 and rother = 0.138. Their T’S before cor-rection were 0.167 and 0.119 respectively. Theseresults are slightly higher than the results reportedin Table 2, which are not significant. However weshould be careful in drawing conclusions as theintegration of cognitive style research is diicultdue to the heterogeneity of the applied constructs(which confirms Hubers statement that measure-ment instruments have not yet been adequatelydeveloped) .

Personality characteristics of the user Per-sonality characteristics cover an even wider rangeof dimensions than cognitive style dimensions.‘Personality variables believed to strongly im-pact MIS success include lotus of control, dogma-tism, ambiguity tolerante, extroversion/introver-sion, need for achievement, risk taking propensity,evaluative defensiveness, and anxiety level’ [105,p. 9671.

Risk averse managers can reduce risk by gath-ering more information [104]. Thus we expectrisk aversion to be positively correlated with MSSusage. l2 On the contrary, as the adoption of newtechnologies can be considered a risk, risk aversionmay be negatively correlated with the adoptionof MSS. After correction for sampling and mea-surement error, Alavi and Joachimsthaler foundrisk aversion to have an influence on satisfactionof T = 0.100 and on DSS performance of T = 0.174.For personality characteristics in general they re-port T’S of 0.124 and 0.132 respectively.

Organizational leve1 of user Users higher inthe organizational hierarchy are less liiely to usecomputers 160, 681. A common argument is thatthey will not use a keyboard as they assume thisto be below their dignity.13 In addition, the influ-ence of this factor wil1 probably reduce over time

121t is not impossible that risk averse managers ouerusetheir MSS.

130n the other hand MSS-InahfeSting itself as an Exec-utive Information System @Is)-may have become a statussymbol.

9

as computers become more ubiquitous. Further-more, tasks of higher leve1 managers also are sup-posed to be less structured and hence less suitedto be supported by computers [68], however, MSSprimarily aim at supporting management tasks.

As predicted, organizational leve1 of the user basa significant and negative influence on usage. Theinfluence on satisfaction, although in the predicteddirection, is not significant.

User expectations of the system Userattitudes-‘expectations, regarding the role of MISwithin the organization’ [105, p. 968]-wil1 influ-ence the probability of MSS success [30, 461. Themotivation to use an MSS is determined by theexpectancy that a particular degree of use will re-sult in a certain quality of decision making, theperceived probability that a decision of a certainquality wil1 lead to a certain outcome, and the at-tractiveness of this outcome [22] or, more simple,‘people will resist an application when the costsoutweigh the benefits’ [66, p. 4301. A similar ar-gument can be made for the motivation to par-ticipate in MSS development. Users with lowerexpectations wil1 tend to be less involved in MSSdevelopment, and if involved there wil1 be less un-derstanding between developers and (prospective)users [lg]. Understanding, in turn, is supposed tobe positively correlated with MSS success. Usagewil1 depend on the expectation that the systemwil1 help one to perform her task. However, whenexpectations are unrealisticly high the user maybecome disappointed in the system and cesse touse MSS [36]. However, differences in expectationsmay be caused by a lack of understanding betweendevelopers and users. In this case realism of expec-tations merely is a ‘reliable indicator of subsequentsuccess or failure’ [36, p. 4591.

A problem with a general definition of expecta-tionslattitude and the relation to system successis that the commonly used success measure in-formation satisfaction also is an attitude towardsthe system [9], thus, by definition, the two con-cepts wil1 be correlated, another occurrence of the‘Easter egg fallacy.’

The results concerning the infmence of user ex-pectations are presented in Table 2. The correla-tions are moderate and significant, the survey re-sults however are highly heterogeneous. Although

10 3. ANALYSE?

based on the results of only two studies, effect sizesconceming the reahsm of user expectations-aspresented in Table 2-show high and fairly ho-mogeneous effect sizes.

3.2.2 External variables

External organizational environmentWhen the design of MSS does not take intoconsideration the external organizational en-vironment the scope of MSS wil1 be limited tothose situations where the organization can beconsidered a closed system-operational controland to a limited extent management control. Tosupport the information needs of managementMSS should both incorporate data necessary formanagement control and strategie planning, andmatch the design of MSS with the characteristicsof the external organizational environment [31].

A distinction may be made between environ-ments that are static and dynamic [31, 381. Inmore dynamic environments more non-financialdata will be needed by management, more fre-quent reporting wil1 be necessary and more fore-cast information wil1 be used [38]. Another dimen-sion is the controllability of the environment. ‘Or-ganizations in different environments, as well asthe same organization in different environmentalconditions, wil1 require different decision processesand, consequently, different information charac-teristics’ [31, p.3081. Success of MSS is determinedby the match between the MSS and the informa-tion needs implied by the enviromnent [31, 381.MSS that do not con6rm to this mix wil1 not beoptimal. Marginal costs and benefits of the MSS

wil1 not be equal.Firrthermore, the strategie position of organiza-

tions might be an important factor. Organizationswhich have a high market share or some other su-perior strategie position are supposed to benefitmore from MSS than organizations who are strate-git inferior to them. In organizations with a poorstrategie position MSS may even reduce managerialproductivity [98].

3.2.3 Organizational variables

Organizational structure When power andresponsibilities in an organization are decentral-ized, MSS for top (headquarters) management bas

to be more sophisticated, since the top of the or-ganizations bas to be able to monitor the resultsof divisions [38]. Montazemi found decentraliza-tion of decision making created a need for moreeffective computer based information systems [72].Ein-Dor and Segev indicate that decentralized or-ganizations are more likely to have incompatiblesystems, thus decreasing the chance of MSS success[301-

Simultaneously, introduction of MSS might influ-ence organizational structure. Evidente on this in-fluence, however, is contradictory. Changes in or-ganizational structure may or may not occur [85].Ginzberg even claims that simultaneous changeof organizationd structure and MSS is a neces-sary condition for MSS success [35]. Centrahza-tion may increase, because reliable informationbecomes available to higher levels, but may alsodecrease as MSS supplies lower leve1 managementwith the capacity to handle less routine decisions.The way MSS should match organizational struc-ture is not clear either. On the one hand, MSS maybe more likely to be succes& when it strengthensthe organizational structure. On the other hand itmay be more likely to be succes&1 when it worksopposite to the organizational structure, thus cor-recting for the flaws of this structure. MSS in adecentralized organization may enhance the possi-bilities of top management to monitor the organi-zation without actively engaging in the activitiesof lower leve1 management echelons. Centralizedcontrol increases in this situation, the flexibility,commonly associated with decentralization maybe preserved. The opposite may occur in a cen-tralied organization where MSS a.llows top man-agement to delegate ‘more routine decisions whoseoutcomes are more closely controlled’ [85, p. 6811.While the organization may benefit, people wholoose power-will be more closely monitored-arelikely to resist implementation of the MSS, peoplewho gain power are more likely to accept the MSS

[66]. As with previous variables we might claimthat the MSS and the organization should fit [35].

Size of organization Lee and Kim [61] statethat the influence of organizational size on MSS

success is indirect. MSS design wil1 be more for-malized in larger organizations. This formaliia-tion will lead to a higher probability of MSS suc-

-

3.2 Environmental variables 11

Size [61, 981n d.f. z P x” P

surGe y 2 165 0.1914satisfaction 2 165 0.2218other 1 115 0.2997

Task chamcteristics [34, 55, 89, 981survey 6 723 0.0584usage 1 213 0.0037satisfaction 5 510 0.0400other 4 484 0.0267

Leadership style 146, 551surveylsatisfaction 2 127 0.2503

0.2382 2.7250.2793 3.1670.2997 3.255

0.0217 0.9070.0037 0.055

-0.0032 0.2720.0299 0.627

0.2233 2.621 0.004 0.19 0.66

0.003 2.16 0.140.001 3.39 0.070.001 - -

0.182 7.41 0.190.478 - -0.393 ll.58 0.020.265 4.88 0.18

Table 3: Unweighted (ru) and weighted (rw) correlation for the influence of organizational characteristicson MSS success. The significante of the findings is reported by z. The homogeneity is reported as XZ.The studies integrated in the research are mentioned after the label. The number of studies is reportedunder n. D.f. indicates the sum of the degrees of freedom of the integrated studies.

cess. This view is shared by Ein-Dor and Segev[301*

Size of organization appears to be positively cor-related with MSS success (see Table 3). Althoughthe effect sizes are significant, the results are some-what heterogeneous. We should be cautious indrawing conclusions.

Task characteristics Structuredness of a task‘can be defined as the extent to which the pro-cedure for dealing with management decision-making relevant to the task is routine and unam-biguous.’ [61, p. 921. The traditional view of DSS

is that they mainly support semi-structured tasks.However, less structured tasks tend to be less ana-lyzable and hence the development of MSS to sup-port these tasks is more di&& [18]. Cox, Zmud,and Clark attribute this problem to a higher prob-ability of misunderstanding between developersand users [lg]. On the other hand, users per-forming less structured tasks may have a higherneed for MSS to support these tasks [89]. Sandersidentified three diiensions of task structuredness:taak newness, task difficulty, and task variability.Task difhculty in turn consists of analyzability andpredictability [89].

Task variability-which can be considered a di-mension of task uncertainty [34]-hss to matchwith the requisite variety of MSS. Hence highlyvariable tasks wil1 need broader MSS than less vari-

able tasks. Tasks analyzability-also a componentof task uncertainty [34]-will influence the rich-ness of the information needed. Thus a close re-lationship between task uncertainty and organiza-tional information processing is likely [34]. Uncer-tain tasks wil1 require more information. The suc-cess of an MSS is determined by the match between‘information needed’, due to task uncertainty and‘information provided’ by the MSS. Thus we ex-peet to find a high correlation between usage andtask uncertainty.

The research findings, however, show that taskcharacteristics have no significant influence on MSS

success. Evidente on the influence of task charac-teristics on satisfaction is mixed. We should notbe too surprised about this finding, as the defi-nitions of task characteristics are rather vague, apoint that is confirmed by the heterogeneity of theeffect sizes.

Leadership style Leaders who have greaterconsideration for their subordinates and who aremore likely to initiate structuring of tasks, tendto have more satisfied subordinates. It is proba-ble that the same will hold for the managementof MSS [46]. The effect sizes presented in Table 3conflrm this hypotheses, are significant and ratherhomogeneous.

12 3. ANALYSIS

3.2.4 IS development and IS operations

IS department Lee and Kim [61] argue thatolder and larger IS departments are more liielyto have developed a higher degree of proceduralformalization, thus age and size are used as prox-ies for maturity. Procedural formalization is supposed to be positively correlated with MSS success,hence older IS departments are more likely to havesuccess in developing MSS.

The position of the IS department in the organi-zation might influence the probability of MSS suc-

cess. One important aspect of this effect mightbe that the organization, by plating the IS depart-ment higher in the organizational hierarchy indi-cates the importante it attaches to this functionP21.

Although the maturity of the IS department basstrong effect sizes which, apart from the influenceof usage, are significant, we should be careful todraw conclusions, as the effect sizes are rather het-erogeneous. As can be concluded from Table 4 thisheterogeneity is mainly due to the results of Joshi[51]. Without these results, effect sizes are smallerand less significant, but stil1 heterogeneous.

Accessibility of hardware and softwareGreater accessibility of hardware and software willbe positively correlated with UIS [46] and usage[68, 771. Deficient accessibiity of hardware andsoftware may cause non-use and dissatisfactionamongst users who are willing to use the system.On the same time it provides users who are notwilling to use the system-for instance because ofa high leve1 of computer anxiety-with an easyexcuse for not doing so [41].

The accessibility of hardware and software doesnot show a significant influence on MSS success.Although, when we look to the results for usageand satisfaction separately we see an interestingeffect. Accessibility has an negative effect on us-age, but not on satisfaction. We could hypothizethat unsatisfied users get more satisfied when theyhave a good excuse for not using the MSS.

User attitude towards IS department Joshimentions the influence of user satisfaction with ISstaff and services on user information satisfaction.Users who are less satisfied with the IS staff and

~ services wil1 mention this point as part of the eval-uation of their information satisfaction [51]. Fur-ther we can imagine an indirect effect through in-fluence on the understanding between user and de-veloper. Joshi indeed found the predicted effect.Some care should be observed however in investi-gating this variable, as another occurrence of theEaster egg fallacy might be at hand.

3.3 Process variables

3.3.1 Use

Resistance to use It is too easy to simplyassume that resistance to use wil1 have a nega-tive iníluence on information system success, andhence is a negative factor. However, other rele-vant factors may cause resistance. Resistance touse may also be a consequente of low quality ofthe MSS. When the MSS produces erroneous infor-mation and use would result in negative outcomesto the organization the intervenience of resistanceto use caused by the low te,chnical validity maybe positive to the organization. Gnly if usage ofthe MSS is critical to .the operation ‘of the systemnon-usage can be called resistance [66].

; ;j,“..

3 . 3 . 2 D e v e l o p m e n t

Formal development methods Lee and Kim[61, p. 901 define formalization as ‘the degree towhich work processes are standardiied’. Formal-ization enhances coorclmation and communicationbetween system developers and users, or amongdevelopers of specific systems. Formalization mayimply strict documentation standards, which wil1enhance the transfer of information over time, thusleadmg to lower operational and maintenance tost[40]. On the other hand, excessive formalizationmay lead to rigidity and disfunctionality. Thus,it is not clear whether formalization will be pos-itively or negatively correlated with MSS success.

Lee and Kim however state that ‘as MIS develop-ment remains unsystematic and unstructured incomparison with other organizational tasks, it isunlikely to be formalized excessively’ [61, p. 901.

Formal development methods tend to be leastdeveloped in smaller organizations, smaller IS de-partments, younger IS departments and for tasksthat are less structured [61]. When the degree

3.3 Process variables 13

n d.f.Maturity of IS department 115, 51, 5; 61, 781

rul Z P x2 P

survey 5 488 0.2674 0.4955 5.380 0.000 31.36 0.00usage 1 24 0.2466 0.2466 1.186 0.117 - -satisfaction 4 410 0.3225 0.5695 5.175 0.000 26.49 0.00other 3 414 0.2770 0.5086 5.538 0.000 16.56 0.00

Idem, with [51] omitted 115, 57, 61, 781survey 4 169 0.1606 0.2020 2.265 0.012 1.86 0.60usage 1 24 0.2466 0.2466 1.186 0.117 - -satisfaction 3 9 1 0.1835 0.1709 1.646 0.050 0.38 0.83other 2 95 0.0948 0.2143 1.480 0.070 2.03 0.15

Accessibility of hardware and software [46, 681survey 2 164 0.0611 0.1331 1.174 0.120 15.30 0.00usage 1 63 -0.2476 -0.2476 2.011 0.978 - -satisfaction 1 1 0 1 0.3585 0.3585 3.671 0.000 - -

Table 4: Unweighted (T,) and weighted (T,,,) correlation for the influence of IS characteristics on MSSsuccess. The significante of the findings is reported by z. The homogeneity is reported as xz. Thestudies integrated in the research are mentioned after the label. The number of studies is reported undern. D.$ indicates the sum of the degrees of freedom of the integrated studies.

of formalization is lower, benefits of additionalformalization Ge higher and the risk of over-formalization is lower. Hence, smaller organiza-tions, as wel1 as smaller and younger IS depart-ments and organizations with less structured taskswil1 benefit more from formalization of the devel-opment process [61].

Similar arguments can be made about the state-ment of objectives. On the one hand it is arguedthat clearly stated objectives are a necessity forsuccessful systems development (e.g. [71]), on theother hand it is claimed that explicitly stating ob-jectives leads to too much rigidity in a too earlystage of the development process. A certain sim-ilarity with proponents of more and less formaldevelopment methods can be observed. The samelogie may apply as well. Clearly stated objectivesmay be necessary in organizations that have fewexperience in information systems. In these orga-nizations no system may be developed at all whenthe goal is not defined beforehand. On the otherhand, when an organization has reached a certainmaturity higher quality systems may be developedby allowing more freedom in the development pro-cess.

Only two studies investigating the relation be-tween the formality of the development method

and MSS success were found. The correlations aresignificant and in the predicted direction. Theheterogeneity may be explained by the fact thatPowers [78] only reports whether his results aresignificant or not, without providing an estimateof the exact effect size.

User involvement User involvement is the ex-tent to which target users have influence onand/or knowledge of-participate in-the designand maintenance of ‘their’ MSS [76, 1051. Userinvolvement can be considered a particular man-ifestation of participative decision making [9, 49,97].14 User involvement is commonly associatedwith MSS success, this association may be calledpopular wisdom [9, 10, 56, 66, 76, 94, 97, 1041.User involvement can also be seen as an ethicalobligation [9].

Ives and Olson state that ‘user participation is

14This definition differs from the common psychologicaldefinition of involvement-‘the number of connections amanager makes between his/her own life and DSS' [104,p. 2091. Thus it has been suggested ‘to use the term “userparticipation” instead of “user involvement” when referringto the sssignments, activities, and behaviors that users ortheir representatives perform during the systems development process’ [7, p. 601. In this paper we wil1 use thetraditional term user involvement.

14 3 . ANALYSXS

d.f.Forma1 development [il, 781

rul .z P XL P

survey -2 67 0.2468 0.3438 2.378 0.009 2.37 0.12satisfaction 2 67 0.1607 0.3030 1.812 0.035 4.75 0.03other 1 17 0.0955 0.0955 0.427 0.335 - -

User involvement (1, 3 , 6 , 9, 10, 27, 28, 36, 51, 56, 64, 68, 76, 78, 86, 84,941total 19 2041 0.1878 0.2509laboratory 3 114 0.1475 0.1614survey 16 1927 0.1953 0.2560usage 12 912 0.1649 0.1246satisfaction 12 1499 0.3115 0.3354other 6 455 0.1608 0.4424

User training [6, 89, 98jsurvey 5 523 0.3358 0.3203usage 1 39 0.2531 0.2531satisfaction 5 523 0.4051 0.3804other 4 484 0.2675 0.2601

Management support [6, 10, 28, 61, 78, 86, 891survey 9 1035 0.2225 0.3080usage 2 548 0.2377 0.3170satisfaction 7 518 0.3174 0.3601other 5 394 0.1379 0.2309

Amount of change [8, 6, 981survey 3 90 0.2092 0.2838usage 2 82 0.2760 0.2760satisfaction 2 82 0.3235 0.3259other 1 8 0.0000 0.0000

Attitude towards change [8, 6]survey 2 82 0.2036 0.1898usage 2 82 0.2151 0.1979satisfaction 2 82 0.1775 0.1662

7.570 0.000 127.72 0.002.003 0.023 1.25 0.547.382 0.000 125.42 0.003.981 0.000 69.07 0 .009.685 0.000 47.61 0.004.374 0.000 24.21 0.00

7.390 0.0001.659 0.0498.895 0.0005.812 0.000

7.400 0.000 6.61 0.585.517 0.000 1.45 0.237.439 0.000 6.16 0.413.532 0.000 4.22 0.38

2.3732.6173.0470.000

1.7051.7191.546

0.0090.0040.0010.500

0.0440.0430.061

2.38 0.67- -

4.21 0.383.79 0.28

0.86 0.650.00 1.000.26 0.61

- -

7.15 0.0111.08 0.004.64 0.03

Table 5: Unweighted (ru) and weighted (rW) correlation for the influence of development characteristicson MSS success. The significante of the findings is reported by z. The homogeneity is reported as ~2.The studies integrated in the research are mentioned after the label. The number of studies is reportedunder n. D.f. indicates the sum of the degrees of freedom of the integrated studies.

advocated when acceptance is critical or when in-formation required to design the system can onlybe obtained from users’ [49, p. 5891 which accord-ing to them wil1 be the case when designing MSS.

Accordmg to Swanson user involvement only ‘co-produces’ MSS success. Implying that user involve-ment is a necessary but not sufficient conditionfor success [94]. DeBrabander and Edström notonly question whether user involvement is a suf-ficient condition for MSS success, they even claim

that ‘in other cases good results can be obtainedwithout involvement of the user’ [21, p. 1911, thusthey consider user involvement neither a sufficient,nor a necessary condition for MSS success. Part ofthe argument DeBrabander and Edström make isthat there are different roles for users and devel-opers in the communication established by userinvolvement, which wil1 infìuence the probabilityof successful communications resulting from userinvolvement [21,20]. Causahty may also be in the

3.3 Process variables

other direction. Users wil1 not be involved in sys-tems development, unless they expect the systemto be successful.

Involvement may be linked to MSS success in two

ways. Firstly, by involvement an information flowfrom users to developers is established. This maylead to better systems, which wil1 imply a higherprobability of success. Thus users are consideredto possess information that is useful for the de-velopment of good systems [3, 21, 49, 64, 69, 971.Alavi and Henderson argue that iterative or evolu-tionary development strategies maximize user in-put for the development process [ll. This, how-ever, requires the manager both to have a goodmental model of the problem situation and to beable to estimate the parameters of this model [14].Consequently, user involvement might work betterwith higher grade staff [43].

Secondly, an information flow from developersto users may be established. Involvement maythus interact with other factors aifecting systemssuccess. Involvement may lead to better informedusers [3, 9, 43, 49, 971, who are more likely to usethe MSS in the right way for the right purposes[56, 971, who are more likely to see the poten-tial benefits of the system and hence wil1 have amore positive attitude towards, and more confi-dence in the MSS [43, 69, 1051, and who finallyhave the possibility to enhance their own knowl-edge of their tasks by participating in systemsdesign [l, 641. User involvement may also influ-ence the user’s perception of the quality of theIS department/developers and hence indirectly in-fluence user information satisfaction [51]. Psychelogical barriers to the system may be reduced aswell. Involvement is likely to create users whohave a commitment to the MSS [43,49] and are lesslikely to resist the change implied by introduction[l, 3, 10, 49, 971. To benefit from this advantage,one should take care that users can remember in-volvement with the development efforts, and relatethis effort to the MSS. Thus it may be necessary toshow quick results, in which case iterative designmethods may be beneficial [64].

User i n v o l v e m e n t can b e measuredobjectively-the amount of user involvement-and subjectively-the amount of user involvementas experienced by the user. Objective mea-sures assess the information flow from user todeveloper, while subjective measures assess the

15

information flow from developers to the users.Research asking the user about the amount ofinvolvement wil1 typically result in the subjectiveresults. Objective involvement is more difficultto measure. One possibility is to look for opera-tionalizations of user involvement in the form of‘participative design methods’ and ‘evolutionary’and ‘user initiated’ development. Another moreor less objective measure of user involvement isinvolvement as reported by IS managers. Olsonand Ives report, a low and negative correlation(r = -0.02; n = 83, Cronbach’s o’s for theused scales are: o,,,, = 0.85; crIs manaser = 0.74)between both measures [76].

User involvement is the most widely investi-gated variable in this meta-analysis. On thewhole, 19 studies report usable effect sizes. Theeffect sizes are significant and in the predicted di-rection. However, only laboratory studies reachan acceptable leve1 of homogeneity. An explana-tion for this heterogeneity is provided by Ives andOlson in their (traditional) review of user involve-ment research. They state that research designsare often faulty and research is not based on strongtheory [49]. Whereas Ives and Olson conclude thatresearch bas not convincingly demonstrated theeffect of user involvement, our meta-analysis maygive food to a somewhat more optimistic view-not al1 operationalizations of user involvement aresucces&& but some are.

When we look at the influence of user involve-ment on usage and satisfaction, it turns out thatuser involvement bas more influence on satisfac-tion, than on usage (z = 2.346 (0.025)). To testfor a differente between outcomes of ‘objective’and ‘subjective’ user involvement, studies men-tioning ‘user involvement’ were compared withstudies mentioning ‘user initiated development’,‘prototyping’, and ‘iterative development’. Sub-jective user involvement appeared to have a some-what larger effect size than objectively measureduser involvement (z = 1.5791). The influence ofuser involvement on satisfaction reported by Alavisnd Joachimsthaler is comparable to our results-results before respectively after correction for sam-pling and measurement error are: T = 0.301,T = 0.369. Iníluence on DSS performance as re-ported by them was r = 0.287 and r = 0.386 re-spectively.

16 3 . ANALYSIS

studies (n)

user proto- userinitiation Wing involvement

2 3 16\ I

unweighted r 0.0538 0.0912 0.2004weighted T 0.0464 0.0936 0.2593z-value 0.3966 1.056 7.647CP> (0.346) (0.145) (0.000)homogeneity (x2) 0.02 3.89 122.79CP> (0.90) (0.14) (0.00)Based on [l, 3, 6, 9 , 1 0 , 27, 28, 36, 51, 56, 64, 68, 76, 78, 86, 84, 941

Table 6: The influence of types of user involvement on MSS success.

Quality of training Quality of training wil1 in-fluence user understanclmg of the system. Henceit wil1 influence the success of the MSS. Qualityof training, when provided by the IS department,wil1 also influence the user’s perception of the ca-pabilities of the IS department, thus indirectly in-fluencing user information satisfaction [51].

The results for the iníluence of training on MSSsuccess are reported in Table 5. Al1 results arein the expected direction, and significant. Effectsizes are homogeneous as well. Noteworthy is thefact that the influence of training on UIS is largerthan the influence on usage. However, this dif-ference is not significant (Z = 1.0211). Alavi andJoachimsthaler found training to have an influenceof r = 0.271 (before correction) and r = 0.295 (af-ter correction for sarnplmg and measurement er-ror) on DSS performance. These result are compa-rable to the findings for ‘other’ independent vari-ables as reported in Table 5.

Management support Management sup-port-not to be confused with management (user)involvement, which usually (see for instance [94])means involvement of the user/manager-can bedefined as the extent to which management abovethe leve1 of the user-who may be a managerherself-supports the development of MSS ingeneral and the MSS under consideration in par-ticular. The importante of management supportis popular wisdom [71]. Management support maylead to increased enthusiasm [lg, 611 or decreasedresistance [66] among users and may guaranteethat minimal conditions-like adequate resources,both in time and money, adequate organizational

infrastructure, and good user documentation-forsuccessful systems development and use are met[26, 611. Management support is also supposedto be positively related to MIS staff quality [lg].It may be concluded that management supportinfluences MSS success indirectly, rather thandirectly. Management support is a substitute foradequate organizational procedures.15

After user involvement, management support isthe most investigated independent variable. Re-sults,, which are presented in Table 5, show anacceptable leve1 of homogeneity, which is slightlysurprising, as the results of Dol1 suggest that topmanagement involvement is not the relevant vari-able, but merely the way in which top manage-ment is involved with MSS [24]. All results are inthe expected direction and are highly significant.

Change People and organizations tend to resistchange [53, 661. The introduction, or modifica-tion, of an MSS implies change; may be even morechange than the introduction of traditional infor-mation technology, as the MSS has to interact witha managers decision making and thus might en-force a change of his mental model of the decisionprocess [8] or may result in redesign of affectedjobs [49].

There is little reason to suppose that (prospec-tive) MSS users are less likely to resist change than

15This argument is the mirror of the point made by Leesnd Kim, who state ‘The positive influence of proceduralformalization in MIS development on MIS success tends tobe greater when top management concern is lower’, im-plying that formalization is an altemative for managementsupport [61].

3.4 Information subsystem variables

‘ordinary’ people16 or that organizations are moreeager to adopt an MSS than to accept other formsof change. Success of MSS wil1 be influenced bythe extent of perceived change implied by intro-duction/modification of the MSS and the attitudesof users towards those changes [8]. There may beinteraction between attitude towards change andperceived change, thus users opposed to changeare likely to perceive more change than users fa-voring change [8].

There wil1 not only be a direct, but also an in-direct effect. Users, being resistant to change, wil1tend to cooperate less with the MSS developers thegreater the expected change implied by systemsdevelopment is. Hence a greater degree of (ex-

. pected) change wil1 lead to lower developer/userunderstanding which wil1 imply a lower quality ofthe MS S [lg].

Understanding between user and devel-oper When user and developer get along bet-ter, we expect a higher probability of systems suc-cess. Kaiser and Srinivasan suggest that attitudesmerely should be compatible, not necessary con-current, to achieve understandmg [52]. Better un-derstanding wil1 lead to higher user involvement,higher quality of the final system [lg, 521, and ahigher correspondence between user expectationsand the final system [36].

3.4 Information subsystem vari-ables

Flexibility Flexibility is traditionally consid-ered to be an important concept in IS in general.The results reported in Table 7 reflect this find-ing. Correlations are amongst the highest foundin this meta-analysis. However, only 2 studies areavailable and results are highly heterogeneous.

Format of output The format of the outputof MSS mainly has been discussed in combina-

r6Keen explains ‘social inertia’ partly by irrationality orbounded rationality of persons who resist change [53]. Itmay be possible that MSS users are more rational than ‘or-dinary man’ . In this case resistsnce to change amongstthem may be lower. Barki even states that ‘DSS users are[ . . .] quite different from the traditional users of MIS, itthen might make sense to expect DSS that bring a greatdeal of change to be more successful than DSS that bringless change’ [8, p. 2631.

17

tion with cognitive characteristics of the users.However, observations of the format of outputsec have also been made. Researchers have in-vestigated whether graphic or tabular output, ora combination of both leads to better results.This research, however, nowadays may seem out-fashioned, as the influence of revolutionary devel-opments like the implementation of CRT terminalsto display graphics is investigated. Another short-coming of these studies is that they sometimescompare adequate tabular presentations of a prob-lem with graphics that are-compared with mod-ern standards-inadequate, and vice versa (e.g.[13]). Another problem is that some situationsmay be more suited for the application of graphics,while others might be more suited for the use oftabular data [12]. A match between data represen-tation and problem enhances the quality of MSS.However, an empirical evahration of this optimalmatch is very diicult. We should thus be verycareful to draw conclusions to easily from theseresearch findings, and remember that there is arich body of literature investigating the applica-tion of the right graph or table for the right data(e.g. [16, 29, 65, 1001).

Expectations of ‘format-of-output-research’ arethat the use of graphics wil1 lead to faster decisionmaking, to a better understanding of the problemsituation [63], and to better decisions [63], the useof graphics and tables together wil1 lead to stil1better decisions [63]. Benbasat proposes to con-duet a number of laboratory experiments within agreat variety of task settings [12].

Interaction effects include that heuristics pre-fer graphics-and their decision making wil1 ben-efit from them-, while analytics wil1 prefer ta-bles [13, 12, 631. An important set of researchfindings in this area are the so-called Minnesotaexperiments. These laboratory experiments inves-tigated the influence of different system character-istics like complexity, the application of CRT’S andpresentation of data. A summary of the results ofthese experiments bas been presented in [23].

Quality of user interface Raskin discussed in-tuitive user interfaces, concluding that what isusually called ‘intuitive’ merely means familiar, re-maining close to the cognitive world of the user-the status quo-, thus leaving little room for im-

18 3. ANALYSIS

n dd. r, rw 2 P x2 PFlexibility [6, 981

survey 2 154 0.5088 0.4386 5.669 0.000 3.92 0.05usage 1 39 0.5673 0.5673 3.707 0.000 - -satisfaction 2 154 0.5619 0.4870 6.274 0.000 5.12 0.02other 1 115 0.3546 0.3546 3.852 0.000 - -

Quality of user documentation [25, 26, 961survey/satisfaction 3 542 0.3775 0.3881 8.401 0.000 4.12 0.13

Table 7: Unweighted (rU) and weighted (rW) correlation for the influence of information subsystemcharacteristics on MSS success. The significante of the findings is reported by z. The homogeneity isreported as ~2. The studies integrated in the research are mentioned after the label. The number ofstudies is reported under n. D.f. indicates the sum of the degrees of freedom of the integrated studies.

provement of the interface, ‘even when it can beshown that a feature that is completely familiar(intuitive) is deficient’. ‘That quality of a newinterface paradigm that is commonly titled “intu-itive” may wel1 turn out to be one of the worstqualities it can have’ [81, p. 181. Bénard and Satirfound spreadsheet-like functionality to have a pos-itive influence on user satisfaction [lol. The tra-ditional EIS user interface might be too simple.Managers, who are supposed to be adult and in-telligent, are sometimes supplied with with userinterfaces that seem to be suited for ten year oldchildren.

Quality of user documentation Good userdocumentation not only trams the user, but alsobas an educational function. It explains the userwhy the MSS is suited for which purposes. It helpsthe user understand how the MSS works and howto use it [25,26,33]. The quality of user documen-tation is defined as the degree to which it commu-nicates this information to the user efficiently andeffectively [33].

Thus a certain miniial quality of user docu-mentation may be necessary to make sure the sys-tem is used at all. Better user documentationallows users to use the system more efficient-thus possibly decreasing usage-and effective [40].Good documentation may allow to spend less ontraining and user assistance [40]. It enhancescommunication from systems developers to users[26, 401. It is expected that better user documen-tation wil1 increase MSS success through increasinguser information satisfaction [33]. Dol1 even notes

that it may wel1 be possible that the success ofspreadsheet packages which allow users to buildtheir own MSS may be explained by the availabil-ity of adequate manuals [25]. Lee found PC usersnot to be satisfied with their user documentation[60]. Guillemette’s findings partially support thisconclusion: users rated documentation of pack-aged software as more appealing than documenta-tion of internally developed software. The latter,however, they found more convincing [40].

User documentation potentially is an importantfactor in establishing MSS success. Results are inthe expected direction, and are highly significant.However, effect sizes are not as homogeneous asone would hope. More problematic is the fact thatthe studies are not independent. Al1 papers are(co)authored by Doll. This fact might introducesome bias.

3.5 F’urther analyses

The data obtained from the meta-analysis havebeen analyzed to investigate whether other avail-able characteristics of research have some influenceon obtained results. This was done by analyzingthe absolute values of the effect sizes.

Laboratory research has been compared withsurvey research-see Table 8. It is not only note-worthy that laboratory research yields smaller av-erage results (z = 1.858), but also that these re-sults are quite homogeneous: al1 laboratory re-search tends to find an effect size from the samepopulation. Of course we should be careful todraw conclusions as the number of laboratory

19

Iaboratory surveystudies (n) 6 33d.f. 276 4222unweighted T 0.1845 0.2579weighted r 0.1888 0.2891s-value 3.173 14.591CP> (0.001) (0.000)homogeneity (x2) 0.29 109.55(PI (1.00) (0.00)

Table 8: The results of survey research and labo-ratory research compared.

studies in this meta-analysis is small.Furthermore, the effect sizes for usage, satis-

faction and other dependent variables were com-pared. The results are summarized in Table 9. Al1effect size are fairly heterogeneous, which in thiscase is a desirable property. Outcomes for satis-faction show larger effect sizes than outcomes withusage as the dependent variable (Z = 2.5119). Theother independent variables are somewhere in be-tween. The weighted results exhibit effect sizes ofabout the same size as the effect sizes for satis-faction. The unweighted results are much smallerhowever. Overall these results suggest that fieldstudies using UIS as an dependent variable aremost promising to pursue.

The influence of time on effect sizes bas beeninvestigated as well. Time influences neither theoverall effect size (Z = 0.5719), nor the effectsize with satisfaction as the dependent variable(z = 0.6923). There is a negative correlation

studies (n) 17d.f. 1817unweighted r 0.1594weighted r 0.1647z-value 5.4070) (0.000)homogeneity (x2) 98.28(PI (0.00)

usage satis-faction

273042

0.26020.303612.807

(0.000)93.48

(0.00)

other

131135

0.16830.2821

6.032(0.000)

59.76(0.00)

Table 9: The results of different dependent vari-ables compared.

(r = -0.1911) between effect sizes with usage asthe dependent variable and time (z = 4.6240).A possible explanation might be that over timefewer systems fall below the threshold where userscease to use the system. A positive developmentof the effect size reported for the other dependentvariables can be seen (r = 0.3851; z = 3.6918).Causes of this tendency may be twofold. First,more survey and Iess laboratory research is car-ried out. Second, the research carried out is basedon a better theoretical basis, hence increasing thechange that research wil1 be carried out in a fruit-ful direction.

4 Discussion and suggestionsfor further research

In this paper an analysis bas been made of previ-ous MSS research. This analysis allows US to gainnew insights in our problem domain. However,not al1 research could be incorporated in our anal-ysis. One reason is that researchers do not alwaysgive adequate information. In order to facilitatefurther analyses of their data researchers shouldtake care to publish covariance matrices and sam-ple sizes along with their research findings. Edi-tors can entourage such practice.

Quite a number of variables investigated in thisanalysis show heterogeneous effect sizes. A pos-sible cause may be that the interrelationships be-tween variables vary during the systems develop-ment and introduction process. A solution for con-flicting outcomes might be to focus more researchon process, instead of factors [35, 741. Newmanproposes an approach related to the evaluation ofservices (SERVQUAL) in marketing research [74].Heterogeneity may also be due to the fact thatmeasurement instruments in information systemsresearch are of relatively low quality. Hence it isnot always clear whether the instruments measurethe constructs they are supposed to assess [70].The development of high-quality measurement in-struments is a critical research priority [27, exec-utive summary]. In some cases however the con-struct to be investigated should be more clearlydefined. Results in the case of user involvement-see Table 6-show that this kind of research maybe worthwhile. Another solution for the ‘prob-

20 5. SUMMARY

lem’ of heterogeneity might be found in carryingout research in which not only the direct influ-ence of contingency variables on MSS success isinvestigated, but also the match between differ-ent contingency variables and MSS design as sug-gested by [35, 671. Results for the realism of userexpectation-which is an instance of matching-suggest that this approach may yield useful re-sults.

varàablematurity of IS department 0.4957 *

Not al1 variables have been investigated empiri-cally. The external environmental variables influ-encing MSS design have only be subject of theo-retical research. Only their derivative-taak char-acteristics which are influenced by environmentalcharacteristics-bas been the subject of empiri-cal research. However, only a limited amount ofwork bas been done in this area and the definitionsfor task characteristics applied in this researchshow some heterogeneity. The area of personalitycharacteristics also shows a lack of research find-ings. Particular the external environmental vari-ables may be suited for further research.

flexibility 0.4386 *realism of expectations 0.4131quality of user documentation 0.3881formal development 0.3438user training 0.3203management support 0.3080expectations 0.3033 *amount of change 0.2838user involvement 0.2509 *size of organization 0.2382leadership style 0.2233attitude towards computers 0.2226attitude towards change 0.1898 *organizational leve1 -0.1393accessibility of HW/SW 0.1331 *computer experience 0.0993 *cognitive style -0.0855as -0.0578education 0.0286taak characteristics 0.0217

Another point that deserves more study are fac-tors for which high effect sizes were found, butwhere more evidente is needed. Especially ‘qual-ity of user documentation’-which hss only beeninvestigated by Doll-and ‘flexibility’-which sur-prisingly bas only been investigated twice-arecandidates worth further research.

Table 10: A global summary of the research íind-ings. Reported is the weighted correlation be-tween the independent variable and the successmeasure. The ssme results have been labeled ‘to-tal’ in prior tables. An * in the final column in-dicates that the null-hypothesis of homogeneousresults was rejected at cy = 0.05.

A general problem in the research findings isthat only a limited variety in success wil1 be found.Systems of which the quality falls below a certainthreshold wil1 soon be ‘forgotten’ and wil1 not beincorporated in survey research [8]. Thus certainvariables may not be traceable by traditional re-search methods at ah.

5 S u m m a r y

Finally, more research investigating both directand indirect influences of the contingency vari-ables and interdependenties amongst them maylead to interesting results. Particular in caseswhere opposite direct and indirect iniluences areexpected this approach is necessary. Structuralequation modeling-using LISREL or EQS-is thesuited methodology to investigate the direction ofcausality in the relations between the different fac-tors, snd to distinguish the direct effects of a fac-tor on success from the indirect effects of and viaother variables. A good example is the study byZinkhan [104].

In this article the state-of-the-art in MSS researchbas b e e n assessed. Theoretical considerationsabout the predicted influence of numerous contin-gency variables from previous research have beensummarized. When possible, an objective inves-tigation of the validity of these theories bas beenmade by applying meta-analytical techniques.

Attitude towards computers, user expectations,realism of user expectations, size of organization,leadership style, maturity of IS department, userinvolvement, training, flexibility of the MSS, qual-ity of user documentation, management supportfor MSS, and application of formal developmentmethods showed positive and significant results.The amount of change implied by the introductionof MSS, and organizational leve1 of the user had a

-

RaEFER.ENCES

significant and negative influence on MSS success.The influence of education, computer experience,age, task characteristics, accessibility of hard- andsoftware and cognitive style showed mixed and/orinsignificant results.

Correlations above r = O.S-called ‘moderate’by Nunnally [75]-were found for user expecta-tions, realism of user expectations, maturity ofIS department, the application of formal develop-ment methods, user involvement, training, man-agement support, the amount of change, and thequality of user documentation. For practitionersit may be encouraging to note that a number ofthese influential variables is controllable.

User involvement was the most widely investi-gated variable. An attempt was made to distin-guish objective user involvement, and user involve-ment as experienced by the user. Findings for thelatter vsriable appeared to be larger than findingsfor the first.

A further analysis of the data showed findingsfor laboratory studies to be lower, and more homo-geneous than findings of field research. Further-more, correlation between the contingency factorsand user information satisfaction is higher thanfindings which used usage as the independent vari-able. The relation between the contingency fac-tors and usage diminishes over time. This maybe caused by the fact that MSS less often fall be-low the leve1 at which managers cease to use thesystem.

Both heterogeneity of findings and a lack of find-ings for some variables cal1 for further research. Inthe latter case more research is needed. In the firstcase more research is needed.

References

[l] Maryam Alavi and John C. Henderson. Anevolutonary approach for implementing a de-cision support system. Management Science,27(11):1309-1323, November 1981.

[2] Maryam Alavi and Erich A. Joachimsthaler. Re-visiting DSS irnplementation research: A meta-analysis of the literature and suggestions for re-seamhers. MIS Quarterly, pages 95-116, March1992.

[3] Kwasi Amoako-Gyampah and Kathy B. White.User involvement and user satisfaction: on ex-

21

ploratory contingency model. Information EiManagement, 25:1-10, 1993.

[4] James E. BaiIey and Sammy W. Pearson. De-velopment of a tool for analyzing and measur-ing computer user satisfaction. M a n a g e m e n tScience, 29(5), May 1983.

[5] M. L. Bariff snd E. J. Lusk. Cognitive andpersonahty tests for the design of managementinformation systems. Management Science,23(8):826-829, April 1977.

[S] H. Barki and S.L. HufI. Implementing decisionsupport systems: correlates of user satisfactionand system usage. INFOR (Canada), 28(2):89-101, May 1990.

[7] Henri Barki and Jon Hartwick. Measuring userparticipation, user involvement, and user atti-tude. MIS Quarterly, pages 59-79, Match 1994.

[S] Henro Barki and Sid L. Huff. Change, attitudeto change, and decision support system success.Information & Management, 9(5):261-268, De-cember 1985.

[9] Jack J. Baroudi, Margrethe H. Olson, aud BlakeIves. An empirical study of the impact ofuser involvement on system usage and informa-tion satisfsction. Communications of the ACM,29(3):232-238, Msrch 1986.

[lO] Roch Benard and Ahmet Satir. User satisfactionwith EISs: Meeting the needs of executive users.Information Systems Management, pages 21-29,Fall 1993.

[ll] Izak Benbasat and Albert S. Dexter. Individ-ual differences in the use of decision supportaids. Jounaal of Accounting Research, 20(1):1-11, Spring 1982.

[12] Izak Benbasat and Albert S. Dexter. An empir-icaI evaluation of graphical and color-enhancedinformation presentation. Management Science,31:1348-1364, November 1985.

[13] Izak Benbasat, Albert S. Dexter, and Paul S.Masulis. An experimental study of the hu-man/computer interface. Communications ofthe ACM, 24(11):752-762, 1981.

[14] Dipankar Chakravarti, Andrew Mitchell, andRichard Staelin. Judgment based marketing de-cision modeIs: an experiment& investigation ofthe decision calculus approach. ManagementScience, 25(3):251-263, Maxh 1979.

[15] Paul H. Cheney and Gary W. Dickson. Orga-nizational characteristics and information sys-tems: An exploratory investigation. Academy ofManagement Joumal, 25(1):170-184, 1982.

-

22 REFEREIVCES

[16] Wihiam S. Cleveland and Robert McGill.Graphical perception: the visual decoding ofquantitative information on graphical displaysof data. of the Royal Statistical SocietyA, 150(3):192-229, 1987.

[17] Harris M. Cooper and Robert Rosenthal. Sta-tistical versus traditiomd procedures for summa-rizing research findings. Psychological Bulletin,87(3):442-449, 1980.

[18] Randolph B. Cooper snd Robert W. Zmud. In-formation technology implementation research:a technological difIùsion approach. ManagementScience, 36(2):123-139, February 1990.

[lg] James F. Cox, Rober W. Zmud, and Steven J.Clark. Auditing an MRP system. Academy ofManagement Joumal, 24(2):386-402, 1981.

[20] B. DeBrabander and G. Thiers. Successful in-formation system development in relation to sit-uational factors which affect effective communi-cations between MIS-users snd EDP-speciahits.Management Science, 30(2):137-155, February1984.

[21] Bert DeBrabander and Anders Edström.Successful information system developmentprojects. Management Science, 24(2):191-199,October 1977.

[22] Gerardine DeSanctis. Expectancy theoryas an explanation of voluntary use of adecision-support system. Psychological Reports,52(1):247-260, February 1983.

[23] Gary W. Dickson, James A. Senn, and Nor-man L. Chervany. Research in management in-formation systems: the Minnesota experiments.Management Science, 23(9):913-923, May 1977.

[24] WiIIiam J . DOR Avenues for top management involvement in successful MIS develop-ment. MIS Quarterly, pages 17-31, March 1985.

[25] WiIIiam J. Dol1 and Mesbah U. Ahmed. DOC-umenting information systems for management:a key to maintaining user satisfaction. Infomaa-tion B Management, 8(4):221-226, 1985.

[26] William J. Dol1 and Gholamreza Torkzadeh.The quahty of user documentation. Informationd Management, 12(6):73-78, 1987.

[27] WiIIiam J. Dol1 and Gholamreza Torkzadeh.The measurement of end-user computing satis-faction. MIS Quarterly, 12:259-274, June 1988.

(281 WiIliam H. Dutton and Kenneth L. Kraemer.Management utilization of computers in Amer-ican 1ocaI govemments. Communications of theACM, 21(3):206-218, Msrch 1978.

[29] A.S.C. Eherenberg. Rudiments of numeracy.Joumal of the Royal Statistical Society A,140(3):277-297, 1977.

[30] Philip Ein-Dor and Eli Segev. Organiza-tional context and the success of managementinformation systems. Management Science,24(10):1064-1077, June 1978.

[31] Kweku Ewusi-Mensah. The external orgsniza-tional environment and its impact on manage-ment information systems. Accounting, Organi-zations d Society, 6(4):301-316, 1981.

[32] Amy W. Gatian. Is user satisfaction a vahdmeasure of system effectiveness. Infornzation dManagement, 26:119-131, 1994.

[33] Leopoldo A. Gemoets and Mo Adam Mahmood.Effect of the quahty of user documentation onuser satisfaction with information systems. In-formation d Management, 18:47-54, 1990.

[34] Jawaid A. Ghani. Task uncertainty and the useof computer technology. Information d Man-agement, 22(2):69-76, 1992.

[35] Michael J. Ginzberg. An organizational con-tingencies view of accounting and informationsystems implementation. Accounting, Organiza-tions d Society, 5(4):369-382, 1980.

[36] Michael J. Ginzberg. Early diagnosis of MISimplementation failure: promising results andunanswered questions. Management Science,27(4):459-478, April 1981.

[37] Gene V. Glsss, Barry McGaw, and Mary LeeSmith. Meta-analysis in social research. Sage,1981.

[38] Lawrence A. Gordon and Danny Mier. A con-tingency framework for the design of accountinginformation systems. Accounting, Organizationst3 Society, 1(1):59-69, 1976.

(391 Penelope Sue Greenberg. The use of meta-analysis techniques in developing dissertationpropos& guidelines to aid supervising facuhy.Issues in Accounting Education, 7(2):145-163,FaIl 1992.

[40] Ronald A. Guillemette. What readers mean by‘good documentation’. Information d Manage-ment, 18:69-77, 1990.

[41] Barbara A. Gutek, Susan A. Winter, andKatherine M. Chudoba. Attituted towards com-puters: when do they predict computer use?In Pwceedings of the Academy of Management,19??

REFEREXCES

[42] Starr Roxane Hiltz and Kenneth Johnson.User satisfaction with computer-mediated com-munication systems. Management Science,36(6):739-764, June 1990.

[43] R.A. Hirschheim. User experience with and as-sessment of participative systems design. MISQuarterly, 9(4):295304, December 1985.

[44] George P. Huber. Cognitive style as a basis forMIS and DSS designs:, much ado about nothing?Management Science, 29(5):567-579, May 1983.

[45] John Hunter, Frank L. Schmidt, aud Gregg B.Jackson. Meta-analysis: cumulating researchfindings across studies. Studying organizations:innovations in methodology, 4. Sage, 1982.

[46] Magid Igbaria end Sidney A. Nachman. Cor-relates of user satisfaction with end user com-puting: an exploratory study. Information E7Management, 19:73-82, 1990.

[47] Juhani Iivari and Irja Ervasti. User informationsatisfaction: IS implementabiiity and effective-ness. Information Ei Management, 27:205-220,1994.

[48] Blake Ives, Scott Hamilton, and Gordon B.Davis. A framework for research in computer-based management information systems. Man-agement Science, 26(9):910-934, September1980.

[49] Blake Ives and Margrethe H. Olson. User in-volvement and MIS success: a review of re-search. Management Science, 30(5):586-603,May 1984.

[50] Blake Ives, Margrethe H. Olson, and Jack J.Baroudi. The messurement of user informa-tion satisfaction. Communications of the ACM,26(10):785-793, October 1983.

[51] Kailash Joshi. A causal path model of overalluser attitudes towards the MIS function: thecase of user information satisfaction. Infowna-tion Ei Management, 22(2):77-88, 1992.

[52] Kate Ksiser a n d A n a t h Srinivasan. User-analyst differences: an empirical investigationof attitudes related to systems development.Academy of Management Joumal, 25(3):630-646, September 1982.

[53] Peter G. W. Keen. Information systems andorgauizational change. Communications of theACM, 24(1):24-33, January 1981.

[54] F.N. Kerlinger. Foundations of behavioral Te-search. Holt, Rinehart, and Winston, 2nd edi-tion, 1973.

[55] Kyung Kyu Kim. Task chzracteristics, decen-trahzation, and the success of hospital infor-mation systems. Information d Management,19:83-93, 1990.

[56] William R. King and Jamue 1. Rodriquez. Par-ticipative design of strategie decision supportsystems. Management Science, 27(6):717-726,June 1981.

[57] WiIIiam R. King and Rajiv Sabherwal. The fac-tors affecting strategie information systems ap-plications: an empirical assessment. Informa-tion & Management, 23:217-235, 1992.

[58] Louis A. LeBlanc. An assessment of DSS per-formance: The impact of utiliation and closure.Infownation B Management, 20:137-148, 1991.

[59] Louis A. LeBlanc and Kenneth A. Kozar. Anempirical investigation of the relationship be-tween DSS usage and system performance1 acase study of a navigating support system. MISQuarterly, 1990.

[60] Dennis M.S. Lee. Usage patterns and sourcesof assistance for personal computer users. MISQuarterly, 10(4):313-325, December 1986.

[61] Jinjoo Lee and Sang-Hoon Kim. The relation-ship between procedural formahzation in MISdevelopment and MIS success: A contingentanalysis. Information d Management, 22:89-111, 1992.

[62] Henry C. Lucas Jr. The use of an accountinginformation system, action and organizationalperformance. The Accounting Review, pages735-746, October 1975.

[63] Henry C. Lucas Jr. An experimentai investiga-tion of the use of computer-based graphics in de-cision makmg. Management Science, 27(7):757-768, July 1981.

[64] Mo A. Mahmood and Jeanette N. Medewitz.Impact of design methods on decision supportsystems success: an empirical assessment. In-fownation ei Management, 9:137-151, October1985.

[65] B.H. Mahon. Statistics and decisions: the im-portance of communication and the power ofgraphical presentation. Joumal of the Royal Sta-tistical Society A, 140(3):298-323, 1977.

[66] M. Lynne Markus. Power, politics, and MISimplementation. Communications of the ACM,26(6):430-444, June 1983.

[67] M. Lynne Markus and Daniel Robey. The orga-nizational validity of management informationsystems. Human Relations, 36(3):203-226, 1983.

24 R.EFER.ENCES

[68] Charles H. Mawhinney and Albert L. Lederer. Astudy of personal computer utilization by man-

d Management, 18:243-253,1990.

[69] Shelby H. McIntyre. An experimental study ofthe impact of judgment-based marketing mod-els. Management Science, 28(1):17-33, January1982.

[70] Nancy Paule Melone. A theoretical assess-ment of the user-satisfaction contruct in infor-mation systems research. Management Science,36(1):76-91, January 1990.

[71] Mare H. Meyer and Kathleen F. Curley. Expertsystem success models. Datamation, 35:35-38,September 1 1989.

[72] Ali Reza Montazemi. Factors affecting informa-tion satisfaction in the context of the smal1 busi-ness environment. MIS Quarterly, 12:239-256,June 1988.

[73] Peter P. Mykytyn, Jr. and Gsry 1. Green. Effectsof computer experience and task complexity onattitudes of managers. Infownation Ei Manage-ment, 23:263-278, 1992.

[74] Michael Newman and Daniel Robey. A sociaiprocess model of user-analyst relationships. MISQuarteriy, pages 249-266, June 1992.

[75] J u m C . Nunnally.McGraw-Hill, 1967.

Psychometrie thwry.

[76] Margrethe H. OIson and Blake Ives. User in-volvement in systems design: an empiricai testof alternative approach. Information Ei Manage-ment, 4(4):183-195, 1981.

1771 William J. Paisley. Information needs and uses.Annual review of information science and tech-nology, 3:1-30, 1968.

[78] Richard F. Powers and Gary W. Dickson. Mis-project management: Myths, opinions, and real-ity. Califomia Management Review, XV(3):147-156, Spring 1973.

[79] K.S. Ramau and C.K. Phoon. Decision supportsystems in Singapore: Issues in their manage-ment and their bene&ial contributions. Infor-mation d Management, 18:153-165, 1990.

[80] Arkalgud Ramaprasad. Cognitive process as abasis for MIS and DSS design. ManagementScience, 33(2):139-148, February 1987.

[81] Jef Raskin. Intuitive equals familiar. Commu-nications of the ACM, 37(9):17-18, September1994.

[82] Louis Raymond. Organizational characteristicsand MIS success in the context of smah business.MIS Quarterly, pages 37-52, March 1985.

1831 Edward Rivard and Kate Kaiser. The benel% ofquahty IS. Datamation, 35:53-58, January 151989.

1841 Daniel Robey. User attitudes and managementinformation system use. Academy of Manage-ment Joumal, 22(3):527-538, 1979.

[85] Daniel Robey. Computer information systemsand organizationai structure. Communicationsof the ACM, 24(10):679-686, October 1981.

[86] Daniel Robey snd Richard L. Zeller. Factors in-fluencing the success and failure of an informa-tion system for product quality. INTERFACES,8(2):70-75, February 1978.

[87] John F. Rockart and Michael E. ‘Ireacy. TheCE0 goes on-hne. Harvard Business Review,January-February 1982.

[88] Robert Rosenthal. Meta-analytic procedures forsocial research, volume 6 of Applied Social Re-search Methods Series. Sage, 2nd revised edition,1991.

[89] G. Lawrence Sanders and James F. Courtney. Afield study of organizational factors iníluencingDSS success. MIS Quarterly, 9(1):77-93, March1985.

[90] Ramesh Shsrda, Steve H. Barr, and James C.McDonnell. Decision support systems effective-ness: a review and empirical test. ManagementScience, 34(2):139-159, Februery 1988.

[91] Sidney R. Snitkin and Wilham R. King. Deter-minants of the effectiveness of personal decisionsupport systems. Infomaation Ei Management,10(2):83-90, 1986.

[92] Ralph H. Sprague and Eric D. Carlson. Buildingeffectiue decision support systems. Prentice Hall,1982.

[93] Ananth Srinivasan. Alternative measures of sys-tem effectiveness: associations and implications.MIS Quarterly, 9(3):243-253, 1985.

[94] E. Burton Swanson. Management informationsystems: appreciation and involvement. Man-agement Science, 21(2):178-188, October 1974.

(951 Hervé Thiriez. Towards a DEISS. EuropeanJoumal of Opemtions Research, 61:72-85, 1992.

[96] Gholamreza Torkzadeh and William J. Doll.The place and value of documentation in end-user computing. Information d Management,24:147-158, 1993.

REF’EREiVCES

[97] Gholamreza Torkzadeh and William J. Doll.The test-retest rehability of user involvementmeasures. d Management, 26:21-31, 1994.

[98] Godwin J. Udo and J. Steve Davis. Factors af-fecting decision support system benefits. Infor-mation d Management, 23:359-371, 1992.

(991 Michael Antal Vasarheiyi. Man-machine plan-ning systems: A cognitive style examination ofinteractive decision making. Joumal of Account-ing Research, 15(1):138-153, Spring 1977.

[lOO] Howard Wainer. How to display data badiy.The American Statistician, 38(2):137-147, May1984.

[lol] Peter Wei11 and Margrethe H. Olson. As assess-ment of the contingency theory of managementinformation systems. Joumal of MIS, 6(1):59-85, Summer 1989.

[102] Fredric M . Wolf. Meta-anaìysis: quantitativemethods for research synthesis, volume 59 ofQuantitative Applications in the Social Sciences.Sage, 1986.

[103] Bayard E. Wynne and Gary W. Dickson. Ex-perienced managers’ performance in experimen-tal man-machine decision system simulation.Academy of Management Joumal, 18(1):24-40,March 1975.

[104] George M. Zinkhan, Erich A. Joachimsthaier,and Thomas C. Kinnear. Individual differencesand marketing decision support system usageand satisfaction. Joumal of Marketing Research,XXIV:208-214, May 1987.

[105] Robert W. Zmud. Individuai differences andMIS success: a review of the empiricai litera-ture. Management Science, 25(10):966-979, Oc-tober 1979.

25