Effects of User Participation in Systems Development: A ... · User Participation Effects of User Participation in Systems Development: A Longitudinal Field ExperimentI By: James

User Participation

Effects of UserParticipation inSystemsDevelopment:A Longitudinal FieldExperimentI

By: James E. HuntonCollege of Business AdministrationSchool of AccountancyUniversity of South Florida4202 East Fowler Avenue, BSN 3043Tampa, FL 33620-5500U.S.A.jhunton @bsn01.bsn.usf.edu

Jesse D. BeelerAccounting DepartmentMillsaps CollegeJackson, MS 39210-0001U.S.A.beelejd @okra.millsaps.edu

research. Participation is manipulated at threeincreasing levels: (1) no voice, (2) non-instru-mental voice, and (3) instrumental voice.

Research findings suggest that users’ pre-experiment level of involvement with and atti-tude toward the present system are positivelyassociated with their desire to participate in thedevelopment of the new system. Study resultsalso indicate that users’ a priori self-efficacybeliefs regarding their perceived abifity toeffectively contribute to the developmentprocess are positively related to desired partic-ipation. Pre- to post-experiment gains in psy-chological and behavioral variables are nextassessed. In the instrumental voice condition,user involvement, user attitude, and perfor-mance gains are significantly highest. Userattitude and involvement gains are significantlyhigher in the non-instrumental voice conditionthan in the no voice condition; however, gainsin user performance are not significantly differ-ent between these treatment conditions.Research findings indicate that user participa-tion can be effective, particularly when usersperceive a noticeable degree of instrumentalcontrol over the decision outcome.

Keywords: Participation, involvement, account-ing, procedural justice, self-efficacy

ISRL Categories: FA05, FA06, FB03, FD02,FD03, GB02, GB03

Abstract

This study examines the efficacy of user par-ticipation in developing an accounting applica-tion. The research takes place over a 19-month time frame, involves 516 clerical-levelaccounting subjects, and includes experimen-tal manipulations in a field setting. The modelof user participation and involvement proposedby Hartwick and Barki (1994) provides thefoundation for the research framework. Theirmodel is augmented by the inclusion of con-cepts from procedural justice and self-efficacy

~Robert Zmud was the accepting senior editor for this paper.

Introduction

This research examines the impact of allowingclerical accounting personnel the opportunityto participate in developing software. User par-ticipation is considered to be an important fac-tor in the successful implementation and ulti-mate performance of information systems(Cushing 1990). Motivated by recent studies user participation and involvement (Barki andHartwick 1989, 1994; Hartwick and Barki1994), the current study focuses on the influ-ence of participation on user involvement, userattitude, and behavior.

MIS Quarterly/December 1997 359

User Participation

The proposition that the success of an infor-mation system (IS) is directly proportional the extent of user participation in developingthe system has guided a variety of researchendeavors (Cushing 1990; Ives and Olson1984; Martin 1984). However, IS researchexamining the consequential effects of userparticipation has produced equivocal results(DeLone and McLean 1992; Hartwick andBarki 1994; Ives and Olson 1984; Lawrenceand Low 1993; Pettingell et al. 1988). It hasbeen suggested that experiments conducted infield settings may provide valuable insight intokey participation factors leading to IS success(DeLone and McLean 1992). In that light, thisstudy incorporates a longitudinal field experi-ment including 516 professional accountingpersonnel working in 162 field offices.

Recent advances in the participation literaturehave refined the predictive and nomologicalvalidity of key participation constructs. In par-ticular, the user participation and user involve-ment constructs have been clarified and differ-entiated (Barki and Hartwick 1989, 1994), anda theoretical model of user participation andinvolvement has been developed (Hartwickand Barki 1994). The current study extendsand complements this line of inquiry in four keyareas. First, the user participation and involve-ment model is expanded by including conceptsfrom procedural justice research (Hunton1996, forthcoming; Lind and Tyler 1988) andself-efficacy literature (Bandura 1986;Compeau and Higgins 1995a, 1995b).Second, this study focuses on mandatoryusers engaged in hands-on activity. It hasbeen suggested that user participation andinvolvement may be unimportant to mandatoryusers (Hartwick and Barki 1994); however, believe that participation by mandatory usersmay be an integral factor in the ultimate suc-cess of an IS. Third, the current research oper-ationalizes a longitudinal experimental designin a field setting, whereas prior research useda survey method (Hartwick and Barki 1994).Finally, that research relied on self-reportedmeasures of user involvement and attitude asconsequence variables, while the current studyenlarges that scope by also including an objec-tive measure of performance.

]he research context is presented in the nextsection since it establishes parameters for theresearch model. The theory and hypothesesare then developed, followed by a descriptionof the research method. Finally, researchresults are provided and key study findings arediscussed.

Research Context

rhe setting of this study is a state agency. Dueto the nature of the agency’s mission, fieldoffices are autonomous and it was the desireof the state legislature for the agency to contin-ue to operate in this fashion. The decentralizedenvironment made it difficult for agency man-agement to coordinate and control centralagency functions. For example, variousaccounting and management information sys-tems had evolved at each field office with littleor no coordination between offices or with thecentral office located at the state capital.Accountants at each field office were familiarwith their local accounting systems; however,they were not exposed to the specific account-ing systems used at other field offices. As aresult of the decentralized environment andfragmented approach to developing account-ing applications, it was difficult for the centralaccounting office to perform necessaryaccounting functions in an accurate and timelyrnanner.

State auditors had performed numerous finan-cial, operational, and management advisoryaudits over the past several years regardingthe agency’s accounting and managementinformation systems. The agency had beenconsistently criticized for its outdated and inef-ficient information systems. In particular, inter-nal controls over the agency’s accounting sys-tems were characterized as seriously deficient.The expenditure cycle application, includingpurchasing, receiving, accounts payable, anddisbursements, particularly disturbed the audi-tors. One specific deficiency concerned latevendor payments. The state had enacted aprompt pay law that required all agencies tornake vendor payments within 30 days afterbeing billed for receipt of goods/services.

360 MIS Quarterly~December 1997

User Participation

Auditors had determined that the agency wasnot in compliance and that delays and errors inthe accounting function were primarily respon-sible for late payments. The agency was in vio-lation of this law to such an extent that vendorsfiled a class action lawsuit against the agencydemanding interest and penalty payments.

The lawsuit grabbed the attention of the statelegislature. A special committee of the legisla-ture found that the agency had made little head-way correcting information systems deficien-cies. As a result, the agency was mandated bythe state legislature to revamp all informationsystems. Immediately thereafter, the agencydirector resigned under pressure. The newlyappointed director hired a chief information offi-cer (CIO) whose primary mission was to stan-dardize and modernize the agency’s accountingand management information systems.

The CIO developed an information systemsdevelopment plan (ISDP) which called for allsystems to undergo dramatic redesign anddevelopment. The first phase of the ISDP con-sisted of revamping the expenditure cycleapplication. The CIO felt that agency account-ing managers and auditors should play animportant role in the systems developmentprocess. To accomplish this, the CIO estab-lished a steering committee comprised ofaccounting managers from the central and dis-trict offices, state auditors, and IS analysts tooversee development of the expenditure cycleapplication. The agency director suggestedthat lower level accounting clerks who will usethe new system on a regular basis also shouldbe involved in the development process.However, neither the CIO nor accounting man-agers were convinced that participation byaccounting clerks would significantly improvethe new system, particularly considering theadditional commitment of agency resourcesrequired by such participative efforts.Successful implementation of the ISDP wascritical to the agency’s strategic plan and thedirector was not sure whether the cost of cleri-cal participation would outweigh the benefits.The director felt that he could not afford to sub-optimize the system’s potential; hence, hehired the researchers as consultants to investi-

gate the efficacy of allowing participation byaccounting clerical personnel.

After reviewing the proposed expenditure cycleapplication development plans and conductingpreliminary interviews with users at variouslevels throughout the agency, the researchersadvised the director and CIO that some degreeof participation by clerical level accounting per-sonnel during development was warranted.The researchers suggested that, aside fromcognitive advantages (e.g., enhanced under-standing and exchange of information), therewould probably be important affective (e.g.,ownership of the new system and trust insuperiors) and motivational (e.g., expectancybeliefs and self-efficacy perceptions) benefitsarising from clerical level participation.

The researchers’ advice was met with skepti-cism by the CIO. He believed that clerical levelpersonnel lacked sufficient knowledge andexpertise to effectively contribute during thesystems development process. Additionally,the CIO was under considerable time pressureand felt that engaging clerical level users indevelopment efforts would seriously jeopardizepre-established deadlines. Instead, he pro-posed developing the system without up-frontparticipation from clerical users and then con-ducting post-development group meetings ateach field location to demonstrate and explainthe new system’s attributes, functions, andbenefits. In essence, he suggested selling andjustifying the system to users after develop-ment but prior to training and implementation.The proposed user meetings would include aquestion and answer session to address anddiscuss concerns over perceived system defi-ciencies and to minimize efforts to resist thenew system. However, according to the CIO’splan, the concerns, preferences, and opinionsexpressed by clerical level users would not befurther incorporated into the IS design.

After reviewing the researchers’ and CIO’s pre-liminary reports, the director was unsure of theappropriate action. However, the director knewthe agency would be revamping numerousaccounting and management information sys-tems over the next decade and he wanted tounderstand and execute the most effective

MIS Quarterly~December 1997 361

User Participation

development technique while keeping a watch-ful eye on time and cost constraints. As a result,the director decided to allow the researchers toconduct a field experiment designed to deter-mine whether participation by clerical level per-sonnel during systems development producedsignificant benefits in terms of improvedemployee attitude and performance.

In field experimentation, there are a host oflegal, economic, political, and other constraints.From an experimental design perspective,there are limitations that restrict the degree ofinternal validity attainable. However, given therich contextual setting and professional subjectpool, field experimentation can neverthelessprovide valuable contributions to extantresearch streams. The theoretical foundationfor the experimental design is next presented.

Theory and Hypotheses

Hartwick and Barki propose a model of userparticipation and involvement reflecting twoantecedent factors: user involvement and atti-tude. User involvement reflects subjective psy-chological beliefs regarding the importanceand personal relevance of the system to theuser. User attitude represents affective or eval-uative feelings toward the system. Hartwickand Barki suggest that higher levels of userinvolvement and attitude concerning a pro-posed system should lead to an increaseddesire to participate in development activities.They delineate three dimensions of user par-ticipation in the information system develop-ment process: (1) overall responsibility,(2) user-information systems relationship, and(3) hands-on activity. Their model proposedthat higher levels of actual participation alongone or more of these dimensions would height-en post-development user involvement andattitude. Finally, psychological benefits of par-ticipation are predicted to increase an individu-al’s intention to use the system which, in turn,should manifest itself in increased system use.

The Hartwick and Barki model of user partici-pation and involvement serves as the corner-stone for our research model (see Figure 1).

Model la (Figure 1, panel A) depicts an asso-ciation between antecedent conditions andclesired participation. Model lb (Figure 1,panel B) illustrates the effect of user participa-tion on user involvement, user attitude, andu~ser performance.

The effect of antecedent conditionson desired participation

Hartwick and Barki suggest that predevelop-ment states of user involvement and attitudetoward a proposed IS are positively related toa user’s desire to participate in upcomingdevelopment activities. They primarily basetheir assertion on the theory of reasonedaction (TRA) (Fishbein 1980; Fishbein Ajzen 1974, 1975). The TRA indicates that priori user involvement beliefs can exert a rela-tively strong influence on users’ desire to par-ticipate, since these beliefs act as a linkingmechanism between an object (i.e., the pro-posed system) and a behavior (i.e., participa-tive activities). A user who believes that anobject is personally relevant and important islikely to form positive attitudes toward thatobject, since attitudes are generally formed onthe basis of beliefs (Fishbein and Ajzen 1975;Lind and Tyler 1988). Conversely, a priori atti-tudes toward an object can influence thedevelopment of beliefs about the object(Hartwick and Barki 1994); however, the influ-ential strength of attitudes on beliefs is rela-tively weak (Fishbein 1980). Hence, there is expected co-variance between user involve-ment and attitude. Finally, Fishbein suggeststhat attitudes toward an object can affect anindividual’s desire to engage in a target behav-ior (e.g., hands-on activity).

In this study, the users’ a priori levels ofinvolvement with and attitude toward the pre-sent IS are assessed, whereas Hartwick andB, arki refer to beliefs and attitudes toward aproposed system. It is our contention that ifusers believe the existing system is personallyrelevant and important, they will desire to par-ticipate in development activities aimed atenhancing the existing system beyond its cur°rent level (i.e., making it more efficient and

362 MIS Quarterly/December 1997

User Participation

Panel A: Research model 1a---effect of antecedent conditions on desired participation

Panel B: Research model lb~effect of user involvement, user attitude, and user participation on userperformance

Figure 1. Research Model


User Participation

effective). If beliefs play a large part in the for-mation of attitudes, as indicated by Fishbeinand Ajzen (1975), we also expect a positiverelationship between the users’ attitude towardthe existing system and their desire to partici-pate in a process that is expected to improvethe system. One could counterargue the exis-tence of a.negative relationship between anexisting system and the desire to participate inthe development process. However, in order toaccept that argument one would have tobelieve that users will desire to expend valu-able time and energy developing a system thatthey feel is personally unimportant and irrele-vant. In this study, a position congruent withHartwick and Barki is taken, as we believe thathigher levels of user involvement and attitudewith the present IS will result in an increaseddesire to improve the IS through participativeactivities. Based on the theoretical linkagesbetween beliefs, attitudes, and desires, thefirst three parts of hypothesis one (alternateform) are presented here and depicted inFigure 1, panel A:

Hla:There is a positive relationshipbetween predevelopment user in-volvement with and user attitude to-ward the present information system.

Hlb:There is a positive relationshipbetween predevelopment userinvolvement with the present informa-tion system and the user’s desire toparticipate in the development of anew information system.

Hlc: There is a positive relationshipbetween predevelopment user attitudetoward the present information sys-tem and the user’s desire to partici-pate in the development of a newinformation system.

An additional antecedent condition included inresearch Model la is user self-efficacy. Theconcept of self-efficacy refers to the extent towhich individuals believe they can effectivelyperform certain behaviors (Bandura 1986).Individuals who believe they can successfully

execute given behaviors typically desire to ¯engage in such behavior. Conversely, individu-als tend to avoid situations where failure isdeemed likely (Bandura 1986). In an IS set-ting, self-efficacy refers to the users’ belief thattl~ey can successfully perform the behaviorsr~ecessary to effectively contribute to the sys-tem development process (Compeau andHiggins 1995a, 1995b; Hunton forthcoming).As suggested by Compeau and Higgins(1995a), based on the theory of reasonedaction (Fishbein 1980), self-efficacy beliefs canbecome an influential link between an object(i.e., the system) and a related behavior (i.e.,participation). Self-efficacy was not included asan antecedent factor in the Hartwick and Barkimodel; however, self-efficacy theory wouldsuggest a positive link between the users’ apriori perceived level of self-efficacy and theirdesire to engage in development activities.Accordingly, part four of the first hypothesis(alternate form) is presented and shown Figure 1, panel A:

Hid:There is a positive relationshipbetween pre-development self-effica-cy perceptions regarding the users’beliefs about their ability to effectivelycontribute to the information systemsdevelopment process and their desireto participate in the development of anew information system.

The effect of user participation onh~volvement, attitude, andperformance

User participation refers to a set of behaviors,activities, and assignments in which users maybe engaged throughout the systems develop-ment process (Barki and Hartwick 1994). Themodel presented by Hartwick and Barki incor-porates the assumption that users will increasetheir actual participation commensurate withdesires. It has been suggested that allowingindividuals to set their actual participation levelproportionate with their desired level likely opti-m, izes consequential perceptions, attitudes,and behavior (Vroom and Jago 1988).


User Participation

However, depending on an individual’s level inthe organizational hierarchy, actual participa-tion may or may not be at the users’ discretion.Users at lower organizational levels, for exam-ple, may be not be allowed participation indevelopment activities regardless of theirdesire to participate, as was the case in thestate agency. Subjects in this study representmandatory users at lower organizational lev-els, where the degree of participation allowedis a choice made by upper level managers.Since some level of participation was not onlyallowed but also required of accounting per-sonnel in this experiment, participation is treat-ed as an independent variable manipulatedwith three decreasing levels: instrumentalvoice, non-instrumental voice, and no voice (acontrol condition).

Participation by Instrumental Voice

One way to operationalize participation is toprovide users with instrumental voice duringthe course of hands-on activity. The terminstrumental voice originates from the control-oriented perspective of procedural justice theo-ry. Procedural justice refers to the perceivedfairness of a decision making process in accor-dance with expected or accepted norms (Lindand Tyler 1988; Thibaut and Walker 1975,1978). Instrumental voice provides users withan opportunity to express their opinions, prefer-ences, and concerns to decision makers. Thisform of participation provides users with asense of control during the developmentprocess, since the expression of instrumentalvoice is expected to become manifest in thedecision outcome (Folger 1977; Houlden et al.1978; Leventhal 1980). Procedural justiceresearchers stress the critical, influential natureof control perceptions on resulting attitudes andbehavior (Greenberg 1990; Lind and Tyler1988). Researchers in the field of IS (Ives andOlson 1984; Lawrence and Low 1993; Lodge1989; Mumford 1981; Mumford et al. 1983;Mumford and Henshall 1979) also suggest thecritical nature of control perceptions during thesystems development process. They indicatethat development strategies providingenhanced levels of perceived control are

expected to yield higher levels of user involve-ment, attitudes, and performance.

In a recent field experiment, unrelated to thecurrent study, instrumental voice was manipu-lated regarding the development of an IS(Hunton 1996). That study found that subjectsin the instrumental voice condition, as opposedto a no voice condition, believed the decisionprocess was more fair, perceived a higherdegree of control over the decision outcome,felt more satisfied with the resulting IS, andperformed at a higher level. The current studyexpands on the procedural justice aspect ofthat research by incorporating a participationmode called non-instrumental voice.

Participation by Non-Instrumental Voice

The value-expressive perspective of procedur-al justice asserts that voice can enhance judg-ments of process fairness for reasons otherthan influence or control (Lerner 1981; Lind etal. 1990; Lind and Tyler 1988). Advocates ofthe value-expressive perspective contend thatparticipation by voice can engender height-ened affective feelings, such as inclusion andself-esteem, thereby promoting perceptions ofbeing treated fairly, even when voice has noinstrumental impact on the outcome. One wayto operationalize the value-expressive theoryis to provide non-instrumental voice to affectedparties.

Non-instrumental voice allows users to evalu-ate a decision outcome after the decision ismade; however, the outcome is unaffected bythe expression of non-instrumental voice. Ithas been suggested that the exercise of non-instrumental voice can create a perception ofillusory control in which people may believethey experience objective increases in controlover the outcome, particularly when non-instrumental voice is expressed throughout thedecision process (Hunton and Price 1994).Overall, non-instrumental voice is considered aweaker form of participation than instrumentalvoice but stronger than no participation (Lindet al. 1990). Non-instrumental voice has beenshown to enhance attitudes of affected parties;


User Participation

however, research evidence regarding theinfluence of non-instrumental voice on individ-ual behavior is equivocal (Lind and Tyler1988).

Consequences

In this study, it is expected pre- to post-experi-ment gains in user involvement and attitudewill be highest in the instrumental voice condi-tion, as this treatment affords the highest levelof user control. Barki and Hartwick (1994) indi-cate that user involvement might be enhancedif users develop a better cognitive understand-ing of the new system and gain awareness ofhow the system can help them in their job. Inthe current study, attempts to justify andexplain the new system in the non-instrumen-tal participation condition are expected to fulfillthese cognitive criteria. As well, the value-expressive perspective of procedural justicetheory would predict enhanced favorable atti-tudes toward the system due to the psycholog-ical, symbolic contributions offered by non-instrumental voice (Lind and Tyler 1988).Finally, a positive relationship between userinvolvement and user attitude is anticipated,based on the same premise articulated forHl a. Hence, the following hypotheses (alter-nate form) are presented and depicted inFigure 1, panel B:

H2a: A positive relationship exists betweenuser involvement pre- to post- experi-ment gains and user attitude pre- topost-experiment gains.

H2b: Preo to post-experiment user involve-ment gains will be highest in theinstrumental voice participation con-dition, followed by the non-instrumen-tal voice participation condition, andlowest in the no voice (i.e., control)condition.

H2c: Pre- to post-experiment user attitudegains will be highest in the instrumen-tal voice participation condition, fol-lowed by the non-instrumental voice

participation condition, and lowest inthe no voice (i.e., control) condition.

Flesearchers suggest positive relationshipsbetween beliefs, attitudes, and behavior (Barkiand Hartwick 1994; Fishbein 1980; Locke andLatham 1990). The theory of reasoned actionindicates that beliefs (i.e., personal relevanceand importance) about an object (i.e., the sys-tem) can exert a positive influence on individ-ual behavior. Attitudes toward an object canalso affect behavior, but this relationship isgenerally less influential than the belief-behav-ior linkage, unless there is complete corre-spondence between the attitude and the targetbehavior. In the current study, user attitudetoward the newly developed system is mea-sured, not toward tasks associated with usingthe system. As such, there is only partial corre-spondence between attitudes and observedbehavior (i.e., using the system more efficient- ly and effectively). Hence, while positive rela-tionships are anticipated between (1) userin, volvement and performance and (2) user atti-tude and performance, it is expected that theattitude-performance link will be relativelyweak.

A direct relationship between the participationmanipulation and user performance is alsoexpected. Although evidence linking participa-tion to individual performance is mixed(l:3aroudi et al. 1986; Ives and Olson 1984;Pettingell et al. 1988; Wong-On-Wing 1988),there is some indication that instrumental voicecan lead to improved performance (Earley andLind 1987; Hunton 1996; Lind and Tyler 1988).Tlhe primary thrust behind the instrumentalvoice-performance relationship is increaseduser control over the outcome. The influenceof non-instrumental: voice on performance isnot predictable with any degree of confidence,since non-instrumental voice offers no objec-tive increase in outcome control. The argu-ment has been made that user participationmay be causally related to performance forreasons other than control (Lind and Tyler1988). For example, participative activities canevoke feelings of work group inclusion, systemownership, and outcome responsibility, there-by motivating users to increase their extant


User Participation

performance levels (Locke and Latham 1990).While expecting the influence of instrumentalvoice on performance to be stronger than non-instrumental voice, it is anticipated that thenon-instrumental voice manipulation will posi-tively impact performance for the reasons justprovided and because non-instrumental voicecan evoke perceptions of illusory outcomecontrol (Hunton and Price 1994). Accordingly,the final hypotheses are presented (alternateform) and illustrated in Figure 1, panel B:

H3a: User involvement pre- to post-experi-ment gains positively influence userperformance pre- to post-experimentgains.

H3b: User attitude pre- to post-experimentgains positively influence user perfor-mance pre- to post-experiment gains.

H3c: The direct influence of user participa-tion on pre- to post-experiment userperformance gains will be highest inthe instrumental voice participationcondition, followed by the non-instru-mental voice participation condition,and lowest in the no voice (i.e., con-trol) condition.

Experimental Design

The state agency was divided into nine dis-tricts; each district director was responsible forfield office operations. At the clerical level,there was little interaction between field officeswithin the same district and virtually no interac-tion between field offices located in differentdistricts. The researchers randomized threedistricts to each of three experimental treat-ment conditions: instrumental voice, non-instrumental voice, and no voice. District direc-tors and accounting managers were aware ofthe experiment; however, other district andfield office employees were unaware that anexperiment was in progress.

Instrumental voice participationmanipulation

The agency had a centralized managementinformation systems (MIS) staff that reportedto the CIO. There were 51 field offices in theinstrumental voice participation manipulation.Small groups of clerical level accountingemployees provided input into four hands-onactivities: defining and designing (1) report for-mats, (2) screen layouts, (3) input and outputforms, and (4) user procedure manuals. MISpersonnel performed the bulk of the technicalwork in these four areas and the accountingclerks made suggestions for improvementsbased on prototype designs of reports,screens, forms, and procedure manuals.Accounting clerks at each field office met insmall groups. Each consecutive Friday foreight weeks, there were two-hour morningmeetings at each field office. By Fridayevening, suggestions from each group wereforwarded to the MIS department. Based onthis input, MIS analysts created an updated setof prototypes.

In each field office, accounting clerks receivedan initial four-hour training session in effectivegroup techniques. Each group learned to usethe consensual conflict resolution (CCR) tech-nique (Hall and Watson 1970; Innami 1994).Group members were taught to maximize theirproductivity by using the six steps of CCR (seeAppendix A). The CCR instruction was intend-ed to increase reasoning orientation anddecrease positional orientation of group verbalbehavior. There were no supervisors in thesegroups. During the initial group meeting, themembers elected a leader and a recordingsecretary.

Since there were 51 field offices, theresearchers did not have enough time to con-duct the initial group meetings. Theresearchers trained 10 human resource per-sonnel in how to conduct the initial meeting atthe field locations. This training session lastedapproximately five hours. The trainers wererandomized to the 51 field offices and theyconducted all initial group meetings in a one-week time frame. Trainers were instructed not


User Participation

to discuss either the identity or number of fieldoffices involved. The trainers were aware thatan experiment was in process, but they wereblind to the manipulations and hypotheses.

During the eight-week instrumental voice par-ticipation period, the groups were self- sustain-ing. In order to maximize the free exchange offeelings and information during meetings, nosupervisors, managers, or observers were pre-sent. However, field office accounting man-agers made certain that all group membersattended all meetings and that meetings lastedapproximately two hours.

Non-instrumental voiceparticipation manipulation

Once the new expenditure cycle applicationwas completed, subjects in the non- instru-mental voice participation condition attendedan orientation session lasting approximatelysix hours. The purpose of the session was todemonstrate the new application, explain thebenefits of the new system, and listen to usercomments. Participants were told their sugges-tions could not be incorporated into the expen-diture cycle application, as it was alreadydeveloped and ready for conversior~/implementation.

There were56 field offices in the non-instru-mental voice participation condition. TwelveMIS analysts conducted the orientation meet-ings, with one analyst per meeting. Prior toholding the orientation meetings, theresearchers, CIO, and 12 analysts first held aneight-hour meeting to standardize the orienta-tion session agenda and procedures. Eachanalyst conducted either three or four orienta-tion meetings. Analysts were aware of theexperiment but blind to the manipulations andhypotheses. Analysts were randomly assignedto field offices and all orientation sessionswere completed within five consecutive work-days.

No participation manipulation

There were 55 field offices in the no participa-tion condition. Accounting clerks in this condi-tion were aware that MIS was developing anew expenditure cycle application, howeverthey were not informed that some accountingclerks were participating during developmentor that other accounting clerks attended an ori-entation meeting.

Training session

Two weeks before implementation, all affectedaccounting clerks at all field offices, regardlessof their participation treatment condition,attended an eight-hour, on-site user trainingsession on how to operate the newly devel-Ol~)ed expenditure cycle application. The 10human resource personnel who initially trainedthe groups and the 12 analysts who conductedthe orientation meetings held the training ses-sions. These trainers were cautioned not tomention their prior involvement during thetraining meetings. The 22 trainers were ran-domly assigned to the 162 field offices.T~raining was completed within nine workingdays.

Timing of manipulations andmeasurement of variables

The experiment was conducted during a 19month period. Ten months prior to implemen-tation of the new system (t.lo), all participatingaccounting clerks at all 162 field officesresponded to a pre-experiment questionnaire(see Appendix B). The survey instrument wascontained on diskette and all items were ran-demized per individual subject. Survey itemsincluded (1) user involvement and attitudetoward the existing expenditure cycle applica-tion, (2) perceived self-efficacy in participatingin hands-on activity regarding new systemsdevelopment, (3) desire to participate hands-on activity, and (4) demographic infor-m.ation. Group effectiveness training also took


User Participation

place during month t.10 for participants in theinstrumental voice participation condition.

One month after administration of the pre-experiment survey, the instrumental voice par-ticipation development meetings commencedand were cor)cluded in two months (t. 9 and t.8).The expenditure cycle system was completedduring the next five months(t.7 through t.3). Thenon-instrumental voice participation orientationsessions were next conducted (t.2). During themonth of system conversion (t.1), training ses-sions were provided to all accounting clerks.

The first three months of operation werelabeled learning curve months (t÷l, t÷2, and t÷3)and observation of the system operation con-tinued for an additional six months (t÷4 throught+9). At the end of the sixth month of systemoperation (t÷9), a post-experiment survey wasadministered (see Appendix C). The surveyinstrument was contained on diskette and allitems were randomized per individual subject.Survey items included user involvement andattitude toward the new expenditure cycleapplication, as well as manipulation checkquestions.

Performance Measurement

A monthly performance measure was calculat-ed for each field office as follows:

Vendor payments in violation of the state’s 30-dayprompt payment law(t)

Total number of vendor payments (t)where t = experimental months - 6 through + 9

This performance indicator, percentage of latevendor payments by month, is directly relatedto the efficiency and effectiveness of theexpenditure cycle application under develop-ment, since vendor payments are the ultimateoutput from this system. While tracking latevendor payment rates by field office, it was notpossible to isolate violations to individualaccounting clerks who took part in this experi-ment; accordingly, a research limitation isacknowledged. That is, while it was possible torestrict the analysis of survey responses to

accounting clerks who were employed by theagency throughout the entire 19 months of theexperiment, the field office level performanceindicator impounds inherent turnover effects.Great effort was taken to ensure there were nochanges in policies or procedures surroundingthe expenditure cycle that may have account-ed for the difference in performance gainscores across experimental conditions. As aresult of a thorough investigation into thisissue, the researchers are confident that theonly differential treatment across experimentalconditions was the extent of participation pro-vided to accounting clerical level personnel.

Results

Sample characteristics

Sample statistics are provided in Table 1. Atthe beginning of the experiment, there were615 accounting personnel in 162 field offices(Table 1, panel A)o At the end of the experi-ment, only 516 (83.9%) of the original 615accounting personnel were still employed bythe agency (Table 1, panel B). Agency person-nel records indicate that the retention rate dur-ing the 19 month experimental period was notunusual, as the retention rate for the samelevel of clerical employees for the 19 monthtime-frame prior to commencement of the cur-rent study was 82.4%.

MANOVA was used to determine whetherthere were significant individual differencesamong treatment conditions for the 99 subjectswho left the agency between pre- and post-experiment testing. The set of dependent vari-ables included age, years of education, yearsof accounting experience, years with stateagency, and gender. No significant differencesamong treatment conditions were found, asthe F-ratio (p-value) for the MANOVA modelwas 0.82 (.6058). Only pre- and post-experi-ment survey responses from the 516 whoremained with the agency throughout theentire experimental period were subsequentlyanalyzed.


User Participation

Table 1. Sample Statistics

Panel A: Employees per region at the beginning of the experiment (t.10)

Instrumental Non-Instrumental NoDemographic Variable Voice Voice Voice

Number of field officesNumber of accounting clerksAverage number of clerks per groupMean age of accounting clerksMean years of education completedMean years of accounting experienceMean years with the State agencyGender (% female)

51193

3.828.213.75.04.3

53.4

56221

3.927.913.65.14.1

52.8

Total

55 162201 615

3.7 3.828.1 28.113.6 13.74.9 5.04.0 4.1

54.1 53.2

Panel B: Regional breakdown of employees present at the end of the six-month post-implementationobservation (t÷9) who were also present at the beginning of the experiment (t.10)

InstrumentalVoiceDemographic Variable

Number of accounting personnel 159 188Percentage Retention Rate 82.38 85.07

Non-Instrumental NoVoice Voice

16984.08

Total

51683.9

Preliminary testing

Reliability estimates were calculated onresponses to the pre- and post-experiment sur-vey items using Cronbach’s alpha (e~) statistic(n = 516). Research indicates that reliabilityestimates exceeding .70 are desirable(Cronbach et al. 1972; Neter et al. 1990).Reliability estimates for pre-experiment surveyresponses are as follows: ~ = .789 (nine itemsrepresenting user involvement), ~ = .946. (fouritems representing user attitude), ~ = .904 (fouritems representing user self-efficacy), and e~ .958 (four items representing desired participa-tion). Post-experiment survey reliability esti-mates are as follows: o~ = .846 (nine items rep-resenting user involvement) and ~ = .959 (fouritems representing user attitude). Because reli-ability estimates were satisfactory, related sur-vey item responses were averaged to obtainpre- and post-experiment variable indices.

Manipulation Checks

Manipulation check survey items are shown inAppendix C. Items assessing the extent towhich users perceived they actually participat-ed in development activities served as a

manipulation check for the instrumental voicecondition. Based on a seven-point scale, meanresponses by treatment condition were 5.54(instrumental voice), 1.79 (non-instrumentalvoice), and 1.82 (no voice). The ANOVAmodel indicated an F-ratio (p-value) of 246.65(:-~ .0001). Scheffe’s multiple pairwise compari-son (~ = .01) revealed that 5.54 was signifi-cantly higher than either of the other twomeans, and the other means were not signifi-cantly different from one another.Consequently, the instrumental voice participa-tion manipulation was considered successful.

The next set of manipulation check items weredesigned to assess the subjects’ degree ofperceived control over various aspects of thedecision outcome (i.e., the new expenditurecycle application). Based on a seven-pointscale, mean perceived control responses bytreatment condition were 6.02 (instrumentalvoice), 3.16 (non-instrumental voice), and 1.15(no voice). The ANOVA model F-ratio (p-value) was 187.49 (-< .0001). Using Scheffe’srnultiple pairwise comparison (o< = .01), thesethree means were all significantly differentfrom one another. These results provide evi-dence that the three treatment conditions


User Participation

evoked varying levels of perceived control, inaccordance with procedural justice theory.Hence, differential attitudinal and behavioralresponses observed during hypothesis testingmight be partially attributable to the manipula-tion of perceived control.

To test the success of the non-instrumentalparticipation manipulation, one of the surveyquestions asked the subjects whether theyparticipated in a six-hour orientation meeting.All subjects in the non-instrumental voice con-dition properly indicated yes (n = 188) whileone of 159 subjects in the instrumental voicecondition and two of 169 subjects in the novoice participation condition indicated yes. Thefew affirmative responses from the instrumen-tal voice and no voice participation conditionslikely arose from confusion over orientationversus training sessions. Overall, the non-instrumental voice participation manipulationwas deemed successful.

Another manipulation check question assesseda constant condition provided to all subjects:user training. All subjects indicated that theyhad attended an eight-hour user training ses-sion. The final manipulation check questionasked participants whether they believedaccounting clerks throughout the agency wereallowed to participate in developing the expen-diture cycle application. Based on a seven-point scale, mean responses were: 6.22(instrumental voice), 0.56 (non-instrumentalvoice), and 0.68 (no voice). Using ANOVA ratio = 388.44, p-value -< .0001) and Scheffe’spost-hoc test (o~ = .01), the instrumental voicemean was significantly different from the othertwo means and the non-instrumental voice andno voice means were not significantly differentfrom one another. Hence, the treatment condi-tions were considered independent.

Systematic Bias Checks

The research tested for differences in demo-graphic factors across experimental conditionsfor the 516 subjects who completed the experi-ment. The demographic factors examinedwere (1) age, (2) years of education, (3)

of experience, (4) years with the state agency,and (5) gender. To test for possible differ-ences, four ANOVA models were used, wherethe dependent variable for each model repre-sented the first four demographic factors listedabove, and the independent variable was theparticipation treatment condition (i.e., no voice,non-instrumental voice, and instrumentalvoice). Additionally, a chi-square test was usedto determine whether the proportion of maleand female subjects differed across the threeconditions. The overall ANOVA model F-ratios(p-values) were 1.01 (.8567) for age, (.5921) for years of education, 0.84 (.9477) years of experience, and 1.49 (.6432) for yearswith the state agency. ANOVA results indicateno significant differences across conditions.Also, there was no significant difference in theproportion of male and female subjects acrossexperimental conditions (chi-square = 0.88,p-value = .46).

The research also tested for differences indemographic factors between the 99 subjectswho left the agency during the experimentaltime period and the 516 subjects whoremained. To accomplish this objective, t testsof each demographic factor were conducted,where those who left the experiment repre-sented one group, and those who completedthe experiment represented the other group.The t-statistic (p-value) for each variable fol-lows: 1.25 (.45) for age, 0.94 (.61) for years education, 1.55 (.37) for years of experience,1.81 (.24) for years with the state agency, and1.02 (.56) for gender. Based on t test results,we find no significance differences in demo-graphic characteristics due to attrition.

The research examined the extent to whichtrainer, field office, or both may have influ-enced post-experiment survey responses.Although trainers were randomized to fieldoffices and field offices were randomized totreatment conditions, there may have beenunintended, systematic influences due to thesefactors. A MANOVA model was calculatedwhere the independent variables were(1) trainer, (2)field office, and (3)interaction trainer and field office, and the set of depen-dent variables included post-experiment userinvolvement and user attitude responses. The


User Participation

overall F-ratio (p-value) was 1.22 (.4873).Based on the overall significance level, sys-tematic effects of trainer, field office, or both onthe two dependent variables were deemedunlikely.

Hypothesis testing

Consistent with Hartwick and Barki, EQS, astructural equation modeling program (Bentler1989), was used to test the hypotheses. Sincethere is no generally accepted measure ofoverall model goodness of fit with structuralequations, leading researchers recommendthat multiple criteria be evaluated and com-pared to determine how well the data fits themodel (Breckler 1990; Wheaton 1987). In thisstudy, three key goodness of fit criteria, sug-gested by Bentler (1989, 1990), were used.

The first goodness of fit indicator is called thenormed fit index (NFI), introduced by Bentlerand Bonett (1980). The NFI transforms the chi-square statistic into a 0 to 1 range, where 1indicates a perfect fit. However, NFI can beaffected by sample size, such that in smallsamples the index may not reach 1. As abenchmark, Bentler and Bonett suggest that aNFI of .90 or above represents a good fit.

[he second goodness of fit indicator, also pro-posed by Bentler and Bonett, is called the non-normed fit index (NNFI). The NNFI modifiesthe NFI by accounting for the degrees of free-dom in the model. As a result, it works well atall sample sizes. A negative feature of theNNFI is that in sampling studies the NNFI issensitive to sampling error; therefore, the vari-ance of NNFI can be considerably larger thanthe variance of NFI. As with the NFI, a NNFI of.90 or greater indicates a good fit.

The third criteria is called the comparative fitindex (CFI). Bentler (1990) indicates that CFI works well at all sample sizes and is morestable than the NNFI because the CFI is lesssensitive to sampling error. Consequently, theCFI is often used as the key goodness of fitindex in structural equation models. PossibleCEI values also range from a low of 0 to a high

of 1. According to Bentler, a CFI of .90 indi-cates a good fitting model.

Hypothesis 1

"l’he first hypothesis predicts positive relation-ships between the antecedent variables anddesired participation (see Figure 1, panel A).Specifically, the four parts of hypothesis oneposit significant associations between userinvolvement and user attitude (Hla), userinvolvement and desired participation (Hl b),user attitude and desired participation (Hlc),and user self-efficacy and desired participation(Hld). The unit of analysis for the first hypoth-esis is at the individual level (n = 516), sinceno experimental treatments had been adminis-tered at the time subjects responded to thepre-experiment survey instrument.

[--QS results provide standardized path coeffi-cients and related significance levels (p-valuesbased on t statistics). Initially, all possiblepaths on research model la were tested andgoodness of fit indices were obtained as fol-low: NFI = .792, NNFI = .826, and CFI = .811.All indices were below the recommendedthreshold of .90. The paths between user self-efficacy and user attitude (path coefficient .02, p-value = .5426) and user self-efficacyand user involvement (path coefficient = .04, p-value = .4177) were not significant. These twopaths were subsequently dropped from themodel and the EQS program was re-executed."][’he resulting model, including all hypothesizedpaths, is depicted in Figure 2.

.][’he goodness of fit indices of the modelshown in Figure 2 all exceed the recommend-ed benchmark of .90 (NFI = .938, NNFI = .954,and CFI = .941). Accordingly, the data appearto adequately fit the model. For purposes ofassessing the reliability of survey items to rep-resent their underlying constructs, the EQSmodel was designed to determine the signifi-cance of paths between the survey items andmodel constructs. To accomplish this, EQSuses factor analysis to determine path coeffiocients and significance levels. All survey itemsloaded high on their respective factors, as the


User Participation

UI-Rl--.96a .87a -- UI-I1 < UA1 --.98a SE1 --.98a

UI-R2--.98a .91a -- UI-I2 < UA2 --.95a SE2 --.92a

UI-R3--.92a .85a -- UI-I3 < UA3 --.95a SE3 --.92a

UI-R4--.94a .90a -- UI-I4 < UA4 --.85a SE4 --.93a

.94a -- UI-I5

]~~~..21a . ~"~ser~’~ ~User~’X~

DP1 -- .96a

DP2 -- .91a

DP3 --.93a

DP4 -- .85a

Notes:1. Pre-experiment survey responses (n = 516) are used in the EQS model.

UI-Rx = User Involvement - Relevance, Ul-lx- User involvement - ImportanceUAx = User Attitude, SEx- Self-Efficacy, DPx = Desired Participation, where x = survey itemnumber shown in Appendix B.

3. Standardized coefficients are shown along paths.4. Path significance levels are as follow: a < .001, b < .01, c < .05.5. EQS uses factor analysis to calculate the path coefficients from survey items into constructs.6. Model goodness of fit indices: NFI = .938, NNFI = .954, CFI = .941.7. Model chi-square, degrees of freedom, p-value = 416.3, 181, < .0001.

Figure 2. Results of EQS Structural Equation Modeling on Research Model la


User Participation

lowest standardized path coefficient was .85and all paths were significant at p-value <.001. Hence, the survey items appear to bereliable.

Paths among model constructs are all signifi-cant at p-value < .001. The standardized pathcoefficients are as follows: user involvementand user attitude equal .21 (Hla), userinvolvement and desired participation equal.53 (Hlb), user attitude and desired participa-tion equal .42 (Hlc), and user self-efficacy anddesired participation equal .31 (Hld). Basedon EQS results, the first set of hypotheses aresupported.

Hypotheses 2 and 3

Due to interactions between subjects in theinstrumental voice participation hands-onmeetings and in the non-instrumental voiceparticipation orientation meetings, there was alack of independence between individual sub-jects at field locations. Therefore, the unit ofcomparison for testing hypotheses two andthree is the group level (i.e., field location) andthe sample sizes are as follows: instrumentalvoice (n = 51), non-instrumental voice (n 56), and no voice (n = 55). The theoreticalmodel developed by Hartwick and Barki repre-sents how user participation and involvementaffects individuals; yet, changes in groupmeans are reported here. It is recognized thatan individual’s beliefs, attitudes, and behaviorlikely do not completely correspond to thegroup mean.

Comparisons between pre- and post-experi-ment beliefs, attitudes, and performance wereanalyzed using group gain scores. To obtaingain scores, the research first calculatedmean user involvement and user attitudeindices by field location using individual pre-and post-experiment survey responses. Next,each group’s mean post-experiment indexwas subtracted from its mean pre-experimentindex to obtain a gain score. Group gainscores are used as independent observationsin statistical testing for the second and thirdhypotheses. Shown in Table 2 are the pre-

experiment mean responses(panel A), post-experiment mean responses (panel B), and(.lain scores (panel C) for user involvementfind user attitude.

Regarding the performance measure (per-centage of late vendor payments), it was notpossible to specifically identify late vendorpayments with individual accounting clerks.However, it was possible to record late ven-dor payment percentages by field office dur-ing each experimental month. Objective mea-sures of performance gain scores are report-ed in Table 2, panel C. In this study, perfor-"rnance gain scores represent the differencebetween post- and pre-experiment late ven-dor payment percentages at each field loca-tion. As requested by agency management,the actual late payment percentages for thepre- and post-experiment periods are notshown.

E--QS and ANOVA models were used to testthe second and third hypotheses. Group gainscores were used as inputs for EQS andANOVA calculations. EQS model results areshown in Figure 3~ The data appear to ade-quately fit the model, as two of the three good-hess of fit indices exceed the recommendedlevel of .90 (NFI = .892, NNFI = .922, and CFI= .914). EQS standardized path coefficientsand ANOVA results are next discussed withrespect to the hypotheses.

Hypothesis 2: The second hypothesis pre-dicts a significant association between userinvolvement and user attitude gains (H2a) andposits increases in user involvement (H2b)and user attitude (H2c) gains as the participa-tion treatment moves from no voice to non-instrumental voice to instrumental voice. Asanticipated, the path between user involve-rnent and user attitude gains (.18) is signifi-(:ant (p-value < .01). The standardized pathcoefficient (p-value) between participation anduser involvement gains is .42 (< .001) and thepath between participation and user attitudegains is .20 (< .01): These significant, positivepaths are consistent with a priori expectations.To further investigate changes in gain scoresacross treatment conditions, ANOVA wasused and the results are presented in Table 3.


User Participation

Table 2. Descriptive Statistics and Gain Scores

Panel A: Response means from pre-experiment survey

Variable

User InvolvementUser Attitude

InstrumentalVoice

4.064.81

Non-InstrumentalVoice

4.045.02

NOVoice

4.284.79

Overall

4.134.87

Panel B: Response means frompost-experiment survey

Variable

User InvolvementUser Attitude

InstrumentalVoice

5.316.07

Non-InstrumentalVoice

4.525.61

NoVoice Overall

4.32 4.723.95 5.21

Panel C: Gain scores

Instrumental Non-Instrumental NoVariable Voice Voice Voice Overall

User Involvement + 1.25 + 0.48 + 0.04 + 0.59User Attitude + 1.26 + 0.59 - 0.84 + 0.34Performance* + 0.0586 + 0.0470 + 0.0472 + 0.0508

*Performance gains were calculated as follows:

SUM [Late Vendor Payments (t-6 through t-l)]

SUM [Total Vendor Payments (t-6 through t-l)]minus

SUM [Late Vendor Payments (t .,~ through t +9)]

SUM [Total Vendor Payments (t +,~through t*9)]

Months t÷l through t+3 were considered learning curve periods. Agency management asked theresearchers not to reveal actual late-pay violation percentages. Accordingly, only the differencebetween post- and pre-experiment late-pay violation rates are shown.

As with the EQS model, mean gain scores byfield location were used as dependent vari-ables in the ANOVA models. This provides aconservative test of differences since the vari-ance among individuals at field locations isaveraged.

Table 3, panel A presents the results of testinghypothesis H2b. The overall model is statisti-cally significant (p-value = .0013), mean gainscores are significantly different across treat-ment conditions (o~ = .05), and the direction gain scores is consistent with the hypothesis.Table 3, panel B shows the results of testinghypothesis H2c. The ANOVA model is signifi-cant (p-value < .0001), treatment means aresignificantly different from one another (o~

.01), and mean gain scores are directionallyconsistent with predictions. Based on the com-bined results of the EQS and ANOVA models,hypothesis two is supported.

Hypothesis 3: The first two parts of the finalhypothesis predict positive associationsbetween user involvement and performance(H3a) and user attitude and perfor.mance(H3b). EQS model results show a significantinvolvement-performance path (standardizedcoefficient = .16, p-value < .01) and a signifi-cant attitude-performance path (standardizedcoefficient = .11, p-value < .05). As anticipat-ed, the attitude-performance link is statisticallysignificant, yet relatively weak. Accordingly,hypotheses H3a and H3b are supported.

MIS Quarterly/Deceml~er 1997 375

User Participation

Participation Condition

No-Voice INon-Instrumental Voice]

Instrumental Vc,ice ]

UI-R1 Gain- .88a

UI-R2 Gain- .85a

UI-R3 Gain- .91a

UI-R3 Gain- .96a

.82a -- UA1 Gain

.93a -- UA2 Gain

.87a -- UA3 Gain

.90a -- UA4 Gain

UserInvolvement

UI-I1 Gain m .88a

UI-I2 Gain --.82a

UI-I3 Gain --.95a

UI-I4 Gain --.91a

UI-I5 Gain --.93a

.18b

.16b .11c

.31a

UserAttitude

Notes:1. Pre-experiment minus post-experiment gain scores (n = 162) are used in the EQS model.2. UI-Rx = User Involvement - Relevance, Ul-lx- User involvement- Importance

UAx = User Attitude, where x = survey item number shown in Appendices B and C.3. Standardized coefficients are shown along paths.4. Path significance levels are as follow: a < .001, b < .01, c < .05.5. EQS uses factor analysis to calculate the path coefficients from gain scores into constructs.6. Model goodness of fit indices: NFI = .892, NNFI = .922, CFI = .914.7. Model chi-square, degrees of freedom, p-value = 230.16, 84, < .0001.8. The participation condition variable was treated as an ordinal scale, where 9 = no voice,

1 = non-instrumental voice, and 2 = instrumental voice.

Figure 3. Results of EQS Structural Equation Modeling on Research Model lb


User Participation

Table 3. Supplemental Analyses of H2b, H2c, and H3c

Panel A: H2b--ANOVA model results on user involvement gain scores

Independent Variable D.F.

Participation Treatment 2Error 159Total 161

Sum-Squares

40.68465.96506.64

Mean-Square

20.342.93

F-ratio

6.94

p-value

.0013

Scheffe’s Pairwise Comparison of Mean Gain Scores [means with the same (different) superscript(s)are not (are) significantly different from one another at (~ = .05].

Instrumental Voice +1.26a

Non-instrumental Voice +0.47b

No Voice +0.03c

Panel B: H2c--ANOVA model results on user attitude gain scores

Independent Variable D.F.

Participation Treatment 2Error 159Total 161

Sum-Squares Mean-Square F-ratio p-value

126.74 63.67 12.18 .0001790,53 4.97917,27

Scheffe’s PainNise Comparison of Mean Gain Scores [means with the same (different) superscript(s)are not (are) significantly different from one another at o~ = .01].

Instrumental Voice +1.26a

Non-instrumental Voice +0.59b

No Voice - 0.84c

Panel C: H3c--ANOVA model results on performance gain scores

Independent Variable D.F. Sum-Squares Mean-Square F-ratio p-value

Participation Treatment 2 .00466 .00233 14.06 .0001Error 159 .02632 .00017Total 161 .03098

Scheffe’s Pairwise Comparison of Mean Gain Scores [means with the same (different) superscript(s)are not (are) significantly different from one another at e~ = .01].

Instrumental Voice +.0586a

Non-instrumental Voice +.0470b

No Voice +.0472b

H3c predicted increasing performance gainsacross the no voice, non-instrumental voice,and instrumental voice treatment conditions.Agency management asked the researchersnot to reveal the actual pre- or post-experi-ment vendor payment violation rates.

Accordingly, only performance gains (i.e.,pre- minus post-experiment violation rates)

are shown. Figure 4 depicts monthly perfor-

mance gains by treatment condition, usingthe following formula:


User Participation

SUM [Late Vendor Payments

SUM [Total Vendor Payments (~

minus

Late Vendor Payments

Total Vendor Payments (~ +x)

where x = experimental months 1 through 9

Months labeled t+l, t+2, and t÷3 reflect the firstthree months of system operation. These threemonths were considered learning curve peri-ods since the gains had not settled into a dis-cernible pattern until t+4, as evidenced inFigure 4.

The dependent variable for hypothesis testingwas obtained by subtracting the mean sixmonth (t. 6 to t.1) pre-experiment late pay viola-tion rate from the mean six month (t÷4 to t÷9)post-experiment rate for each field office usingthe following formula:

SUM [Late Vendor Payments (t-6 th.’ough t-l)]

SUM [Total Vendor Payments (~ .6 t~,ough ~-1)]

minus

SUM [Late Vendor Payments (t +~ ~,,oug,~ +9)]

SUM [Total Vendor Payments (~ ÷4 ~,,oug, t ÷9)]

EQS model results, shown in Figure 3, reveala significant path between the participationtreatment and user performance (standardizedcoefficient = .31, p-value < .0001 ). Additionally,ANOVA was used to examine mean gainscore differences among experimental treat-ment conditions. The results are shown inTable 3, panel C. The overall ANOVA modelwas significant at p-value --- .001. The highestgain was in the instrumental voice condition(+.0586), followed by the no voice (+.0472)and non-instrumental voice (+.0470) treat-ments. All conditions experienced positive per-formance gains, presumably due to animproved expenditure cycle application; how-ever, the gain in the instrumental voice partici-pation condition was significantly higher than

in the other two conditions. While the perfor-mance gain in the non-instrumental conditionwas expected to be significantly higher thanthe no voice condition, ANOVA results do notsupport this proposition. As a result of hypoth-esis testing, H3c was partially supported.

IDiscussion

In this study, the efficacy of hands-on partici-pation by accounting clerical personnel in theprocess of developing an accounting applica-tion was examined. The field experimentinvolved administering pre- and post-experi-ment surveys, manipulating participation atthree increasing levels (instrumental voice,non-instrumental voice, and no voice), andmonitoring performance data for 15 months.Three psychological antecedent factors weremeasured: user involvement with the presentIS, user attitude toward the present IS, anduser self-efficacy beliefs regarding perceivedability to effectively contribute during the ISdevelopment process. All three antecedentswere significantly associated with the userdesire to engage in systems developmentactivities. After the system was implemented, apost-experiment survey assessed userinvolvement with and attitude toward the newIS.

Mean gain scores (post- minus pre-experimentresponses) for user involvement and user atti-tude were highest in the instrumental voiceparticipation condition, followed by the non-instrumental voice Condition, and lowest in theno voice condition. The gain in user perfor-mance (i.e., change in percentage of late ven-dor payments) was significantly highest in theinstrumental voice participation condition,while performance gains in the non-instrumen-tal voice and no voice participation conditionswere not significantly different from each other.

-this study enhances the Hartwick and Barkimodel of user participation and involvement byintegrating theory from self-efficacy (Bandura1986) and procedural justice (Hunton 1996,forthcoming; Lind and Tyler 1988; Thibaut andWalker 1975) into the research framework.

378 MIS Quarter/y/December 1997

User Participation

0.06

0.05

0.04

0.03

0.02

0.01

t +1 t +2 t +3 t +4 t +5 t +6 t +7

I---~---Instrumental Voice --ll--Non-lnstrumental Voice A, No Voice

t +8 t +9

Learning Curve -- Post-Experiment Performance Period

Figure 4. Performance Gains

Research findings support the Hartwick andBarki model of user participation and involve-ment. Additionally, results suggest that partici-pation by mandatory users in systems devel-opment activities may be more effective thanHartwick and Barki conclude.

Hartwick and Barki (p. 458) state the following:’q’hus, for mandatory users, user participationand involvement would seem unimportant."They arrive at this conclusion because theirdata showed that, for mandatory users, there isno significant relationship between user partici-pation, user involvement, user attitude, andintention to use the system. The primary rea-son behind their assertion is that mandatoryusers may not have felt a sense of overallresponsibility for the newly developed ISbecause their participation was directed, limit-ed, and controlled by their superiors. However,

Hartwick and Barki recognize the importance ofmandatory users in the development processand call for more research in this area.

In our study, the accounting clerks weremandatory users and the nature and degree oftheir participation was established by theirsuperiors. Yet, significant gains in userinvolvement, user attitude, and performancewere found. There could be several explana-tions for the differences found in this study ascompared to the Hartwick and Barki research.For example, study participants, contextualfactors, and research methods were not thesame. However, there may be a commonthread between these two studies.

Hartwick and Barki conclude that mandatoryusers in their sample did not feel a sense ofoverall responsibility for the proposed IS. In

MIS Quarter/y/December 1997 379

User Participation

the current study, subject perception of controlover the newly developed IS was manipulatedand measured and this concept of control issimilar to Hartwick and Barki’s notion of overallresponsibility. Assuming that these two con-structs are somewhat comparable, the resultshere reveal that participation by mandatoryusers is important if the participation strategyis implemented in such a way that users rec-ognize the instrumentality of their participativeinput on the resulting IS. Hence, Hartwick andBarki are correct in concluding that participa-tion by mandatory users may be ineffective,particularly if the users do not gain a sense ofoverall responsibility (i.e., control). The resultsof the current study support this assertion, as apositive relationship was found between con-trol perceptions and consequential beliefs, atti-tudes, and behavior.

The current study predicted that the non-instrumental voice manipulation would result inimproved user involvement, attitude, and per-formance. While the first two propositions aresupported, the latter is not. That is, perfor-mance gains between the non-instrumentaland no voice treatment conditions are not sig-nificantly different. One explanation for the lackof a performance effect between these twoconditions could be the way in which non-instrumental voice was manipulated in thisstudy. Hunton and Price indicate that the exer-cise of non-instrumental voice can evoke per-ceptions of illusory control. In fact, subjects inthis study indicated a higher level of perceivedcontrol than did subjects in the no voice condi-tion. However, Hunton and Price emphasizethat the impact of non-instrumental voice isstrongest when such voice is expressedthroughout the decision process and usersbelieve they actually participated in the devel-opment process.

In this study, the non-instrumental voice treat-ment was administered after the IS had beendeveloped. When subjects were asked theextent to which they participated in hands-onactivities, mean responses across the non-instrumental voice groups were relatively lowsince they did not express their voice through-out the entire development process.Consequently, for subjects in the non-instruo

mental voice condition, an incongruent situa-tion exists where control perceptions are ele-vated but participation perceptions are not.-I’herefore, it is likely that the one-time expres-sion of non-instrumental voice dampened itspotential effect and that increased control per-ceptions were not strong enough to evoke sig-nificant performance differences.

]there are some limitations to this research.[External validity of study findings may besomewhat limited since this research wasrestricted to a single state agency and culturaland other contextual factors specific to thisorganization may constrain generalizability."][’here is a possibility that training in groupeffectiveness for subjects in the instrumentalvoice condition may be confounding the resultsto some extent. Mean group differences in pre-to post-experiment assessments of userinvolvement and attitude were examined.Changes in individual responses due to anindependence problem were not examined.Hence, it cannot be said that individuals willreact to the experimental manipulations in thesame way as did the groups as a whole.Objective performance data from each fieldoffice were collected~ but it was not possible toidentify performance criteria to individuals with-i,n each field office; accordingly, the perfor-rnance measure impounds various effectsinherent with personnel turnover.

Future research in user participation shouldc, ontinue to test and elaborate the frameworkpresented in this study. In particular, it wouldbe interesting to provide users with non-instru-rnental voice opportunities throughout thedevelopment process to determine the behav°ioral impact. Also, one could examine theeffect of providing various levels of user con-trol at different phases of the systems devel-opment life cycle, such as planning, analysis,design, and implementation. Another fruitfularea for future research would be to examinegroup decision processes and techniques inan effort to maximize group contributions indeveloping accounting and management infor-rnation systems. In particular, researchersc.ould study process gains and losses inherentin group decision making to determinewhether group participation strategies demon-


User Participation

strate significant gains as compared to individ-ual strategies.

There are three broad managerial implicationsthat can be drawn from this study. First, animportant antecedent condition leading to asuccessful participation strategy seems to bethe user’s desire to participate in the develop-ment process. User involvement is a key factorinfluencing desired participation. Post-develop-ment cognitive, affective, and motivational fac-tors can be optimized by providing meaningfulparticipation opportunities to users who deemthe IS to be personally relevant and important.Second, even in situations where user involve-ment is relatively high, low self-efficacy per-ceptions may inhibit the user’s desire to partici-pate in development activities. It is interestingto note that unless and until affected partiesengage in participation activities, self-efficacyperceptions are unlikely to change. Hence,managers should provide users with opportuni-ties to participate in IS development activities,especially where the users are likely to judgethe experience as successful. In this manner,experientially enhanced self-efficacy percep-tions should increase the user’s level of desireto participate in subsequent IS developmentprojects. The final, and perhaps most impor-tant, message from this research is that userparticipation in developing information systemsmay revolve around one core issue: user con-trol. It appears as though a successful partici-pation strategy is one that maximizes theuser’s instrumental control over the proposedsystem. The appropriate participation strategyto employ is contingent on many factors,including the IS context, organizational con-straints, and resources limitations. Judiciouslyselecting a participation strategy that best pro-vides users with a sense of overall responsibil-ity and system ownership is a key IS successfactor.

References

Bandura, A. Social Cognitive Theory, PrenticeHall, Englewood Cliffs, N J, 1986.

Barki, H., and Hartwick, J. "Rethinking theConcept of User Participation," MISQuarterly (13:1 ), March 1989, pp. 53-63.

Barki, H., and Hartwick, J. "Measuring UserParticipation, User Involvement, and UserAttitude," MIS Quarterly (18:1 ), March 1994,pp. 59-82.

Baroudi, J. J., Olson M. H., and Ives, B. "AnEmpirical Study of the Impact of UserInvolvement on System Usage and UserSatisfaction," Communications of the ACM(29:1), 1986, pp. 232-238.

Bentler, P. M. EQS Structural EquationsProgram Manual, BDMP StatisticalSoftware, Los Angeles, 1989.

Bentler, P. M. "Comparative Fit Indexes inStructural Models," Psychological Bulletin(107:2), 1990, pp. 238-246.

Bentler, P. M., and Bonett. D. G. "SignificanceTests and Goodness of Fit in the Analysisof Covariance Structures," PsychologicalBulletin (88:2), 1980, pp. 588-606.

Breckler, S. J. "Applications of CovarianceStructure Modeling in Psychology: Causefor Concern?" Psychological Bulletin(107:2), 1990, pp. 260-273.

Compeau, D. R., and Higgins, C. A. "ComputerSelf-Efficacy: Development of a Measureand Initial Test," MIS Quarterly (19:2),1995a, pp. 189-211.

Compeau, D. R., and Higgins, C. A."Application of Social Cognitive Theory toTraining for Computer Skills," InformationSystems Research (6:2), 1995b, pp.118-143.

Cronbach, L., Gleser G., Nanda H., andRajaratnam N. The Dependability ofBehavioral Measurement: Theory ofGeneralizability for Scores and Profiles,Wiley, New York, 1972.

Cushing, B. E. "Frameworks, Paradigms, andScientific Research in ManagementInformation Systems," Journal ofInformation Systems (4), Spring 1990, pp.38- 59.

DeLone, W. H., and McLean, E. R."Information Systems Success: The Questfor the Dependent Variable," InformationSystems Research (3:1), 1992, pp. 60-95.

Doll, W. J., and Torkzadeh, G. "A DiscrepancyModel of End-User Computing Par-


User Participation

ticipation," Management Science (35:10),1989, pp. 1151-1171.

Earley, P. C., and Lind, Ao E. "ProceduralJustice and Participation in Task Selection:The Role of Control in Mediating JusticeJudgments," Journal of Personality andSocial Psychology (52:6), 1987, pp.1148-1160.

Fishbein, M. "A Theory of Reasoned Action:Some Applications and Implications," inNebraska Symposium on Motivation, (27),H. Howe and M. Page (eds.), University Nebraska Press, Lincoln, NE, 1980.

Fishbein, M., and Ajzen, I. "Attitudes TowardObjects as Predictors of Single and MultipleBehavioral Criteria," Psychological Review(81:1), 1974, pp. 59-74.

Fishbein, M., and Ajzen, I. Belief, Attitude,Intentions and Behavior." An Introduction toTheory and Research, Addison-Wesley,Reading, MA, 1975.

Folger, R. "Distributive and Procedural Justice:Combined Impact of ’Voice’ andImprovement on Experienced Inequity,"Journal of Personality and SocialPsychology (35:2), 1977, pp. 108-119.

Greenberg, J. "Organizational Justice:Yesterday, Today and Tomorrow," Journalof Management (16:1 ), 1990, pp. 399-432.

Hall, J., and Watson, W. H. "The Effects of aNormative Intervention on Group Decision-Making Performance," Human Relations(23:1), 1970, pp. 299-317.

Hartwick, J., and Barki, H. "Explaining the Roleof User Participation in Information SystemUse," Management Science (40:4), 1994,pp. 440-465.

Houlden, P., LaTour, S., Walker, L., andThibaut, J. "Preferences for Models ofDispute Resolution as a Function ofProcess and Decision Control," Journal ofExperimental Social Psychology (14:2),1978, pp. 13-30.

Hunton, J. E. "Involving Information SystemUsers in Defining System Requirements:The Influence of Procedural JusticePerceptions on User Attitudes andPerformance," Decision Sciences (27:4),1996, pp. 617-669.

Hunton, Jo E. "User Participation in DefiningSystem Interface Requirements: An Issue

of Procedural Justice," Journal ofInformation Systems, forthcoming.

Hunton, J. E., and Price, K. H., "A Frameworkfor Investigating Involvement Strategies inAccounting Information SystemsDevelopment," Behavioral Research inAccounting (6), Supplement 1994, pp.128-157.

Innami, I. "The Quality of Group Decisions,Group Verbal Behavior, and Intervention,"Organizational Behavior and HumanDecision Processes (60:3), 1994, pp.409-430.

Ives, B., and Olson, M. H. "User Participationand MIS Success: A Review of Research,"Management Science (30:5), May 1984, pp.586-603.

Lawrence, M., and Low, G. "ExploringIndividual User Satisfaction Within User-Led Development," MIS Quarterly (17:2),June 1993, pp. 195-208.

Lerner, S. C. "Adapting to Scarcity andChange: Stating the Problem," in TheJustice Motive in Social Behavior, M. J.Lerner and S. C. Lerner (eds.), PlenumPress, New York, 1981, pp. 1187-1198.

Leventhal, G. S. "What Should be Done WithEquity Theory? New Approaches to theStudy of Fairness in Social Relationships,"in Social Exchange: Advances in Theoryand Research, K. Gergen, M. Greenberg,and R. Willis (edso), Plenum Press, NewYork, 1980, pp. 27-55. "

Lind, E. A., Kanfer, R., and Earley, P. C."Voice, Control and Procedural Justice:Instrumental and Non-instrumentalConcerns in Fairness Judgments," Journalof Personality and Social Psychology(59:5), November 1990, pp. 952-959.

Lind, E. A., and Tyler, T. R. The SocialPsychology of Procedural Justice, PlenumPress, New York, 1988.

Locke, E. A., and Latham, G. P. A Theory ofGoal Setting and Task Performance,Prentice-Hall, Englewood Cliffs, N J, 1990.

Lodge, L. J. "A User Led Model of SystemsDevelopment," in Participation in SystemsDevelopment, K. Knight (ed.), Kogan Page,London, 1989, pp. 108-153.

Martin, J. An Information Systems Manifesto,Prentice-Hall, Englewood Cliffs, N J, 1984.


User Participation

Mumford, E. "Participative Systems Design:Structure and Method," Systems,Objectives, Solutions (1:1), January 1981,pp. 5-19.

Mumford, E., Bancroft, N., and Sontag, B."Participative Design--Success andProblems," Systems, Objectives, Solutions(6:2), August 1983, pp. 133-141.

Mumford, E., and Henshall, Do A ParticipativeApproach to Computer Systems Design: ACase Study of the Introduction of a NewComputer System, Halsted Press, NewYork, 1979.

Neter, J., Wasserman, W., and Kutner, M. H.Applied Linear Statistical Models, Irwin,Homewood, IL, 1990.

Pettingell, K., Marshall, T., and Remington, W."A Review of the Influence of UserParticipation on System Success,"Proceedings of the Tenth InternationalConference on Information Systems, J. I.DeGross, J. C. Henderson, and B. R.Konsynski (edso), Boston, MA, 1988, pp.227-236.

Thibaut, J., and Walker, L. Procedural Justice:A Psychological Analysis, Erlbaum,Hillsdale, N J, 1975.

Thibaut, J., and Walker, L. "A Theory ofProcedure," California Law Review (66:1),1978, pp. 541-566.

Vroom, V. H., and Jago, A. G. The NewLeadership: Managing Participation inOrganizations, Prentice-Hall, EnglewoodCliffs, N J, 1988.

Wheaton, B. "Assessment of Fit in OverIdentified Models with Latent Variables,"Sociological Methods Research (16:1 ),1987, pp. 118-154.

Wong-On-Wing, B. "User Participation inSystems Development: An Attributional

Approach," Journal of Information Systems(2), Spring 1988, pp. 3-14.

About the Authors

James E. Hunton, Ph.D., C.P.A., is an assis-tant professor of accounting at the Universityof South Florida’s School of Accountancywhere he teaches accounting information sys-tems and information systems auditing cours-es at the undergraduate and graduate levels.Dr. Hunton brings an extensive professionalaccounting and information systems back-ground into his teaching and research endeav-ors. His primary research thrust involvesbehavioral facets of developing, implementing,and auditing accounting information systems.He has research papers accepted and pub-lished in a variety of journals including TheAccounting Review, Management Science,Decision Sciences, Journal of InformationSystems, Advances in Accounting InformationSystems, Journal of Applied Psychology,Journal of Behavioral Decision Making, andBehavioral Research in Accounting.

Jesse D. Beeler, Ph.D., C.P.A., is an assistantprofessor of accounting at Millsaps Collegewhere he teaches financial accounting andaccounting information systems courses at theundergraduate and graduate levels. Dr.Beeler’s research emphasis focuses on behav-ioral decision making in accounting contexts.He has published articles in such journals asAdvances in Accounting, Journal of BehavioralDecision Making, IS Audit & Control Journal,Internal Auditing, Government Accountant’sJournal, and Journal of Accounting Theory andPractice.


User Participation

Appendix AConsensual Conflict Resolutiion (CCR) Instructions

(Hall and Watson 1970; Ilnnami 1994)

The CCR instructions were provided to all team members to increase the reasoning and decrease thepositional orientation of group verbal behavior. Team leaders were requested to continually refer teammembers to the CCR instructions throughout deliberations.

1. Present your position as lucidly and logically as possible.

2. Avoid "win-lose" stalemates in the discussion.

3. Avoid changing your mind only to avoid conflict and to reach agreement and harmony. Withstandpressures to yield.

4. Avoid conflict-reducing techniques such as majority voting, averaging, bargaining, coin flipping, andthe like. If you do not understand an issue or you desire additional discussion, press for additionalsharing of information.

5. View differences of opinion as both natural and helpful rather than as a hindrance in decision making.

6. View initial agreement as suspect. Explore the reasons underlying apparent agreements.


User Participation

Appendix BPretest Instrument Administered at Beginning of Month t_lo

Introductory Statement: Thank you for participating in this survey. Agency management is genuinelyinterested in your answers to the following statements/questions. Your answers will become a criticalpart of the planning process regarding the development of a new expenditure cycle application for theagency. We ask you not to write your name on this survey, as your responses will be completelyanonymous. Your specific responses will not be revealed, rather responses from all participants in thissurvey will be averaged. You are encouraged to be completely open and honest in your answers.

User Involvement (Barki and Hartwick 1994): The following items used a seven-point, bipolar scale.The first set of five items represents the importance subscale of involvement. The second set of fouritems represents the personal relevance subscale of involvement.

Please circle the answer that best describes your beliefs toward the existing expenditure cycle applica-tion used at your field office. I believe the current expenditure cycle application is:

1.* 1 = Nonessential 7 = Essential2. 1 = Trivial 7 = Fundamental3.* 1 = Insignificant 7 = Significant4.* 1 = Unimportant 7 = Important5. 1 = Not Needed 7 = Needed

1. 1 = Irrelevant to me 7 = Relevant to me2. 1 = Of no concern to me 7 = Of concern to me3.* 1 = Doesn’t matter to me 7 = Matters to me4. 1 = Means nothing to me 7 = Means a lot to me

*These items were reverse-scored in the survey instrument.

User Attitude (Barki and Hartwick 1994): The following items used a seven-point, bipolar scale.

Please circle the answer that best describes your attitude toward the existing expenditure cycle appli-cation used at your field office. I believe the current expenditure cycle application is:

1.* 1 = Useless 7 = Useful2.* 1 = Bad 7 = Good3. 1 = Worthless 7 = Valuable4. 1 = Terrible 7 = Terrific



User Participation

User Self-Efficacy (Bandura 1986; Hunton forthcoming): The following items used a seven-point,bipolar scale.

If you were allowed to participate in developing accounting and management information systemsapplications, to what extent do you believe you could effectively contribute in the following areas:

1.* Defining report formats (1 = Not at all, 4 = Much, 7 = A great deal)2. Defining screen layouts (1 = Not at all, 4 = Much, 7 = A great deal)3. Defining input and output forms (1 = Not at all, 4 = Much, 7 = A great deal)4.* Creating user procedural manuals (1 = Not at all, 4 = Much, 7 = A great deal)


Desired Participation (Barki and Hartwick 1994; Doll and Torkzadeh !989): The following itemsused a seven-point, bipolar scale with a mid-point reference.

As the new expenditure cycle application is being developed, to what extent do you desire to partici-pate in each of the following activities:

Defining report formatsDefining screen layoutsDefining input and output formsCreating user procedural manuals

(1 = Not at all, 4 = Much, 7 = A great deal)(1 = Not at all, 4 = Much, 7 = A great deal)(1 = Not at all, 4 = Much, 7 = A great deal)(1 = Not at all, 4 = Much, 7 = A great deal)

*This item was reverse-scored in the survey instrument.

Demographic Questions:

Please indicate your age. __

Please indicate your gender ~ (Female) __ (Male)

What is the highest year of education you have completed? __

How many years experience do you have working in the fiield of accounting? __

How many years have your worked for this state agency? ~


User Participation

Appendix CPost-Test Instrument Administered at the End of Month

Introductory Statement; Thank you for participating in this survey. Agency management is genuinelyinterested in your answers to the following statements/questions. Your answers will become a criticalpart of the planning process for future development of new accounting and management informationsystems applications throughout the agency. We ask you not to write your name on this survey, asyour responses will be completely anonymous. Your specific responses will not be revealed, ratherresponses from all participants in this survey will be averaged. You are encouraged to be completelyopen and honest in your answers.

User Involvement (Barki and Hartwick 1994):

Please circle the answer that best describes your beliefs toward the recently developed expenditurecycle application used at your field office. I believe the new expenditure cycle application is:

Reader Note: The same nine adjective pairs used in the pre-experiment questionnaire (see AppendixB) were also used in the post-experiment survey.

User Attitude (Barki and Hartwick 1994):

Please circle the answer that best describes your attitude toward the recently developed expenditurecycle application used at your field office. I believe the new expenditure cycle application is:

Reader Note: The same four adjective pairs used in the pre-experiment questionnaire (see AppendixB) were also used in the post-experiment survey.

Manipulation Checks:

1. Manipulation check for Instrumental Voice Participation (Barki and Hartwick 1994; Doll andTorkzadeh 1989):

During the process of developing the new expenditure cycle application, to what extent did you actuallyparticipate in each of the following activities:

Defining report formatsDefining screen layoutsDefining input and output formsCreating user procedural manuals




User Participa~on

2. Manipulation check for Perceived Control (Leventha11980):

To what extent do you feel you had personal control over the defining the following attributes of thenew expenditure cycle application?

1. Report formats2. * Screen layouts3. Input and output forms4.* User procedural manuals


3. Manipulation check for Non-Instrumental Voice Participation treatment condition:

Did you participate in a six-hour meeting called "Expenditure Cycle Application Orientation Meeting?"(Yes/No)

4. Manipulation check for Training manipulation administered to all users:

Did you participate in an eight-hour meeting called "Expenditure Cycle User Training Meeting?"(Yes/No)

5. Manipulation check administered to determine independence between treatments:

As far as I know, accounting clerks throughout the agency were not allowed to participate in theprocess of developing the new expenditure cycle application (1 = Agree, 4 = Not Sure, 7 = Disagree).


388 MIS Quarter/y/December 1997

Documents

Effects of User Participation in Systems Development: A ... · User Participation Effects of User Participation in Systems Development: A Longitudinal Field ExperimentI By: James