Self-efficacy, learning method appropriation and software skills acquisition in learner-controlled CSSTS environments

Self-efficacy, learning method appropriationand software skills acquisition inlearner-controlled CSSTS environmentsisj_12016 3..27

Andrew M. Hardin,* Clayton A. Looney† & Mark A. Fuller‡

*Lee Business School, University of Nevada, Las Vegas, Nevada, USA, email:[email protected], †College of Business, University of Montana, Missoula,Montana, USA, email: [email protected], and ‡School of Management,University of Massachusetts, Amherst, Massachusetts, USA, email: [email protected]

Abstract. A computer-simulated software training system (CSSTS) delivers aspecific form of computer-based training in which participants are allowed to selectvarious training features within a simulated software environment. Given thegrowing use of these systems as end-user training (EUT) aids, there is a need forgreater understanding of how participants use these systems, as well as whetherparticipant-controlled learning environments are truly effective. The presentresearch examines how a particular learner characteristic, software self-efficacy,drives appropriation in a high learner control, CSSTS environment. Contrary tonotions in the literature, results from spreadsheet and database software trainingcourses reveal that pre-training specific software self-efficacy constitutes a signifi-cant, negative predictor of faithful appropriations of the CSSTS. This research alsoestablishes a positive relationship between faithful appropriation and increases insoftware self-efficacy (SSE). In essence, faithful appropriations lead to greaterincreases in SSE, which influences software skills performance. In addition, theresearch validates prior EUT research by extending prior findings to a databasetraining environment. A psychometrically sound measure is put forth to capturedatabase self-efficacy.

Keywords: computer-simulated software training system, end-user training,adaptive structuration theory, appropriation, software self-efficacy, databaseself-efficacy.

INTRODUCTION

Organisations continue to make substantive investments in employee training, highlighting apersistent desire to develop and maintain a highly skilled workforce in today’s competitivemarketplace. In 2009, organisations with over 100 employees spent in excess of $52.8 billion

doi:10.1111/isj.12016

Info Systems J (2014) 24, 3–27 3

© 2013 Wiley Publishing Ltd

on employee training, or $1041 per employee. Fourteen per cent of these expenditures wereused to fund computer skills training [Anonymous, 2010]. While these numbers were slightlyhigher than 2008, instructor-led training fell sharply from the previous year (from 47% to27.7%). To reduce costs associated with instructor-led training, organisations expended$562 934 on learning tools and technologies, up nearly $100 000 from the prior year. Corre-spondingly, organisations reduced training professionals salaries by an average of $2000[Anonymous, 2010].

These data reflect the pace at which organisations offset accelerating expenditures forsoftware skills instruction through the implementation of e-learning self-study software appli-cations, which accounted for over 30% of student training hours in 2009 [Anonymous, 2010].A computer-simulated software training system (CSSTS) represents a specific type ofe-learning self-study system that has become immensely popular for facilitating softwareinstruction. One CSSTS was used by over 86 000 trainees in 2005 alone [Speckler, 2006].CSSTSs facilitate the simultaneous instruction of large numbers of participants, reducinginstructor preparation time1 and allowing organisations to cost-effectively train participants withvarious degrees of prior software experience [Speckler, 2006]. When using these systems,participants are presented with instructional features, which they are free to use as frequentlyor infrequently as they wish, employing their own strategy for practicing the targeted softwareskills. Given the vast flexibility provided to users during the learning process, such systemsoffer a high degree of learner control.

Despite the widespread use of CSSTSs, relatively little is known about their impact onsoftware training success [Brown, 2001; Yi & Davis, 2001; DeRouin et al., 2004]. Although agreat deal of end-user training (EUT) research has been conducted [Olfman & Bostrom, 1991;Davis & Bostrom, 1993; 1994; Bostrom et al., 1994; Olfman & Mandviwalla, 1994; 1995], muchof this research involves human-led instruction followed by hands-on use of the target system.CSSTSs constitute qualitatively different settings, as these systems represent a specifictechnology-mediated learning (TML) application that facilitates a ‘learning from computers’environment [Gupta & Bostrom, 2009, p. 694]. To our knowledge, only one study has exam-ined learning outcomes in a CSSTS setting [Gupta, 2008]. Because Gupta’s [2008] studyfocused exclusively on spreadsheet training, it remains to be seen whether the findings can beextended to other software training contexts. Moreover, no study to date has investigated howand why users adopt particular instructional features in CSSTS training environments, orwhether software skills acquisition depends on such interactions. This research addressesthese gaps in the literature.

Laying the conceptual groundwork for this effort, Gupta & Bostrom [2009] put forth a modelof TML that invokes both social cognitive theory (SCT) and adaptive structuration theory (AST)to explain how users acquire software skills. The current study builds on this research byinvestigating software self-efficacy, an important component within SCT, as both a predictor ofCSSTS feature use and as a learning outcome in high learner control environments. Comple-

1Although not required to teach the material, in CSSTS environments, instructors must set up student access, implement

the respective training modules, track computer-generated test scores, etc.

4 A M Hardin et al.

© 2013 Wiley Publishing Ltd, Information Systems Journal 24, 3–27

menting SCT, AST provides an overarching perspective for understanding the relationshipbetween software self-efficacy and faithfulness of appropriation (FOA), defined in this contextas the combined use of the CSSTS features. Further, AST helps explain how this faithfulappropriation influences subsequent software self-efficacy perceptions and, ultimately, soft-ware skills acquisition. Examining the interplay among software self-efficacy, FOA and traineeperformance informs and extends the conceptual relationships proposed in the TML literature,while simultaneously addressing the need for research investigating the agents of appropria-tion in EUT settings [Gupta, Bostrom, & Huber, 2010].

The following questions are addressed by this research: (1) Does pre-training softwareself-efficacy influence appropriation in high learner control, CSSTS environments? (2) Doesfaithful appropriation in high learner control CSSTS environments lead to increases in softwareself-efficacy? (3) Can findings from prior research in high learner control CSSTS environmentsbe generalised beyond spreadsheet software training?

The remainder of this paper is organised as follows. First, brief overviews of SCT and ASTare provided. Hypotheses are then developed. Next, we describe our research design, includ-ing a methodology section and a discussion of our analyses and results associated with twoseparate studies. Finally, we review the theoretical implications of our findings, discuss thestudy’s limitations, and explore potential avenues for future research.

THEORY

SCT

SCT proposes a triadic reciprocal relationship between the person, their behaviour and theenvironment. Residing within the person element of SCT, self-efficacy is defined as ‘a belief inone’s capability to organize and execute the courses of action required to produce givenattainments’ [Bandura, 1997, p. 3]. The specificity of self-efficacy beliefs can range fromgeneral (e.g. computer self-efficacy) to specific (e.g. software self-efficacy) [Compeau &Higgins, 1995; Compeau, Gravill, Haggerty, & Kelly, 2006]. As a self-regulatory process,people’s self-efficacy beliefs influence the actions they pursue, the effort they put forth, theirpersistence against repeated failure and control of their cognitive functioning, such as sus-ceptibility to self-debilitating thoughts [Bandura, 1997].

The literature identifies four primary sources of self-efficacy [Gist & Mitchell, 1992; Bandura,1997]. Enactive mastery, considered the strongest source of self-efficacy information[Bandura, 1997; Pajares, 2004], is acquired through prior experience in a given domain[Compeau & Higgins, 1995] or through hands-on training in which the participant is performingthe target behaviour [Bandura et al., 1977]. Vicarious experience, the second source ofself-efficacy information, can be delivered in two ways: through behavioural modelling or verbalguidance. Behavioural modelling provides training information by observing another whodemonstrates the desired skills or behaviours [Yi & Davis, 2001]. Verbal guidance providestraining information by giving either verbal or written instructions about how to accomplish a

Learning method appropriation 5


behaviour. Thus, verbal guidance ‘describes rather than shows how to perform given activities’[Bandura, 1997, p. 371]. Because verbal guidance can also be presented in written form, it canbe used as a means of retention enhancement following behavioural modelling demonstrations[Yi & Davis, 2001]. Not to be confused with verbal guidance, verbal persuasion represents thethird source of efficacy information. Verbal persuasion provides evaluative feedback throughverbal cues. Depending on how the feedback is conveyed, self-efficacy beliefs can either beweakened or bolstered. In contrast to the instruction-oriented focus of verbal guidance, verbalpersuasion focuses on the provision of performance feedback. Finally, physiological states actas the fourth source of self-efficacy information by providing cues that influence a person’sbelief in his or her capabilities. For example, muscle pain during physical activities or stresscan reduce one’s sense of self-efficacy. These four sources of self-efficacy informationcombine through a complex cognitive integration process, where individuals weigh thesesources and use them in the development of self-efficacy perceptions [Bandura, 1997].

AST

AST was originally proposed to integrate the decision-making and institutional schools ofthought guiding information systems (IS) research [DeSanctis & Poole, 1994]. Drawing uponstructuration theory [Giddens, 1984], AST proposes that advanced IS are designed such thatthey reinforce certain social structures. A system’s ‘spirit’, defined as ‘the general intent withregard to the values and goals underlying a given set of structural features’ [DeSanctis &Poole, 1994, p. 126], represents one method for describing the social structures embedded inadvanced information technologies. Originally designed for understanding how teams appro-priate group decision support systems, AST has more recently been applied in individual-levelTML models. Depending on factors such as aptitude, motivation, or self-efficacy, learners maychoose to appropriate an advanced technology either faithfully or unfaithfully to the system’sspirit [Gupta & Bostrom, 2009].

Appropriation moves provide a useful way of organising how users adopt structural features.DeSanctis & Poole [1994] identify four general appropriation moves: directly adopting thestructures, relating structures to other structures, constraining or interpreting the structures ormaking judgments regarding the usefulness of the structures. Within these general categories,specific types of appropriation moves are further defined. These include directly adoptingstructures through direct appropriation, relating structures to other structures through combi-nation, substitution, enlargement, and contrast, constraining or interpreting the structuresthrough constraint or making judgments regarding the usefulness of the structures through theuse of affirmation, negation and neutrality.

Learning dimensions, features and efficacy sources

SIMNET™ represents one CSSTS frequently used for software training. Unlike traditional TML,SIMNET™ provides a simulated experience with the software package in which various trainingfeatures are available for use as the trainee deems appropriate. In this simulated environment,

6 A M Hardin et al.


functions normally included in the actual software package (e.g. the help feature) are renderedinactive. SIMNET™ does not limit the number of times a particular feature may be used, nordoes it restrict the order in which the various features can be accessed. Thus, the systemfacilitates a high learner control, learning from computers environment [Gupta & Bostrom,2009; Gupta, Bostrom, & Huber, 2010].

Technology structures can be described in terms of their features and dimensions[DeSanctis & Poole, 1994]. Learning dimensions provided by these technology structures arescalable [Gupta & Bostrom, 2009]. Gupta & Bostrom [2009] map the scalability of specificlearning dimensions to video- and web-based training. Specifically, the findings of Gupta[2008] are used to propose that scalability of the learning dimensions production pattern (thetiming of declarative knowledge), structuredness of practice (specificity of practice), restric-tiveness of practice (whether or not the practice was guided), feedback (immediacy of feed-back), and guidance (how closely the guidance was mapped to the leaning goals) is generallyhigher for web-based training compared with video training. We build on and extend thesepropositions by more precisely examining the respective learning dimensions, their scalabilityand the features used to deliver them within CSSTS environments. In particular, we breakdown the learning structures delivered by the CSSTS2 to show that the learning dimensionsdescribed by Gupta [2008] vary in scalability across the respective features.

SIMNET™ targets MS Office™ software skills acquisition. The CSSTS includes Teach Me,Show Me and Let Me Try features. The Teach Me feature provides written instructions usingtext-based sidebars and graphical callouts. Through text-based instruction, this feature pro-vides an experience similar to the self-efficacy source, verbal guidance. The Show Me featuredemonstrates skills using animation and voice, delivering an experience similar to behaviouralmodelling training. The Show Me feature also manipulates the cursor to visually demonstratethe training exercises for participants. Finally, the Let Me Try feature allows learners to attemptthe simulated software exercises unaided, and provides participants with system-generatedmessages, indicating whether the exercise has been completed successfully or unsuccess-fully. Because the exercises are similar to completing unassisted exercises using an actualsystem, the Let Me Try feature provides a learning experience that delivers the most potentform of self-efficacy building source, enactive mastery (i.e. unaided hands-on experience),followed by verbal persuasion (i.e. feedback acknowledging whether the task was completedcorrectly). Table 1 outlines the relationship between the SIMNET™ features, the respectivesources of self-efficacy that the features provide and the learning dimensions proposed byGupta & Bostrom [2009].

Learning dimensions, features, appropriation moves and frequency of use

AST suggests that designers’ intentions partly determine a system’s spirit. Additional factors,such as the design metaphor, system features, user interface and training materials, should

2The system described by Gupta [2008] appears to include features similar to those provided by the CSSTS used in the

current study. In particular Gupta [2008] describes features called ‘Show Me’ and ‘I do’. These features appear similar to

the SIMNET™ features Show Me and Let Me Try described in the subsequent paragraph.



also be considered by the researcher when determining the system’s spirit. Based upondiscussions with the instructional designers, an extensive review of the system, and a carefulexamination of the associated user manuals and training materials, SIMNET™ was determinedby the authors to portray a spirit consistent with the theoretical underpinnings of SCT, providinga theoretical linkage between the training system and learning effectiveness. In addition, thereading of the system spirit suggested that the respective features should be used incombination to optimise the benefits of the learning dimensions described in Table 1. Thus,

Table 1. Learning dimensions, features and efficacy sources

Learning dimensions*

Teach Me (verbal

guidance)

Show Me (behavioural

modelling)

Let Me Try (enactive

mastery) Feature combination

ProductionPattern

Delayed Production Delayed Production Delayed Production Immediate Production

The lag between

demonstration of an

action and practice

by the learner.

Feature provides written

instructions using

text-based sidebars and

graphical callouts.

Students do not practice

with feature.

Demonstrates skills using

animation and voice.

Manipulates the cursor to

visually demonstrate the

training exercises.


with feature.

Learners practice specific

tasks, but can complete

them on their own using

techniques not

demonstrated by the

system.

Students can use the Let

Me Try feature

immediately following

instruction by the Teach

Me or Show Me features.

Structuredness ofPractice

Low Low Low High

The extent to which

technology imposes

its procedures on

the learner.

Feature provides textual

guidance using specific

steps. Students do not

practice with feature.

Feature demonstrates the

skill using specific steps.


with feature.

Learners practice specific

tasks, but can complete

them on their own using

techniques not

demonstrated by the

system.

Students can follow

system instructions when

completing tasks.

Restrictiveness ofPractice

Low Low Low High

The degree to

which a system

limits an action.


instructions only. Students

do not practice with

feature.


skill only. Students do not


Students can complete

the exercises on their own

using techniques not

demonstrated by the

system.

Students can follow

system instructions when

completing tasks.

Feedback Low Low Moderate High

The degree to

which a system

provides a

response, including

correction, addition

or approval and

speed of response.


instructions. Students do

not practice with feature.


skill. Students do not


Participants are provided

feedback that the exercise

has been completed

correctly or incorrectly.

Does not enforce

procedures taught using

Show Me and Teach Me.

The Let Me Try Feature

provides feedback that the

exercise has been

completed correctly or

incorrectly. Students can

use procedures taught

using Show Me and

Teach Me.

Guidance Moderate Moderate Moderate High

The degree to

which a system

provides direction

or advice towards a

course of action.


instructions that are

directly linked to learning

goals.

Feature demonstrates

skills directly linked to

learning goals.

Students can complete

the exercises on their own

using techniques not

demonstrated by the

system.

Teach Me and Show Me

provide specific guidance

on how to complete a

task. Let Me Try allows

students to complete

tasks using those

procedures.

*Gupta & Bostrom [2009].

8 A M Hardin et al.


FOA in the current context can be captured by assessing how frequently participants rely onall of the features.

AST suggests that users may invoke one of several appropriation moves. Appropriationmoves provide a useful method for understanding whether features are appropriated faithfullyor unfaithfully according to the system’s spirit. However, AST has generally focused onunderstanding group level appropriation in decision support settings [DeSanctis & Poole,1994]. Given the inability to analyse group level discourse in individual-level training settings,assessing appropriation requires a different approach. One method that may be well suited forthese environments involves analysing participants’ behaviours during training sessions. Inparticular, how appropriation moves relate to the frequency of feature use. Table 2 details therelationships between learning dimensions, CSSTS features, appropriation moves and fre-quency of use. We refer to these respective relationships when developing our hypotheses inthe following.

HYPOTHESES AND THEORY DEVELOPMENT

Software self-efficacy, feature selection and use

Individuals with low general computer self-efficacy are less willing to attempt hands-on trainingthan those with high general computer self-efficacy [Gist et al., 1989]. Although less studied,the relationship can also be expected to hold for software self-efficacy. General computerself-efficacy and software self-efficacy operate on similar principles, merely at different levelsof abstraction [Marakas et al., 1998]. Because low pre-training software self-efficacy (PreSSE)participants doubt their abilities, they will likely gravitate towards features that guide themthrough the learning process. Rather than attempting to complete a task unaided, tentativelearners will utilise features that demonstrate tasks or deliver written explanations. Through theuse of these guided features, low PreSSE participants will become more confident in theirsoftware skills. As confidence builds, these users will eventually migrate towards the use of thehands-on, enactive mastery feature, which allows them to perform the task on their own. Anexample of an individuals with low presentation self-efficacy can help illustrate this behaviour.Because of low self-efficacy, such individuals will rely heavily on supporting aids such astext-laden slides or detailed speaker notes. Based on successful presentation experiencesover time, presentation self-efficacy beliefs will increase. As the individuals become moreconfident in their abilities, they will be more assured about delivering presentations withoutaids.

Trainees with robust PreSSE beliefs are unlikely to be fazed by the prospect of completingtasks unassisted. Given their highly confident ability beliefs, they will likely bypass features thatmerely explain or demonstrate software skills. Rather than reviewing skills they believe theyalready possess, they will seek out hands-on, enactive learning training methods. In essence,PreSSE will be inversely related to faithful appropriations of the CSSTS. Weaker PreSSEbeliefs will lead to a greater utilisation of the training features embedded in a CSSTS than willstronger PreSSE beliefs.



H1: PreSSE will be a significant, negative predictor of faithfully appropriating the CSSTS.

CSSTS training effectiveness

Gupta & Bostrom [2009] distinguish between structural features and dimensions. Structuralfeatures in an IS context reflect functionalities, whereas structural dimensions reflect whatthe features do for the actor/learner. In the current context, the set of features are Teach

Table 2. Learning dimensions, features, appropriation moves and frequency of use

Learning dimensions* Feature combination Appropriation moves Frequency of use

Production Pattern Immediate Production Based on the reading of the

SIMNET™ spirit,

constraining the structures

by favouring one structure

over the other results in

unfaithful appropriation.

Relying primarily on the use

of one feature is consistent

with constraining the

structures by favouring one

structure over the other.

The lag between

demonstration of an action

and practice by the learner.

Students can use the Let

Me Try feature immediately

following instruction by the

Teach Me or Show Me

features.

Based on the reading of the

SIMNET™ spirit, faithful

appropriation requires

combining the learning

structures by relating the

structures to other

structures.

Relying on the use of all

three features allows

participants to combine the

learning structures by

relating structures to other

structures.Structuredness of Practice High

The extent to which

technology imposes its

procedures on the learner.

Students can follow system

instructions when

completing tasks.

Restrictiveness of Practice High

The degree to which a

system limits an action.

Students can follow system

instructions when

completing tasks.

Feedback High


system provides a response,

including correction, addition

or approval and speed of

response.

Students receive feedback

on whether a task has been

completed correctly or

incorrectly when following

the instructions provided by

the Teach Me and Show Me

features.

Guidance High


system provides direction or

advice towards a course of

action.

Students can follow the

instructions provided by the

Teach Me and Show Me

features when completing

the exercises using the Let

Me Try feature.

*Gupta & Bostrom [2009].

10 A M Hardin et al.


Me, Show Me and Let Me Try, and these features deliver structural dimensions affiliatedwith the self-efficacy building sources of verbal persuasion, vicarious experience and enac-tive learning. As described in Table 1, using a combination of instructional features producesmore profound results across each learning dimension (production pattern, structuredness ofpractice, restrictiveness of practice, feedback and guidance). Therefore, faithfully appropri-ating the CSSTS should produce a positive effect on learning outcomes [Gupta & Bostrom,2009; Gupta, Bostrom, & Huber, 2010; Thomas et al., 2008]. By relating the structural fea-tures to others structural features, combined feature use delivers a high level of the struc-tural dimensions positively related to leaning outcomes.

Prior research indicates that some learners initially overestimate their abilities [Zimmerman,1995]. Episodes of overestimation occur more frequently when learner control is increased[Piccoli et al., 2001]. Participants who initially overestimate their ability tend to exclude impor-tant facts in high learner control environments [Williams, 1996]. We embrace and extend thisline of reasoning by suggesting that because of an overestimation of ability, high PreSSEparticipants will make inaccurate judgments regarding the usefulness of the learning structuresprovided by the CSSTS. Given robust beliefs in their abilities, these individuals may mistakenlyshun the use of the Teach Me and Show Me vicarious experience features, instead choosingto complete the exercises relying primarily upon the Let Me Try, enactive mastery feature,which allows them to tackle the task on their own. By invoking appropriation moves that favourthe enactive mastery feature over the vicarious experience features, high PreSSE participantswill likely miss important pieces of self-efficacy building information delivered by the respectivelearning dimensions. Because high PreSSE participants overestimate their abilities, they willconstrain the CSSTS structures by favouring one structure over the other, resulting in anunfaithful appropriation. Because low PreSSE participants are more likely to faithfully appro-priate the CSSTS than high PreSSE participants, low PreSSE users will experience a greaterincrease in post-training software self-efficacy (PostSSE).

H2: Through faithful appropriation of the CSSTS, low PreSSE participants will realise agreater increase in software self-efficacy than will high PreSSE participants.

Over three decades of SCT research confirm that when properly isolated within the domain ofinterest, the self-regulatory process of self-efficacy predicts performance [Bandura, 1997].Although well established in the self-efficacy literature, we evaluate this relationship to assessthe meaningfulness of the changes in software self-efficacy following the computer simulatedsoftware training.

H3: PostSSE will be a significant, positive predictor of objective software skills performance.

METHODOLOGY

Two studies were undertaken to test the research hypotheses. Study 1 examines the hypoth-esised relationships during SIMNET™ Excel spreadsheet training, whereas Study 2 attempts to



replicate these relationships during SIMNET™ Access database training. Such an approachenables us to assess whether the proposed relationships hold across disparate types ofsoftware, lending additional credence to our results.

Study 1

Participants included 207 undergraduates enrolled in a pre-certification IS course at a largeNorth American university. Participants utilised SIMNET™ as their exclusive means for learningspreadsheet skills. The spreadsheet training took place 1 hour and 15 minutes per day, 3 daysa week, for 2 weeks. Each day, participants entered a computer training facility in whichSIMNET™ had been installed. After entering their respective login credentials, participantswere provided with the opportunity to complete a series of spreadsheet exercises. Consistentwith high learner control environments, students could practice the exercises as many times asthey wished, using any of the available features (i.e. Teach Me, Show Me, Let Me Try) duringthe process. Upon completion of the training session, participants were required to take anobjective hands-on spreadsheet assessment, which was designed by the faculty and deliveredby the system. The assessment represented a significant portion of the students’ final grade inthe course. While graduate assistants were present during lab sessions, their role involvedtaking attendance, providing non-task-related technology support and dismissing the class forthe day. Other than through the CSSTS, no instructional advice was provided during thesoftware training.

Task

Participants were given access to a series of computer-simulated spreadsheet exercisesprovided by the CSSTS. Exercises varied in difficulty and were designed specifically to prepareparticipants for an objective, hands-on spreadsheet skills assessment. Practice exercisesranged from the generation of simple graphs to the creation of complex formulas. Collectively,the exercises cultivated the skills necessary for using spreadsheets in a business setting.Providing an objective measure of performance, the post-training spreadsheet skills assess-ment consisted of hands-on exercises, which evaluated participants’ cumulative understandingof the software. Arrived at through faculty consensus to provide a sufficient level of challengefor business users, the difficulty of the spreadsheet training was consistent with expectationsof learning for the spreadsheet software portion of the course.

Sample demographics

The sample consisted of participants enrolled in a business school pre-certification courserequired of all business majors. The average age of the participants was 20.1 years. Genderwas distributed as 46% female and 54% male. Self-reported majors were approximately equalacross the disciplines, with ‘undecided’ being reported slightly more often than the others.



Participants reported an average spreadsheet software experience level of 57.44, measuredon a 0–100 scale anchored by ‘no experience’ (0) and ‘very experienced’ (100).

Measures

Surveys were administered both pre- and post-training. PreSSE was measured before thespreadsheet training commenced, while the measures for FOA and PostSSE were adminis-tered after the 2-week training, immediately prior to the spreadsheet skills assessment.

To control for individual differences as proposed by Gupta & Bostrom [2009], computeranxiety (affective states), outcome expectancies (motivation), grade point average (GPA;aptitude) and gender were entered as covariates in the respective analyses. Computer anxietyand outcome expectancies were measured using validated measures [Compeau & Higgins,1995; Compeau, Higgins, & Huff, 1999].

PreSSE and PostSSE were measured using an existing spreadsheet self-efficacy measure[Johnson & Marakas, 2000]. The response format for the spreadsheet self-efficacy measureutilised a two-part question typical of efficacy measures [Bandura, 2005].

FOA was measured using a three-item, seven-point Likert-type scale developed by theauthors. Consistent with our conceptualisation of appropriation, the items captured the degreeto which the SIMNET™ features were collectively utilised.

Finally, user performance was measured objectively by the spreadsheet skills assessment,administered and scored by the CSSTS. This test entailed the hands-on completion of a set ofspreadsheet tasks designed to evaluate participants’ cumulative spreadsheet abilities. Theinstrument items are listed in Table 3.

Results

Measurement model results. Other than FOA, study 1 utilised previously validated existingmeasures. As evidenced by the factor loadings provided in Table 4, exploratory factor analysisrevealed that all items loaded higher on their target constructs than on any other construct.Cronbach’s alpha was 0.71, 0.96 and 0.93 for FOA, PreSSE and PostSSE, respectively.Respective reliabilities for computer anxiety and outcome expectancies were 0.78 and 0.91.Consequently, the measures demonstrated acceptable levels of validity and reliability. Tofacilitate the analyses of the relationships among variables, composite scores were created foreach multi-item measure.

Hypothesis 1. Hypotheses 1 proposed a significant, inverse relationship between PreSSE andFOA. Specifically, the hypothesis predicted that individuals with weaker PreSSE beliefs wouldmore faithfully appropriate the CSSTS than trainees with robust PreSSE beliefs. Given theanticipated linear relationships between PreSSE and FOA, the data were examined viaregression. PreSSE and the covariates identified by Gupta & Bostrom [2009] were entered asindependent variables, whereas FOA served as the dependent variable. Referring to Table 5,one covariate, computer anxiety (b = 0.16, t (201) = 2.17, p < 0.05) and PreSSE (b = -0.27, t



(201) = 3.77, p < 0.001) proved to be significant predictors of FOA. As evidenced by thenegative beta weight, participants with more fragile PreSSE beliefs more faithfully appropriatedthe CSSTS features than trainees with stronger PreSSE beliefs. Thus, H1 was supported.

Hypothesis 2. Hypothesis 2 proposed that through faithful appropriation of the CSSTS, lowPreSSE participants would realise a significantly greater increase in spreadsheet self-efficacythan high PreSSE participants. To evaluate the influence of FOA on spreadsheet self-efficacyacross the low and high PreSSE groups, a dichotomisation of the sample was necessary.

Table 3. Measurement items

Construct Item

Spreadsheet

self-efficacy

I believe I have the ability to manipulate the way a number appears in a spreadsheet.

I believe I have the ability to use and understand the cell references in a spreadsheet.

I believe I have the ability to enter numbers in a spreadsheet.

I believe I have the ability to summarise numeric information using a spreadsheet.

I believe I have the ability to use a spreadsheet to communicate numeric information to others.

I believe I have the ability to use a spreadsheet to display numbers as graphs.

I believe I have the ability to use a spreadsheet to assist me in making decisions.

I believe I have the ability to write a simple formula in a spreadsheet to perform mathematical

calculations.

I believe I have the ability to use a spreadsheet to share numeric information with others.

Database

self-efficacy

I believe I have the ability to customise the display format for a field in a table.

I believe I have the ability to build a report.

I believe I have the ability to write a query using the query design wizard.

I believe I have the ability to customise the data type of a field in a table.

I believe I have the ability to import data into a database.

I believe I have the ability to password protect a database.

I believe I have the ability to update records in a table.

I believe I have the ability to build a form.

Faithfulness

of appropriation

How often did you use the SIMNET ‘Show Me’ feature when preparing for the Excel (Access)

skill assessment?

How often did you use the SIMNET ‘Teach Me’ feature when preparing for the Excel (Access)

skill assessment?

How often did you use the SIMNET ‘Let Me Try’ feature when preparing for the Excel (Access)

skill assessment?

Computer

anxiety

I feel apprehensive about using computers.

It scares me to think that I could cause the computer to destroy a large amount of information

by hitting the wrong key.

I hesitate to use a computer for fear of making mistakes I cannot correct.

Computers are somewhat intimidating to me.

Outcome

expectancies

I would be better organised.

I would increase my effectiveness on the job.

I would spend less time on my routine job tasks.

I would increase my sense of accomplishment.

I would increase my chance of obtaining a promotion.

I would increase my chance of getting a raise.



Table 4. Study 1 factor loadings

Component

1 2 3 4

Excel 1 0.817 0.127 -0.043 -0.136

Excel 2 0.812 0.178 -0.093 -0.127

Excel 3 0.807 0.109 -0.127 -0.100

Excel 4 0.799 0.055 -0.092 -0.031

Excel 5 0.789 0.075 -0.120 -0.027

Excel 6 0.783 0.149 -0.124 0.067

Excel 7 0.780 0.104 -0.118 -0.040

Excel 8 0.767 0.039 -0.152 0.033

Excel 9 0.691 -0.054 -0.054 -0.083

OE 1 0.123 0.849 -0.066 0.011

OE 2 0.045 0.833 0.033 -0.014

OE 3 0.160 0.807 -0.017 0.033

OE 4 0.119 0.789 -0.132 0.026

OE 5 0.102 0.781 -0.074 0.116

OE 6 0.006 0.751 0.059 -0.241

CA 1 -0.095 -0.045 0.812 0.134

CA 2 -0.192 -0.022 0.808 0.093

CA 3 -0.189 -0.080 0.789 0.025

CA 4 -0.097 -0.011 0.658 0.129

FOA1 -0.150 0.010 0.147 0.818

FOA2 -0.177 -0.080 0.102 0.793

FOA3 0.057 0.046 0.113 0.724

Bold represents relevant construct item loadings.

Table 5. Hypothesis 1 – regression results

b t Supported

Study 1

Hypothesis

H1 -0.270 3.77*** Yes

Covariates

Computer anxiety 0.160 2.17*

Study 2

Hypothesis

H1 -0.260 3.44*** Yes

Covariates

Outcome expectancies 0.137 2.34*

GPA -0.131 2.24*

Gender 0.200 3.22**

***p < 0.001.

**p < 0.01.

*p < 0.05.



Similar to Gist et al. [1989], who investigated the influence of alternative training methods onthe development of computer self-efficacy by comparing the upper and lower sample thirds[Gist et al., 1989], we dichotomised the sample based upon PreSSE scores. A median splitwas used to retain the entire sample, as well as to provide a more conservative test of theresearch hypotheses than would have a split based upon quartiles or thirds.

An analysis of covariance (ANCOVA) compared the differences in spreadsheet self-efficacyacross the low and high PreSSE groups. Table 6 contains the results. No covariates weresignificant. Low PreSSE participants realised a greater increase in spreadsheet self-efficacy(PreSSE = 54.47, PostSSE = 69.53) than did high PreSSE participants (PreSSE = 89.63,

Table 6. Results for study 1 – Hypothesis 2

Independent variable n

Software self-efficacy Type III fixed effects results

M(pre) M(post) df F p h2

Participant type 201 64.08 0.000 0.24

Low PreSSE 103 54.47 69.53

High PreSSE 104 89.63 85.50

Covariate n Estimate SE df t p h2

Gender 207 -0.209 2.401 201 -0.087 0.931 0.000

GPA 207 1.230 1.088 201 1.131 0.259 0.006

Computer anxiety 207 -0.278 0.959 201 -0.290 0.772 0.000

Outcome expectancies 207 2.303 1.561 201 1.476 0.142 0.000

Independent variable n M(pre) M(post) df F p h2

Appropriation (low PreSSE) 94 2.900 0.092 0.030

Low 48 58.36 69.95

High 52 50.88 69.15


Gender 100 1.397 4.282 94 -0.326 0.745 0.001

GPA 100 1.899 2.066 94 0.919 0.360 0.009

Computer anxiety 100 0.013 1.603 94 0.008 0.994 0.000



Appropriation (high PreSSE) 101 0.061 0.806 0.001

Low 57 91.35 87.17

High 50 87.67 83.60


Gender 107 -0.231 2.517 101 -0.092 0.927 0.000

GPA 107 0.569 1.084 101 0.525 0.600 0.003





PostSSE = 85.50). This difference was significant (F (1, 201) = 64.08, p < 0.001, h2 = 0.24),supporting Hypothesis 2.

Despite a significant difference, a simple comparison of PostSSE across the low and highPreSSE participants may not represent a complete picture of appropriation effects. Specifi-cally, examining the effects of the level of appropriation within the low and high PreSSE groupsmay lead to a deeper understanding of the results.3 As such, we further dichotomised the lowand high PreSSE groups based on FOA. Changes in spreadsheet self-efficacy were thenassessed between participants who more faithfully and less faithfully appropriated the CSSTS.Referring to Table 6, ANCOVA results reveal that within the low PreSSE group, high appro-priation participants experienced a greater increase in spreadsheet self-efficacy (PreSSE =50.88, PostSSE = 69.15) than low appropriation participants (PreSSE = 58.36, PostSSE =69.95). Although only marginally significant (F (1, 94) = 2.900, p < 0.10), this result reinforcesthe assertion that faithful appropriation leads to greater increases in PostSSE. For the highPreSSE participants, no such difference was observed (F (1, 101) = 0.061, p = 0.806).However, the non-significant difference for the high PreSSE participants can be readilyexplained, as the level of faithful appropriation among these users was low overall. Becausethe high PreSSE group failed to fully reap the benefits of the Teach Me and Show Me features,the analysis essentially compares two groups of individuals with relatively low FOA levels. Inessence, PostSSE increases when FOA levels are high. These results further bolster theassertion that increases in PostSSE surface only when users faithfully appropriate the CSSTS.

Hypothesis 3. Hypothesis 3 predicted that PostSSE would be a significant, positive predictorof objective software skills performance. The results are illustrated in Table 7. After controllingfor the significant covariate GPA (b = 0.266, t (201) = 4.304, p < 0.001), the independent

3We would like to thank an anonymous reviewer for bringing this to our attention.

Table 7. Hypothesis 3 – regression results

b t Supported

Study 1

Hypothesis

H1 0.344 5.165*** Yes

Covariates

GPA 0.266 4.304***

Study 2

Hypothesis

H1 -0.260 3.440*** Yes

Covariates

Computer anxiety -0.155 2.656**

GPA -0.180 3.152**

***p < 0.001.

**p < 0.01.

*p < 0.05.



variable PostSSE (b = 0.344, t (201) = 5.165, p < 0.001) was a significant, positive predictor ofperformance. This result supports Hypothesis 3.

Study 2

Study 2 served two purposes. First, although one previous study has examined the effects ofspreadsheet training on CSSTS outcomes [Gupta, 2008], no study has assessed whetherfindings can be extended beyond spreadsheet software training. Second, study 1 showedrelatively high levels of PreSSE. As a result, ceiling effects represent an alternative explanationfor the results. To address these possibilities, a second study was conducted during adatabase software training course, where PreSSE levels were expected to be lower. Partici-pants included 280 undergraduates enrolled in an IS course at a large North Americanuniversity. The training session was conducted in exactly the same manner as study 1; the onlydifference was the focus on database rather than spreadsheet software.

Sample demographics

The sample consisted of participants enrolled in a business school pre-certification courserequired of all business majors. The average age of the participants was 21.4 years. Genderwas distributed as 42% female and 58% male. Self-reported majors were approximately equalacross the disciplines, with ‘undecided’ being reported slightly more often than the others.Participants reported an average database software experience level of 26.63, measured ona 0–100 scale anchored by ‘no experience’ (0) and ‘very experienced’ (100). As expected,participants reported much less database experience (M = 26.63) compared with spreadsheetexperience (M = 57.44).

Measures

Surveys were administered both pre- and post-training. Like study 1, PreSSE was measuredbefore the database training commenced, while the FOA and PostSSE measures wereadministered after the 2-week training, immediately prior to the database skills assessment. Asthe task involved acquiring skills with database software, PreSSE and PostSSE were mea-sured using a database self-efficacy measure developed by the authors. The response formatfor the database self-efficacy measure required a two-part question typical of efficacy mea-sures [Bandura, 2005], exactly like the spreadsheet self-efficacy scales utilised in study 1.Appendix 1 describes the instrument development process used to validate the databaseself-efficacy measure. All other items were the same as study 1, although some were minimallytailored to focus on database software rather than spreadsheets.

Results

Measurement model results. Referring to Table 8, all items loaded higher on their targetconstructs than on any other construct. Cronbach’s alphas for FOA, PreSSE, PostSSE,



computer anxiety and outcome expectancies were 0.71. 0.96, 0.93, 0.75 and 0.91, respec-tively. Given evidence for acceptable levels of validity and reliability, composite scores werecreated for the multi-item measures to facilitate the subsequent analyses.

Hypothesis 1. Hypotheses 1 proposed that PreSSE would be significantly, inversely related toFOA. A regression model was used to analyse the data. PreSSE and the covariates identifiedby Gupta & Bostrom [2009] were entered as independent variables, whereas FOA served asthe dependent variable. Table 5 contains the results. After controlling for the significantcovariates outcome expectancies (b = 0.137, t (274) = 2.34, p < 0.05), GPA (b = -0.131, t (274)= 2.24, p < 0.05) and gender (b = 0.200, t (274) = 3.22, p < 0.01), PreSSE proved to be asignificant predictor of FOA (b = -0.260, t (274) = 3.44, p < 0.001). As expected, the significantrelationship between PreSSE and FOA was negative, indicating that participants with weakerPreSSE beliefs more faithfully appropriated the CSSTS than participants with stronger PreSSEbeliefs. These results support H1 and replicate the findings of study 1.

Hypothesis 2. Hypothesis 2 proposed that through faithful appropriation of the CSSTS, lowPreSSE participants would realise a greater increase in PostSSE than would high PreSSEparticipants. To evaluate the influence of FOA on PostSSE across the low and high PreSSEgroups, the same dichotomisation procedure described in study 1 was utilised. An ANCOVA

Table 8. Study 2 factor loadings

Study 2

1 2 3 4

ACC 1 0.886 0.118 -0.024 0.056

ACC 2 0.873 -0.057 0.017 0.040

ACC 3 0.871 0.057 0.001 0.127

ACC 4 0.868 0.064 -0.062 0.003

ACC 5 0.865 0.070 -0.015 0.106

ACC 6 0.846 -0.040 0.002 0.103

ACC 7 0.827 0.128 -0.089 0.040

ACC 8 0.824 0.066 -0.034 0.105

OE 1 0.036 0.856 -0.040 0.043

OE 2 0.093 0.847 -0.132 0.043

OE 3 0.002 0.841 0.052 0.002

OE 4 0.037 0.768 -0.044 0.101

OE 5 0.107 0.729 -0.094 0.037

OE 6 0.106 0.612 0.138 -0.071

CA 1 -0.016 -0.080 0.858 0.040

CA 2 -0.023 -0.008 0.858 0.033

CA 3 -0.086 -0.066 0.784 0.042

CA 4 0.039 0.049 0.649 0.006

FOA1 0.038 0.064 0.090 0.836

FOA2 0.145 0.068 0.041 0.812

FOA3 0.203 0.020 -0.021 0.704



compared the differences in database self-efficacy gains across the two participant groups. Nocovariates were significant. As shown in Table 9, a significant difference between the groupsemerged (F (1, 274) = 104.15, p < 0.001, h2 = 0.28). Specifically, low PreSSE participantsrealised a greater increase in database self-efficacy (PreSSE = 9.56, PostSSE = 53.29) thandid high PreSSE participants (PreSSE = 52.31, PostSSE = 64.69).

Consistent with study 1, additional analyses were performed to confirm FOA drives PostSSEincreases. Specifically, FOA levels within the low and high PreSSE groups detected whetherappropriation affected subsequent changes in database self-efficacy. Table 9 provides theANCOVA results. Within the low PreSSE group, high appropriation participants experienced a

Table 9. Results for study 2 – Hypothesis 2

Independent variable n

Software self-efficacy Type III fixed effects results

M(pre) M(post) df F p h2

Participant type 274 104.15 0.000 0.28

Low PreSSE 137 9.56 53.29

High PreSSE 143 52.31 64.69


Gender 280 -2.306 3.184 274 -0.724 0.470 0.002

GPA 280 1.777 1.491 274 1.192 0.234 0.005




Appropriation (low PreSSE) 131 12.205 0.001 0.085

Low 64 11.64 47.18

High 73 7.74 56.81


Gender 137 5.062 4.516 131 1.121 0.264 0.009

GPA 137 3.668 2.152 131 1.705 0.091 0.022


Outcome expectancies 137 -0.604 2.516 131 -0.240 0.811 0.000


Appropriation (high PreSSE) 137 0.808 0.370 0.006

Low 66 51.29 59.68

High 77 55.00 68.98


Gender 143 -5.705 4.486 137 -1.272 0.206 0.012

GPA 143 0.973 2.033 137 0.479 0.633 0.002





greater increase in database self-efficacy (PreSSE = 7.74, PostSSE = 56.81) than low appro-priation participants (PreSSE = 11.64, PostSSE = 47.18). The difference was significant (F (1,131) = 12.205, p = 0.001), reinforcing the notion that faithful appropriation leads to greaterincreases in SSE. Regarding the high PreSSE participants, no significant difference emerged(F (1, 137) = 0.808, p = 0.370), arguably for the same reasons described in study 1. In sum,these results not only support Hypothesis 2, but also reproduce study 1 findings. Faithfulappropriation appears to lead to greater increases in subsequent self-efficacy beliefs.

Hypothesis 3. Hypothesis 3 predicted that PostSSE would serve as a significant, positivepredictor of objective database skills performance. Given the expected linear relationship, aregression model was executed. Table 7 illustrates the results. PostSSE, computer anxiety,outcome expectations, GPA and gender were entered as independent variables, whereas theobjective assessment score was entered as the dependent variable. After controlling for thesignificant covariates GPA (b = 0.180, t (274) = 3.152, p = 0.002) and computer anxiety (b =-0.155, t (274) = 2.656, p = 0.008), PostSSE was a significant, positive predictor of theobjective measure of database performance (b = 0.259, t (274) = 4.554, p < 0.001). Theseresults support Hypothesis 3 and confirm the findings of study 1.

DISCUSSION

The purpose of this research involved answering three important research questions. The firstsought to examine whether PreSSE influenced appropriation in high learner control CSSTSenvironments. Across spreadsheet and database training sessions, the results revealed aninverse relationship between PreSSE and FOA. Trainees with weaker PreSSE beliefs morefaithfully appropriated the CSSTS than participants with stronger PreSSE beliefs. In essence,those who doubted their capabilities leveraged more of the CSSTS features during therespective training sessions. More confident trainees, on the other hand, appropriated theCSSTS less faithfully, essentially constraining the learning dimensions by favouring onestructural feature over the others. These findings extend and inform Gupta & Bostrom [2009]by establishing that individual differences, such as software self-efficacy, play a criticallyimportant role in influencing appropriation in high learner control environments. This finding,however, runs contrary to conventional AST wisdom. Rather than making more faithful appro-priation moves, participants with stronger PreSSE beliefs tended to constrain the learningstructures rather than using them in combination. Strong PreSSE beliefs appear to underminea trainee’s motivation to utilise the entire set of structural features, despite their documentedbenefits.

The second research question asked whether faithful appropriations in high learner control,CSSTS environments leads to increased software self-efficacy beliefs. Our findings suggestthat participants who faithfully appropriated the CSSTS realised greater increase in softwareself-efficacy (PostSSE) than those who appropriated the CSSTS less faithfully. To furtherexplore this issue, appropriation levels within the low and high PreSSE groups were assessed.



Within the low PreSSE group, more faithful appropriation led to greater increases in PostSSE.The high PreSSE group, on the other hand, often failed to utilise the CSSTS features set incombination, restricting the structural dimensions available to them. Given the low levels offaithful appropriation, no significant increases in PostSSE emerged. These patterns wereconsistent across spreadsheet and database training sessions, serving to emphasise theprominent role that appropriation plays in CSSTS environments.

These findings not only shed light on EUT research, but also inform the high learner controlliterature, which has typically found that individuals with stronger self-efficacy beliefs tend toperform better in such settings. A credible explanation exists for this contradiction. Most highlearner control research has relied on general learning self-efficacy, while the current studyfocused on task-specific software self-efficacy. To facilitate predictiveness, the generality ofself-efficacy beliefs should match the relevant outcome variables they intend to predict[Bandura, 1997]. When examining outcomes confined to a particular task, software self-efficacy should prove to be a stronger predictor compared with decontextualised generalbeliefs. Understanding the nature of these differences in self-efficacy generality provides anexciting avenue for future research in high learner control training environments.

The third research question sought to understand whether findings from prior EUTresearch can be generalised beyond spreadsheet training to other applications. Answeringa call for research targeting computer-based training in new software settings [Gupta,Bostrom, & Huber, 2010], the results obtained from database training were remarkablysimilar to spreadsheet training. In fact, all hypotheses were supported across the studies.Nonetheless, the effects of faithfully appropriating the structural features, and the dimen-sions they provide, appear to be more pronounced in settings where trainees have hadminimal exposure to the software. Consequently, examining database software skills acqui-sition moved the literature beyond the examination of training outcomes in spreadsheetenvironments, where many participants possess relatively extensive experience. Expandingon this contribution, a psychometrically sound measure of database self-efficacy was devel-oped. Researchers can now confidently apply this measure in future research. While theseresults hold promise, the extent to which they generalise to other software applications (e.g.application development, web design) provides an interesting avenue for research incomplex software training environments.

In addition to addressing three important research questions, this research contributes to theliterature in a number of other ways. Built upon AST, Gupta & Bostrom [2009] anticipated thepotential of self-efficacy as an influential factor in training method appropriation. To ourknowledge, no empirical evidence existed to support such claims. Our results clearly show thatthrough faithful appropriation, CSSTSs enhance learning outcomes. Combining the comple-mentary elements of AST and SCT provided a clearer picture of the mechanisms underlyingtrainee learning processes in CSSTS environments and ultimately software training success.

In addition, this research put forth a new conceptualisation of appropriation well suited forstudying individual-level phenomena in CSSTS contexts. Establishing a link between learningdimensions and FOA, defined in this context as frequency of use across all features, providesa novel conceptualisation of FOA. This theoretical connection was established by linking



learning dimensions to the respective CSSTS features. The establishment of these theory-based relationships provides researchers with a potential new method for assessing trainingsystem appropriation at the individual level.

Finally, to our knowledge this research is the first study to examine appropriation in a highlearner control CSSTS environment. As the results attest, FOA plays a crucial role in softwareskills acquisition. Understanding appropriation in these environments is vital, as it affordsacademicians and practitioners alike the opportunity to develop interventions that help usersutilise the structural dimensions, improving learning outcomes in organisational training con-texts. Our studies clearly show that high levels of appropriation are essential to developing thesoftware skills demanded in today’s highly competitive marketplace. These findings providevaluable insights into how theoretical propositions associated with EUT operate in real worldsettings. Not only were the studies conducted in a realistic setting, but university studentsconstituted the actual users for which the CSSTS was designed. For all intents and purposes,the current context represents an organisational setting, allowing us to examine whether priorexperimental findings could be generalised to a real training environment.

Limitations and future research

Like any research undertaking, our study is not without limitations. While this research used asample appropriate to our research question – examining a software artefact and its userswithin the context of a typical computer skills training course – using students enrolled in abusiness pre-certification course may be viewed as a limitation in terms of generalising ourfindings. However, our intent was not to generalise our findings to contexts outside of similartraining environments. Rather, our purpose was to generate a theoretical perspective thatcould be evaluated for its generalisability in other settings [Lee & Baskerville, 2003]. None-theless, future research should attempt to replicate our results using training participants withvarious backgrounds, as well as in different settings.

A field study methodology was used to investigate learning in a high learner control CSSTSenvironment. Although the implementation of a controlled experimental design would haveeliminated alternative explanations for our findings, it would have artificially restricted the veryhigh learner control environment that trainees in these environments typically confront.Lending credence to our findings, the results were replicated across two separate trainingsessions using two different software packages. Nonetheless, no methodology can be deemedperfect. While field studies often sacrifice rigorous controls, experimental designs have theirown disadvantages, such as the creation of artificial environments [Saks, 1995]. In addition,previous studies of computer self-efficacy have used similar designs [Compeau & Higgins,1995; Agarwal et al., 2000], where users interact with systems in a field setting. Moreover, theimportance of confirming theoretical relationships using non-experimental methods has beenrecognised elsewhere [Dennis et al., 2001]. Consequently, we chose a methodology thatmaximised a realistic setting rather than forcing participants to utilise an artificial CSSTS thatmay not reflect reality. Regardless, future research should use alternative methodologies tobuild upon our findings. For example, experimental studies could be used to compare partici-



pants who were free to select any feature, with participants who were guided through featureselection as predicted by learning theories. Such a design could lead to a better understandingof how the ordered progression of feature selection and use may affect computer skillsacquisition in computer-based training environments. This could also help ascertain whethertrainees, who utilise all the features without guidance, select features in a manner consistentwith prior research, which has demonstrated the superiority of behavioural modelling followedby retention enhancement and hands-on enactive mastery. The limitation of this approach,however, is that the artificial setting would no longer constitute a high learner control environ-ment, as the use of CSSTS features would be dictated by the experimental environment.Future research should consider the balance of these approaches.

Measuring FOA using a three-item, seven-point Likert-type scale is subject to recall bias. Inessence, participants may have forgotten how often they used particular features during thetraining sessions. However, a self-reported measure was adopted to maintain consistency withour research goals, namely, observing learners using CSSTS in a natural setting as opposedto an experimental one. Overt monitoring possesses the very real potential of altering thebehaviours we wanted to observe [Shadish et al., 2002]. Supporting this decision, priorresearch has highlighted the benefits of self-reports for investigating intercorrelations amongperceptions in the early stages of research [Spector, 1994], criteria applicable to our effort. Inaddition, our FOA measure differs from those previously used in the AST literature. Forinstance, neither attitude nor consensus was assessed. However, we believe our measure ofFOA contributes to the field by offering a novel method for assessing appropriation in highlearner control EUT environments. Nonetheless, future research should both examine ourconceptualisation of FOA and investigate our findings using alternative measurementapproaches.

An additional limitation involves our assessment of training effectiveness for the highPreSSE participants. Because PreSSE scores approached the scale’s upper bound, ceilingeffects could have occurred. In essence, the measure may not have provided sufficient roomto detect increases in software self-efficacy. Study 1 showed a non-significant increase insoftware self-efficacy for the high PreSSE group. Several students who entered the spread-sheet training session reported high PreSSE, meaning that ceiling effects could haveaccounted for the result. Countering this notion, however, the same pattern of results wasobserved in the database training context, where participants reported much lower levels ofPreSSE. In this case, the self-efficacy measure contained plenty of room to detect increases.Therefore, ceiling effects can be confidently ruled out as a rival explanation. Nonetheless,future research should account for other types of software applications to identify the boundaryconditions of our findings.

CONCLUSION

This study investigated the viability of CSSTSs for improving software skills acquisition duringtwo separate software training sessions. Theoretical linkages between specific CSSTS fea-



tures and the sources of self-efficacy information were established. Using AST and SCT as atheoretical guide, PreSSE played an important role in influencing appropriation in high learnercontrol CSSTSs environments. Contrary to previous AST research, self-efficacy beliefs wereinversely related to faithful appropriation. Participants with weaker PreSSE more faithfullyappropriated the CSSTS than participants with stronger PreSSE beliefs. Consistent with thereading of the system’s spirit, low PreSSE participants combined the learning structures byrelating the structures to other structures, while high PreSSE participants constrained thestructures by favouring one structure over the other. In turn, those participants who morefaithfully appropriated the system enjoyed greater increases in software self-efficacy. Inessence, faithfully appropriating CSSTS features is crucial for users to acquire the advancedsoftware skills necessary to succeed in a highly competitive marketplace. Armed with thisknowledge, IS researchers and practitioners alike are now better positioned to improvelearning outcomes in organisational training contexts.

REFERENCES

Agarwal, R., Sambamurthy, V. & Stair, R. (2000) The

evolving relationship between general and specific com-

puter self-efficacy – an empirical assessment. Informa-

tion Systems Research, 11, 418–430.

Anonymous (2010) Industry report. Training, 44, 8–24.

Bandura, A. (1997) Self-Efficacy: The Exercise of Control.

W. H. Freeman/Times Books/Henry Holt & Co, New York.

Bandura, A. (2005) Guide for constructing self-efficacy

scales. In: Adolescence and Education, Self-Efficacy

Beliefs of Adolescents, Vol. 4, Pajares, F. & Urdan, T.

(eds), pp. 1–39. Information Age Publishing, Greenwich.

Bandura, A., Adams, N.E. & Beyer, J. (1977) Cognitive

processes mediating behavioral change. Journal of Per-

sonality and Social Psychology, 35, 125–139.

Bostrom, R.P., Olfman, L. & Sein, M.K. (1994) The impor-

tance of learning style in end user training. MIS Quar-

terly, 14, 101–119.

Brown, K.G. (2001) Using computers to deliver training:

which employees learn and why? Personnel Psychol-

ogy, 54, 271–296.

Compeau, D.R., Gravill, J., Haggerty, N. & Kelly, H. (2006).

Computer self-efficacy: A review. In D. Galletta & P.

Zhang (Eds.), Human-Computer Interaction in Manage-

ment Information Systems: Foundations, Advances in

Management Information Systems (Vol. 5, pp. 225–

261). Armonk, NY: M. E. Sharpe.

Compeau, D.R. & Higgins, C.A. (1995) Application of

social cognitive theory to training for computer skills.

Information Systems Research, 6, 118–143.

Compeau, D.R., Higgins, C.A. & Huff, S. (1999). Social

cognitive theory and individual reactions to computing

technology: A longitudinal study. MIS Quarterly, 23(2),

145–158.

Davis, S.A. & Bostrom, R.P. (1993) Training end users: an

experimental investigation of the roles of the computer

interface and training methods. MIS Quarterly, 17,

61–85.

Davis, S.A. & Bostrom, R.P. (1994) The effects of an

intrinsically motivating instructional environment on soft-

ware learning acceptance. Information Systems Journal,

4, 3–26.

Dennis, A., Wixom, B. & VandenBerg, R. (2001) Under-

standing fit and appropriation effects in group support

systems via meta-analysis. MIS Quarterly, 25, 167–193.

DeRouin, R.E., Fritzsche, B.A. & Salas, E. (2004) Optimiz-

ing learning: research-based guidelines for learner-

controlled training. Human Resource Management, 43,

147–162.

DeSanctis, G. & Poole, M.S. (1994) Capturing the com-

plexity in advanced technology use – adaptive structu-

ration theory. Organization Science, 5, 121–147.

Giddens, A. (1984) The Constitution of Society. University

of California Press, Los Angeles, CA.

Gist, M. & Mitchell, T.R. (1992) Self-efficacy: a theoretical

analysis of its determinants and malleabilty. Academy of

Management Review, 17, 183–211.

Gist, M., Schwoerer, C. & Rosen, B. (1989) Effects

of alternate training methods on self-efficacy and



performance in computer software training. Journal of

Applied Psychology, 74, 884–891.

Gupta, S. (2008) New Approaches to End-User Training.

VDM Verlag, Saarbrucken, Germany.

Gupta, S., Bostrom, R.P. & Huber, M. (2010). End User

Training: What we Know, Need to Know, The Database

for Advances in Information Systems, 42, 9–39.

Gupta, S. & Bostrom, R.P. (2009) Technology-mediated

learning: a comprehensive theoretical model. Journal of

the Association for Information Systems, 10, 686–

714.

Johnson, R.D. & Marakas, G.M. (2000) Research report:

the role of behavioral modeling in computer skills acqui-

sition: toward refinement of the model. Information

Systems Research, 11, 402–417.

Lee, A.S. & Baskerville, R.L. (2003) Generalizing general-

izability in information systems research. Information

Systems Research, 14, 221–243.

Marakas, G.M., Yi, M.Y. & Johnson, R.D. (1998) The mul-

tilevel and multifaceted character of computer self-

efficacy: toward clarification of the construct and an

integrative framework for research. Information Systems

Research, 9, 126–163.

Olfman, L. & Bostrom, R.P. (1991) End-user software

training: an experimental comparison of methods to

enhance motivation. Information Systems Journal, 1,

249–266.

Olfman, L. & Mandviwalla, M. (1994) Conceptual versus

procedural software training for graphical user inter-

faces: a longitudinal field experiment. MIS Quarterly, 18,

405–426.

Olfman, L. & Mandviwalla, M. (1995) An experimental

analysis of end-user training manuals. Information

Systems Journal, 5, 19–36.

Pajares, F. (2004) Current directions in self-efficacy

research. In: Advances in Motivation and Achievement,

Vol. 10, Maehr, M. & Pintrich, P.R. (eds), pp. 1–49. JAI

Press, Greenwich.

Piccoli, G., Ahmad, R. & Ives, B. (2001) Web-based virtual

learning environments: a research framework and pre-

liminary assessment of effectiveness in basic IT skills

training. MIS Quarterly, 25, 401–426.

Saks, A.M. (1995) Longitudinal field investigation of the

moderating and mediating effects of self-efficacy on the

relationship between training and newcomer adjust-

ment. Journal of Applied Psychology, 80, 211–235.

Shadish, W.R., Cook, T.D. & Campbell, D.T. (2002)

Experimental and Quasi-Experimental Designs for

Gneralized Causal Inference. Houghton Mifflin

Company, New York.

Speckler, M.D. (2006) A Report on the Success of Train

and Assess ITTM in Higher Education Microsoft OfficeTM

Instruction. Prentice Hall, Boston, MA.

Spector, P.E. (1994) Using self-report questionnaires in

OB research: a comment on the use of a controversial

method. Journal of Organizational Behavior, 15, 385–

392.

Thomas, M.T., Gupta, S. & Bostrom, R.P. (2008) A meta-

theory for understanding IS in socio-technical systems.

Paper presented at the Hawaii International Conference

on System Sciences, Hawaii.

Williams, M.D. (1996) Learner control and instructional

technologies. In: Handbook of Research for Educational

Communications and Technology, Jonassen, D.H. (ed.),

pp. 170–198. Macmillan, New York.

Yi, M.Y. & Davis, F. (2001) Improving computer training

effectiveness for decision technologies: behavior mod-

eling and retention enhancement. Decision Sciences,

32, 521–544.

Zimmerman, B.J. (1995) Self-regulation involves more

than metacognition: a social cogntive perspective. Edu-

cational Psychologist, 30, 217–211.

Biographies

Andrew M. Hardin is the Director of the Center for Entre-

preneurship and an Associate Professor in the Lee Busi-

ness School at the University of Nevada, Las Vegas.

Professor Hardin’s research is focused on organizational

collaboration and virtual work, financial decision support

systems, and research methodologies. His work has

appeared in journals such as Management Science, MIS

Quarterly, Organizational Behavior and Human Decision

Processes, Journal of Management Information Systems,

European Journal of Information Systems, Information

Systems Journal, Journal of the Association for Informa-

tion Systems, The DATA BASE for Advances in Informa-

tion Systems, Group Decision and Negotiations, Small

Group Research, and Educational and Psychological Mea-

surement. Hardin currently serves as Senior Editor for the

Information Systems Journal and The DATA BASE for

Advances in Information Systems, Senior Associate Editor

for the European Journal of Information Systems, and

Guest Associate Editor for MIS Quarterly.

Clayton A. Looney is an Associate Professor and the

Ron and Judy Paige Faculty Fellow in the School of Busi-

ness Administration at The University of Montana. He

earned his Ph.D. in Information Systems from Washington

State University. Leveraging expertise in human-computer

interaction, cognitive psychology, and behavioral econom-



ics, his cross-disciplinary research focuses on designing

technologies to overcome decision-making biases. His

work has appeared in Management Science, Organiza-

tional Behavior and Human Decision Processes, Decision

Sciences, Information Systems Journal, Communications

of the ACM, Communications of the Association for Infor-

mation Systems, Journal of Computer Information

Systems, Journal of Information Technology Education,

and Group Dynamics:Theory, Research, and Practice as

well as various international conferences.

Mark A. Fuller is the Dean and the Thomas O’Brien

Endowed Chair of the Isenberg School of Management at

the University of Massachusetts Amherst. Professor Fuller

received his Ph.D. in Management Information Systems

from the University of Arizona. His research focuses on

virtual teamwork, technology supported learning, and trust

and efficacy in technology-mediated environments, and

has appeared in outlets such as Information Systems

Research, Management Information Systems Quarterly,

Journal of Management Information Systems, Decision

Sciences, Journal of the Association for Information

Systems, Journal of Organizational Behavior, IEEE Trans-

actions of Engineering Management, and Decision

Support Systems. Professor Fuller has won multiple teach-

ing awards, has published a textbook on Information

Systems Project Management, and has taught graduate

and undergraduate courses on a variety of topics, includ-

ing global information systems and strategy, information

systems project management, and collaborative

technology.

APPENDIX 1

Instrument development

Fourteen items were developed by the authors based upon a review of the computer self-efficacy literature. Consistent with the existing measure of spreadsheet self-efficacy used instudy 1, items were generated that assessed participants beliefs in their abilities to completespecific database exercises. For example, I believe I have the ability to build a report.Consistent with the self-efficacy literature, a two-part, 0–100 scale anchored by ‘cannot do’ (0)and ‘totally confident’ (100) was used [Bandura, 2005]. Content validity was established byhaving self-efficacy researchers review the items.

Measurement model results

To establish the factor structure of the database self-efficacy measure, the instrument wasadministered to 476 participants during various database training sessions. The factor struc-ture was examined by conducting a confirmatory factor analysis using AMOS 19. Initial fitwas unsatisfactory due to the presence of correlated error terms. The associated itemswere deleted and a model with eight items was re-evaluated. Fit for the new model was good,c2 (20, n = 476) = 55.98, p < 0.001), CMIN/DF = 2.79, CFI = 0.990, NFI = 0.985, GFI = 0.970,AGFI = 0.945, RMSEA = 0.065 (confidence interval = 0.045–0.087).

As a final step, the measurement model was revaluated using the study 2 data. Once again,model fit was good, c2 (20, n = 280) = 37.98, p = 0.009), CMIN/DF = 1.90, CFI = 0.991, NFI =0.982, GFI = 0.968, AGFI = 0.943, RMSEA = 0.057 (confidence interval = 0.028–0.084). Basedon these respective tests, the eight-item measure of database self-efficacy was used in thesubsequent analysis. As a final test, support for Hypotheses 3 in study 2 established thepredictive validity of the database self-efficacy measure. The database self-efficacy items arelisted in Table 3.



Documents

Self-efficacy, learning method appropriation and software skills acquisition in learner-controlled CSSTS environments