A Longitudinal Experiment on the Effects of Post Training Transfer - Gaudine Et Al

A Longitudinal Quasi-Experimenton the Effects of PosttrainingTransfer Interventions

Alice P. Gaudine, Alan M. Saks

A longitudinal quasi experiment tested the effects of a relapse preventionand transfer enhancement posttraining intervention on the self-efficacy,transfer behavior, and performance of a sample of nurses who attended atwo-day training program on the McGill Model of Nursing. ANCOVAresults failed to support the effectiveness of the intervention; in fact,participants in the transfer enhancement condition had the lowest transferbehavior and performance except when it was combined with relapseprevention. However, all trainees showed significant increase in self-efficacy, behavior, and performance. Results are explained based ontraining program effectiveness, organizational context, and transfersystem. A positive transfer climate and factors in the transfer system likelycontributed. Implications discussed include the need to conduct a transferof training needs analysis (TTNA) and a contingency approach toposttraining transfer interventions.

In order for organizations to benefit from their investments in training anddevelopment and remain competitive, trainees must apply, generalize,and maintain over time what they learn in training on the job (Salas & Cannon-Bowers, 2001). This is known as transfer of training, and it involves the gener-alization of learned material to the job and the maintenance of trained skills andlearned material over a period of time (Baldwin & Ford, 1988). In addition tobeing an important criterion of a training program’s success, transfer of trainingis also considered to be the primary leverage point by which training caninfluence organizational-level outcomes (Kozlowski, Brown, Weissbein,

57HUMAN RESOURCE DEVELOPMENT QUARTERLY, vol. 15, no. 1, Spring 2004Copyright © 2004 Wiley Periodicals, Inc.

Note: This research was conducted for the first author’s doctoral dissertation andsupervised by the second author at Concordia University in Montreal. The study wassupported by grant 410-93-0277 from the Social Sciences and Humanities ResearchCouncil of Canada.

hrd15105.qxp 2/10/04 1:42 PM Page 57

58 Gaudine, Saks

Cannon-Bowers, & Salas, 2000). Until recently, research on transfer of traininghas emphasized learning principles and conditions of practice in the design oftraining programs. However, these design factors have been primarily studiedin laboratory settings and have only been found to influence immediate learn-ing and retention rather than the generalization and maintenance of trainedskills on the job (Baldwin & Ford, 1988). This approach to transfer of traininghas also been criticized as being part of a mental model that limits research ontransfer of training in organizational settings (Latham & Seijts, 1997).

In response to this criticism, there has been a trend toward designing train-ing programs with a specific transfer component or intervention that takesplace subsequent to the actual program and provides trainees with guidanceand strategies on how to transfer their newly acquired knowledge and skills tothe work environment (Haccoun & Saks, 1998). In recent years, researchershave called for more research on posttraining transfer interventions usingactual employees in corporate training settings (Burke, 1997).

The purpose of this study was to test the effects of two posttraining trans-fer interventions—relapse prevention and transfer enhancement—on the self-efficacy, transfer behavior, and performance of a sample of nurses who receivedtraining on a complex interpersonal skill.

Posttraining Transfer Interventions

Studies on the effects of posttraining transfer interventions on the transfer oftraining have been an important development in training research. Twenty yearsago, Marx (1982) suggested relapse prevention (RP), a cognitive-behavioralstrategy for reducing the likelihood of a relapse and increasing the transfer ofmanagerial training programs. The basic idea was to make trainees aware of therelapse process and teach them to identify high-risk situations, develop copingresponses to avoid a relapse in the transfer environment, and maintain behav-ioral change following training. One of the main goals of RP is to develop highlevels of self-efficacy for identifying high-risk situations and using appropriatecoping strategies (Machin, 2002).

Over the last decade, a number of studies have investigated the effective-ness of relapse prevention as a posttraining transfer intervention. In one of thefirst studies, Wexley and Baldwin (1986) compared an RP intervention withan assigned goal-setting, participative goal-setting, and control condition in alaboratory study in which students were trained in time-management skills.Eight weeks after training, subjects in the assigned goal-setting conditionscored higher on learning than subjects in the other three groups, and subjectsin both of the goal-setting groups scored higher on a behavioral self-reportmeasure than subjects in the relapse prevention and control conditions.

In the first field study on a posttraining RP, Noe, Sears, and Fullenkamp(1990) tested the effects of an RP intervention in a training program onsupervisory skills for managers. In a follow-up two to three months after the

hrd15105.qxp 2/10/04 1:42 PM Page 58

Posttraining Transfer Interventions 59

program, participants in the relapse group had significantly higher scores thana control group on five of fourteen questionnaire items that measured the use ofRP strategies rather than the actual transfer of supervisory skills.

Tziner, Haccoun, and Kadish (1991) added a relapse prevention moduleto a two-week training program for military instructors in the Israeli defenseforces. Compared with a control group that received only the two-week train-ing program, trainees who also received a two-hour RP module at the end ofthe program had significantly higher knowledge of the course contents, andreported making greater use of the transfer strategies that had been taught inthe course, ten weeks after completing it. In addition, supervisors rated the RPtrainees higher in their use of the actual trained skills.

More recently, Burke (1997) examined the effects of relapse prevention onthe maintenance of assertive communication knowledge and skills among asample of undergraduate students. However, neither a full nor a modified versionof relapse prevention had an effect on retention of the course content, use oftransfer strategies, or use of trained skills compared with a control group threeweeks after training. Burke and Baldwin (1999) tested the effects of two versionsof relapse prevention in a sample of researchers attending a supervisory-skillstraining program on coaching. Although neither RP intervention had a maineffect on transfer outcomes, an interaction between relapse prevention and trans-fer climate was significant. The full RP intervention had a significant effect on theuse of transfer strategies and trained skills in an unsupportive transfer climatefour weeks after the training program. A modified, scaled-down RP module andthe control group showed improved transfer in supportive climates.

In summary, to date very few studies have investigated the effects of relapseprevention as a posttraining transfer intervention and only three have been fieldstudies with actual employees. Furthermore, the results have been mixed andinconsistent: Tziner et al. (1991) found some support, Burke (1997) as well asWexley and Baldwin (1986) found no support, and Burke and Baldwin (1999)found a significant interaction effect with the transfer climate but not a maineffect. Furthermore, although self-efficacy is considered to be the key mecha-nism through which RP improves transfer, no study to date has examined theeffect of an RP intervention on self-efficacy.

Self-Efficacy and Transfer of Training

During the last decade, self-efficacy—or one’s belief that one can perform spe-cific tasks and behaviors—has been found to be a critical variable in trainingresearch. In the context of training, self-efficacy refers to a trainee’s belief thathe or she can master training tasks and behaviors and perform them onthe job.

Research over the last decade has consistently shown that trainingincreases trainees’ self-efficacy and that self-efficacy predicts trainee learningand performance (Salas & Cannon-Bowers, 2001; Tannenbaum, Mathieu,

hrd15105.qxp 2/10/04 1:42 PM Page 59

60 Gaudine, Saks

Salas, & Cannon-Bowers, 1991). Furthermore, self-efficacy has been shown tomediate the effect of training on training outcomes such as job attitudes,turnover, absenteeism, and performance (Gist, Stevens, & Bavetta, 1991;Latham & Frayne, 1989; Mathieu, Martineau, & Tannenbaum, 1993; Saks,1995). Thus, it is now generally accepted that self-efficacy plays a central rolein understanding training and in enhancing training effectiveness (Mathieuet al., 1993).

Self-efficacy has also been found to be an important variable in transfer-of-training research (Machin, 2002). For example, Ford, Quinones, Sego, andSorra (1992) found that trainee self-efficacy was related to opportunities toperform trained tasks on the job. Airmen with high self-efficacy performedmore of the trained tasks and reported performing more complex and difficulttasks. In addition, Gist et al. (1991) found that trainee self-efficacy was relatedto the acquisition and maintenance of a complex interpersonal skill.

Finally, self-efficacy is also considered to be an important variable inrelapse prevention. According to Marx (1982), relapse prevention providestrainees with an enhanced sense of self-efficacy that they can cope with diffi-cult and high-risk situations. Marx also notes the difficulty for most traineesto master complex skills following training, something that requires monitor-ing and coping skills. An increase in self-efficacy will increase trainees’ likeli-hood of persisting, mastering, and performing complex skills on the job.

Purpose of the Study

The primary purpose of the present study was to test the effects of an RP andtransfer-enhancement posttraining transfer intervention. The study extendsprevious research in a number of important ways. First, the training materialin the present study involved a highly complex interpersonal skill. As indicatedearlier, RP is likely to be particularly helpful in preventing relapses after learn-ing complex skills because such skills take more time to master and apply onthe job (Marx, 1982). The training program in the present study requiredtrainees to learn to work in collaboration with families to structure learningenvironments and help the families explore issues and develop over time.Trainees learn how to help families cope and attain their goals. This is a highlycomplex interpersonal task that involves a significant number of componentparts, a high degree of uncertainty, and numerous steps if it is to be performedsuccessfully (Wood, 1986).

Second, although self-efficacy is considered to be an important mechanismfor the effects of relapse prevention, no previous study has tested the effectsof RP on trainees’ self-efficacy. As suggested by Tziner et al. (1991), researchon RP should include process measures to understand better how RP worksand if in fact it enhances self-efficacy. Burke and Baldwin (1999) also called forresearch on potential mediating variables such as self-efficacy to understandthe effects of RP on transfer behavior. In the present study, we examined the

hrd15105.qxp 2/10/04 1:42 PM Page 60

effects of RP on trainee self-efficacy immediately after training as well as aftertwo and six months.

Third, previous research on posttraining transfer interventions suffers froma number of measurement limitations. With few exceptions, transfer outcomeshave been measured by self-report, which means they can be deficient andcontaminated (Baldwin & Ford, 1988). Baldwin and Ford have stated thatthere is a critical need for research with relevant criterion measures of transfer.A second problem has been the failure to distinguish between transfer behav-ior and transfer performance. Most studies have focused more on behaviorthan on performance, even though an important purpose of training and devel-opment is to improve performance (Yamnill & McLean, 2001). Kraiger (2002)recently noted the importance of separating changes in behavior from changesin performance that result from training, an ambiguity inherent in Kirkpatrick’straining evaluation model. According to Kraiger, trainees may change theirbehaviors but not improve their job performance, or vice versa. Holton (1996)also called attention to the importance of individual performance measures inhis evaluation model.

Another limitation has been the relatively short time frame over whichtransfer outcomes have been measured, usually several weeks to one month.Furthermore, previous studies have only measured transfer once after train-ing. Thus, most studies have not truly measured maintenance of transfer.Although it is not clear what the best time frame is for measuring transfer fol-lowing training, a good strategy is probably to have a short-term and long-termmeasure. In the present study, we address these issues by having supervisorsevaluate trainees’ transfer behavior and performance in the short term (twomonths after training) and in the long term (six months after training).

Finally, previous research on relapse prevention has seldom tested andcompared the effectiveness of RP to other posttraining interventions. In fact,only one study (Wexley & Baldwin, 1986) compared an RP intervention withother interventions, and it was a laboratory study in which student subjectslearned time-management skills. One of the objectives of the present studywas to compare RP with a new posttraining transfer intervention that we calltransfer enhancement.

Transfer Enhancement Intervention

An implicit assumption in transfer research is that trainees know when to usenew skills on the job and when these skills will be most effective. This mightbe the case in situations where trainees learn routine tasks. However, fortrainees who are trained in a complex interpersonal skill, when to use the newskill may not be as straightforward.

In the present study, we designed and tested the effects of a new post-training transfer intervention called transfer enhancement. The transferenhancement intervention is designed to help trainees identify situations in


hrd15105.qxp 2/10/04 1:42 PM Page 61

62 Gaudine, Saks

their work environment where the application of newly trained skills is likelyto be most useful and effective.

The potential value of this intervention can be surmised based on severalearlier studies. For example, Mayer and Russell (1987) noted that behavioralmodeling may provide trainees with the ability and motivation to implementnew skills, but transfer of training will be minimal if they are unable to recog-nize cues or opportunities in their workplace to use the new skills.

Another relevant study comes from the literature on realistic job previews.Meglino, De Nisi, Youngblood, and Williams (1988) developed an enhance-ment job preview in order to dispel commonly held negative views of new-comers. Rather than focusing on the negative aspects of the job—as a realisticjob preview does—the enhancement job preview tries to make employees feelthat the job is not overly difficult, and includes positive aspects of the new joband organization. The transfer enhancement intervention is similar to theenhancement job preview in that both take a positive view of what the employeewill be able to do rather than of what will be difficult.

As indicated earlier, relapse prevention is expected to increase trainees’self-efficacy for coping with high-risk situations for transfer. In a similar man-ner, we expected the transfer enhancement intervention to increase trainees’self-efficacy by identifying situations in which they will be able to apply thetraining on the job effectively.

Therefore, based on the extant literature and theory, we expected therelapse prevention and transfer enhancement interventions to increase trainees’self-efficacy, transfer behavior, and performance in comparison with a controlgroup. Trainees who received both the relapse prevention and transferenhancement interventions were expected to show the greatest improvementsin self-efficacy, transfer behavior, and performance.

Method

The method was as follows.Participants. Participants were nurses from a medium-size hospital in a

large Canadian city who attended a two-day training program on the McGillModel of Nursing. Most of the nurses worked in one of eleven different nurs-ing units in the hospital. The mean age of the sample was 42.5 years and themean number of years worked at the hospital was 12.4.

Study Design. A longitudinal quasi-experiment was conducted to test theeffects of relapse prevention and transfer enhancement posttraining transferinterventions on trainees’ self-efficacy, transfer behavior, and performance. Thestudy consisted of a 2 � 2 factorial design that crossed relapse prevention(received or not received) and transfer enhancement (received or not received).This resulted in four conditions with approximately equal numbers of partic-ipants in each. In addition, the design also included informal discussions withthe hospital’s nurse managers.

hrd15105.qxp 2/10/04 1:42 PM Page 62

The training program was given twelve times over a five-month periodwith approximately twelve nurses in each session. Scheduling difficulties at thehospital prevented us from assigning nurses randomly to the twelve trainingsessions. Therefore, participants for each session were chosen by the managersresponsible for each of the eleven nursing departments. The managers wereasked to select one or two nurses for each training session. This type of assign-ment to conditions is typical in hospital settings, where it is impossible for allnurses in a unit or department to attend a program at the same time.

Each training session was then randomly assigned to one of the experi-mental conditions. In other words, we randomly assigned each of the twelvetraining sessions to one of the four conditions. Neither the managers nor thenurses were aware of the experimental conditions and all believed that alltwelve sessions were the same. Furthermore, because the trainees in each ofthe sessions came from different nursing departments, they did not representexisting or naturally assembled groups.

This type of design, in which the training group is randomly assigned toa posttraining transfer intervention, has been used in previous studies ofrelapse prevention (Burke, 1997; Burke & Baldwin, 1999). Further, Noe et al.(1990) recommended the use of quasi-experimental designs with pretrainingand posttraining measures and comparison groups to evaluate the effective-ness of relapse prevention strategies. This type of design is also one of the mostcommon forms of quasi-experimental designs (Cook, Campbell, & Peracchio,1990).

Six of the sessions were held in English and six were held in French. TheEnglish and the French workshops were offered alternatively. Each conditionhad English and French participants, and participants completed question-naires in the same language as the session that they attended. A French and anEnglish instructor planned the training program together, and both attendedthe first few sessions, including the transfer interventions, in order to ensurethat the training given to all groups was the same. In addition, it was decidedby the hospital’s director of nursing, who is the first author of this paper, thatthe nurses’ supervisors should attend the training before the nurse trainees inorder for them to be able to rate their nurses’ performance on the McGill modelas well as to facilitate the transfer of training.

Procedure. All participants were informed that a training program on theMcGill Model of Nursing was being provided as part of a university researchstudy. They were also told that their participation would involve completingseveral questionnaires, each taking twenty to thirty minutes of their time. Theywere told that their participation in the study was voluntary, their responseswould be kept confidential, and no one working at the hospital would see theirquestionnaires.

Participants were asked to complete a questionnaire immediately beforethe training; it measured their self-efficacy and asked about background anddemographic variables. A posttraining questionnaire was completed at the end


hrd15105.qxp 2/10/04 1:42 PM Page 63

64 Gaudine, Saks

of the two-day training for the control group and immediately following thetransfer intervention for the experimental groups. In addition, participantswere asked to complete questionnaires at approximately two and six monthsafter the training program.

To measure transfer behavior and performance, participants were alsoasked to give a questionnaire to their head nurse or supervisor at the end ofthe training program (to measure their pretraining behavior and performance)and two and six months later. The head nurse or supervisor returnedthe questionnaire directly to the researchers by mail. The instructions for thisquestionnaire informed the head nurse or supervisor that no one at the hos-pital would see the questionnaire after it had been completed.

The first follow-up questionnaire was completed between two and threemonths after the training. Participants in all four conditions attended a one-hour follow-up session, approximately two months after the training. Thefollow-up sessions were held by the same instructors who had giventhe workshops. This session began with participants completing the studyquestionnaire to measure their self-efficacy. Following this, there was a groupdiscussion, allowing participants to share situations in which they had usedthe McGill Model of Nursing. Those who received a posttraining transferintervention were asked if their plan to increase transfer of training was use-ful. Last, participants were asked to provide their head nurse or supervisorwith a questionnaire to complete. The second follow-up occurred betweensix and seven months after attending the workshop. At this time, participantscompleted a second follow-up questionnaire and were asked to give theirhead nurse or supervisor a questionnaire to complete as part of the six-monthfollow-up.

The pre- and posttraining questionnaires were completed by 147 nurses.The two- and six-month follow-up questionnaires were completed by 118 and95 nurses, respectively. Supervisor reports were received for 110, 102, and 63nurses at each of the three time periods. To test for any significant differencesin behavior and performance between respondents and non-respondents attwo and six months, t tests were performed. Only one significant differencewas found: those who responded at six months had a mean score of 2.32 forpretraining behavior, whereas those who did not respond at six months had amean score of 2.02. Further, the results of a MANOVA on age, number ofyears of experience at the hospital, shift worked, as well as the pretrainingmeasures of self-efficacy, verified the equivalency of the four treatment con-ditions. In addition, the supervisor ratings for behavior and performance weretested by MANOVA and confirmed equivalency of the four groups. Session ofthe workshop that the nurse attended, unit the nurse worked in, language,shift the nurse worked, and tenure were tested as potential covariates, and itwas found that these variables did not have a direct effect on any of the studyvariables.

hrd15105.qxp 2/10/04 1:42 PM Page 64

McGill Model of Nursing Program. Participants in all four conditionsreceived the identical two-day training program on the McGill Model ofNursing. The nurses in this hospital had no or very little knowledge of thismodel prior to the study. Models of nursing serve to guide nursing practice byproviding a framework for nursing care. This framework includes aspects ofcare related to the health of the person, family, and community that are partof the unique knowledge base of the discipline of nursing. When nurses prac-tice without a model of nursing, the care they provide may be restricted to afocus on a patient’s presenting illness and psychosocial problems. Thus, thisworkshop’s training required that nurses question and change their currentbehaviors that were related to how they provided nursing care. Thus, this train-ing was for a complex interpersonal skill, requiring participants to integratelearning and to generalize learning so that they could apply it in the workplace.

The two-day workshop consisted of lectures, small and large group dis-cussions, role plays, group activities, and videos that demonstrated patientassessments and cases. Participants practiced making assessments and devel-oping nursing care plans based on the McGill Model of Nursing.

Posttraining Transfer Interventions. Participants who received aposttraining transfer intervention received it at the end of the second day oftraining. These interventions took approximately thirty to sixty minutes.Participants in the control group only received training on the McGillModel of Nursing. Those in the combined condition received the relapse pre-vention intervention followed immediately by the transfer enhancementintervention.

Relapse Prevention. The relapse prevention intervention was based on theone used by Tziner et al. (1991) and includes the main strategies of Marx’sRP (1982) model: (1) awareness of the relapse process and problem;(2) identification of high-risk situations; and (3) development of copingresponses. The intervention began by informing trainees of the transfer-of-training problem. After this, trainees were asked to write down situationswhere they anticipated problems with transfer of training. These difficultsituations were then discussed in the group. Next, the instructor gave a brieflecture on three potentially useful coping strategies for dealing with transfer oftraining: time management, assertiveness, and reliance on peers for support.Trainees were then asked to document coping strategies that they could use tohelp them in the difficult situations they identified, so that they could developan individualized plan to deal with transfer-of-training problems. Finally, abrief lecture informed them that they should not give up when they wereunable to follow their plan, but rather reflect on the reasons for their failure tosee if they could do anything differently the next time. Trainees kept a copy oftheir plan and submitted a copy to their instructor. Trainees were informedthat they would meet with their instructor in approximately two months todiscuss how they had implemented their plan.


hrd15105.qxp 2/10/04 1:42 PM Page 65

66 Gaudine, Saks

Transfer Enhancement. Like the relapse prevention intervention, thetransfer enhancement intervention began by informing trainees of the transfer-of-training problem. Trainees were then asked to reflect on future oppor-tunities for using their new knowledge, instead of on future problematicsituations, and these opportune times and situations were then discussed bythe group. Trainees then wrote down a personal plan for ensuring thatopportune times would be employed for transfer of training. A brief lecturethen instructed them to monitor for opportune times and to continuedeveloping strategies to ensure that they took advantage of these occasions touse the McGill model.

Trainees kept a copy of their plan to make use of opportune times andsubmitted a copy to their instructor. They were informed that they would meetwith their instructor in approximately two months to discuss how they hademployed their plan.

Measures. All measures were developed in English by the researchers andtranslated into French by a nurse expert. A second nurse expert, who also hasa research background, verified that the translation was accurate. Finally, thetranslation was reviewed for accuracy of meaning by one of the researchers.

Self-Efficacy. The self-efficacy measure developed for this study con-sisted of twenty-nine items based on the content of the workshop and theMcGill Model of Nursing. The instructors reviewed the items to makesure they covered what was taught in the workshop. For each of the twenty-nine items, participants were asked to indicate their degree of confidencethat they could perform the task on a ten-point scale ranging from not at allconfident (1) to totally confident (10). A sample item is this: “Write a nurs-ing care plan that includes concepts from the McGill model.” The Cronbachalpha of the self-efficacy scale was .95 at pretraining, .97 immediately fol-lowing training, and .98 after two and at six months.

Transfer Behavior. The transfer behavior measure consists of actualbehaviors of the McGill Model of Nursing. Trainees’ supervisors were askedto indicate how frequently trainees performed each of twenty-nine behaviors.Supervisors responded to a five-point scale that ranged from never (1) toalways (5). A sample item is this: “Create a collaborative relationshipwith the patient’s family.” The Cronbach alpha was .98 at all three timeperiods.

Transfer Performance. The measure of transfer performance was designedto assess trainees’ performance of the McGill Model of Nursing—that is, howwell they performed the actual behaviors taught in training. Eight items weredeveloped and a ninth was added on the two- and six-month follow-up toevaluate overall performance on the use of the model. Supervisors were askedto rate trainees’ performance on a five-point scale ranging from unsatisfactory(1) to very good (5). A sample item is this: “Writing nursing care plans for hisor her patient’s family.” The Cronbach alpha was .93 at the time of training,.85 at two months, and .90 at six months.

hrd15105.qxp 2/10/04 1:42 PM Page 66

Results

Arvey and Cole (1989) have argued that the use of analysis of covariance(ANCOVA) with a pretest measure covariate is the best approach for testingthe effects of training because it increases power and is superior to other sta-tistical approaches, such as repeated measures ANOVA and posttest-onlydesigns, especially when sample size is small and the reliability of the depen-dent measures is high, as was the case in the present study. Therefore, to testthe effects of the two interventions on the three dependent variables, weconducted ANCOVA at each time period with the corresponding pretrainingmeasure as a covariate.

We tested the data for the assumptions of ANCOVA and found that theassumptions were met, including the assumptions of ANOVA (groups aremutually exclusive, there is homogeneity of variance, dependent variable isnormally distributed) and the assumptions that the covariate is a continuousvariable, there is linearity between covariates and the dependent variable, andthere is homogeneity of variance (Munro, 2001).

The use of ANCOVA is especially appropriate when the power to reject thenull hypothesis is weak. In the present study, the power was in the low to mod-erate range because of the small sample size, especially for the posttraininganalysis at six months. One way to increase the power to reject the null hypoth-esis and decrease the probability of a Type II error is to use a less conservativealpha level (Arvey & Cole, 1989; Sackett & Larson, 1990). Therefore, we usedan alpha level of .10 rather than .05 for this purpose. The power of the signif-icant results reported here was in the range of .42 to .61. Table 1 presents themeans, standard deviations, and sample sizes for all of the dependent variablesby condition.

Effects for Self-Efficacy. We tested the effects of the interventions on self-efficacy immediately after training and at two and six months after training. Ineach case, the pretraining self-efficacy covariate was significant. For posttrain-ing self-efficacy, neither of the intervention main effects was significant; how-ever, the interaction was significant, F (1, 140) � 3.61, p � .10 (eta-square �.025). The interaction effect is due to the fact that the group receivingboth interventions had higher self-efficacy than the other conditions. Attwo months, the effect for the relapse prevention intervention was significant,F (1, 111) � 3.75, p � .10 (eta-square � .03). Contrary to expectations,trainees who received the relapse prevention intervention had lower self-efficacy. Neither the main effects nor the interaction effect were significant atsix months.

Effects for Transfer Behavior. We tested the effects of the interventionson transfer behavior at two and six months with the pretraining behavior mea-sure as a covariate. At two months, the main effect for the transfer enhance-ment intervention was significant, F (1, 68) � 3.17, p � .10 (eta-square �.05), and so was the interaction, F (1, 68) � 5.12, p � .05 (eta-square � .07).


hrd15105.qxp 2/10/04 1:42 PM Page 67

Tab

le 1

.M

ean

s, S

tan

dar

d D

evia

tion

s, a

nd

Sam

ple

Siz

e fo

r Se

lf-E

ffica

cy, B

ehav

ior,

an

d P

erfo

rman

ce b

y C

ond

itio

n

Con

ditio

n Rel

apse

Pre

vent

ion

and

Rel

apse

Pre

vent

ion

Tran

sfer

Enh

ance

men

tTr

ansf

er E

nhan

cem

ent

Con

trol

Vari

able

MSD

NM

SDN

MSD

NM

SDN

1.Pr

etra

inin

g se

lf-ef

ficac

y6.

431.

033

6.57

1.4

396.

721.

237

6.45

1.3

382.

Post

trai

ning

sel

f-ef

ficac

y7.

241.

043

7.25

1.1

387.

661.

137

7.46

1.1

373.

Self-

effic

acy,

tw

o m

onth

s7.

391.

124

7.56

1.1

307.

251.

332

7.62

1.1

314.

Self-

effic

acy,

six

mon

ths

7.24

1.1

197.

531.

222

7.57

1.2

267.

871.

026

5.Pr

etra

inin

g be

havi

or2.

28.7

232.

13.8

242.

15.8

282.

08.8

306.

Tran

sfer

beh

avio

r, t

wo

mon

ths

2.83

.520

2.52

.723

2.89

.727

3.06

.827

7.Tr

ansf

er b

ehav

ior,

six

mon

ths

3.04

.511

3.29

.813

3.22

.718

3.40

.720

8.Pr

etra

inin

g pe

rfor

man

ce2.

49.7

232.

33.7

242.

21.8

272.

27.9

319.

Tran

sfer

per

form

ance

, tw

o m

onth

s3.

08.5

202.

58.6

233.

01.8

273.

11.7

2710

.Tr

ansf

er p

erfo

rman

ce, s

ix m

onth

s3.

16.7

113.

21.9

133.

46.8

193.

55.8

20

hrd15105.qxp 2/10/04 1:42 PM Page 68

So again, contrary to expectations, trainees who received the transfer enhance-ment intervention had lower transfer behavior. Furthermore, the interactioneffect indicates that the transfer enhancement intervention was more effectivewhen it was combined with relapse prevention. Neither the main effects northe interaction effect were significant at six months.

Effects for Transfer Performance. We tested the effects of the interven-tions on transfer performance at two and six months, with the pretrainingperformance measure as a covariate. At two months, the main effect for thetransfer enhancement intervention was significant, F (1, 68) � 3.31, p � .10(eta-square � .05), as was the interaction, F (1, 68) � 3.77, p � .10 (eta-square � .05). Similar to the results for transfer behavior, trainees whoreceived the transfer enhancement intervention had the lowest transfer per-formance. And once again, the interaction effect indicates that the transferenhancement intervention was more effective when it was combined withrelapse prevention. Neither the main effects nor the interaction effect weresignificant at six months.

Discussion

The purpose of this study was to test the effects of a relapse prevention and trans-fer enhancement posttraining transfer intervention. Contrary to expectations,neither intervention improved trainees’ self-efficacy, transfer behavior, or per-formance compared with that of a control group. We believe that there are at leasttwo likely explanations for these results.

First, the effects of a posttraining intervention might depend on the effec-tiveness of the training program itself. For example, if a training program isbased on a needs analysis and is effectively designed and delivered, it is morelikely to result in transfer, thus negating the potential effects of a posttrainingtransfer intervention. In contrast, a poor training program might not resultin transfer whether or not there is a posttraining transfer intervention.

Second, the effect of a posttraining transfer intervention might depend onthe organizational context. For example, transfer is more likely to occur in apositive transfer climate whether or not there is a posttraining transfer inter-vention (Rouiller & Goldstein, 1993). In fact, there is some evidence that a fullRP intervention is not effective in supportive climates (Burke & Baldwin,1999). To interpret the results more clearly, we conducted additional analysesand had informal discussions with the hospital’s nurse managers.

First, we tested the effectiveness of the training program by conductingrepeated measures analysis for each of the dependent variables. This is tanta-mount to a pretest-posttest design, with two posttest measures for transferbehavior and performance and three for self-efficacy. The results indicated asignificant effect for self-efficacy, F (3, 79) � 22.80, p � .001, transfer behav-ior, F (2, 33) � 25.63, p � .001, and transfer performance, F (2, 33) � 27.64,p � .001.


hrd15105.qxp 2/10/04 1:42 PM Page 69

70 Gaudine, Saks

As can be seen in Table 1, the means for all three dependent variables ineach condition show a similar pattern of improvement following training rightup to six months. These results indicate that trainees in all four conditionsshowed a marked improvement in their self-efficacy, transfer behavior, andtransfer performance. Furthermore, the increased levels of self-efficacy, behav-ior, and performance remained high after six months, providing evidenceof transfer generalization and maintenance. It is also worth noting that self-efficacy immediately after training was positively related to transfer behaviorand performance at two months (.21, p � .05 and .24, p � .05) and sixmonths (.36, p � .01 and .34, p � .05). Thus, the training program increasedtrainees’ self-efficacy, and self-efficacy predicted transfer up to six months later.

Second, we used the qualitative data obtained from our informal discus-sions with the hospital’s nurse managers to understand the role that the orga-nizational context played in the study. It appears that a number of factors inthe organizational context as well as the transfer system might have contributedto the transfer of the training program. According to Holton, Bates, and Ruona(2000, pp. 335–336), the transfer system refers to “all factors in the person,training, and organization that influence transfer of learning to job perfor-mance,” such as supervisor support, peer support, perceived content validity,transfer design, and opportunity to use new skills on the job. Some of thesefactors are particularly relevant in the present study.

First, the training program was selected by a committee composedprimarily of staff nurses with representation from most nursing units. Thiscommittee chose the McGill Model of Nursing for use by the nursing depart-ment in response to a professional inspection report on the nursingdepartment made by a governing body of nursing that suggested the hospi-tal implement a model. Thus, trainees had some choice in the decisionprocess that led to the training program. Second, many aspects of the orga-nizational environment were favorable to and supportive of the training. Forexample, the committee recommended to the director of nursing thatinstruction be given on the McGill Model of Nursing. The hospital educa-tion coordinator discussed the use of education money for this project withthe hospital’s nursing union representatives, and they agreed about the valueof the project. Thus, the hospital’s nurses agreed in principal with the train-ing project even before attending the workshop. The nursing supervisors and147 nurses received the same training. The director of nursing was knownto be a McGill nurse graduate, and the stream of research questionnairesthe nurses and their supervisors were given demonstrated her interest in theimplementation of the training. As a result, there was a great deal of supporton the part of nurses, supervisors, management, and the union for the train-ing. This provided a very positive and supportive climate for the trainingprogram and for transfer, something that has been shown to be extremelyimportant for transfer (Rouiller & Goldstein, 1993; Tracey, Tannenbaum, &Kavanagh, 1995).

hrd15105.qxp 2/10/04 1:42 PM Page 70

Third, trainees had frequent opportunities to transfer immediately uponreturning to the work environment, which has also been shown to be a criti-cal factor for transfer (Ford et al., 1992). As well, because all nurse employeesreceived the training, regardless of the experimental condition, there existed acritical mass, thereby building in support and acceptance for the trainingprogram and its application on the job (Broad & Newstrom, 1992).

In summary, we believe that many of the critical factors required for suc-cessful transfer were operating in the organizational context and the transfersystem. Consequently, the training program itself resulted in a high degree oftransfer generalization and maintenance, and therefore negated the potentialeffects of the posttraining transfer interventions. This interpretation is consis-tent with contemporary models of transfer of training (Machin, 2002) as wellas training effectiveness, in which training is conceptualized in the organiza-tional context and the transfer system (Holton et al., 2000; Kozlowski et al.,2000).

Implications for Practice

Our results, combined with the results of previous studies, are mixed andinconsistent on the effectiveness of relapse prevention. We suspect that this isnot because RP is ineffective, but rather because there is a lack of attention tothe training situation and the organizational context. Transfer research almostalways begins with the premise that transfer of training is a problem and atransfer intervention is the solution. However, it now appears that thisapproach is far too simplistic. Instead, the extent to which transfer is a prob-lem is likely to vary across training situations and organizational contexts, andas a result, the need for a posttraining transfer intervention as well as the typeof intervention is likely to depend on a number of factors.

In fact, two recent studies demonstrate how the effectiveness of a post-training transfer intervention depends on the organizational context. Burkeand Baldwin (1999) found that a full relapse prevention intervention was onlyeffective in climates that were unsupportive and concluded that “there is lessneed for RP tools as the transfer climate becomes more supportive and that notransfer intervention may even be optimal in these climates” (p. 235). Inanother study, Richman-Hirsch (2001) found that a posttraining goal-settingintervention was most effective for trainees who worked in a supportive workenvironment. Thus, on the basis of these two studies, it appears that someposttraining interventions will be more effective in unsupportive environments(that is, relapse prevention) whereas others will be more effective in support-ive environments (that is, goal setting).

In addition to the organizational context, factors in the training situationare also likely to influence the type of posttraining transfer intervention thatwill be most effective. For example, there is some evidence that the effective-ness of relapse prevention depends on trainees’ locus of control (Tziner et al.,


hrd15105.qxp 2/10/04 1:42 PM Page 71

72 Gaudine, Saks

1991). Thus, some trainees might need and benefit from relapse prevention orsome other posttraining intervention more than others. The nature of the taskmight also be relevant for the effects of posttraining transfer interventions. Forexample, relapse prevention might be more effective with tasks that are morecomplex and difficult to learn and transfer. Complex tasks take longer to learnand transfer and are therefore more likely to result in a relapse than more sim-ple and routine tasks, which can be learned and applied more rapidly. In addi-tion, the effect of RP and other posttraining strategies might depend on thetype of learning outcome (Noe et al., 1990).

Thus, an important practical implication of this study is that a transfer oftraining needs analysis (TTNA) should be conducted in order to determine thekinds of transfer obstacles that exist in the organizational context and for a par-ticular training program (Hesketh, 1997). This should be a starting point inthe determination of how best to deal with transfer problems. In addition toexamining the organizational context, information about trainees and the train-ing task should also be obtained in order to determine the best approach fordealing with transfer obstacles. In some cases, the best solution might be pro-grams and activities that take place before, during, or after a training programand might involve management, trainers, or trainees (Burke, 2001; Machin,2002). Thus, a posttraining transfer intervention might not be the best or onlysolution for improving the transfer of training.

If a posttraining transfer intervention is to be part of the solution to atransfer problem, it is important to determine what type of posttraining inter-vention will be most effective. In this regard, we recommend a contingencyapproach based on the information obtained from the TTNA. As indicated ear-lier, some of the factors to consider include trainee characteristics, nature ofthe task, and transfer climate. This should then enable trainers to choose themost appropriate posttraining transfer intervention for a particular training sit-uation and organizational context. As well, an instrument such as the Learn-ing Transfer System Inventory (LTSI) (Holton et al., 2000) might be useful forconducting a diagnosis of the transfer system to identify transfer barriers andthe most appropriate posttraining transfer intervention. As noted by Holtonet al., it is easy to choose the wrong intervention if the transfer system is notdiagnosed first.

Implications for Research

Although we have recommended a contingency approach for posttrainingtransfer interventions, much more research is needed to identify when a par-ticular intervention will be most effective. Therefore, future research needs toinclude measures of trainee and task characteristics as well as the transfer sys-tem when testing the effectiveness of posttraining transfer interventions.Research along these lines is necessary in order to learn when an interventionwill be effective and to provide practitioners with specific guidelines about

hrd15105.qxp 2/10/04 1:42 PM Page 72

when to use a particular posttraining transfer intervention, and what factorsare most important to consider when choosing one.

We also recommend future research on the transfer enhancement inter-vention introduced in this study. Although our results did not support its effec-tiveness, this might be the result of the factors we discussed earlier, whichresulted in transfer generalization and maintenance across all conditions. How-ever, it is worth noting that participants in the transfer enhancement conditionhad the lowest means for transfer behavior and performance at two monthsafter training. Interestingly, this was not the case for trainees who receivedtransfer enhancement combined with relapse prevention. Thus, the transferenhancement intervention without relapse prevention might have inflatedtrainees’ expectations for transfer.

Although this explanation is speculative, since we did not measure trainees’transfer expectations, it is consistent with realistic job preview theory and train-ing research. Several studies have found that expectations about training arerelated to training outcomes and that unmet expectations about trainingcan have negative consequences for performance (Hoiberg & Berry, 1978;Tannenbaum et al., 1991). Furthermore, Hicks and Klimoski (1987) found thattrainees who received a realistic preview of a training program believed theworkshop was more appropriate for them to take, were more motivated to learnand committed in their decision to attend, and were better able to profit fromthe training.

The finding that transfer enhancement was more effective when it wascombined with relapse prevention is noteworthy because it suggests thatrelapse prevention might actually be most effective for trainees who have hightransfer expectations, in the same way that a realistic job preview is most effec-tive for new hires who have inflated job expectations. Thus, future researchmight test the hypothesis that relapse prevention works best with traineeswho have high expectations about transfer, whereas transfer enhancementmight be more effective for trainees who have low expectations for transfer.Future research on transfer might also consider the role of transfer expecta-tions in the transfer process. In particular, research is needed on the effects ofposttraining transfer interventions on trainees’ transfer expectations as well asother process variables that might explain how posttraining transfer interven-tions work.

Study Limitations and Conclusions

A number of limitations should be kept in mind when interpreting the resultsof this study. First, this study involved experienced members of one occupa-tion, in one organization, and used a specific, complex training program. As aresult, the results may only be generalized to other training programs, occu-pations, and organizations with caution. In terms of the internal validity of thisresearch, one might argue that a demand effect occurred to the extent that


hrd15105.qxp 2/10/04 1:42 PM Page 73

74 Gaudine, Saks

participants and their supervisors wanted to provide evidence to support thetraining program. However, anecdotal observation of actual changes thatoccurred in the nursing department (for example, surgical nurses began tophone patients a few days prior to scheduled surgery, as well as a day or soafter surgery, in an attempt to address patient concerns and need for informa-tion) provided further support for the effectiveness of the training program.

It is also possible that contamination of treatment and control groups mayhave occurred. Although no evidence of this was found, it is possible thatparticipants using relapse prevention strategies were observed and emulatedby their coworkers. Such an effect would have attenuated the effects of theposttraining transfer interventions.

Finally, the results of this study must also be considered in light of the con-struct validity of the relapse prevention manipulation and concerns aboutstatistical conclusion validity. Relapse prevention is a general approach forpreparing trainees for high-risk situations in the transfer environment. RPinterventions tend to vary from study to study, and as shown by Burke andBaldwin (1999) the effectiveness of RP depends in part on how the interven-tion has been operationalized. Thus, a different operationalization of the RPconstruct, such as the full RP condition tested by Burke and Baldwin, mightlead to different results from those reported here. In addition, the results ofthis study must also be interpreted with caution given the low power to detectsignificant effects for the posttraining transfer interventions.

In conclusion, the results of this study suggest that in order to advanceboth the theory and practice of posttraining transfer interventions, a newapproach to research is required. In particular, much more attention must begiven to the organizational context and the transfer system in order to deter-mine when posttraining transfer interventions will be most effective and whattype of intervention will be effective in different situations. Only then will webe able to provide sound advice to practitioners about the use and effective-ness of posttraining transfer interventions. This is a fundamental requirementin meeting the challenge of translating training science into practice (Salas &Cannon-Bowers, 2001).

References

Arvey, R. D., & Cole, D. A. (1989). Evaluating change due to training. In I. L. Goldstein (Ed.),Training and development in organizations (pp. 89–117). San Francisco: Jossey-Bass.

Baldwin, T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research.Personnel Psychology, 41, 63–105.

Broad, M. L., & Newstrom, J. W. (1992). Transfer of training. Reading, MA: Addison-Wesley.Burke, L. A. (1997). Improving positive transfer: A test of relapse prevention training on trans-

fer outcomes. Human Resource Development Quarterly, 8, 115–128.Burke, L. A. (2001). Training transfer: Ensuring training gets used on the job. In L. A. Burke

(Ed.), High-impact training solutions: Top issues troubling trainers. Westport, CT: Quorum.Burke, L. A., & Baldwin, T. T. (1999). Workforce training transfer: A study of the effect of relapse

prevention training and transfer climate. Human Resource Management, 38, 227–242.

hrd15105.qxp 2/10/04 1:42 PM Page 74

Cook, T. D., Campbell, D. T., & Peracchio, L. (1990). Quasi experimentation. In M. D. Dunnette &L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 1, 2nd ed.,pp. 491–576). Palo Alto, CA: Consulting Psychologists Press.

Ford, J. K., Quinones, M. A., Sego, D. J., & Sorra, J. S. (1992). Factors affecting the opportunityto perform trained tasks on the job. Personnel Psychology, 45, 511–527.

Gist, M., Stevens, C., & Bavetta, A. (1991). Effects of self-efficacy and post-training interventionon the acquisition and maintenance of complex interpersonal skills. Personnel Psychology, 44,837–861.

Haccoun, R. R., & Saks, A. M. (1998). Training in the 21st century: Some lessons from the lastone. Canadian Psychology, 39, 33–51.

Hesketh, B. (1997). W(h)ither dilemmas in training for transfer. Applied Psychology, 46, 380–385.Hicks, W. D., & Klimoski, R. J. (1987). Entry into training programs and its effects on training

outcomes: A field experiment. Academy of Management Journal, 30, 542–552.Hoiberg, A., & Berry, N. W. (1978). Expectations and perceptions of navy life. Organizational

Behavior and Human Performance, 21, 130–145.Holton, E. F., III. (1996). The flawed four-level evaluation model. Human Resource Development

Quarterly, 7, 5–21.Holton, E. F., III, Bates, R. A., & Ruona, W.E.A. (2000). Development of a generalized learning

transfer system inventory. Human Resource Development Quarterly, 11, 333–360.Kozlowski, S.W.J., Brown, K. G., Weissbein, D. A., Cannon-Bowers, J. A., & Salas, E. (2000). A

multilevel approach to training effectiveness: Enhancing horizontal and vertical transfer. InK. J. Klein & S.W.J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations(pp. 157–210). San Francisco: Jossey-Bass.

Kraiger, K. (2002). Decision-based evaluation. In K. Kraiger (Ed.), Creating, implementing, andmanaging effective training and development (pp. 331–375). San Francisco: Jossey-Bass.

Latham, G. P., & Frayne, C. A. (1989). Self-management training for increasing job attendance:A follow-up and a replication. Journal of Applied Psychology, 74, 411–416.

Latham, G. P., & Seijts, G. H. (1997). Overcoming mental models that limit research on transferof training in organizational settings. Applied Psychology, 46, 371–375.

Machin, M. A. (2002). Planning, managing, and optimizing transfer of training. In K. Kraiger(Ed.), Creating, implementing, and managing effective training and development (pp. 263–301).San Francisco: Jossey-Bass.

Marx, R. (1982). Relapse prevention for managerial training: A model for maintenance of behav-iour change. Academy of Management Review, 7, 433–441.

Mathieu, J. E., Martineau, J. W., & Tannenbaum, S. I. (1993). Individual and situational influ-ences on the development of self-efficacy: Implications for training effectiveness. PersonnelPsychology, 46, 125–147.

Mayer, S., & Russell, J. (1987). Behavior modeling training in organizations: Concerns andconclusions. Journal of Management, 13, 21–40.

Meglino, B., De Nisi, A., Youngblood, S., & Williams, K. (1988). Effects of realistic job previews:A comparison using an enhancement and a reduction preview. Journal of Applied Psychology,73, 259–266.

Munro, B. (2001). Statistical methods for health care research. Philadelphia: Lippincott.Noe, R., Sears, J., & Fullenkamp, A. (1990). Relapse training. Journal of Business and Psychology,

4, 317–328.Richman-Hirsch, W. L. (2001). Posttraining interventions to enhance transfer: The moderating

effects of work environments. Human Resource Development Quarterly, 12, 105–120.Rouiller, J. Z., & Goldstein, I. L. (1993). The relationship between organizational transfer climate

and positive transfer of training. Human Resource Development Quarterly, 4, 377–390.Sackett, P. R., & Larson, J. R. Jr. (1990). Research strategies and tactics in industrial and organi-

zational psychology. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and orga-nizational psychology (Vol. 1, 2nd ed., pp. 419–489). Palo Alto, CA: Consulting PsychologistsPress.


hrd15105.qxp 2/10/04 1:42 PM Page 75

76 Gaudine, Saks

Saks, A. M. (1995). Longitudinal field investigation of the moderating and mediating effects ofself-efficacy on the relationship between training and newcomer adjustment. Journal of AppliedPsychology, 80, 211–225.

Salas, E., & Cannon-Bowers, J. A. (2001). The science of training: A decade of progress. AnnualReview of Psychology, 52, 471–499.

Tannenbaum, S. I., Mathieu, J. E., Salas, E., & Cannon-Bowers, J. A. (1991). Meeting trainees’expectations: The influence of training fulfilment on the development of commitment,self-efficacy, and motivation. Journal of Applied Psychology, 76, 759–769.

Tracey, J., Tannenbaum, S. I., & Kavanagh, M. (1995). Applying trained skills on the job: Theimportance of the work environment. Journal of Applied Psychology, 80, 239–252.

Tziner, A., Haccoun, R., & Kadish, A. (1991). Personal and situational characteristics influenc-ing the effectiveness of transfer of training improvement strategies. Journal of OccupationalPsychology, 64, 167–177.

Wexley, K., & Baldwin, T. (1986). Post-training strategies for facilitating positive transfer: Anempirical exploration. Academy of Management Journal, 29, 503–520.

Wood, R. E. (1986). Task complexity: Definition of the construct. Organizational Behavior andHuman Decision Processes, 37, 60–82.

Yamnill, S., & McLean, G. N. (2001). Theories supporting transfer of training. Human ResourceQuarterly, 12, 195–208.

Alice P. Gaudine is an associate professor of nursing in the School of Nursing atMemorial University of Newfoundland, Canada.

Alan M. Saks is professor of organzational behavior and human resourcemanagement in the Division of Management and the Joseph L. Rotman Schoolof Management at the University of Toronto, Ontario, Canada.

hrd15105.qxp 2/10/04 1:42 PM Page 76

Documents

A Longitudinal Experiment on the Effects of Post Training Transfer - Gaudine Et Al