TRAINEES’ REACTIONS TO TRAINING:

SHAPING GROUPS AND COURSES FOR HAPPIER TRAINEES

REFEREED PAPER 1009, Streams 10: WORKPLACE LEARNING

Antonio Giangreco Iéseg School of Management

3, Rue de la Digue F-59800 Lille, France

e-mail: a.giangreco@ieseg.fr

Andrea Carugati Aarhus School of Business

University of Aarhus Fuglesangs Allé 4

8210 Århus V, Denmark e-mail: andreac@asb.dk

Antonio Sebastiano CREMS Centro di Ricerca in Economia e Management in Sanità

Università Carlo Cattaneo – LIUC Corso Matteotti, 24

21053 Castellanza (VA), Italy e-mail: asebastiano@liuc.it

Davide Della Bella CSFU - Consorzio Sistemi Formativi UCIMU

V.le Fulvio Testi, 128 20092 Cinisello Balsamo (MI), Italy e-mail: davide.dellabella@probest.it

Please send all correspondence to Antonio Giangreco

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

2

TRAINEES’ REACTIONS TO TRAINING:

SHAPING GROUPS AND COURSES FOR HAPPIER TRAINEES

Abstract

The extensive use of training as a managerial tool in today’s companies carries a

significant need of evaluating its real effects. In the last fifty years, the great debate on

training evaluation has interested both HR scholars and practitioners producing however a

tangible gap. In fact, while HR scholars tend to develop multidimensional training evaluation

models, often ignored by training managers, a large majority of HR practitioners, above all in

small and medium size companies, stops at the level of reactions evaluation, which is the first

dimension assessed by several models. Focusing on the evaluation of reactions, this study

aims at testing a set of four course-related and two participant-related variables that may

maximise the trainees’ satisfaction with the training. This is done by inserting these six

variables into a research model present in the literature on the reactions evaluation, in which

the trainees’ overall satisfaction with training (OST) is the result of three main antecedents

namely perceived usefulness of training, perceived trainer performance and perceived

training efficiency. To test these hypotheses we used a structured survey data from a sample

of 13,753 trainees grouped in 1,230 courses and involving 3,047 companies financed by an

Italian inter-professional fund. The results confirmed the role of all three antecedents of OST.

Moreover, of the six course- and participant-related variables, the percentage of female

among trainees (also as a moderator variable), the length of the course and the type of

occupational group as target audience showed an impact on trainees’ overall satisfaction with

the training. These results may provide useful implications for practitioners in shaping

courses and audiences to maximise the trainees’ reactions.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

3

Keywords: training evaluation, reactions evaluation, trainee perceptions, moderator effects,

courses’ and participants’ characteristics.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

4

Introduction

In a business world that is changing very quickly, training is considered among the

most effective tools to enhance employees’ knowledge, skills and behaviour (Seyler et al.,

1998; Tan et al., 2003). Due to its increasing critical role in times of change, companies rely

on and invest a significant amount of resources on training (Kraiger et al., 2004). As a

consequence, we may expect that the need to fully evaluate training in terms of efficiency and

effectiveness would increase with the same intensity (Santos and Stuart, 2003), above all in

times of economic contraction when the need for accountability and control is higher

(Tannenbaum and Woods, 1992; Taylor et al., 1998) or when training is outsourced (Gainey

and Glass, 2003).

Training evaluation, however, is a complex process (Mckillip, 2001). In fact, over the

last fifty years, training evaluation has been at the centre of the debate between HR scholars

and practitioners. Indeed, training evaluation is among those topics that register a discrepancy

between what HR scholars developed in their literature and what HR practitioners put into

action (Salas et al., 1999). On the one hand, in fact, following the direction taken by the

Kirkpatrick’s (1959a, 1959b, 1960a, 1960b, 1967 and 1996) hierarchical model of training

outcomes, HR scholars (e.g. Warr et al., 1970; Hamblin, 1974; Cannon-Bowers, 1995;

Kaufman et al., 1995; Molenda et al., 1996; Phillips, 1997 and 2003; Cascio, 1999) have put a

lot of effort into the design of articulated multi-dimension and multi-level training evaluation

models. On the other hand, HR practitioners have only partially adopted those models

(Pershing and Pershing, 2001), so that a large majority of companies, especially small and

medium size firms, stops at the level of reactions evaluation (Mann and Roberston, 1996), due

to the lack of knowledge (Kraiger et al, 2004), interest (Swanson, 2005), time and financial

resources (Plant and Ryain, 1992; Sims, 1993; Wang and Wilcox, 2006) or due to a

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

5

costs/benefits analysis where the tangible and intangible costs seem to be higher than the

related benefits (Cullen et al., 1978; Green, 1996).

All these have also produced an additional effect on the literature on training

evaluation. In fact, although reactions evaluation is by far the most popular and frequently the

only dimension assessed by companies (Olsen, 1998; Santos and Stuart, 2003), journals tend

to be reluctant to publish studies mainly based on this (Arthur et al., 2003). So published

studies material that has focused on full training evaluation increasingly struggles with results

as they move from reactions to more complex dimensions (e.g. learning, behaviours and

organisational results, Level 2, 3 and 4 for the Kirkpatrick model), calling (and wishing) for

more companies apply for full evaluation training models (Arthur et al., 2003). The

consequence of these differences is that HR scholars and practitioners are de facto ignoring

each other to the extent that research focusing on more advanced tools is practically ignored

by many companies (Giangreco et al., 2008a).

Our efforts here are directed towards making these positions closer. We may argue, in

fact, that, as the reactions evaluation is greatly used, HR practitioners may be interested in

identifying some trainee and course characteristics that can help to predict the level of

satisfaction to training. The aim of our paper is twofold. Firstly, a more specific aim is to

identify and test a number of alternative hypotheses on the effects of overall six course-related

and participant-related (length of the course, type of content, number of participants, type of

audience, female participation and age variance) variables on the overall level of satisfaction

with the training of individuals participating in the same training initiative. These variables

are placed within a model that was recently proposed in the literature on reactions evaluation

(Giangreco et al., 2008b). In this model, the overall satisfaction to training is thought to be

linked to three main antecedents. These are, firstly the extent to which trainees perceive

training as being properly organised in terms of planning, support and premises; secondly, the

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

6

extent to which trainees perceive training as being useful for their present job and future

development (Rowold, 2007); and thirdly, the extent to which trainees perceive the trainer as

adequate in terms of knowledge, time management and their trainees’ involvement (Russ-Eft

et al., 2005). To test all these hypotheses we used more than 13,000 individual observations

grouped in more than 1,200 courses from seven training programmes in the private sector that

took place in the western part of Lombardy region (northern Italy), between 2005 and 2006.

Starting from that, a second aim of our paper oriented to practitioners is to maximise the

reactions evaluation of trainees through choices to be taken in the phase of planning of the

training.

The rest of the article is organised as follows. We first present the research model and

the hypotheses to be tested in the study. We then provide a brief description of the research

context, together with the data set, measures and analytical procedures used in the study. We

conclude by presenting the key findings and the discussion of the theoretical implications and

policy.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

7

Theory and Research Models

In the present section we first debate the importance of trainees’ reactions in training

evaluation, keeping in mind the limitations that HR scholars attribute to the first level of the

Kirkpatrick model. We then present the research model and the hypotheses to be tested

adding six variables between course-related and participant-related characteristics from a

previous study (Giangreco et al., 2008b). These are length of the course, type of content,

number of participants, and target audience - as course characteristics - and female

participation and age variance - as participant characteristics. Subsequently female

participation and age variance are tested as moderator variables of the impact of what

individuals perceive about the usefulness of training and the performance of the trainer on

OST.

Reactions Evaluation and Overall Satisfaction to Training

The debate on the usefulness of reactions evaluation is still alive and intense. Several

scholars, indeed, most of the time as critics of the Kirkpatrick model (for example

Tannenbaum and Woods, 1992; Holton, 1996; Bates, 2004; Brown, 2005; Blanchard and

Thacker, 2007), attribute a lack of a proven cause-effect relationship, as well as a lack of

proven progressive importance of information moving from Level 1 (reactions), to Level 2

(learning), Level 3 (behaviours) up to Level 4 (results).

Little reflection is necessary on these two limits. First of all, different empirical

research show contradictory findings on the cause-effect issue between reactions and other

constructs (Alliger et al. 1997). For example, Russ-Eft et al. (2005) demonstrate that negative

reactions predict lower learning, while Tan et al. (2003) surprisingly find that negative

reactions imply higher learning. According to Dixon (1990) and Rowold (2007), positive

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

8

reactions don’t automatically mean positive learning. McEvoy (1997), instead, raises the

timing issue in that trainees usually leave the training in a status of euphoria, showing much

higher positive reactions than if done later. In a recent nomological study on reactions

evaluation, Sitzmann et al. (2008) underline that it is not correct to establish that reactions

don’t have any impact on learning. Moreover, besides this, it has been proven that positive

training experiences have positive consequences on employees’ attitudes and behaviour such

as organisational commitment and job motivation (Meyer and Allen, 1997; Rhoades and

Eisenberger, 2003). In such a scenario, the need of proven progressive importance of

information between the different levels becomes less critical for two reasons: firstly, it would

not be realistic to think that the effect of a training initiative would stop at a certain level

without influencing the following levels or part of them (Seyer et al., 1998); secondly,

training is not always meant to produce output at all levels and sometimes aims at achieving

objectives – for example integration for new employees (Tannenbaum et al., 1991), retain of

knowledge workers (Alvesson, 2000), higher motivation of workers or stronger team spirit -

that are not necessarily detected by most training evaluation models used. “Consistent with

Kraiger (2002), we suggest that the goal of an evaluation effort should guide the selection of

the outcome measures. Within this context, reactions are one potentially outcome, depending

on the goal of training” (Sitzmann et al., 2008: 23).

Reactions evaluation, however, is useful for several reasons well identified by Morgan

and Casper (2000). Firstly, reactions evaluation can provide useful feedback for the design

and improvement of training initiatives (also Lee and Pershing, 2002); secondly, it develops a

function of customer relations with employees showing that the organisation pays attention to

training (Santos and Stuart, 2003); thirdly, it may potentially forecast the use of more

sophisticated evaluation tools (Sims, 1993).

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

9

Following the directions taken by Casper and Morgan (2000), our intention is to

enhance the knowledge of trainee reactions for a better understanding of training effectiveness

in order to make the interests and mutual benefits of HR scholars and practitioners on the

subject of training evaluation closer to each other. With these premises, trainee reactions as

“subjective evaluations of the training experience” (Rowold, 2007: 23) represent an overall

attitudinal measure of satisfaction and evaluation with regards to the training which they took

part in (Alliger and Janak, 1989).

The Antecedents of OST

In building our research models we started from the findings of Giangreco et al.

(2008b). In their study, they identified three key determinants of overall satisfaction with

training. The first, namely perceived training efficiency (PTE) refers to the perception of the

organisational aspects of the training initiative in terms of materials, tools and premises; the

second, namely perceived usefulness of training (PUT) is related to the individual perception

of the utility of training in the present work situation and future development (Rowold, 2007);

the third, namely perceived trainer performance (PTP) concerns the individual perception of

the quality of the trainer’s performance in terms of content and process.

In relation to PTE, Giangreco et al. (2008b) showed that the more the trainees (i) felt

that the teaching materials used was valid (Kidder and Rouiller, 1997), (ii) thought the course

was administered properly so that the location, the premises and the didactical supports were

adequate (Lee and Pershing, 2002), and (iii) found the course planning was well balanced and

compatible with the work style of trainees (Amietta, 2000; Morgan and Casper, 2000), the

more the participants were satisfied overall with the training.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

10

In reference to PUT, Giangreco et al (2008b) confirmed the strong link between the

perception of the utility of training and the levels of satisfaction with it. In fact, they

confirmed that the more trainees (i) found the topics developed as useful for present work

responsibilities (Brown and Reed, 2002), (ii) thought that the content was relevant to their

present individual learning needs (Webster and Martocchio, 1995), (iii) believed subjects

taught will help their future development (Noe, 1986), (iv) considered the training coherent

with the objectives declared, and (v) positively evaluated the balance between theoretical and

practical aspects (Morgan and Casper, 2000; Sebastiano and Bellet, 2005), the more the

participants found the training satisfying.

Efficiency and usefulness of training, however, don’t necessarily ensure high levels of

satisfaction due to the fact that the perception of the trainer’s performance (namely perceived

trainer performance, PTP) may have an important effect on the overall satisfaction with the

training (Morgan and Casper, 2000; Russ-Eft et al., 2005). Specifically, the more trainees find

that the trainer (i) masters the topics, (ii) delivers properly the subjects, (iii) uses time

adequately, (iv) adopts suitable methodologies, and (v) is capable of involving the audience

(Morris, 1984; Steiner et al., 1991; Hook and Bunce, 2001), the more the participants are

satisfied with the training. On this basis, we therefore test the following hypotheses

(Giangreco et al., 2008b: 10):

H1. The perceived efficiency of training will be positively related to trainees’ overall

satisfaction with training. The more individuals perceive that the training is effectively

organised and managed, the greater their level of overall satisfaction with the training

will be.

H2. The perceived usefulness of training will be positively related to trainees’ overall

satisfaction with training. The more useful individuals perceive the training to be, the

greater their level of overall satisfaction with the training will be.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

11

H3. Perceived trainer performance will be positively related to trainees’ overall

satisfaction with training. The more highly individuals rate the performance of

trainers, the greater their level of overall satisfaction with the training will be.

In building up our model we follow the direction taken by some scholars who included

in the evaluation of training some course-related and participant-related characteristics (e.g.

Warr et al., 1999; Holladay and Quinones, 2005; Lorenzet et al., 2005; Combs and Luthans,

2007).

In relation to the course characteristics we included length of the course, type of

content, number of participants and target audience. Length of the course refers to the

number of hours for each course. A view shared by HR practitioners is that individuals tend to

be more satisfied with short courses rather than longer ones (Sitzmann et al., 2008). This may

be associated to lower disruption of work activities (related to Amietta, 2000) – after all it is

only a day out of the office! - or to reduced reciprocal exposition of dysfunctional dynamics –

it is too short to see the awkward side of you! -. Type of content concerns the nature of topics

taught. Giangreco et al. (2008b) found that ‘soft’ behavioural and relational topics were

associated with higher level of satisfaction with the training than courses dealing with ‘hard’

technical and managerial issues. In relation to the number of participants, it is necessary to

underline that usually HR practitioners identify 25-30 the maximum number of participants in

an effective training initiative. More than that can not be considered as continuing education

or vocational training. So, a higher number of participants is associated to a higher level of

satisfaction due to the better class atmosphere, interaction and participation of trainees. Target

audience refers to a single occupational group, with participants coming from the same

company, rather than multiple occupational groups, with trainees coming from different

companies. This is also attributed to the fact that people of different age, seniority, race,

gender, ability and experience will provide a positive and enriching contribution to the

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

12

organisation (Combs and Luthans, 2007). By extension, therefore, the increasing diversity of

training audience can be associated to higher levels of satisfaction with training. Based on

these arguments, we propose the following hypotheses:

H4. The length of a course is negatively related to trainees’ overall satisfaction with

training. The shorter the course is in terms of hours, the greater the individual level of

overall satisfaction with the training will be.

H5. The type of content in terms of nature of topics taught is positively related to trainees’

overall satisfaction with training. The more the content concerns behavioural and

relational issues (rather than technical and managerial ones), the greater the

individual level of overall satisfaction with the training will be.

H6. The number of participants is positively related to trainees’ overall satisfaction with

training. The higher the number of trainees (up to 25), the greater their level of

overall satisfaction with the training will be.

H7. The presence of a multiple occupational group (Vs a single occupational group) is

positively related to trainees’ overall satisfaction with the training.

Traditionally, studies on trainee characteristics were mainly focused on the selection

of trainees who were supposed to take training, rather than understanding what kind of effect

trainee characteristics can have on training (Tannenbaum and Yukl, 1992), training evaluation

and more specifically on reactions evaluation. In our model we consider the gender

distribution in terms of female participation and age distribution in terms of age variance for

each course audience. In relation to gender distribution, HR practitioners tend to believe that

female workers show higher levels of overall satisfaction with the training. A possible

explanation is provided by the fact that lower job involvement and discriminatory behaviours

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

13

against female workers (and also minorities) still exist (Tam, 1997; Robertson and Block,

2001). In fact, being that “female are matched to positions that have a shorter duration of

training …” (Barron et al. 1993: 336), as reaction female workers who are involved in

training initiatives, approach training with more commitment and are willing to maximise the

benefits of the experience. With reference to the age variance, HR practitioners tend to

assume that the more diversified the audience, the more trainees will be satisfied with the

training experience. This is based on similar considerations developed for the target audience,

given that also differences in age will offer the organisation better conditions (Combs and

Luthans, 2007). Based on these statements, we propose the following hypotheses:

H8. The female participation is positively related to trainees’ overall satisfaction with the

training.

H9. The age variance is positively related to trainees’ overall satisfaction with the

training.

The Moderator Effects

In the previous section, we presented the major issues and related hypotheses that link

OST to its potential antecedents. However, the characteristics related to the participants –

female participation and age variance - can be also seen as a moderator of the impact of

perceived usefulness of training and perceived trainer performance on OST. In studies on

diversity training (for example Holloday and Quinones, 2005), gender (in this case of the

trainer) is taking into account as a potential moderator variable of training usefulness and

instructor effectiveness and their respective antecedents. This is a more articulated issue

mainly based on HR practitioners’ experiences and beliefs. Our aim is to investigate whether

these two characteristics of trainees may have an effect on the impact of the two strongest

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

14

determinants – PUT and PTP respectively according to Giangreco et al. (2008) – of overall

satisfaction with the training.

We capture this possibility in the following hypotheses:

H10. The female participation will positively moderate the effect of perceived usefulness of

training on OST.

H11. The female participation will positively moderate the effect of perceived trainer

performance on OST.

H12. The age variance will positively moderate the effect of perceived usefulness of training

on OST.

H13. The age variance will positively moderate the effect of perceived trainer performance

on OST.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

15

Methods

The Research Context

Fondimpresai is an Italian national agency created by a joint inter-professional fund by

the most important confederation of Italian enterprises (Confindustria, 126,590 companies

associated with 4,771,000 employees) and the major trade union confederations (CGIL, CISL

and UIL, all together count about 3 millions members). The main aim of Fondimpresa is to

make training accessible to companies and workers as a tool to enhance innovation and

development. This is very important not only for large companies, but also for small and

medium size firms which represent a great part of the Italian economy, as well as playing a

significant role in the success of the “made in Italy”.

Fondimpresa gathers 52,212 companies involving 2,686,319 workers, of which 27,938

(53.51%) and 2,021,002 (75.23%) are respectively the companies and workers belonging to

the manufacturing sector. Specifically, in the three main traditional sectors of the Italian

economy (production of metal, production of machine tools and textile and fashion),

Fondimpresa gathers respectively 19.76%, 17.41% and 11.01% of companies, involving

16.08%, 23.84% and 8.11% of workers.

Once set up, Fondimpresa run a start-up training programme at a regional level called

PISTE with the intention of promoting innovative training in the field of market relations,

new technologies, process innovation, human resource management and development of

management systems. The programme is intended to finance in-house courses and inter-

company courses on technical, managerial and relational issues and problems. For our study,

we considered seven training programmes that were developed in the western part of the

Lombardy region run by three different partnerships composed of universities and training

centres. Lombardy has the most developed economy in Italy, given that its gross regional

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

16

product represents 20.9% (year 2006) of the Italian GNP. Accordingly, in the Lombardy

region, the companies subscribing to Fondimpresa are 13,946 and the workers involved are

858,965, 27% and 32% respectively of the figures of Fondimpresa at a national level.

The seven programmes took place from March 2005 to April 2006. Overall it included

overall 1,780 courses for a total of 30,507 training hours, involving 3,047 companies and

13,197 workers for a total of 14,635 participations.

Of the 3,047 companies taking part in the PISTE project in west Lombardy, 25% were

large companies employing more than 250 people; 32% were medium size firms with a

workforce of between 50 and 250 workers and a yearly turnover of less than 50 million euros;

30% were small companies employing between 10 and 49 workers and an annual turnover of

less than 10 million euros; 13% were micro-firms with a staff fewer than 10 people and a

yearly turnover of under 2 million euros. The companies belonged mainly to the mechanical

sector (25%), construction (22.4%), chemical sector (8.7%), services (5.9%), textile (5.5%),

retailing (4.1%), food and agro industrial (3.4%), and others (21.5%).

Of the 1,780 training initiatives developed (23.68 hours, the average length of the

course; 10.72 average number of participants; and 5.71, average number of women), 70.78%

of them (equal to 66% of overall hours) were in-company courses (single occupational group

as target audience) specially designed on the demand of a specific company; the remaining

29.22% (equal to 34% of overall hours) were inter-company courses (multiple occupational

group as target audience) where the participants came from different firms. With reference to

the content of the 1,780 courses, approximately 33% interested the technical dimension of the

role – which refers to the knowledge and professional methodologies that are necessary to

reach the expected results linked to a specific role (Goeta, 1992; Rebora, 2001) - such as

information technology, foreign languages, safety, and other specific related topics, etc.; 24%

was relative to the managerial dimension of a role – which refers to the level of responsibility

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

17

on the assigned resources and its degree of discretion (Goeta, 1992; Rebora, 2001) - for

example human resources management, audit and control, marketing, strategic planning, etc.;

and 43% was related to the relational/behavioural dimension of a role – which refers to the

complexity of managing relationships to obtain resources and information, consensus and

support (Goeta, 1992; Rebora, 2001) - for instance interpersonal communication, cross-

cultural management, resistance to change and so on. Moreover, the average number of hours

was lower (19, 23 and 26 hours respectively) moving from technical, to managerial and then

to relational topics mainly due to the different content and nature of the courses which require

different methodologies as well as in terms of training load. The average number of hours for

in-house courses (23.5 hours) was similar to that of the inter-company courses (24.0 hours) in

spite of the complexity that companies and workers have in participating to off work-site

training.

Of the 14,635 participants, 41.53% of them were female and 58.47% were male. The

female participation was about the same (43%, 24% and 43%) respectively for technical,

managerial and relational courses. The average age of trainees was 39 years, both for female

and for male trainees. The average age was 39.5, 38.6 and 38.9 for technical, managerial and

relational courses respectively. The average number of participants was 11.3, 11.4 and 9.9

respectively for technical, managerial and relational courses respectively. The average

number of participants was 10.5 for in-house courses and 11.2 for inter-company courses.

Sample

The empirical investigation took place simultaneously for the development of the

seven training programmes from March 2005 to April 2006. For the survey we used a self-

completion questionnaire that was distributed at the end of each course. The instrument was

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

18

designed as a traditional ‘happy sheet’ or ‘smile sheet’, or as originally named by Lee and

Pershing (2002) as ‘reactionnaire’.

Of the 14,635 participants to the training programme 13,754 returned a completed

questionnaire, for a response rate of 94%. Those observations were relative to 1,230 courses

out of the 1,780 courses developed for the Fondimpresa training programmes in the

Lombardy region.

Measures

Both measures for the independent variables (perceived training efficiency, perceived

usefulness of training and perceived trainer performance) and the dependent variable (overall

satisfaction with training) were designed for the study of these training programmes,

following the compulsory directives of Fondimpresa training management handbook. The

same scales have been previously used and tested in a previous study based on a smaller

sample of trainees participating to one of the seven training programmes analysed in our study

(Giangreco et al., 2008). All responses to the items used to build the scales in the

questionnaire were scored on a five-point Likert scale measuring trainees’ assessment of the

items in question (from 1 = poor to 5 = high/excellent).

Dependent variable. The dependent variable overall satisfaction with training was measured

with a single item scale meant to tap the general satisfaction of people with the training they

participated in. This choice was based on Wanous and Reichers (1997) argument that a single

item scale is an adequate measure of an overall construct.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

19

Independent variables. The independent variables, namely perceived training efficiency,

perceived usefulness of training and perceived trainer performance, were measured with 3, 5

and 5 items respectively used by Giangreco et al. (2008b). The factor analysis, run at the

course level (principal components with varimax rotation), of the 13 items loaded properly on

their respective factors. For reason of synthesis, the factor analysis results (KMO = 0.95) are

not reported here but they are available from the authors upon request. Once scaled, all factors

showed good levels of internal reliability with an alpha coefficient of .76, .87 and .90 for

perceived training efficiency, perceived usefulness of training and perceived trainer

performance respectively.

Course characteristics. The course characteristics were measured as follows: through the

absolute value for length of the course and number of participants; type of content or issue (1

= technical; 2 = managerial; 3 = behavioural/relational) and target audience (1 = single

occupational group; 2 = multiple occupational groups).

Participants’ characteristics. The participant characteristics were measured through the

percentage of female trainees based on a single course (female participation) and the age

variance. The demographic data of participants could not include other variables due to the

need of being able to guarantee the anonymity of respondents and therefore the impossibility

of matching the responses to individuals.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

20

Analysis Procedures

The research models were tested using Ordinary Least Squares (OLS) multiple

regression analysis using the variables presented above. We grouped the 13,753 responses in

1,230 observations in order to develop the analysis on a single course.

The moderator hypotheses were tested putting in order into the regression the

participant-related and course-related characteristics first, the antecedents of overall

satisfaction to training as seconds and, in the end, the multiplicatory variables on OST. The

multiplicatory variables were created to capture the ‘perceived usefulness of training X female

participation’, ‘perceived trainer performance X female participation’, ‘perceived usefulness

of training X age variance’ and ‘perceived trainer performance’ X age variance’ interactions.

Following Jaccard et al. (1990) procedure, all variables involved by the moderator hypotheses

were first mean-centred in order to decrease multi-collinearity.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

21

Results

Descriptive Results

Means, standard deviations and correlations for the variables considered in the study

are summarised in Appendix A.

On average trainees showed higher level of overall satisfaction with training (mean =

4.17; SD = 0.55). A similar trend showed perceived trainer performance with a mean = 4.42,

a SD = 0.49, whilst perceived usefulness of training and perceived training efficiency were

slightly lower (mean = 3.94 and 3.86; SD = 0.53 and 0.54 respectively).

As expected, the dependent variable OST was positively correlated with its three main

antecedents. PTE, PUT and PTP were correlated among them with a mean correlation of 0.75.

An additional analysis, however, demonstrated that although considerable, the

intercorrelations between the OST antecedents did not show any problem of multicollinearity.

In fact, the variance inflation factor VIF diagnosis showed a tolerance statistic of (1/VIF) of

0.42, 0.26 and 0.28 for PTE, PUT and PTP respectively, indicating therefore the absence of a

multicollinearity problem also according to Field’s (2005) more severe parameter threshold of

0.10.

Concerning the participant characteristics it is worth noting the relatively low female

participation among trainees (mean = 0.40; SD = 0.28), while between the course

characteristics, the length of the course (mean = 23.67; SD = 12.01) pinpoints the

development of relatively long courses.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

22

Tests of Main Hypotheses

The results of the regression analysis for hypotheses H1, H2, H3, H4, H5, H6, H7, H8

and H9 are shown in Table 1. As displayed, results provided significant support for

hypotheses H1, H2 and H3. In fact, perceived training efficiency, perceived usefulness of

training and perceived trainer performance were all significantly positively correlated to OST

(betas = .14, p < .001; .53, p < .001; .33, p < .001 respectively).

PLEASE INSERT TABLE 1 HERE

With reference to the course characteristics, Hypothesis H4 found empirical

confirmation so that the length of the course was negatively correlated to OST (beta = -.04; p

< .01). Hypothesis H7, contrary to what was expected, showed a negative correlation between

the target audience and OST (beta = -.04; p < .001), so single occupational groups have the

tendency to be overall more satisfied with the training than multiple occupational groups. On

the other hand, hypotheses H5 and H6 did not find any support, given that the type of content

and number of participants were not correlated to OST.

Concerning the participants characteristics, female participation (hypothesis H8) was

found to be positively correlated to OST (beta = .03; p < .01), while age variance (hypothesis

H9) did not find any support in the analysis. Overall the model showed a noteworthy

explanatory capacity being able to explain more than four fifth of the variance (Adjusted R

square = 0.87; p < .001).

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

23

Tests of Moderator Hypotheses

As displayed in Table 2, only one of the four moderator hypotheses (H10, H11, H12

and H13) found support in the analysis. Indeed, hypothesis H13 found confirmation of the

positive moderator effect of the presence of female trainees on the impact of perceived trainer

performance on OST (beta = .04; p < .05). Meaning that the more female are among the

trainees, the more the perception of the trainer performance influences the overall satisfaction

of trainees with the training.

PLEASE INSERT TABLE 2 HERE

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

24

Discussion

Although methodologies and practices in training are facing innovations due to new

tendencies – for example more participation of trainees or transfer of behavioural/relational

contents – or new technologies – for example e-learning – the debate on training evaluation

seems still to struggle over the same issues. On the one hand, practitioners tend mainly to

measure the reactions of trainees, practically ignoring more advanced levels of training

evaluation. On the other hand, scholars emphasise the importance of more advanced levels of

analysis developing multidimensional and multi level training evaluation models. However,

some researchers have underlined the risk of such a tendency: “the idea of being able to

calculate the return on investment of training is enticing. An absolute number is a neat

package, a trainer’s dream! In some cases, it can be obtained. In other cases, while seductive,

it may not be worth the effort” (Lilly, 2001: 1). As a consequence, the research on trainees’

reactions still has areas of potential development given that reactions on training may also

give positive contributions to the organisational climate and to the quality of relationships at

work (Rhoades and Eisenberger, 2003). Based on these facts, this study aims at providing a

contribution to the evaluation of reactions by analysing some situational factors that may have

an impact on the overall trainees’ satisfaction with the training. Starting from the findings of a

previous study (Giangreco et al., 2008b), and after re-testing their main hypotheses, we

focused on four course characteristics (namely length of the course, type of content, number

of participants and target audience) and two participant characteristics (female participation

and age variance). Hypotheses related to each of these factors were then tested using a large

sample of trainees and courses coming from a large variety of companies located in western

Lombardy (northern Italy).

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

25

Theoretical and Policy Implications

The findings of our study may stimulate the debate on training evaluation as well as

providing some interesting policy implications for HR practitioners.

The first point refers to the confirmation of the set of OST antecedents indicated by

Giangreco et al. (2008b). Indeed, our results provide strong support in the fact that the overall

satisfaction of trainees with the training depends on the extent to which individuals perceive

the training as useful for their present post and their future development, the extent to which

participants perceive that the trainer was performing well in terms of content and process, and

the extent to which trainees perceive that the course was adequately organised and

administered. Although interrelated, our findings also confirm the same order of intensity of

the original theoretical framework, so that perceived usefulness of training has the strongest

impact on OST, followed by perceived trainer satisfaction and then perceived training

efficiency.

The second point refers to the role played by the course characteristics. At the same

time our study confirms the shared belief of HR practitioners and disconfirms another. In fact,

on the one hand we notice that the shorter the course, the higher the overall satisfaction of

trainees with the training will be. This may relate to minor work disruptions that the trainees

have to experience, as well as the minor likelihood of detecting negative aspects in such a

short experience. On the other hand, contrary to what was expected, it is the single

occupational group that is associated to higher levels of OST, rather than the multiple

component groups. This may be explained by the fact that in single occupational groups the

familiar class climate could favour a tendency to be more satisfied with the training

experience. The business compatibility in terms of organisational similarities (Moss Kanter

and Corn, 1993) probably represents an important issue in explaining this unattended finding.

It is also interesting the failure of our study in supporting the hypotheses related to the other

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

26

two courses’ characteristics. In fact, neither the number of participants, nor the type of content

– the latter in contrast with Giangreco et al. (2008b) findings – found support from the

analysis.

The third point concerns the role of participants’ characteristics. Of the two variables,

only female participation was supported, showing therefore that a higher presence of female

trainees may be associated to higher levels of satisfaction with the training. This result is

somehow expected in light of the fact that women are more committed to training and willing

to put extra efforts into as a counterbalance to existing discriminatory policies.

The fourth point is relative to the results concerning the moderator effect of age

variance and female participation on PUT and PTP impacting OST. We only found a

moderator effect. This is that the more women present in the training, audience tends to lead

an increase in the impact of the perception of the trainer performance on OST. Although it

was not possible to include the gender of the trainers in the analysis for practical reasons, this

was probably expected considering the circumstances in that about three quarters of trainers

were male.

The final point has a more practical perspective which implies some interesting

implications for HR practitioners dealing and managing training programmes. First of all, our

study shows that in order to maximise the reactions of trainees, systematic monitoring is

important for the three key antecedents tested earlier by Giangreco et al. (2008b). Although

training organisers and managers tend to be driven by the perception of the performance of

the trainer – is she/he a star in the class room? – our study confirms that the extent to which

individuals perceive training as helpful for their work and functional for their future

development is one of the most critical and important aspects of training. Moreover, the

results related to the course and the participant characteristics suggest to shape the audience

by a significant presence of female trainees whenever possible, and to deliver short training

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

27

initiatives with single occupational groups therefore coming from the same company. These

last two findings picture a need of more homogeneity of the audience somehow not in

agreement with the literature that considers diversity as an enriching characteristic for training

activities. Rather than a matter of individuals, however, it could be related to the need of more

contextualisation of topics and issues developed by the training.

Study Limitations and Future Research

The findings of the present study have to be treated with caution before making further

generalisations. This is due to various reasons. First, although the sample is verified in terms

of courses, companies and individuals coming from the same region could cause some

problems of extension of the results to other type of context. Second, the cross-sectional

nature of the empirical investigation limits the possibility of the rigorous testing of the cause-

effect relationship between OST and its antecedents. Third, the self assessed scale that we

used may present problems of common method variance although they showed proper

psychometric capabilities. Fourth, there were only two participant related characteristics due

to the required anonymity of respondents.

Whenever possible, future research should attempt to overwhelm the limitations

addressed above. In fact, a longitudinal and comparative analysis between different cultural

systems taking into account a wider set of control variables regarding the courses, the trainees

and also the trainer would be particularly important for the assessment of such articulated and

extensive training programmes. Furthermore, future research should seek to extend our

findings trying to identify variables that will help training planners in shaping audiences and

designing courses for the maximisation of the trainees’ satisfaction with the training. “Further,

planning decisions about the breadth and depth of training should be based in part on

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

28

knowledge of trainees’ characteristics” (Kraiger et al., 2004: 341). This would be especially

helpful for all those companies (the majority) that don’t go over reactions evaluation.

Conclusions

Although the first level of the Kirkpatrick model, reactions evaluation, has received

many criticisms on its capability of influencing the following levels, we cannot ignore that it

may provide information and benefits regarding at least two aspects. First, it gives feedback to

training designers whether enhance some corrections (Lee and Pershing, 2002); second, if the

feedback is positive at least we may assume that some ‘soft’ dimensions - climate, quality of

relationship, motivation of workers – would benefit from it (Meyer and Allen, 1997).

Based on these points, the aim of our study has been to investigate reactions

evaluation also in reason of the fact that it is the most and, very often, the only tool used to

evaluate training, above all in small and medium size companies. This trend reveals a major

importance especially in Europe, where the weight of SMEs is particularly relevant (Gray and

Lawless, 2000). We specifically identified and tested a panel of 4 course-related and 2

participant-related variables that could have an impact on the trainees’ perception of the

training. These variables were meant to extend Giangreco et al. (2008b) model, a framework

in which the trainees’ overall satisfaction with training is the result of namely perceived

usefulness of training, perceived trainer performance and perceived training efficiency. The

analysis was carried out on data coming from a sample of some 13,753 participants grouped

in 1,230 courses of seven training programmes financed by an Italian inter-professional fund.

The study confirmed the role of the three main antecedents of OST. Moreover, three of

the six variables shown had an impact on OST. In fact, the percentage of female trainees, the

length of the course and the type of target audience had an impact of the trainees’ overall

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

29

satisfaction with the training. Finally, female participation showed having a moderator effect

on the relationship perceived trainer performance and OST.

Our study also aims at providing a contribution to the debate between HR scholars and

practitioners trying to make their respective positions closer so that we, the scholars, could be

more useful to the perspective of the end-users of training. Indeed, practitioners may gather

(or have confirmed) from our study that a higher presence of female trainees, shorter courses

and a target audience made of a single occupational group will maximise the level of trainees’

overall satisfaction with the training.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

30

References

Alliger, G.M. and Janak, E.A. (1989), “Kirkpatrick’s Levels of Training Criteria: Thirty

Years Later”, Personnel Psychology, Vol. 42, pp. 331-342.

Alliger, G.M., Tannenbaum, S.I., Bennett Jr., W., Traver, H. and Shotland, A. (1997), “A

Meta-Analysis of the Relations Among Training Criteria”, Personnel Psychology, Vol. 50,

pp. 341-358.

Alvesson, M. (2000), “Social Identity and the Problem of Loyalty in Knowledge-Intensive

Companies”, Journal of Management Studies, Vol. 37(8), pp. 1101-1123.

Amietta P. (2000) I Luoghi dell'Apprendimento: Metodi, Strumenti e Casi di Eccellenza,

Milan: FrancoAngeli.

Arthur Jr., W., Bennet Jr., W., Edens, P.A., and Bell, S.T. (2003), “Effectiveness of Training

in Organizations: A Meta-Analysis of Design and Evaluation Features”, Journal of Applied

Psychology, Vol. 88(2), pp. 234-245.

Barron, J.M., Black, D.A. and Loewenstein, M.A. (1993), “Gender Differences in Training,

Capital and Wages”, The Journal of Human Resources, Vol. 28(2), pp. 343-364.

Bates, R. (2004), “A Critical Analysis of Evaluation Practice: the Kirkpatrick Model and the

Principle of Beneficence”, Evaluation and Program Planning, Vol. 27(3), pp. 341-348.

Blanchard, N.P. and Thacker, J.W. (2007), Effective Training – Systems, Strategies, and

Practices, 3rd edition, Pearson Prenctice Hall, Upper Saddle River (NJ, USA).

Brown, K.G. (2005), “An Examination of the Structure and Nomological Network of Trainee

Reactions: A Closer Look at ‘Smile Sheets’”, Journal of Applied Psychology, Vol. 90(5), pp.

991-1001.

Brown, R.E. and Reed, C.S. (2002), “An Integral Approach to Evaluating Outcome

Evaluation Training”, American Journal of Evaluation, Vol. 23(1), pp. 1-17.

Cannon-Bowers, J.A., Salas, E., Tannenbaum, S.I. and Mathieu, J.E. (1995), “Toward

Theroretically Based Principles of Training Effectiveness: a Model and Initial Empirical

Investigation”, Military Psychology, Vol. 7, pp. 141-164.

Cascio, W.F. (1999), Costing Human Resources: the Financial Impact of Behavior in

Organizations, 4th edition, Kent Publishing, Boston.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

31

Combs, G.M. and Luthans, F. (2007), “Diversity Training: Analysis of the Impact of Self-

Efficacy”, Human Resources Development Quarterly, Vol. 18(1), pp. 91-120.

Cullen, J.C., Sawzin S.A., Sisson G.R. and Swanson, R.A. (1978), “Cost Effectiveness: A

Model for Assessing the Training Investment”, Training and Development Journal, Vol. 1,

pp. 24-29.

Dixon, N.M. (1990), “The Relationship Between Trainee Responses on Participant Reaction

Forms and Posttest Scores”, Human Resources Development Quarterly, Vol. 1(2), pp. 129-

137.

Field A. (2005) Discovering Statistics Using SPSS, London: Sage Publications.

Ganey T.W. and Klaas B.S. (2003) “The Outsourcing of Training and Development: Factors

Impacting Client Satisfaction”, Journal of Management, 29(2): 207-229.

Giangreco A., Sebastiano A. and Carugati A. (2008a), “Are We Doing the Right Thing? Food

for Thought on Training Evaluation”. Paper accepted at the Workers Rights Protection in a

New World of Work. The case for a comparative and interdisciplinary approach to Labour

Relations. 6th international conference in Memory of Professor Marco Biagi Modena (Italy),

15-19 March 2008.

Giangreco, A., Sebastiano, A. and Peccei, R. (2008b), “Trainees’ Reactions to Training: An

Analysis of the Factors Affecting Overall Satisfaction with Training”, The International

Journal of Human Resources Management, forthcoming.

Goeta, G.F. (1992), “Il Performance Management Interattivo”, Sviluppo e Organizzazione,

Vol. 3.

Gray, C. and Lawless, N. (2000), “Innovations in the Distance Development of SME

Management Skills”, European Journal of Open, Distance and E-Learning, pp. 1-18.

Green, F. (1997), “Review of Information on the Benefits of Training for Employers”,

Research Report n.7, DfEE, Sheffield.

Hamblin, A.C. (1974), Evaluation and Control of Training, McGraw-Hill, New York.

Holladay, C.L. and Quinones, M.A. (2005), “Reactions to Diversity Training: An

International Comparison”, Human Resource Development Quarterly, Vol.16(4), pp. 529-

545.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

32

Holton, F.H. (1996), “The Flawed Four-Level Evaluation Model”, Human Resources

Development Quarterly, Vol. 7(1), pp. 5-20.

Hook, K. and Bunce, D. (2001), “Immediate Learning in Organisational Computer Training

as a Function of Training Intervention Affective Reaction, and Session Impact Measures”,

Applied Psychology: An International Review, Vol. 50(3), pp. 436-454.

Jaccard J., Choi K.W. and Turrisi R. (1990) “The Detection and Interpretation of Interaction

Effects Between Continuous Variables in Multiple Regression”, Multivariate Behavioural

Research, 25(4): 467-478.

Kaufman, R., Keller, J. and Watkins, R. (1995), “What Works and What Doesn’t: Evaluation

Beyond Kirkpatrick”, Performance & Instructions, Vol. 35(2), pp. 8-12.

Kidder P.J. and Rouiller J.Z. (1997) “Evaluating the Success of a Large-Scale Training

Effort”, National Productivity Review, Spring: 79-89.

Kirkpatrick, D.L. (1959a), “Techniques for Evaluating Training Programs: Part 1 –

Reactions”, Journal of ASTD, Vol. 13(11), pp. 3-9.

Kirkpatrick, D.L. (1959b), “Techniques for Evaluating Training Programs: Part 2 –

Learning”, Journal of ASTD, Vol. 13(12), pp. 21-26.

Kirkpatrick, D.L. (1960a), “Techniques for Evaluating Training Programs: Part 3 –

Behavior”, Journal of ASTD, Vol. 14(1), pp. 13-18.

Kirkpatrick, D.L. (1960b), “Techniques for Evaluating Training Programs: Part 1 – Results”,

Journal of ASTD, Vol. 14(12), pp. 28-32.

Kirckpatrick, D.L. (1967), “Evaluation of Training” in Craig R.L. and Bittel, L.R. (Ed.),

Training and Development Handbook, McGraw-Hill, New York.

Kirckpatrick, D.L. (1996a), “Great Ideas Rivisited”, Training and Development, Vol. 50, pp.

54-59.

Kraiger, K. (2002), “Decision-based Evaluation”, in Kraiger, K. (Ed.), Creating,

Implementing, and Maintaining Effective Training and Development: State-of-the-Art Lessons

for Practice, Jossey-Bass: San Francisco.

Kraiger, K., McLinden, D. and Casper, W.J. (2004), “Collaborative Planning for Training

Impact”, Human Resources Management, Vol. 43(4), pp. 337-351.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

33

Lee, S.H. and Pershing, J.A. (2002), “Dimensions and Design Criteria for Developing

Training Reactions Evaluations”, Human Resources Development International, Vol. 5(2),

pp. 175-197.

Lilly, F. (2001), “Four Steps to Computing Training ROI’, Working Paper, SPHR,

Vancouver.

Lorenzet, S.J., Salas, E. and Tannenbaum, S.I. (2005), “Benefits from Mistakes: the Impact of

Guided Errors on Learning, Perfomance, and Self-Efficacy”, Human Resource Development

Quarterly, Vol. 16(3), pp. 301-322.

Mann, S. and Robertson, I.T. (1996), “What Should Training Evaluations Evaluate?”, Journal

of European Industrial Training, Vol. 20(9), pp. 14-20.

McEvoy, G.M. (1997), “Organizational Change and Outdoor Management Education”,

Human Resource Management, Vol. 36(2), pp. 235-250.

McKillip, J. (2001), “Case Studies in Job Analysis and Training Evaluation”, International

Journal of Training and Development, Vol. 5(4), pp. 283-289.

Meyer, J. P. and Allen, N. J. (1997). Commitment in the Workplace, Thousand Oaks (CA,

USA): Sage.

Molenda, M., Pershing, J.R. and Reigeluth, C.M. (1996), “Designing Instructional Systems”

in Craig, R.L. (Ed.), The ASTD Training and Development Handbook: a Guide to Human

Resource Development, McGraw-Hill, New York.

Morgan, R.B. and Casper, W.J. (2000), “Examining the Factor Structure of Participant

Reactions to Training: A Multidimensional Approach”, Human Resources Development

Quarterly, Vol. 11(3), pp. 301-317.

Morris M. (1984) “The Evaluation of Training”, ICT, March/April: 9-16.

Moss Kanter, R. and Corn, R.I. (1993), “Do Cultural Differences Make a Business

Difference? Contextual Factors Affecting Cross-Cultural Relationship Success”, Journal of

Management Development, Vol. 1”(2), pp. 5-23.

Noe, R.A. (1986), “Trainees’ Attributes and Attitudes: Neglected Influences on Training

Effectiveness”, The Academy of Management Review, Vol. 11(4): pp. 736-749.

Olsen, J.H. (1998), “The Evaluation and Enhancement of Training Transfer”, International

Journal of Training and Development, Vol. 2(1), pp. 61-75.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

34

Pershing, J.A. and Pershing J.L. (2001), “Ineffective Reaction Evaluation”, Human Resources

Development Quarterly, Vol. 12(1), pp. 73-90.

Phillips, J.J. (1997), Handbook of Training Evaluation and Measurement Methods, Gulf

Publishing, Houston.

Phillips, J.J. (2003), Return on Investment in Training and Performance Improvement

Programs, Butterworth-Heineman, Boston.

Plant, R.A. and Ryan, R.J. (1992), “Training Evaluation: A Procedure for Validating an

Organization’s Investment in Training”, Journal of European Industrial Training, Vol.

16(10), pp. 22-31.

Rebora, G. (2001), Manuale di Organizzazione Aziendale, Carocci Editore, Milan.

Robertson, L. and Block, C.J. (2001), “Racioethnicity and Job Performance: A Review and

Critique of Theoretical Perspectives on the Causes of Group Differences”, Research in

Organizational Behavior, Vol. 23, pp. 245-325.

Rhoades, L. and Eisenberger R. (2002) “Perceived Organizational Support: A Review of the

Literature”, Journal of Applied Psychology, 87: 698-714.

Rowold, J. (2007), “Individual Influences on Knowledge Acquisition in a Call Center

Training Context in Germany”, International Journal of Training and Development, Vol.

11(1), pp. 21-34.

Russ-Eft, D.F., Dickinson, P.D. and Levine, R. (2005), “Instructor Quality Affecting

Emergency Medical Technician (EMT) Preparedness: a LEADS Project”, International

Journal of Training and Development, Vol. 9(4), pp. 256-270.

Salas, E., Prince, C., Bowers, C.A., Stoudt, R.J., Oser, R.L. and Janis, A. (1999), “A

Methodology for Enhanicing Crew Resource Management Training”, Human Factors, Vol.

41(1), pp. 161-172.

Santos, A. and Stuart, M. (2003), “Employee Perceptions and Their Influence on Training

Effectiveness”, Human Resources Management Journal, Vol. 13(1), pp. 27-45.

Sebastiano A. and Bellet I. (2005) “Sviluppo Professionale Continuo nelle Aziende Sanitarie:

Criticità del Processo Formativo”, Sanità Pubblica e Privata, 2: 29-39.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

35

Seyler, D.L., Holton III, E.F., Bates, R.A., Burnett, M.F. and Carvalho, M.A. (1998), “Factors

Affecting Motivation to Transfer Training”, International Journal of Training and

Development, Vol. 2(1), pp. 2-16.

Sims, R.R. (1993), “Evaluating Public Sector Training Programs”, Public Personnel

Management, Vol. 22(4), pp. 591-615.

Sitzmann, T., Brown, K.G., Casper, W.J., Ely, K. and Zimmerman, R.D. (2008), “A Review

and Meta-Analysis of the Nomological Network of Trainee Reactions, Paper presented at the

2007 AMC, Journal of Applied Psychology, forthcoming.

Steiner D.D., Dobbins G.H. and Trahan W.A. (1991) “The Trainer-Trainee Interaction: An

Attributional Model of Training”, Journal of Organizational Behavior, 12(4): 271-286.

Swanson, R.A. and Sleezer, C.M. (1987), “Training Effectiveness Evaluation”, Journal of

European Industrial Training, Vol. 11(4), pp. 7-16.

Tam, T. (1997), “Sex Segregation and Occupational Gender Inequality in the United States:

Devaluation or Specialized Training?”, American Journal of Sociology, Vol. 102(6): 1652-

1692.

Tan, J.A., Hall, R.J. and Boyce, C. (2003), “The Role of Employee Reactions in Predicting

Training”, Human Resource Development Quarterly, Vol. 14(4), pp. 397-411.

Tannenbaum, S.I., Mathieu, J.E., Salas, E. and Cannon-Bowers, J.A. (1991), “Meeting

Trainees’ Expectations: The Influence of Training Fulfillment on the Development of

Commitment, Self-Efficacy and Motivation”, Vol. 76(6), pp. 759-769.

Tannenbaum, S.I. and Woods, S.B. (1992), “Determining a Strategy for Evaluating Training:

Operating Within Organizational Constraints”, Human Resources Planning, Vol. 15(2), pp.

63-81.

Tannenbaum, S.I. and Yukl, G. (1992), “Training and Development in Work Organizations”,

Annual review of Psychology, Vol. 43, pp. 399-441.

Taylor, P.J., O’Driscoll, P. and Binning, J.F. (1998), “A New Integrated Framework for

Training Needs Analysis”, Human Resource Management Journal, Vol. 8(2), pp. 29-50.

Wang, G.G. and Wilcox D. (2006), “Training Evaluation: Knowing More than Is Practiced”,

Advances in Developing Human Resources, Vol. 8(4), pp. 528-530.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

36

Wanous J.P. and Reichers A.E. (1997) “How Good Are Single-Item Measures?” Journal of

Applied Psychology, 82(2): 247-252.

Warr, P., Allan, C. and Birdi, K. (1999), “Predicting Three Levels of Training Outcome”,

Journal of Occupational and Organizational Psychology, Vol. 72, pp. 351-375.

Warr, P., Bird, M. and Rackham, N. (1970), Evaluation of Management Training: a Practical

Framework, with Cases, for Evaluating Training Needs and Results, Gower, London.

Webster J. and Martocchio J.J. (1995) “The Differential Effects of Software Training

Previews on Training Outcomes”, Journal of Management, 21(4): 757-787.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

37

Table 1 – Test of Research Model: Regression Analysis Results

OST

1. female participation .032**

2. age variance .001

3. number of participants .017

4. type of audience (multiple occupational group) -.038***

5. length of the course -.039**

6. course content (behavioural/relational) .017

7. perceived training efficiency .132***

8. perceived usefulness of training .530***

9. perceived training performance .331***

Adjusted R Square 0.871

Number of Observations (courses) 1230

Note:

Figures in table are standardised beta coefficients.

*p < 0.5; **p < .01; *** < .001.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

38

Table 2 – Test of Moderator Hypotheses: Regression Analysis Results

OST

1. female participation .031**

2. age variance .001

3. number of participants .017

4. type of audience (multiple occupational group) -.036**

5. length of the course -.039**

6. course content (behavioural/relational) .016

7. perceived training efficiency (PTE) .136***

8. perceived usefulness of training (PUT) .533***

9. perceived training performance (PTP) .328***

10. PTP X female participation .037*

11. PTP X age variance -.001

12. PUT X female participation -.021

13. PUT X age variance -.002

Adjusted R Square 0.871

Number of Observations (courses) 1,230

Note:

Figures in table are standardised beta coefficients.

*p < 0.5; **p < .01; *** < .001.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

Appendix A – Means, Standard Deviations and Correlations for the Main Variables,

Course-related and Participant-related Characteristics

Variable Mean SD 1 2 3 4 5 6 7 8 9

1. female participation 0.40 0.28 --- --- --- --- --- --- --- --- ---

2. age variance 50.30 28.88 0 --- --- --- --- --- --- --- ---

3. number of participants 10.57 3.87 0.09 0.12 --- --- --- --- --- --- ---

4. type of audience (multiple

occupational group) 1.29 0.45 0 0.49 0.10 --- --- --- --- --- ---

5. length of the course 23.67 12.01 -0.10 -0.01 -0.22 0.02 --- --- --- --- ---

6. course content

(behavioural/relational) 1.87 0.85 -0.02 -0.12 0.17 -0.5 -0.24 --- --- --- ---

7. perceived training efficiency 3.86 0.54 -0.02 0.01 -0.01 0.02 0.02 -0.05 --- --- ---

8. perceived usefulness of training 3.94 0.53 -0.04 -0.03 -0.04 -0.03 -0.01 0.07 0.72 --- ---

9. perceived training performance 4.42 0.49 0.02 -0.01 -0.03 -0.03 -0.02 0.02 0.70 0.83 ---

10. overall satisfaction to training 4.17 0.55 0.04 -0.02 -0.01 -0.06 -0.06 0.07 0.74 0.90 0.86

Note: all correlations significant at p < .001

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

40

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm

i Source: www.fondimpresa.it; data updated at September 2007.

PDF créé avec la version d'essai FinePrint pdfFactory http://www.gs2i.fr/fineprint/pdffactory.htm