CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 2

CONTENTS

3 | Introduction

5 | The Learner Experience

7 | Perceptions of Training Consistency

11 | Impact of Consistency on Learners

15 | Conclusions

16 | Study Demographics

18 | About This Research

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 3

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE

Consistency in training is a concept that touches nearly everything in L&D. Not surprisingly, maintaining consistency comes up again and again in Training Industry research as one of the biggest challenges to delivering an effective training portfolio to a population of learners. These learners may be employees on opposite sides of the country who need to gain proficiency in the same set of skills or learners outside the organization, such as customers seeking information about how to use a company’s products or services. No matter where they hail from, however, a common underlying objective of providing learning opportunities for these individuals is that their experience doesn’t vary from one day to the next, from one location to the next or from one instructor to another. Consistency is a driver – and sometimes a requirement – for a variety of training initiatives. For instance, certifications need reliable presentation of materials to meet specific learning goals. Learning technologies can facilitate consistency by requiring instructional designers to design content and courses within the framework of

the technology platform. Considering that a company’s learning management system (LMS) represents a large portion of an organization‘s L&D budget, one would think this functionality would be a high priority in the selection process. Unfortunately, this is not always the case.

Despite all the time and money that companies invest to try to deliver a consistent learning experience, are learners able to recognize consistency from training event to training event, and do they value its importance? There is an argument that it might not matter that much – that learners are simply the “end users,” and consistency is a strategic consideration that manifests in ways not obvious to most learners. The counterargument is that it matters quite a bit. The more learners experience training courses that are structured, the more likely they are to approach their next training expecting to learn from the structure in the same way. Drastic changes in how content is organized, accessed or delivered from event to event can confuse learners and lead to poorer retention and

Introduction

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 4

transfer of knowledge and skills. This is true not because the content is bad, necessarily, but because the learners have to begin each training by trying to determine what to expect in terms of the amount of pre-work involved, the type of learning format they will use, the number of participants (small group versus lecture), and how much participation is required or expected on their part. This statement assumes that the learner is motivated to put in this effort to engage with the training.

So, what kind of effect does training consistency really have on learners? To examine these issues in greater detail, Training Industry, Inc. and The Training Associates Corporation conducted a study to understand how learners feel about the consistency of the training provided by their employers. In early 2017, 283 learners completed a confidential survey that explored how they prefer to be trained, their opinion of how consistent their learning experience is across training programs and the impact that they see on their job as a result of training.

WHAT KIND OF EFFECT DOES TRAINING CONSISTENCY REALLY HAVE ON LEARNERS?

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 5

First, learners were asked a series of questions about the effectiveness of the training they received at their respective companies. As shown in Figure 1, 31 percent of learners rated their company’s training as “always” or “almost always” effective. In contrast, 32 percent of learners rated their company’s training as “sometimes effective” or “not effective.”

Effectiveness, however, is not the only marker for the quality of the cumulative learning experience. As shown in Figure 2, 38 percent of learners rated their company’s training as “always” or “almost always” consistent. However, 12 percent rated their company’s training as “not at all consistent.” Between Figures 1 and 2, it is clear that one-third of learners may have less than stellar opinions of the training they frequently encounter.

Next, learners were asked about the different types of modalities used in the

training events they participated in during the last year. As shown in Figure 3, the two most common modalities learners experienced in the past 12 months were e-learning (52%) and instructor-led training (ILT; 49%).

For more context, each modality was counted to gauge how common multimodal training is in learning experiences. Figure 4 shows the number of different modalities used by learners in the past year. As shown, 52 percent of companies used two to four different modalities to deliver training. For

The Learner Experience

12%

20%

30%

29%

9%Always consistent

Almost always consistent

Usually consistent

Sometimes consistent

Not at all consistent

Figure 2. Learner Ratings of Overall Training Consistency

Figure 3. Use of Training Modalities in the Past 12 Months

E-learning 52%

ILT 49%

Webinars/Podcasts 30%

VILT 28%

Videos 28%

Mentoring/Coaching 24%

Social/Collaborative learning 17%

Blended (ILT + E-Learning/VILT) 15%

Microlearning 13%

Mobile learning 9%

Game-based learning 4%

Figure 4. Frequency of Multiple Modality Use

1

32%

2

24%

3

17%

4

11%

5

6%

6

4%

7

4%

8

1%

9+

1%

Figure 1. Learner Ratings of Overall Training Effectiveness

7%

25%

35%

11%Always effective

Almost always effective

Usually effective

Sometimes effective

Not effective

20%

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 6

the 32 percent that used a single delivery method, the data showed that e-learning was by far the most prevalent, followed by ILT and webinars, mirroring the pattern in Figure 3.

The modalities that learners participate in most frequently, however, may not necessarily be the modalities that learners would like to use if given a choice. As such, survey respondents were asked to indicate which training delivery method they felt was the single most effective and impactful modality. As shown in Figure 5, learners generally prefer ILT over and above all other modalities. This preference was found regardless of whether their company is effective or consistent at delivering training.

Next, respondents were asked to identify what types of training they had participated in over the past 12 months. As shown in Figure 6, learners were

most likely to participate in job-specific technical training. Between one-third and one-quarter of learners completed a certification program, regulatory/compliance training, diversity or HR-related training, or health/occupational safety training.

In sum, the collective perspectives of the learners providing data for this study represent a varied cross-section of experiences as they relate to training effectiveness, modalities, topics and overall training consistency. However, the devil is in the details, and the next section further explores the ingredients of training consistency.

Figure 5. Learning Modality Preferences

ILT 38%

Mentoring/Coaching 15%

Blended (ILT + E-Learning/VILT) 9%

E-learning 9%

Social/Collaborative learning 7%

Webinars/Podcasts 7%

Videos 4%

Microlearning 2%

Game-based learning 2%

Mobile learning 0%

VILT 4%

Figure 6. Participation in Training Topics

Job-specific technical 45%

Certification programs 30%

Regulatory/Compliance 28%

Diversity/HR 27%

Health/Occupational safety 25%

Customer Service 24%

Soft Skills 23%

IT 22%

Process/Quality 19%

Leadership development 18%

Product/Service knowledge 16%

New employee orientation 15%

Management/Supervisory skills 9%

Sales skills 8%

Sales methodology 5%

Market education 5%

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 7

Perceptions of Training ConsistencyAlthough most discussions surrounding training consistency occur with an understanding that everyone is talking about the same thing, consistency can be much more multifaceted. In this research, there were four areas of consistency for which respondents rated their experiences at their company:

1. Training content(i.e., related to the creation and management of instructional material, generally referred to as courses, curriculum, workshops, seminars, webinars or programs)

2. Training delivery(i.e., methods associated with the transfer of information)

Figure 7. Consistency Perceptions of Training Content

3. Learning technology(i.e., technologies used for creating, managing and delivering training)

4. Training administration(i.e., activities and processes associated with managing the logistics and day-to-day operations of the training function)

Look and feel of

training content

Structure of training

content

Appropriate level of detail in training content

Appropriate amount of

training content in any given program

Training content

appears up to date

Training content

seems well researched

Training content is

job-relevant

Always effective Almost always effective Usually effective Sometimes effective Not effective

11%

11%

6%

6% 8

%

5%

5%

4%

4%

11%13

% 14%

22%

22%

29

%

29

% 33

%3

0%

25%2

8%

25

%

25

%

24%

17%

17%

15% 17

%

17% 19

%

40

%

36

%

36

%

32%

39

%

16%

GREAT CONTENT CAN LANGUISH IN POOR DELIVERY.

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 8

For each specific area, respondents were asked to rate the effectiveness of their companies at delivering a consistent learning experience across training programs. The first area of consistency concerned the training content that learners encounter. As shown in Figure 7, learners felt their companies were most effective at ensuring training content is related to their jobs (a combined 51 percent of learners rated their company as “always effective” and “almost always effective”). About 45 percent of learners also agreed that the training content they’re exposed to is consistently both well researched and current. However, when it comes to the look, structure and amount of content, learners were less emphatic. Roughly 25 percent of learners rated their companies as marginally effective or ineffective at providing content that is consistent across these factors.

Some learning leaders, when considering the consistency of their training programs, may think of consistency as only applying to content. However, great content can languish in poor delivery. Accordingly, respondents next rated the consistency of the delivery of training they have experienced at their companies. As shown in Figure 8, the strengths of most companies’ training delivery centers around aspects of classroom learning. Between 43 and 46 percent of learners rated the engagement and expertise of instructors, along with the quality of classroom facilities, as “always effective” or “almost always effective.” Companies

Figure 8. Consistency Perceptions of Training Delivery

Instructor delivering training is engaging

Expertise of the

instructor

Quality of inperson training facilities

Quality of virtual classroom platform

Matching training modality

with content

Collecting feedback on

improvements

Physical surroundings

conducive to virtual training

Always effective Almost always effective

Usually effective Sometimes effective Not effective

6%

6%

4%

4%

3%

2%

13%

13%

12%

12%

14%

14%

29%

29%34%

34%

30%

28%

28%

28%

27%

26%

24%22%

20%

23%

15%

16%

17%

19%

18%

32%

16%

16%

16%

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 9

were seen as less consistent when it came to aspects of virtual training, but learners were particularly split on their employers’ effectiveness at collecting feedback for improving future training programs.

Next, respondents rated the consistency of their experience with the learning technologies their companies use. As shown in Figure 9, companies fare best when using forms of virtual training that function as delivery platforms and when offering training without technology issues caused by company intranets

Figure 9. Consistency Perceptions of Learning Technology

Using/navigating

an LMS

Using virtual training as

delivery platforms

(e.g., videos, e-learning)

Online social and

collaboration tools to support

training programs

Reinforcement tools (e.g.,

post-training quizzes/

activities)

The ability to complete

training without experiencing technology

issues

Always effective Almost always effective Usually effective Sometimes effective Not effective

24%

24%

23%

21%22

%

20

%

28% 31%

27% 29

%

8%

17%

17% 18%

18%

14%15%

12%

12%

6%

12%

13%

11%

11%

10%

or software limitations. When it comes to using social tools for collaboration, however, many companies appear to be lagging behind, as only 25 percent of learners rated their companies as “always effective” or “almost always effective.” It is also notable that across all four categories of consistency, learning technology garnered the lowest endorsements of effectiveness from learners.

Lastly, ratings were made on the consistency of the administrative activities concerning training. As shown

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 10

Figure 10. Consistency Perceptions of Training Administration

The ability to schedule/register for

training

Navigating an individual learning plan

Tracking training completion

The use of pre-/post-test to

support learning

Internal marketing

to publicize training

programs

24%

23%

21%22

%

26% 28

%

29%

29% 32%

25%

25%

8%

5%

18%19%

14%

16%

16%

16%

15%

7%

11%

11%

11%

10%

Always effective Almost always effective Usually effective Sometimes effective Not effective

LEARNERS GENERALLY PREFER ILT OVER AND ABOVE ALL OTHER MODALITIES AS THE MOST IMPACTFUL AND EFFECTIVE MODALITY.

in Figure 10, registering for training (48%) and tracking subsequent course completion (54%) were the two areas where most companies seem to be providing a consistent experience for learners.

Cumulatively, Figures 7 through 10 show that consistency can be meaningfully broken down into many constituent parts and that learners are capable of making distinctions among different aspects of their learning experience and the consistency thereof. But why should companies care?

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 11

To this point, this report has described learners’ perceptions about the effectiveness and areas of consistency of training at their respective companies. But are such perceptions merely an interesting footnote to training strategy, or do they have a tangible impact on employee performance? This section makes the case for the latter by illustrating the insights gained from an analysis of the relationships between consistency and training outcomes.

First, survey respondents were asked about the ongoing learning and development opportunities at their organizations. As shown in Figure 11, less than one-fifth of companies are “always” providing professional development opportunities. It is notable that 15 to 24 percent of learners

responded “rarely” across all items, and, in particular, 12 percent of learners felt that their company does not provide them with training that could prepare them for

positions in the future. On a five-point scale, with 5 being “always” and 1 being

Impact of Consistency on Learners

Figure 11. Perceptions of Professional Development Opportunities

1 Bulut, C., & Osman, C. (2010). The effects of organizational training on organizational commitment. International Journal of Training and Development, 14(4), 309-322.

10%

25%

24% 27

%

22%

20

%

38

%

31%

34

%

34

%

19%

19%

17%

15%

5% 6%

6%

33

%16

%

12%

7%

25%

11%

Always effective Almost always effective Usually effective Sometimes effective Not effective

Provides professional development

opportunities that help me be more effective at my daily job tasks

Makes professional development opportunities available that

“upskill” me for more advanced positions in the

company

Provides professional development

opportunities for all employees

Provides professional development

opportunities for employees with

jobs similar to mine

Provides professional development opportunities

primarily for senior/

management positions

12%

CONSISTENCY IS SIGNIFICANTLY RELATED TO A LEARNER’S

MOTIVATION, PERCEPTIONS OF TRAINING EFFECTIVENESS

AND DISPOSITION TOWARD PROFESSIONAL DEVELOPMENT.

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 12

“never,” the combination of responses averaged 3.20. These results indicate that across all survey respondents, the majority of companies “sometimes” offer professional development opportunities to learners.

Learners’ motivation for training is the extent to which they are willing to improve their knowledge, skills and abilities (KSAs) by engaging with training offerings. This concept was measured using eight items1. As shown in Figure 12, most learners are highly motivated to engage with training, as evidenced by the relatively high proportion of “strongly agree” and “agree” responses across items. On a five-point scale, with 5 being “strongly agree” and 1 being “strongly disagree,” the combination of responses averaged 3.84. These results suggest that most learners are motivated to improve their KSAs.

Why describe the perceived opportunities and level of motivation of learners? Because, as we’ll see, these measures are related to the consistency of training as it is understood by learners in their companies. The results that follow are derived from a correlational analysis of many of the collected ratings and scales used in this research to test whether significant relationships exist between training consistency and the attitudes and experiences of learners. This analysis averaged each of the component sets of ratings for training consistency to create four values that align to each component of training consistency (i.e., companies’ consistency in the content, delivery, technology and administration of training). These averaged ratings are

Figure 12. Motivation for Training

I try to learn as much as I can from training

programs.

I tend to learn more from training programs than

most people.

I am willing to exert considerable effort in

training programs in order to improve my skills.

I am usually motivated to learn the skills emphasized

in training programs.

I believe I can improve my skills by participating in

training programs.

I believe I can learn the material presented in most

training programs.

Participation in training programs is of little use to me because I have all the

knowledge and skills I need to successfully perform my job.

I am willing to invest effort to improve skills and

competencies related to my current job.

I am willing to invest effort to improve skills and

competencies in order to prepare myself for a

promotion.

33%

33%

49%14%

13%

13%

16%

12%35%

36%

29%

42%

48%

50%

51%

48%

48%

49%

8%

5%

2%

4%

22%

16%

18%

18%18%

30%

30%

7%

22%31%

23%

1%

1%

1%

2%

1%

1%

1%

1%

1%

1%

1%

0%

Strongly Agree Agree Neutral

Disagree Strongly disagree

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 13

based on the data presented in Figures 7 through 10.

All four of these areas of consistency were found to be significantly related to learners’ ratings of professional development opportunities, with correlations ranging from 0.57 to 0.63. The size of these correlations denotes “large” relationships,2 which usually implies that such results are not a hiccup in the numbers. In other words, there is a correlational relationship between consistency across all facets of training and learners’ perceptions that their company provides professional development through training. So, when learners see frequent opportunities to better themselves, these learners are likely to work at companies that offer a consistent L&D experience.

All four of the areas of consistency were also found to be significantly related to learners’ ratings of overall training effectiveness, with correlations ranging from 0.43 to 0.58. These are considered medium-to-large correlations, demonstrating a significant relationship between aspects of training consistency and the general effectiveness of a company’s L&D offerings. In other words, the more consistent a company’s training is, the more effective it tends to be.

Lastly, all four of the areas of consistency were found to be significantly related to learners’ ratings of motivation for training,

LEARNERS AT COMPANIES WITH EFFECTIVE AND

CONSISTENT L&D PRACTICES WERE SIGNIFICANTLY MORE

LIKELY TO BE MORE SATISFIED IN THEIR JOBS.

2 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 14

with correlations ranging from 0.28 to 0.35. These are considered medium relationships, so while they are not as strong as the relationships between consistency and professional development opportunities, they nonetheless represent a significant relationship between consistency and learner motivation. Further, the relationships between consistency and motivation are stronger than the correlation between training effectiveness and motivation, which was significant but comparatively small in magnitude (r = 0.15). This finding suggests that a learner’s motivation may have more to do with the experience of training than with the ultimate effectiveness of the KSAs acquired.

Next, survey responses were entered into a series of analysis of variance (ANOVA) tests, which determined whether there were significant differences among the average scores for groups of learners. This analysis created the groups of learners by crossing ratings of training effectiveness (Figure 1) with ratings of

training consistency (Figure 2). This resulted in a group of 68 learners who rated their respective company’s training as both effective and consistent and

an opposing group of 72 learners who rated their company’s training as both ineffective and inconsistent.

Compared to companies that provide ineffective and inconsistent training programs, results of this analysis showed that learners at companies with effective and consistent L&D practices were significantly more likely to:

• Be motivated to make efforts to improve themselves and their job performance by participating in training3

• Be more satisfied with their jobs4

• Have much greater awareness of the benefits and opportunities afforded by training for their own professional development5

To summarize, consistency is significantly related to a learner’s motivation, perceptions of training effectiveness and disposition toward professional development. Moreover, companies whose training programs are both consistent and effective are more likely to have motivated learners who are satisfied with their jobs and see training as a means to professional development.

Consistency in training and L&D practices is a systemic endeavor, and its effects stretch across entire organizations.

3 F(1,138) = 17.53, p < .001, averages by group: 4.09 (consistent/effective) vs. 3.71 (inconsistent/ineffective) 4 F(1,136) = 44.58, p < .001, averages by group: 4.44 (consistent/effective) vs. 3.43 (inconsistent/ineffective) 5 F(1,138) = 116.10, p < .001, averages by group: 3.91 (consistent/effective) vs. 2.39 (inconsistent/ineffective)

TRAINING CONSISTENCY IS MULTIFACETED.

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 15

Consistency has multiple levels because it can be applied in many ways: consistency in a specific training event; learning path consistency for an employee in a particular department; and enterprise consistency at the strategic, cross-functional level. Consistency is ultimately systemic, because it cannot be removed from the context in which it occurs; consistency both affects and is affected by the organization providing the training.

Throughout the results of this research, it is clear that learners understand consistency as something worthy of their attention. Nine percent saw the training they’re offered as being always consistent, and 12 percent saw their

training as anything but. This leaves nearly four-fifths of learners’ experiences falling somewhere in between the poles of this continuum. In other words, companies have ground to make up when it comes to the consistency of their training.

Why should companies close the gap? What benefit do learners’ perceptions of consistency hold for the companies they work for? As we’ve shown in the preceding pages, the benefits are plenty, based on the statistical findings. For instance, ratings of training consistency (i.e., companies that are more consistent in the content,

delivery, technology and administration of training) were significantly related to learners’ ratings of the professional development opportunities they see available to them and to their motivation to learn. These findings suggest that a company that invests in the consistency of its training will tend to have learners who are more engaged both with the training and with their career trajectory at the company. This consistency also tends to be prevalent in companies with effective training programs. Further, companies with effective and consistent training have learners who are more motivated and satisfied with their work and have a positive regard for professional development, compared to learners at companies struggling with effectiveness and consistency in their L&D offerings.

If learners were generally unaware of or indifferent to their training experiences, the data would have painted a drastically different picture. As stated in the introduction to this report, consistency has been repeatedly identified in Training Industry research as a challenge for learning leaders. The results of this report underscore the downstream effects of meeting this challenge (or falling short) from the viewpoint of the learners at the receiving end of training programs.

So, how consistent has your company’s training been lately?

Conclusions

GREATER CONSISTENCY CAN BOOST LEARNERS’ ENGAGEMENT.

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 16

Study Demographics

Figure 13. Organizational Size

Figure 14. Industries Represented

Figures 13 through 16 provide context on the 283 survey respondents who participated in this research. These figures summarize the companies the respondents represent and the functional areas/departments supported by their roles within the organization.

Health Care/Medical/Pharma 20%

3%

< 100 101-500 501-1k 1k-5k 5k-10k 10k-20k 20k-50k 50k+

6%

10%

24% 24%

15%

11%

8%

11%

12%

10%

8%

8%

4%

3%

2%

2%

1%

1%

1%

17%

Banking/Finance/Insurance

Durable Goods/Consumables

Manufacturing

Government

Entertainment/Hospitality

Training and Development

Education

Nonprofit

Utilities

Technology/Telecom

Business Services/Consulting

Construction

Other

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 17

Figure 15. Functional Areas/Departments Represented

Figure 16. Job Roles Represented

Customer Service

Associate

18%

18%

L&D

11%

15%

11%

SalesHR Marketing/ Advertising

IT

Specialist Analyst Manager Trainer Consultant Instructor Executive Level

Instructional Designer

12%

Operations

8%

Other

33%

40%

Finance/Accounting

4% 4%

4%

4%

1%

1%1%

R&D

6%

5% 5%

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE ©2017 | 18

About Training Industry

Our focus is on helping dedicated business and training professionals get the information, insight and tools needed to more effectively manage the business of learning. Our website, TrainingIndustry.com, spotlights the latest news, articles, case studies and best practices within the training industry.

For more information, visit www.trainingindustry.com, call 866.298.4203, or connect with us on Twitter and LinkedIn.

Training Industry, Inc. research captures the collective wisdom of learning professionals, revealing fresh data on trends and practices in the evolving training market.

Copyright © 2017 by The Training Associates and Training Industry, Inc. All rights reserved. No materials from this study can be duplicated, copied, republished or reused without written permission from The Training Associates or Training Industry, Inc. The information and insights contained in this report reflect the research and observations of Training Industry, Inc. and the Training Associates Corporation.

About The Training Associates

Founded in 1994, The Training Associates is the industry recognized leader of learning and development talent and solutions. TTA’s talent offerings feature over 25,000 L&D resources specializing in adult learning for all technologies and business professional skills. Our services include our ability to undertake and manage projects of any size or scope related to the design, development, and delivery of training.

For more information, visit www.thetrainingassociates.com, call 800.241.8868, contact us at info@ttacorp.com, or connect with us on Twitter and LinkedIn.

About This Research

Copyright © 2017 by The Training Associates and Training Industry, Inc. All rights reserved. No materials from this study can be duplicated, copied, republished or reused without written permission from The Training Associates or Training Industry, Inc. The information and insights contained in this report reflect the research and observations of Training Industry, Inc. and the Training Associates Corporation.