MEASUREMENT OF EFFECTIVENESS OF TRAINING PROGRAMS IN INFORMATION
TECHNOLOGY INDUSTRY
Submitted by J.R.VAISHNAVI PRASAD 7NBMD002 Under the guidance of
MS. SHILPA KULKARNI
ICFAI NATIONAL COLLEGE, JP NAGAR, BANGALORE
1
DECLARATION
I hereby declare that the Management Thesis entitled A STUDY ON
MEASUREMENT OF EFFECTIVENESS OF TRAINING PROGRAMS IN INFORMATION
TECHNOLOGY INDUSTRY submitted to ICFAI National College in partial
fulfillment of the requirements for the award of degree of MASTER
OF BUSINESS ADMISTRATION is a record of the original work carried
out by me during the period of study at ICFAI National College and
this work has not formed the basis for the award of any
degree/diploma/associate ship/ fellowship of similar title to any
candidate of any university.
Signature of the Faculty Guide
Signature of the student
Place: Date:
2
TABLE OF
CONTENTSACKNOWLEDGEMENT..............................................................................................................4
INTRODUCTION.......................................................................................................................9
Objectives............................................................................................................................9
Limitations.........................................................................................................................10
Research
methodology......................................................................................................10
.........................................................................................................................................10
Measurement of training
effectiveness..............................................................................12
Kirkpatricks training evaluation
model..............................................................................13
INDUSTRY
PROFILE................................................................................................................16
Information Technology
industry........................................................................................16
COMPANIES
PROFILE.............................................................................................................19
Cisco Systems
Inc..............................................................................................................19
Aztecsoft............................................................................................................................21
Huawei
Technologies.........................................................................................................23
Symphony
Services............................................................................................................25
General Motors Technical
Centre.......................................................................................26
DATA
ANALYSIS.....................................................................................................................28
I PRE TRAINING
PHASE.......................................................................................................28
II MEASUREMENT OF
REACTIONS......................................................................................34
III MEASUREMENT OF
LEARNING.......................................................................................35
IV MEASUREMENT OF
BEHAVIOR........................................................................................37
V MEASUREMENT OF
RESULTS...........................................................................................38
VI OVERALL TRAINING
EFFECTIVENESS:.............................................................................40
Findings.............................................................................................................................45
Suggestions.......................................................................................................................46
Conclusion.........................................................................................................................47
REFERENCES.........................................................................................................................48
ANNEXURES.......................................................................................................................50
3
ACKNOWLEDGEMENT'Gratitude is the memory of the heart' - Jean
Baptiste Massieu, translated from French
I thank The Almighty for his immense grace and blessings
showered throughout this period of management thesis
preparation.
My sincere thanks to Ms. Shiny Samuel, Learning and Development
Manager, Cisco Systems; Ms. Roopashri, HR Manager, Aztecsoft; Ms.
Swathi shetty, Assistant Training Manager, Huawei Technologies; Ms.
Swathi Srivastava, Assistant HR Manager, Symphony Services and Ms.
Rituparna Dasgupta, Training Manager, GMTC; for their valuable
information on training and development on which this report is
based.
I extend my heartfelt gratitude to my Faculty Guide Ms. Shilpa
Kulkarni for her expert guidance and encouragement in all my
endeavors of preparing the management thesis. But for her keep
interest and constructive criticism, this study would not have been
as successful as it is now.
Finally, I would like to thank my family and friends for their
constant support throughout the study.
J.R.VAISHNAVI PRASAD
4
LIST OF ILLUSTRATIONSFigure 1 PURPOSE OF TRAINING
EVALUATION......................................................................13
Figure 2 KIRKPATRICKS TRAINING EVALUATION
MODEL......................................................14
Figure 3 LEVELS OF TRAINING
NEEDS.................................................................................29
Figure 4 KIRKPATRICK MODEL FOR EVALUATING EFFECTIVENESS OF TRAINING
PROGRAMS34 Figure 5 KIRKPATRICK'S 4 LEVELS OF EVALUATION AND
PHILLIPS' ROI METHODOLOGY
..............................................................................................................................................39
LIST OF TABLESTable 1 EFFECTIVENESS OF REACTIONS IN MEASURING
TRAINING SUCCESS.......................41 Table 2 EFFECTIVENESS OF
LEARNING IN MEASURING TRAINING SUCCESS.........................41
Table 3 EFFECTIVENESS OF BEHAVIOR IN MEASURING TRAINING
SUCCESS.........................41 Table 4 EFFECTIVENESS OF RESULTS
IN MEASURING TRAINING SUCCESS...........................42
5
SUMMARY
According to Planty, Cord M.C. and Efferson, Training is the
continuous, systematic development among all levels of employees of
that knowledge and those skills and attitudes which contribute to
their welfare and that of the company. As organizations have
started to realize the need for human resource development for the
survival of the business in the ever changing competitive world,
the importance of training and development has increased. Training
makes a very important contribution to the development of an
organizations human resources. To achieve the purpose of training,
it has to be effectively managed so that the right training is
given to the right people, in the right form, at the right time and
at the right cost. Training is a systematic process which involves
identification of training needs, formulation of training
objectives, designing of training program, administration of
training and evaluation of training effectiveness. The various
resources of the organizations are put to use to train the
employees. Therefore, justification of resource utilization has to
be made by measuring the effectiveness of training programs to
ensure achievement of desired results. This process of comparing
the post training results to the objectives expected by
administrators, trainers and trainees is known as training
evaluation. Measurement of training effectiveness can be done by
different methods. This study entitled Measurement of effectiveness
of training programs in IT industry tries to understand the
training effectiveness measurement in IT companies based on
Kirkpatricks model. According to this model, effectiveness of
training programs can be measured at four levels: reaction,
learning, behavior and results.
6
The present study is a qualitative research on training
effectiveness through semi structured interviews with training and
development managers of five IT organizations. The five IT
organizations are into different facets of IT such as hardware,
telecommunication solutions, software engineering services and
research and development. The study attempts to explore the
training and development function in IT industry with respect to
measurement of training effectiveness.
7
INTRODUCTION
8
MEASUREMENT OF EFFECTIVENESS OF TRAINING PROGRAMS IN INFORMATION
TECHNOLOGY INDUSTRY
INTRODUCTIONTraining is a learning process that involves the
acquisition of knowledge, sharpening of skills, concepts, rules, or
changing of attitudes and behaviors to enhance the performance of
employees. It can be considered as the systematic process which has
its organizational purpose to impart and provide learning
experience. The objectives of training include increase of
productivity, improvement of quality, prevention of technological
obsolescence and enhancement of employees personal growth. The
project entitled Measurement of effectiveness of training programs
in IT industry is an attempt to understand the process of training
effectiveness evaluation followed by different companies in the
Information Technology industry based on the Kirkpatricks model. 5
IT companies are studied for the management thesis.
ObjectivesThe following are the objectives of the study: To
understand the prerequisites of training of various Information
Technology companies To get a better insight on the training
effectiveness measurement levels adopted by different Information
Technology companies according to the Kirkpatricks model
9
LimitationsThe limitations of the study are: The sample selected
for the study might not be representative of the universe The study
would be conducted from the perspective of the organization;
employee perspective would not be considered The study covers the
pre-training phase and post-training phase not considering the
training phase.
Research methodologya) STATEMENT OF THE PROBLEM: As training has
become imperative for the development of human resources in an
organization, the study attempts to understand the parameters
considered in designing a training program. The criteria for
measuring training effectiveness across the IT industry are
studied. b) DATA COLLECTION: The data is collected through primary
and secondary sources. Semi structured interview method is used for
data collection The secondary data sources include books and
websites. c) TARGET POPULATION: The target population consists of
Information technology companies, both hardware and software, in
Bangalore. d) SAMPLING METHOD: Convenient sampling method is
adopted for the study. The sample size is 5 and the sampling unit
is an IT company.
10
The process of training is as follows: Identification of
training needs: The first step in the process of training is the
identification of training needs. The need for training arises when
there is a gap between expected performance and existing
performance. The results of training needs analysis will highlight
the content and scope of training. Development of training design:
A training design is a blue print of a training program which
outlines the training objectives, contents and methods. Stating the
purpose of training and ways and means of achieving the purpose is
of prime importance as they direct the entire training program.
Selection of trainer: As the trainer acts as the medium for
delivering the training content, trainer has to be selected after
considering the participant profile, trainers proficiency and other
similar factors. Administration of training: Training
administration refers to the design and development of training
programs. It includes various activities ranging from information
dissemination of program availability to supporting training
evaluation. Evaluation of training: Measuring the effectiveness of
training so as to ensure that the training objectives have been met
is the last step in the training process. Although different
methods of evaluation are in existence, the purpose of training
effectiveness measurement is to determine the level of success of
the training program in terms of human resource development.
11
Measurement of training effectivenessAs organizations have
started to realize the importance of human resource development,
the investments made on training the employees has increased. As a
result, measuring the effectiveness of training programs has gained
significance in the recent times.
Purposes of Training Evaluation The five main purposes of
training evaluation are: Feedback: It helps in giving feedback to
the candidates by defining the objectives and linking it to
learning outcomes. Research: It helps in ascertaining the
relationship between acquired knowledge, transfer of knowledge at
the work place, and training. Control: It helps in controlling the
training program because if the training is not effective, then it
can be dealt with accordingly. Power games: At times, the top
management (higher authoritative employee) uses the evaluative data
to manipulate it for their own benefits. Intervention: It helps in
determining that whether the actual outcomes are aligned with the
expected outcomes.
12
Figure 1
PURPOSE OF TRAINING EVALUATION
Source: www.
traininganddevelopment.naukrihub.com/training-evaluation
Kirkpatricks training evaluation modelThe effectiveness of
training programs can be evaluated by different means. One of the
widely used models for measuring training effectiveness is the
Kirkpatricks learning evaluation model proposed by Donald L
Kirkpatrick in the year 1959. The model consists of four levels
reaction, learning, behavior, results through which the
effectiveness of training programs can be measured. Reaction: The
reaction level evaluates the feelings of the participants towards
the training program. Feedback can be obtained through various
tools such as questionnaires on the training content, material,
method, trainer style and facilities. The reaction of participants
is a critical factor in the continuance of training programs. Most
trainers feel that initial receptivity provides a good atmosphere
for learning the material in the instructional program but not
necessarily cause high levels of training.
13
Learning: The increase in knowledge, skills, and attitude is
measured in the learning level of Kirkpatricks model. Tests,
surveys, interviews are the commonly used tools of evaluating
learning. A before-and-after approach is usually used to trace
learning to the training program. It is relatively easy to measure
the learning that takes place in technical trainings rather than
behavioral trainings. Behavior: At the behavior level, the transfer
or application of learning through training is measured. A
comparison of performance before and after training would help in
understanding the behavioral changes that have taken place as a
result of training. The behavioral implications are normally
measured after a few months of the training. Control group studies
are commonly used for measuring training effectiveness. Results:
The impact of improved employee performance on the business
environment is evaluated at the results level. Quality improvement,
attrition ratio and organizational growth are some of the measures
of evaluating training results. The challenge is to identify the
degree of the trainee's input and influence on the results of the
organization. Figure 2 KIRKPATRICKS TRAINING EVALUATION MODEL
Source:
tryout.questionmark.com/learningcafe/1919374234/Assessments_and_Kirkpatrick_Model.ppt
14
INDUSTRY PROFILE
15
INDUSTRY PROFILE
Information Technology industryInformation technology is an
integral part of nearly every major global industry. Unlike other
common industries, the IT industry is knowledge-based. Efficient
utilization of skilled labor forces in the IT sector can help an
economy achieve a rapid pace of economic growth. The IT industry
helps many other sectors in the growth process of the economy.
Information Technology (IT), as defined by the Information
Technology Association of America (ITAA), is "the study, design,
development, implementation, support or management of
computer-based information systems, particularly software
applications and computer hardware." IT deals with the use of
electronic computers and computer software to convert, store,
protect, process, transmit, and securely retrieve information.
Today, the term information technology encompasses many aspects of
computing and technology, and the term is more recognizable than
ever before. IT professionals perform a variety of duties that
range from installing applications to designing complex computer
networks and information databases. A few of the duties that IT
professionals perform may include data management, networking,
engineering computer hardware, database and software design, as
well as the management and administration of entire systems.
Information Technology (IT) is a general term that describes any
technology that helps to produce, manipulate, store, communicate,
and/or disseminate information. The Indian IT sector is growing at
a very fast pace. In 2006, it has earned revenue of about US $ 40
billion with a growth rate of 30%. According to NASSCOM (National
Association of Software and Service Companies) IT sector is
expected to generate 2.3 million jobs by 2010.
16
According to the recent review by Harvard Business Review, there
is a direct link between training investment of the companies and
the market capitalization. Those companies with higher training
investment had higher market capitalization. It clearly indicates
that the companies which have successfully implemented training
programs have been able to deliver customer goals with effective
results. It shows that good training results in enhancement of
individual performance, which in turn, helps the organization in
achieving its business goals. Training is a tool that can help in
gaining competitive advantage in terms of human resource. With the
growing investment by IT companies in the development of their
employees many companies have now started their own learning
centers. Accenture has Internet based tool by the name of My
Learning that offers access to its vast learning resources to its
employees. Companies are investing in both the technical training,
which has always been an essential part in IT industry, as well as
in managerial skills development. Companies now keep aside 3-5% of
revenue for training programs. As an example, TCS has decided to
allocate 4% of its revenue towards training of employees.
17
COMPANIES PROFILE
18
COMPANIES PROFILE
Cisco Systems Inc.
Cisco Systems, Inc. is the worldwide leader in networking for
the Internet. Today, networks are an essential part of business,
education, government and home communications, and Cisco Internet
Protocol-based (IP) networking solutions are the foundation of
these networks. Cisco hardware, software, and service offerings are
used to create Internet solutions that allow individuals,
companies, and countries to increase productivity, improve customer
satisfaction and strengthen competitive advantage. The Cisco name
has become synonymous with the Internet, as well as with the
productivity improvements that Internet business solutions provide.
Cisco was founded in 1984 by a small group of computer scientists
from Stanford University. Since the company's inception, Cisco
engineers have been leaders in the development of Internet Protocol
(IP)-based networking technologies. This tradition of innovation
continues with industry-leading products in the core areas of
routing and switching, as well as advanced technologies in areas
such as Home Networking, IP Telephony, Optical, Network Security,
Storage Networking and Wireless LAN. The vision of Cisco is to
change the way people work, live, play and learn. The organization
has close to 65,225 employees worldwide John T. Chambers is the
Chairman and Chief Executive Officer of Cisco Systems Inc. During
the financial year 2007-2008, Cisco Systems posted a revenue of
$39.5 billion. Cisco Systems India Pvt Ltd. commenced operations in
1995. India, as a region, is part of the APAC theater. APAC theater
contributes to about 11 percent of the global revenue of Cisco
Systems. It has 7 sales offices across the region in New Delhi,
Mumbai, Bangalore, Chennai, Pune, Kolkata and Hyderabad. The
headcount in India is over 4000, including R&D, sales and
19
business support staff. Cisco Global Development Center in
Bangalore is the largest outside of the US. The market share of
Ciscos core technologies in India is as follows: Routers 81.1%
Switches 74.4% Total LAN 81.6% Some of the key customers of Cisco
Systems in India include Ford India, Dr. Reddys Laboratories, Tata
Group, Mahindra & Mahindra, Larsen & Toubro, Gas Authority
of India, Indian Oil Corporation, State Bank of India, Life
Insurance Corporation, Future Group Infosys, Wipro, IBM, Adobe,
Intel, Cognizant Technologies, Tata Consultancy Services and Texas
Instruments.
20
Aztecsoft
Aztecsoft is a 2,200 people strong software engineering services
partner for software product companies. It provides full lifecycle
product engineering, independent testing, professional services and
sustained engineering services in various domains such as
application and Infrastructure Software, new media and internet,
financial services software and life sciences software. The company
has developed and tested over 1600 products. It has its presence in
Unites States of America, United Kingdom and Australia apart from
India. Mission: We delight customers by engineering software
solutions in a collaborative environment of innovation, growth and
relentless execution. Vision: To be the fastest growing, most
efficient and reliable global software engineering services company
in the world. Values: Integrity, Passion for Excellence, Fairness,
Teamwork, Transparency, Empowerment, Respect for the Individual.
Aztecsoft provides 4 key advantages to its customers software
engineering needs which are increased speed to market, improved
quality, managing uncertainty, and access to specialists. Some of
the awards and recognitions won by Aztecsoft are as follows:
'Best IT/Telecom Recruitment Advertisement' and 'Best use of
Visuals in an Advertisement' at the Pink Slip Awards 2008
"Innovative Network Award 2007" from Airtel Enterprise
Services21
22nd in Business World's list of Top 100 mid-size companies in
India. Testing Leadership Award 2007" by QAI India Limited 28th in
the Deloitte 2005 Technology Fast 50 India listing International
Organization for Standardization (ISO) 27001:2005 certification for
information security Assessed at Software Engineering Institute
(SEI) Capability Maturity Model Integration Software (CMMI-SW)
Version 1.1 Maturity Level 5 (the highest, Optimizing Level of
Maturity)
A few customers of Aztecsoft include Microsoft, AOL, Fiberlink,
Cadence, Simdesk technologies and Embarcadero technologies.
22
Huawei Technologies
Huawei is a world-leading telecom solutions provider focused on
building long-term partnerships with telecom operators. It has
built up product portfolios including mobile network, broadband
network, IP-based, optical network, telecom value-added service,
and terminal. The products and solutions offered by Huawei are
deployed in over 100 countries and serve over one billion users
worldwide. In 2007, 72% of contract sales were from the
international market, and the organization has set up more than 100
branch offices in order to provide quick services to our customers.
It has established 12 R&D centers around the world. In
addition, there are 31 training centers worldwide to help customers
and local people to study advanced management, technologies and so
on. Huawei believes in localizing its global operations, and makes
it a point to hire local employees. This not only enhances
understanding of the local market, but also contributes to the
local economy by increasing employment, especially in less
developed regions.
Vision: To enrich life through communication. Mission:
23
To focus on our customers' market challenges and needs by
providing excellent communications network solutions and services
in order to consistently create maximum value for customers. Huawei
Technologies has fulfilled the needs of 70% of the world's top 50
telecom companies. As a singular mark of recognition by the world's
leading mobile operator, Vodafone awarded Huawei a 2007 Global
Supplier Award citing delivery of highly competitive products and
services across the group's network. Huawei was the only network
equipment supplier to be awarded this specific accolade. According
to the World Intellectual Property Organization (WIPO), Huawei
moved up 9 places to become the 4th largest patent applicant under
the WIPO Patent Cooperation Treaty (PCT), with 1,365 applications
published in 2007. By the end of 2007, Huawei had deployed over
100,000 GSM green base stations, saving approximately 570 million
kilowatt-hours (kWh) or 170,000 tons of coal. This effort Huawei
technologies effort towards environmental protection and energy
conservation shows its corporate social responsibility.
24
Symphony Services
Symphony Services is a global provider of product engineering
outsourcing services dedicated to helping clients compress time to
market, achieve higher innovation yields and improve productivity
to increase the value from their R & D investments. From
strategic consulting guidance to complete product lifecycle
capabilities, Symphony is distinguished by a singular focus on
product development and collaborative, end-to-end solutions.
Symphony Services was founded in 2002 with initial funding from
Romesh Wadwhani, Chairman and CEO of Symphony Technology Group. In
2003 Symphony Services raised growth capital from TH Lee Putnam
Ventures. There are 3500+ professionals presently working with
Symphony Services. It has been recognized as #1 Software
Engineering Services Provider and Top 50 Best Managed Companies for
2007 by Black Book of Outsourcing Mission To create unprecedented
value for all stakeholders through Innovation, Collaboration and
Operational Excellence in partnership with clients that embrace
globalization to boost engineering and R&D performance and
optimize value across the full product and service lifecycle. The
values of Symphony Services are client delight, integrity, employee
empowerment, esprit de corps, measurable value, innovation and
entrepreneurial spirit.
Symphony Services has a delivery record of more than 1000
products released during 2007. It has witnessed a 48% growth
year-over-year.
25
Symphony Services has 150+ clients. Some of them are Avaya, AMD,
Juniper Networks, Motorola, Qualcomm, Siemens, McGraw-Hill
Companies and SUN Microsystems.
General Motors Technical Centre
General Motors Technical Centre - India (GMTC-I) is a 100%
subsidiary of the automobile legend, General Motors Corporation
USA. This centre at International Tech Park Ltd. (ITPL) Bangalore
houses the R&D and Engineering Centre that provides
Computer-Aided Design/Manufacturing/Engineering (CAD / CAM / CAE)
and Electrical technology support to GMs global automotive
engineering efforts. This organization adds to GMs engineering
capability in Asia Pacific, complementing similar technical centres
in Australia and China. General Motors India, incorporated in 1994
as a 50:50 joint venture company with the CK Birla Group of
Companies, became a fully owned subsidiary of GM in 1999. The
company was restructured in 1999 and was converted from a Public
Limited company to a Private Limited company. General Motors
Corporation, the world's largest automaker, has been the annual
global industry sales leader for 77 years. Founded in 1908, GM
today employs about 266,000 people around the world. With global
headquarters in Detroit, GM manufactures its cars and trucks in 35
countries. General Motors India is now a wholly-owned subsidiary of
General Motors Corporation, the largest automaker in the world. It
offers products under the Chevrolet brand in the country.
Introduced in India in 2003, under the banner For a special journey
called life, Chevrolet has emerged as one of the fastest growing
brand names in India. Today, the Chevrolet brand clearly
26
is established for quality, reliability, safety, innovation,
value for money proposition, environment-friendliness and customer
care. There are 1658 employees working for General Motors Technical
Centre in Bangalore. This centre provides research assistance to
its internal customers by deploying technology
DATA ANALYSIS
27
DATA ANALYSISAnalysis and interpretation of dataThe data for the
study was collected by interviewing the training managers of 5 IT
organizations. The following analysis of data is based on
pre-determined dimensions of training and commonalities of answers
obtained from the training managers. The dimensions considered for
training evaluation analysis are pre training phase, measurement of
reactions, measurement oflearning, measurement of behavioral
changes, measurement of results and overall training
effectiveness based on Kirkpatricks levels.
I
PRE TRAINING PHASE The first phase of training is the pre
training phase which comprises of training needs identification,
selection of participants and trainer, formulation of training
strategy and training design. This stage is of utmost importance as
it serves as the basis for successful training implementation and
evaluation.1.
Frequency of training needs analysis: Training needs can be
identified by deducting the existing skills from the job
requirements. A training need exists when there is a gap between
what is required of a person to perform his/her work competently
and what he/she know. A training needs analysis is the method of
determining if a training need exists and if it does, what training
is required to fill the gap. Inferences: According to the study, it
can be noted that in IT organizations, training needs analysis is
conducted once in a year. However, the introduction of a new
technology or assignment of a new project to a team might result in
conducting a need-based training need analysis.
28
2.
Sources of information for training needs analysis: There are
three levels of training needs assessment organizational level,
task level and individual level. Organizational level assesses the
organizational effectives to determine training needs. Task level
provided data on a particular job to identify the knowledge, skills
and abilities required for optimum performance. Individual level
assesses the performance of employees so as to determine the
training needs. Figure 3 LEVELS OF TRAINING NEEDS
Source: www.traininganddevelopment.naukrihub.com
Inferences: Individual level: The study reveals that the
performance appraisal of employees serves as the primary source of
information to identify training needs. Cisco systems follow a
systematic process of training needs assessment. At the beginning
of every fiscal year, the employees draft their development plan
after discussing with their respective supervisors. This is managed
by an automated system known as E-Performance Management (EPM)
which acts as an input for training design development.
29
The competency assessment performed by Symphony Services and
Aztecsoft also provides input to training needs identification. The
skill metrics of GMTC is an important source of training needs
analysis. On a scale of 5, an employee is supposed to score 3 and
above failing which he/she will have to undergo training in the
particular skill enhancement. Operational level: The business plan
of Huawei Technologies is considered as a primary source of
training needs assessment as the project demands for a specific
skill has to be fulfilled through training. Organizational level:
The training needs assessment of Aztecsoft includes surveillance of
internal as well as the external environment. Aztecsoft conducts a
market analysis to identify emerging technologies and the probable
clients who might adopt those technologies. This acts as an input
to train employees. 3. Basis of participant selection: Selection of
participants is one of the steps in the pre-training phase. The
question of participant selection arises due to ceiling on the
number of participant for a training program. The urgency of
training requirement is an important consideration in participant
selection. Inferences: According to the study, the participants of
training programs are selected based on the nominations given by
managers. The employees might nominate themselves for the training
which after being approved by the managers is taken forward by the
training department. In GMTC, engineers within a team act as
training coordinators who send nominations with the managers
approval.
30
4.
Mandatory training man days: Training mandays indicate the
number of days an employee has to spend in a year on training.
Training man days are determined after considering various factors
such as the objectives of the organization, employees profile, job
competencies and budget. Inferences: From the study, it is inferred
that Cisco Systems and Huawei Technologies do not specify mandatory
training man days to be completed by their employees during the
fiscal year. Symphony Services and Aztecsoft has a training policy
which makes it mandatory for its employees to undergo a minimum of
10 man days and 40 man days respectively. At GMTC, the number of
hours an employee has to undergo training in a year is 2 for
engineers and 6 for managers in addition to the 10 hours web-based
mandatory training to be undergone by all employees.
5. Training design considerations: A training design is a blue
print of a training event or exercise. It gives a detailed plan of
what to do, why to do and how to do. The following are the steps
involved in designing a training program: Choose a training
strategy Break the general training objectives into constituent
parts Mention the specifications of different training programs
Decide on different training packages Design a training program
with detailed training events and schedules Some of the factors
considered in designing a training program are suitability of the
training to meet identified training needs, availability of
resources and expected quality standards. Inferences: Training
objectives and participant profile form the basis of a training
design according to the study. While the cost of training is also
considered by Aztecsoft and Symphony31
Services, the feedback obtained from participants on previous
trainings serve as an input of training design at GMTC.
6.
Classifications of trainings: Training programs can be
classified into different kinds based on different factors. The
nomenclatures used differ from one organization to another.
Inferences: The study reveals that Cisco Systems and Aztecsoft
classify their training into technical and behavioral trainings.
The training programs at Huawei Technologies are classified into
role-based trainings, project-based trainings, compulsory trainings
and soft skills trainings. Compulsory trainings include induction,
quality management and testing trainings. GMTCs trainings are
classified under four heads which are functional, soft skills,
process and mandatory trainings. Symphony Services classifies its
trainings into technical and non-technical where non-technical
trainings encompass soft skills, behavioral and people management
trainings.
7.
Tools of trainings: Training methods differ depending upon the
objective, participant profile, cost and time availability. The two
classifications of training methods are on-the-job trainings and
offthe-job trainings. While on-the-job trainings are used for
technical trainings, off-the-job trainings are suitable for
behavioral trainings. Inferences: On-the-job training tools are
widely used by the organizations under study for technical
trainings. Role plays and case studies are the commonly used tools
of behavioral learning. Cisco believes that learning takes place
through 3es: experience, exposure and education. Hence, it
emphasizes on experiential learning. Self-learning programs form an
integral part of the learning tools of Aztecsoft as the initiative
of the employees indicates their interest towards learning.32
8. Hiring of external trainer: Trainer is one of the human
factors that have a bearing on training effectiveness. The
knowledge, skills and attitude of the trainer should suit the
requirements of the training. Trainers can be internal to the
organization or hired from outside. Inferences: The study reveals
that non-availability of internal experts is the prime factor
leading to hiring of external trainers. At Cisco Systems, all
trainings are outsourced. Among the different vendors with whom
Cisco deals, the best vendor for the particular training is chosen.
Only 10% of the trainers are external at Aztecsoft. This is mainly
due to the low cost factor inherent in trainings provided by
internal experts.9.
Methods of ensuring similarity in training across branches:
Organizations try to ensure quality standards in all its processes.
Training is no exception. It is imperative for the organizations to
ensure similarity in delivery of trainings across different
branches. Inferences: It is inferred from the study that most of
the times, the same training material for a particular training is
used for trainings conducted at different branches of the
organizations. Same trainers might also travel to the various
branches of the organization to ensure similarity in delivery of
training.
33
II
MEASUREMENT OF REACTIONS
The training would be delivered after considering the various
factors discussed above. The third phase of training involves
training evaluation. According to Mathis and Jackson Evaluation of
training compares the post-training results to the objectives
expected by administrators, trainers and trainees. The process of
examining a training program is called training evaluation.
Training evaluation checks whether training has had the desired
effect. Training evaluation ensures that whether candidates are
able to implement their learning in their respective workplaces, or
to the regular work routines. There are different models proposed
to evaluate the effectiveness of training. The commonly used model
is Kirkpatricks model. Donald Kirkpatrick developed the model of
training evaluation in 1959. This 4-level model is considered to be
simple, flexible and complete. The four levels of Kirkpatricks
model are reaction, learning, behavior and results. Figure 4
KIRKPATRICK MODEL FOR EVALUATING EFFECTIVENESS OF TRAINING
PROGRAMS
Level 4 Results Level 3 Behavior Level 2 Learning Level 1
Reaction
- What organizational benefits resulted from the training? To
what extent did participants change their behavior back in the
workplace as a result of the training? To what extent did
participants improve knowledge and skills and change attitudes as a
result of the training? How did participants react to the
program?
-
-
Source: www.businessperform.com
34
1. Feedback on trainings: Level one of Kirkpatricks model
measures the feelings of the participants towards the training
program. The reactions of the participants can be measured by
seeking feedback on the training. It is important to include
closed-ended items (including rating scales) as well as open-ended
items in the feedback form. Mentioning of names in the feedback
forms can be made optional to avail the advantages of anonymity
over confidentiality.
Inferences: According to the study, the organizations obtain
feedback on all training programs. The usual way of obtaining
feedback from participants is through questionnaires which might be
in the paper form or electronic form. 2. Aspects covered in a
feedback form: A questionnaire can identify what to keep, delete,
or improve in a training program design. As a result, changes may
be made in the areas of content, methods, media, trainer style,
facilities and/or course materials. Inferences: The study reveals
that the feedback questionnaires of organizations seek feedback on
course content, trainer, facilities and training administration.
The organizations are of the view that the participants reaction on
the success of the training to meet its objectives is a prime
factor to measure training effectiveness.
III
MEASUREMENT OF LEARNING
The change in knowledge, skills and attitudes as a result of
training is measured in level two of Kirkpatricks model. Learning
evaluation is the measurement of the increase in knowledge or
intellectual capability from before to after the learning
experience. Interviews, tests, surveys and observations are a few
methods of measuring learning.
35
1. Trainings involving measurement of learning: The measurement
of learning through training is restricted to a few trainings only.
This is mainly because of the difficulty in measuring changes in
attitude. Inferences: It is inferred from the study that learning
through training is measured for technical trainings only. Learning
resulting from behavioral training is usually not measured due to
the complexity in measuring soft skills acquired and attitudinal
changes. 2. Time of measuring learning: The time period of
measuring learning is equally important. In order to find out the
exact increase in knowledge, skills, attitude, a pre and post
analysis is performed. Inferences: The study reveals that a pre and
post training test is conducted by most of the organizations.
Huawei Technologies conducts a test only after the training to
assess the knowledge or skill level of the participants resulting
from training.3.
Methods of measuring learning: There are different methods of
measuring learning. Interviews, tests, surveys and observations are
a few methods of measuring learning. Use of control groups enhances
the accuracy of learning measurement. Inferences: Test is the
commonly used method of measuring learning. The skill metrics of
GMTC is used to assess the learning through training. On a scale of
5, an employee is supposed to score 3 and above for successful
completion of training. Aztecsoft relies on feedback from managers
on knowledge increase in employees who had undergone the training
to measure learning through training.
36
IV
MEASUREMENT OF BEHAVIOR Level three of Kirkpatricks model
measures the transfer or application of learning on the job. The
changes in the performance of the employees as a result of training
are measured at this level. 1. Measurement of impact of training on
performance: Impacts are an expression of the changes produced in a
situation as a result of training activities undertaken with
certain training objectives. The three possible impacts of training
are i. ii. iii. Positive impacts in desirable direction Neutral or
no impact in any direction Negative impact in the undesirable
direction
The organizations objective is to create a positive impact on
the performance of employees in the desirable direction through
training. Inferences: It is inferred from the study that feedback
from managers is obtained by most of the organizations to measure
the transfer or application of training by employees. Aztecsoft
conducts a 60 day assessment where the employees performance after
60 days of training is evaluated by the manager to assess the
impact of training on performance. Huawei Technologies and Symphony
Services are yet to implement level three of Kirkpatricks
model.2.
Trainings involving measurement of behavior: Measurement of
behavior is vital to assess the effectiveness of training. As the
goal of the organization is to improve employee performance, the
transfer of training to the real work environment has to be
measured. However, it is difficult to trace the behavioral changes
to training by eliminating other impacts. Therefore, organizations
measure application of training only to a few training programs
after considering the feasibility of measuring transfer of training
on the job.37
Inferences: The study reveals that behavioral changes are
measured for technical trainings. Due to the difficulty of
attributing behavioral changes in managerial skills to training,
level three evaluation is restricted to technical trainings.
V
MEASUREMENT OF RESULTS Measurement of results tries to measure
the contribution of training towards the bottomline results of the
organization. Level four outcomes are either changes in financial
outcomes (such as positive ROI or increased profits) or changes in
variables that have a relatively direct effect on financial
outcomes at some point in the future. Examples of different kinds
of outcomes are increased quality of work, higher productivity,
improved human relations, reduction in wastage of resources and
increased profits. 1. Methods of measuring results: Results
evaluation is the effect on the business or environment resulting
from the improved performance of the trainee. Measures of results
include key performance indicators in terms of volume or values,
attrition rate, quality compliance and achievement of standards.
Inferences: The study reveals that most of the organizations do not
measure the results of training due to the difficulty involved in
establishing firm evidence that a training program was the key or
only source that produced the level four results or outcomes. Cisco
Systems measures results of training by considering successful
delivery of projects on time. Results or outcomes of training are
measured at GMTC by calculating the project hours taken before and
after the training and increase in productivity. The measurement of
results is restricted to technical trainings.
38
2. Measurement of ROI and its scope in the future: According to
Som Mittal, President, NASSCOM, IT companies in India spend $5000
in training a fresh engineer and $3000 in training a fresh BPO
employee. A study of 15 countries in the Organization for Economic
Cooperation and Development found that the majority of enterprises
believe employee training is responsible for "productivity
improvements, greater workforce flexibility, savings on material
and capital costs, improved quality of the final product or service
and a more motivated workforce. However, many companies have not
measured the benefits and the cost of training in a way that
reveals the rate of return on a firm's investment. When
organizations spend a significant amount of their revenues on
training, it is imperative to measure the return on investment. The
key proponent of training ROI is Jack Phillips. He expressed ROI as
a ratio of net benefits to cost. It can be calculated using the
following formula: %ROI = (Total benefits program costs/ Program
costs) * 100 Figure 5 KIRKPATRICK'S 4 LEVELS OF EVALUATION AND
PHILLIPS' ROI METHODOLOGY
Source: www.5leveltraining.com
39
Inferences: It is inferred from the study that all the
organizations are still in level three or four of Kirkpatricks
model. A few of them opined that there is scope for measuring ROI
in the future. However, considerable time would be required to
devise a proper methodology for measuring training ROI.
VI
OVERALL TRAINING EFFECTIVENESS: Kirkpatricks model prescribes
four levels of measuring training effectiveness. The ultimate
objective of training evaluation is to find out if the training
program had met its goals and was of benefit to the organization.
The overall training effectiveness is measured by summing up the
evaluation of training effectiveness under various levels. 1.
Effectiveness of different levels of Kirkpatricks model: The four
levels of Kirkpatricks model provide valuable information to the
organization in evaluating training effectiveness. However, the
organizations consider only a few levels as most effective in
measuring the effectiveness of trainings. Inferences: The IT
organizations surveyed have implemented Kirkpatricks model of
training evaluation at different levels. Some organizations are yet
to move to level three while some are experimenting on
implementation of level four of Kirkpatricks model. The following
table shows the effectiveness of the four levels of Kirkpatricks
model in evaluating training success as perceived by the training
managers:
40
Table 1
EFFECTIVENESS OF REACTIONS IN MEASURING TRAINING SUCCESS
ORGANIZATION Cisco Systems Aztecsoft Huawei Technologies Symphony
Services GMTC EFFECTIVENESS Effective Very effective Effective Very
effective Not so effective Source: Primary data
Table 2
EFFECTIVENESS OF LEARNING IN MEASURING TRAINING SUCCESS
ORGANIZATION Cisco Systems Aztecsoft Huawei Technologies Symphony
Services GMTC EFFECTIVENESS Effective Very effective Effective Not
applicable Very effective Source: Primary data
Table 3
EFFECTIVENESS OF BEHAVIOR IN MEASURING TRAINING SUCCESS
41
ORGANIZATION Cisco Systems Aztecsoft Huawei Technologies
Symphony Services GMTC
EFFECTIVENESS Very Effective Effective Effective* Not applicable
Effective Source: Primary data
* - If implemented on a full-fledged basis
Table 4
EFFECTIVENESS OF RESULTS IN MEASURING TRAINING SUCCESS
ORGANIZATION Cisco Systems Aztecsoft Huawei Technologies Symphony
Services GMTC EFFECTIVENESS Very Effective* Effective* Effective*
Not applicable Very effective * Source: Primary data
* - If implemented on a full-fledged basis
As Symphony Services has not implemented levels two to four of
Kirkpatricks model, the Assistant Training Manager opined that the
effectiveness of levels two to four can be assessed only after
implementation.
2. Measures taken if training objectives are not met: The first
step in designing a training program is to define the training
objectives. The other aspects of the training design are decided
based on the training objectives. If the training objectives are
not met at the end of the training, all the resources utilized for
the42
training might go waste. Hence, the organizations should ensure
that the objectives of the training are achieved. Inferences: It is
inferred from the study that when the training objectives are not
met, the organizations conduct a post training analysis to identify
the cause of the problem. Refresher training course is the common
solution to the problem. In case of critical trainings, the
organizations repeat the training if training goals are not
achieved. Changes in training design and hiring of external experts
are other measures taken in such circumstances.
43
FINDINGS AND SUGGESTIONS
44
FINDINGS AND SUGGESTIONSThe study on measurement of
effectiveness of training programs in Information Technology
industry has revealed a lot of information on pre-requisites of
training and training evaluation. The responses of various IT
organizations represent the training practices of the Information
Technology industry.
FindingsThe following are the important findings of the
study:1.
In IT industry, training needs analysis is conducted annually.
Performance appraisal is the important source of training needs
analysis. The participants for trainings are usually nominated by
the managers. The objective or purpose of the training acts as an
important training design consideration.
2.
3.
4.
5.
Non-availability of internal expert compels IT organizations to
hire external experts for a few trainings.
6.
To ensure similarity in the delivery of training across
different branches, organizations in the IT industry make use of
the same training material.
7.
At the end of all trainings, feedback is sought from
participants on the course content, trainer and facilities.
45
8. The learning through training is measured for all technical
trainings. 9. Feedback from managers is obtained after a certain
period of time to measure the behavioral changes or transfer of
learning to training. 10. If the training objectives are found to
have not met, the cause of the problem is identified and necessary
steps are taken to rectify it.
Suggestions1.
Training needs analysis can be performed on a periodic basis so
as to address critical training needs at the earliest. This would
help the organizations gain an edge over competitors by responding
to the environmental changes such as introduction of new
technology, at the right time.
2.
Apart from questionnaires, focus group sessions can be conducted
by the organizations to measure reactions to training.
3.
To measure the effectiveness of trainings in increasing
knowledge and skills, a continuous assessment can be conducted
during the training after each session apart from a pre and post
training analysis.
4.
Learning through behavioral and soft skills trainings can be
measured using psychometric tests and personality tests. A pre and
post assessment might give an accurate picture of training
effectiveness.
5.
In order to overcome subjectivity in assessing trainees on the
job behavior, a 360 degree feedback can be obtained from the
participants, supervisors, peers and subordinates.
46
6.
To measure training return on investment for behavioral
trainings, increased efficiency demonstrated in solving problems
and decision making can be considered.
ConclusionIn the current business scenario of cut throat
competition, the need of the hour is efficient utilization of
resources to achieve optimum results. As human capabilities can be
developed by trainings, organizations use human resources as a
competitive advantage. By evaluating training effectiveness and
ensuring achievement of training objectives, there is no doubt that
organizations can emerge successful in their efforts to develop
human resources.
47
REFERENCES
Reddy,Rathan.B, Effective Human Resource Training and
Development Strategy, Edition 2, Mumbai: Himalaya Publishing House,
2005 Lynton, R.P. and Pareek, U. Training for Development, 2
Vistaar Publication, 2002 Training and Development, The ICFAI
University, 2006 www.traininganddevelopment.naukrihub.com
www.msu.edu www.businessperform.com www.businessballs.com
www.cisco.com www.aztecsoft.com www.huawei.com
www.hinduonnet.comnd
Edition, New Delhi:
48
ANNEXURES
49
ANNEXURESSEMI STRUCTURED INTERVIEW QUESTIONS MEASUREMENT OF
EFFECTIVENESS OF TRAINING PROGRAMS IN IT INDUSTRY
NAME: DESIGNATION: ORGANIZATION:I. PRE TRAINING PHASE:
1. What is the frequency of assessing employee training
needs?2.
How do you assess training needs of employees? (source of
TNA)
3. On what basis are the participants of training programs
selected?4.
Is there a minimum and maximum ceiling on training programs (man
days) an employee should attend? If yes, what are they?
5. Please name the different parameters considered in designing
a training program. 6. What are the different kinds of training
programs conducted in your organization? 7. Can you name some of
the tools used for these training programs? 8. Under what
circumstances do you hire external trainers?50
9. In case of generic training, how do you ensure similarity in
training design across the different branches of the
organization?
II. MEASUREMENT OF REACTIONS:
10. Do
you obtain feedback from participants for all training programs?
If no, what are the training programs for which feedback is not
sought? 11.How do you collect feedback from participants of
training program? 12.What are the general aspects covered in the
feedback collected from participants?II. MEASUREMENT OF
LEARNING:
13.Which are the trainings for which learning gained through
training Programs are measured?14. When
is the learning through training measured?
15.How is the learning gained through training program
measured?
III. MEASUREMENT OF BEHAVIOURAL CHANGES:
16.Is the impact of training on the performance of the job
measured? a) If yes, how? 17.What are the training programs for
which the application or transfer of training on the job is
measured?
51
V. MEASUREMENT OF RESULTS:
18.Are the organizational benefits gained through training
measured? If yes, how? 19.Do you measure the return on investment
in training? a) If yes, how? b) If no, is there any scope for
measuring return on investment in training?
VI. OVERALL TRAINING EFFECTIVENESS:20. Rate
the different levels of evaluating training programs according
to its effectiveness VERY EFFECTIVE EFFECTIVE NOT EFFECTIVE
LEVEL Reaction Learning Behavior Results
21.If the training is found to have not met its objectives, what
are the corrective steps taken?
THANK YOU
52
ANALYSIS OF DATA
DIMENSIONS Pre training phase
Measurement of reactions Measurement of learning
THEMES Frequency of TNA Sources of TNA Basis of participants
selection Mandatory training mandays Training design considerations
Types of trainings Training methods/tools Circumstances leading to
hiring of external trainers Method of ensuring similarity in
generic training
Method of feedback collection Aspects covered in feedback form
Trainings for which learning is measured Time of measuring learning
through training Method of measuring learning through training
Measurement of impact of training on performance
Measurement of behavior
Measurement of results
Trainings for which transfer of training is measured Measurement
of organizational benefits through training Measurement of ROI and
its scope in53
the future Overall training effectiveness Effectiveness of
different levels of Kirkpatricks model Measures taken if training
objectives are not met
54
