Embed Size (px)
Citation preview
Title: Instructional Designer Competencies: A Validation Study Author(s) & contact: Darlene Russ-Eft Professor & Chair Adult Education & Higher Education Leadership College of Education Oregon State University 301K Furman Hall Corvallis, OR 97331 U.S.A. Email: [email protected] Tiffany A. Koszalka Professor Instructional Design, Development and Evaluation Syracuse University Syracuse, NY U.S.A. Email: [email protected] Catherine Sleezer Manager, Human Resource Baker Hughes, Inc. Email: [email protected] Breannon Babbel School of Public Policy Oregon State University Corvallis, OR Email: [email protected] Fernando Senior Leaning Strategist Universidad Andrés Bello
Santiago, Chile Email: [email protected]
Stream: Vocational education, training and workplace learning [Stream leader: Dr. Ronan Carbery, Limerick University, Ireland] Submission type: Fully refereed paper Submission contact: [email protected]
Instructional Designer Competencies: A Validation Study
In 1986, the International Board of Standards for Training, Performance and
Instruction (ibstpi®) presented the first set of validated instructional design
competencies (See Appendix A in Richey, Fields, & Foxon [2001]). In 2001, ibstpi
published its second, updated set of validated instructional design competencies,
reflecting practices in the early 21st century (Richey et al, 2001). This paper provides
details on the latest set of internationally validated instructional designer competencies,
which reflect today’s practices and the current explosion of technologies entering the
designers’ and educators’ communities of practices.
The presentation begins by briefly summarizing changes in design practice over
the last decade. Then, the ibstpi competency development model is examined and the
methods for revising and validating the 2012 set of competencies are reviewed. The
paper presents some of the key results of the validation effort and concludes with
implications for HRD research and practice.
Brief Review on Instructional Design
It was suggested that formal instructional design work began as a result of
military training demands during World War II (Dick & Reiser, 1989; Seels, 1989).
Gustafson and Branch (1997) credited the Barson model used at Michigan State
University between 1961 and 1965 as being one of the first models proposed to guide
the practice of instructional design. However, it was not until the 1970’s that the term
“instructional design” was commonly used (Dick & Carey, 1978). Since the 1980’s, the
preponderance of instructional design (ID) practice has taken place within the private
sector, primarily in business and industrial settings. This use of ID coincides with the
steady growth of employee training as an integral part of most organizations. In the
United States, for example, the training industry was a $62.5 billion endeavor in 1999
(Training Industry Report, 1999), and by 2009, the training industry boasted over $125
billion investment in employee training (Patel, 2010).
In the late 1990s instructional design was seen as a generic process for
analyzing human performance problems and determining appropriate solutions (Pieters,
1997). This orientation served as the foundation of the 2001 edition of IBSTPI design
competencies (Richey et al, 2001).
Today, instructional designers have moved from working as individuals to
working collaboratively in design teams and with stakeholders, both in person and
through distributed communication channels (Knox & Wilmott, 2008; Koszalka & Wu,
2010). In addition to the new work environment, instructional designers are now
required to possess knowledge of various learning and instructional theories,
technology competencies e.g., visual literacy, message design, screen design,
interactivity design (Richey, Klein, & Tracey, 2010), as well as business acumen and
more sophisticated evaluation skills (Russ-Eft, Bober, de la Teja, & Koszalka, 2008).
Finally, today’s instructional design field is no longer primarily an American endeavor.
Instructional designers are working and being educated worldwide. As organizations
expand beyond individual country boundaries, designers are addressing the issue of
preparing and adapting instructional materials for different cultures and of having to
offer materials in distributed platforms (Rothwell & Kazanas, 2008). This is done both by
internationalizing the materials and processes to make them more culture-free and by
localizing products and processes to make the more culturally dependent (Morrison,
2000).
Competencies and the ibstpi Competency Development Model
It is critical to select the appropriate methodology that will help to formulate
competencies that will inspire higher standards of professional practice and
organizational effectiveness. McClelland (1973) outlined an approach for the
identification of competencies that provided non-biased ways of predicting job
performance. McCelland’s competency approach was applied in the organizational
human resources functions of employee selection, career pathing, performance
appraisals, and development. Later, Spencer and Spencer (1993) portrayed
competency as with core or surface entities, with skills and knowledge being surface
variables that are easier to develop than core characteristic such as attitudes. McLagan
(1997) identified six different approaches to competency definition. She noted that
competencies have been viewed as job task, as results of work efforts, as outputs, as
knowledge, skills, and attitudes, as qualities that describe superior performers, and
finally as bundles or attributes.
ibstpi defines a competency as:
… a set of related knowledge, skill, or attitude that enable an individual to
effectively perform the activities of a given occupation or job function to
the standards expected in employment (Richey et al. 2001, p. 31).
This orientation combines two of the McLagan competency definition models – that of
job tasks and of an accumulation of knowledge, skills, and attitudes. The ibstpi
competencies are statements of behavior – not personality traits or beliefs, but they
often do reflect attitudes. ibstpi competencies are correlated with performance on a job
and are typically measured against commonly accepted standards.
The ibstpi Competency Development Model
The set of competencies for a defined occupation or organizational role, such as
the ibstpi Instructional Designer Competencies, can be represented in a competency
model. According to Marrelli (1998), a competency model refers to “the organization of
identified competencies into a conceptual framework that enables the people in an
organization to understand, talk about, and apply the competencies” (p.10). Another
definition specifies that a competency model “includes the critical competencies that
drive success for an occupation or a specific job, definitions of each competency, and a
list of behavioral indicators for each competency that describe how the competency is
demonstrated in the workplace.” (HR Guide to the Internet, n.d.). Thus, a competency
model gives structure to a collection of competencies by organizing the knowledge,
skills, and attitudes into a specific framework. Furthermore, it includes performance
indicators describing how each competency is demonstrated in a job context.
Figure 1 presents the ibstpi competency development model, as described by
Richey (2002) and Russ-Eft et all (2008). ibstpi uses a generic competency
development model that leverages input from input from research, theory, and practice,
and the input is synthesized using expert review. The final model consists of domains,
competencies, and performance statements.
___________________________________________________________________
______________________________________________________________________
Figure 1. IBSTPI Competency Model
This material is reproduced with permission of John Wiley & Sons, Inc.
© 2008 by John Wiley & Sons, Inc. All rights reserved.
Russ-Eft, D., Bober, M. J., de la Teja, I., Foxon, M. J., & Koszalka, T. A. (2008). Evaluator competencies: Standards for the practice of evaluation in organizations. San Francisco, CA: Jossey-Bass.
______________________________________________________________________
Figure 2 depicts the relationships of the domains, competencies, and
performance statements.
______________________________________________________________________
______________________________________________________________________
Figure 2. Structure of the ibstpi Competency Model for the Instructional Designer
This material is reproduced with permission of John Wiley & Sons, Inc.
© 2008 by John Wiley & Sons, Inc. All rights reserved.
Russ-Eft, D., Bober, M. J., de la Teja, I., Foxon, M. J., & Koszalka, T. A. (2008). Evaluator competencies: Standards for the practice of evaluation in organizations. San Francisco, CA: Jossey-Bass.
______________________________________________________________________
Domains are clusters of related competencies. Competencies are the core component
of the model. Finally, each competency is supported by a list of performance
statements that provide a fuller description of how the competency is demonstrated. As
an example, a domain in the set of ID competencies is Professional Foundations. One
of the competencies for this domain includes Communicate effectively in visual, oral,
and written form. Then, a specific performance statement within that competency is
Write and edit messages that are clear, concise, and grammatically correct, and a
second performance statement within that same competency is Deliver presentations
that effectively engage audiences and communicate clear messages.
Methods
The following paragraphs describe the methods used for the present study. The
first section describes the procedures. This is followed by a description of the
instrument used for data collection and then a description of the sample.
Procedures
The procedures included three major phases: (a) review of the foundational
research, (b) competency drafting, and (c) competency validation and rewriting.
Phase 1: Identification of Foundational Research. The foundation of the ibstpi
Instructional Designer Competency model was based on the work of Richey (1986) and
Richey et al (2001). This was supplemented by an extensive review of current
programs, courses, and training modules in instructional design provided by universities
and professional associations around the world. A literature review of current work in
instructional design was also undertaken. Finally, a pilot study seeking information from
ID practitioners worldwide who were actively working in the field (e.g., business and
industry, higher education, K-12, etc) was conducted to identify both new behaviors,
values, ethical concerns, and future visions in instruction design field and what
remained the same (Reiser, 2006).
Phase 2: Competency drafting. The base list of competencies was analyzed
and debated by ibstpi directors who have particular expertise in different areas of
instructional design. This led to the development of additional competencies and
performance statements. Furthermore, a new classification level also was added based
on the emerging role of the ID function manager and increased importance of managing
ID functions and projects identified in both the pilot study and newer literature.
We requested a review of the proposed competencies from a small group of well-
known and published ID scholars and experts worldwide. Their feedback was
incorporated into the draft competencies and performance statements that would be
validated. The entire set was reviewed and approved by the ibstpi Board.
Phase 3: Competency validation and rewriting. The draft performance
statements created in Phase 2 were prepared in survey form. A validation instrument
was designed, developed, pilot tested, and administered worldwide via the Web. In
addition to the English version, translations and back-translations were undertaken and
survey versions were made available in Chinese, Japanese, Korean, and Spanish. The
instrument asked about the criticality of each competency and each performance
statement.
Following the validation survey, a second survey was implemented to identify the
level of classification for each statement as to essential, advanced, or managerial.
Specifically we requested 24 experts in the instructional design field from outside ibstpi
to undertake this classification. We provided the experts with the definition of novices,
advanced, and management and asked them to indicate a level for each competency
and performance statement as well as provide any further comments. A total of 10
experts completed both tasks: (a) an indication of essential, advanced, and
management for each competency and performance statement and (b) review and
comment on the updated competencies and performance statements.
Instrumentation
The instrument used in the validation study began with an introduction to the
study. It described the purpose and use of the data, and it included the informed
consent materials approved by the Institutional Review Board of Oregon State
University. Three separate sections of the instrument were then presented: (a) the
criticality statements, (b) the respondent background or demographic characteristics,
and (c) any additional comments.
Sections 1 and 3. The first section contained five question seeking background
information related to the respondent’s education and experience in instructional design.
Section 3, appearing after the criticality statements described below, included further
items on the organizational setting, audience for the designer, and the respondent’s
role.
Section 2: Criticality statements. The 22 competencies and 105 performance
statements were listed. Respondents were asked to assign an importance rating to
each competency and each performance statement in relation to their work role. The
rating used a 5-point Likert-type scale, where 5 meant “a very high level of importance”
in the respondent’s work as an instructional designer and 1 meant “no importance.”
Competencies were distinguished from the performance statements with a shaded bar,
and the performance statements related to a specific competency were numbered
sequentially below the competency. Figure 3 shows the validation survey items for one
performance statement related to the competency, Communicate effectively in visual,
oral, and written form.
_____________________________________________________________________
PROFESSIONAL FOUNDATIONS: Use a scale of 1-5 to indicate how important this COMPETENCY
'Communicate effectively in visual, oral, and written form' and its ten PERFORMANCE
STATEMENTS are in relation to your work as an instructional designer.
1 No
importance 2
3 Moderate
importance 4
5 High
importance
COMMUNICATE EFFECTIVELY IN VISUAL,
ORAL, AND WRITTEN FORM.
a)Write and edit messages that are clear,
concise, and grammatically correct
b)Deliver presentations that effectively engage
audiences and communicate clear messages
Figure 3. An example of the items in the validation survey.
______________________________________________________________________
Section 4: Additional comments. The survey contained two open-ended
questions seeking information on any competencies or performance statements that
should be added or reworded. A final question asked for any other comments or
suggestions.
Deployment. The survey was initially created in English in SurveyMonkey. The
English version was translated into Japanese, Chinese, Korean, and Spanish by
researchers fluent in those languages. The translations were back-translated by other
independent researchers. Any discrepancies were then resolved; and those separate
language surveys were created in Survey Monkey as well.
Sampling
The ibstpi Board used a snowball sampling method that sent out information to at
least 60 universities, ID program alumni groups, ID/HR/Training/Teacher/Higher
Education Professional Associations, Professional ID Consulting and Practice Groups,
military and government groups worldwide. Past ibstpi directors also responded and
forwarded the surveys to their professional associates who practice or teach ID.
Specifically, international associations in Australia, Canada, Korea, Japan, Malaysia,
Thailand, several African Nations, several European nations including The Netherlands,
UK, Germany, France, Spain, and Middle Eastern associations and universities were
also sent the surveys. Because the sample was not selected on a random or
systematic basis, the profile data cannot be assumed to be truly representative of
practitioners for whom these standards have been developed. Nevertheless, diversity
of language, locations, and experience appears among the respondents.
Results
A total of 1097 respondents entered the survey. Of these 989 responded to the
survey questions. Of these 771 (or 78%) responded to the English survey, 100 (or
10%) to the Japanese survey, 55 (or 6%) to the Chinese survey, 48 (or 5%) to the
Korean survey, and 15 (or 2%) to the Spanish survey. Some of the demographic
characteristics of the separated language participants appear in Table 1. Table 2
provides the combined demographics
Table 1 Profile of Instructional Designers by Language
English (79%) Japanese (10%) Chinese (6%) Korean (4%) Spanish (1%)
Respondents Respondents Respondents Respondents Respondents
Characteristic N % N % N % N % N %
Gender
Female 316 62% 16 30% 32 73% 9 56% 5 71%
Male 196 38% 38 70% 12 27% 7 44% 2 29%
Primary Region of Work
U.S./Canada 635 81% 3 3% 1 2% 1 2% 1 7%
Asia, Australia & New Zealand 71 9% 92 94% 52 96% 44 94% 0 0%
Europe 48 6% 0 0% 0 0% 1 2% 2 13%
Other 33 4% 3 3% 1 2% 1 2% 12 80%
Portion of Job with ID Focus
20% or less 178 24% 35 37% 11 20% 10 23% 0 0%
21-40% 139 19% 19 20% 20 37% 6 14% 2 14%
41-60% 156 21% 19 20% 16 30% 12 27% 3 21%
61-80% 140 19% 13 14% 5 9% 12 27% 5 36%
81-100% 135 18% 9 9% 2 4% 4 9% 4 29%
Type of Organization (multiple selections allowed)
Business/Industry 222 28% 32 42% 6 11% 8 33% 1 8%
Health Care 51 6% 5 7% 0 0% 0 0% 0 0%
Education 289 37% 24 32% 44 79% 13 54% 4 33%
K-12 Education Institution 50 (17%) 1 (4%) 24 (55%) 2 (15%) 0 (0%)
Higher Education Institution 239 (83%) 23 (96%) 20 (45%) 11 (85%) 4 (100%)
Govt./Military 101 13% 2 3% 2 4% 1 4% 3 25%
Other 127 16% 13 17% 4 7% 2 8% 4 33%
Highest Level of Education
Secondary/High school or equivalent 3 0% 6 6% 2 4% 0 0% 0 0%
Undergraduate degree or equivalent 74 10% 41 41% 31 56% 4 8% 4 29%
Master's degree or equivalent 424 56% 37 37% 19 35% 27 56% 4 29%
Doctoral 232 30% 13 13% 3 5% 15 31% 4 29%
Other 28 4% 3 3% 0 0% 2 4% 2 14%
Degree in ID
Yes 581 77% 35 35% 46 85% 41 87% 9 64%
No 177 23% 64 65% 8 15% 6 13% 5 36%
Level of ID Expertise
Novice 58 11% 14 26% 15 34% 2 12% 1 13%
Intermediate 198 38% 28 52% 23 52% 7 41% 5 63%
Expert 264 51% 12 22% 6 14% 8 47% 2 25%
Type of Instruction
Traditional (face-to-face) 55 11% 8 15% 16 36% 2 13% 0 0%
Self-Study 8 2% 1 2% 3 7% 0 0% 0 0%
e-Learning 134 26% 13 24% 7 16% 5 31% 2 29%
Combination of the Above 321 62% 32 59% 18 41% 9 56% 6 86%
Primary Audience (multiple selections allowed)
Executive 70 8% 4 5% 0 0% 1 4% 2 15%
Middle Managers 181 22% 19 22% 1 2% 2 8% 3 23%
Clerical Staff 21 3% 7 8% 0 0% 2 8% 0 0%
Front Line Production/Service Employees 160 19% 8 9% 0 0% 0 0% 2 15%
K-12 Students 54 6% 2 2% 30 60% 6 25% 0 0%
Higher Education Students 213 25% 20 23% 14 28% 9 38% 4 31%
General Public 40 5% 16 18% 2 4% 4 17% 1 8%
Other 101 12% 11 13% 3 6% 0 0% 1 8%
Role
Employee/Internal Consultant 381 74% 41 77% 35 83% 12 75% 6 86%
External Consultant 134 26% 12 23% 7 17% 4 25% 1 14%
Average Years of ID Work Experience
11.95 Yrs (n = 751) 7.07 Yrs (n = 91) 6.73 Yrs (n = 53) 6.52 Yrs (n=39) 6.21 Yrs (n = 14)
Table 2 Profile of the Combined Instructional Designer Respondents
Respondents
Characteristic N %
Gender (n = 633)
Female 378 60%
Male 255 40%
Primary Region of Work (n = 1001)
U.S./Canada 641 64%
Asia, Australia & New Zealand 259 26%
Europe 51 5%
Other 50 5%
Portion of Job with ID Focus (n = 955)
20% or less 234 25%
21-40% 186 19%
41-60% 206 22%
61-80% 175 18%
81-100% 154 16%
Type of Organization (multiple selections allowed)
Business/Industry 269 28%
Health Care 56 6%
Education 374 39%
K-12 Education Institution - 77 (21%)
Higher Education Institution - 297 (79%)
Govt./Military 109 11%
Other 150 16%
Highest Level of Education (n = 978)
Secondary/High school or equivalent 11 1%
Undergraduate degree or equivalent 154 16%
Master's degree or equivalent 511 52%
Doctoral 267 27%
Other 35 4%
Degree in ID (n = 972)
Yes 712 73%
No 260 27%
Level of ID Expertise (n = 643)
Novice 90 14%
Intermediate 261 41%
Expert 292 45%
Type of Instruction (n = 640)
Traditional (face-to-face) 81 13%
Self-Study 12 2%
e-Learning 161 25%
Combination of the Above 386 60%
Primary Audience (multiple selections allowed)
Executive 77 8%
Middle Managers 206 20%
Clerical Staff 30 3%
Front Line Production/Service Employees 170 17%
K-12 Students 92 9%
Higher Education Students 260 26%
General Public 63 6%
Other 116 11%
Role (n = 633)
Employee/Internal Consultant 475 75%
External Consultant 158 25%
Average Years of ID Work Experience (n = 956)
10.82 Years for 956 respondents
Some demographic differences did emerge. For example, larger percentages of the
Japanese-language respondents, in particular, worked in business and industry. Over
85% of the English-language and Korean-language respondents reported holding a
master’s or doctoral degree. Over 50% of the English-language respondents claimed
status as an “Expert,” and furthermore, the average years of work in instructional design
for these respondents was 11.95 years, while the other language respondents reported
6 to 7 years of experience. Over 55% of all the language groups, except for the
Chinese respondents, reported working in a combination of face-to-face, self-study, and
e-learning modes.
In reviewing the combined, overall demographics we found that 64% of the
respondents came from the United States or Canada, with 26% from Asia, 5% from
Europe, and 5% from other regions. Primary employers appeared to be education
(39%) and business and industry (28%). As for educational level, 52% reported having
a master’s degree or equivalent and 27% reported holding a doctoral degree. It is worth
noting that 73% of the respondents held a degree specifically in instructional design,
and 45% considered themselves to be “expert” with the average number of years of
experience being greater than 10 years. The respondents indicated a wide range of
primary audiences for their work. These respondents worked primarily as employees or
internal consultants (75%).
Domain Ratings
In the following section the results of the ratings of criticality and importance will
be presented. Twenty four instructional design experts from outside ibstpi were asked
to undertake this classification. We will begin with an overview of the ratings of the
domains and then turn to the specific competencies and performance statements.
Overall, the respondents supported the Instructional Designer Competencies at a
high level as indicated by high criticality ratings to most of the items. The scale ranged
from 1 indicating “no importance to 5 indicating “high importance.”
Table 3 presents the ratings for the five competency domains, showing the
number and percentage of competencies within each domain receiving the various
average ratings of 4.5 to 5.0, 4.0 to 4.49, or 3.5 to 3.99. It should be noted that none of
the competencies received a rating of lower than 3.5.
Table 3 Criticality Ratings of Competencies Within Domains
Competency Domain
Criticality Rating Range Total Across Competency
Domains 4.5 - 5.0
Very High- 4.0 - 4.49
High + 3.5 - 3.99
High -
N % N % N % N %
Professional Foundations 0 0% 5 100% 0 0% 5 100%
Planning & Analysis 0 0% 4 100% 0 0% 4 100%
Design & Development 1 14% 4 57% 2 29% 7 100%
Evaluation & Implementation 0 0% 2 67% 1 33% 3 100%
Management 0 0% 2 67% 1 33% 3 100%
Total Across Competencies 1 5% 17 77% 4 18% 22 100%
Table 4 shows the ratings for the performance statements included within the five
domains. Of the 105 performance statements, only two were rated at 3.49 or below –
one performance statement among those in the Design and Development domain and
one in the Management domain. The domains of Professional Foundations, Planning &
Analysis, and Design & Development received the largest percentages of highly rated
performance statements.
Table 5 displays the distribution of competencies according to levels of expertise,
namely “essential,” “advanced,” and “managerial” by the five domains. Table 6 shows a
similar distribution but presents the performance statements. Both tables indicate that
Planning & Analysis, Design & Development, and Professional Foundations include
large numbers of “essential” competencies and performance statements
Table 4 Criticality Ratings of Performance Statements Within Domains
Competency Domain
Criticality Rating Range Total Across
Competency Domains 4.5 - 5.0 Very
High- 4.0 - 4.49 High
+ 3.5 - 3.99 High
- 3.49 & Below Moderate +
N % N % N % N % N %
Professional Foundations 7 25% 14 50% 7 25% 0 0% 28 100%
Planning & Analysis 3 15% 17 85% 0 0% 0 0% 20 100%
Design & Development 5 19% 15 58% 5 19% 1 4% 26 100%
Evaluation & Implementation 0 0% 6 43% 8 57% 0 0% 14 100%
Management 0 0% 10 59% 6 35% 1 6% 17 100%
Total Across Competencies 15 14% 62 59% 26 25% 2 2% 105 100%
Table 5
An Overview of Competencies by Domains and Level of Expertise
Level of Expertise
Mean Competency
Criticality Rank
Frequency by Competency Domain
Professional Foundations
Planning & Analysis
Design & Development
Evaluation & Implementation
Management Total by Level of Expertise
N % N % N % N % N % N %
Essential 4.22 3 60% 3 75% 5 71% 1 33% 0 0% 12 55%
Advanced 4.07 2 40% 1 25% 2 29% 2 67% 1 33% 8 36%
Managerial 3.87 0 0% 0 0% 0 0% 0 0% 2 67% 2 9%
TOTAL 4.13 5 100% 4 100% 7 100% 3 100% 3 100% 22 100%
Table 6
An Overview of Performance Statements by Domain and Level of Expertise
Level of Expertise
Frequency by Competency Domain Total by Level of
Expertise Professional Foundations
Planning & Analysis
Design & Development
Evaluation & Implementation
Management
N % N % N % N % N % N %
Essential 13 46% 10 50% 17 65% 4 29% 0 0% 44 42%
Advanced 15 54% 10 50% 9 35% 6 43% 4 24% 44 42%
Managerial 0 0% 0 0% 0 0% 4 29% 13 76% 17 16%
TOTAL 28 100% 20 100% 26 100% 14 100% 17 100% 105 100%
Competency Ratings and Rankings
Table 7 provides the mean criticality ratings and the related variance for each
competency. The variances across the domains appeared to be similar, except for
some of the “advanced” and “managerial” competencies that showed somewhat larger
variances. The table also displays the ranking of the competencies based on the mean
criticality rating.
The three top ranked competencies were found within the Design & Development
domain: Design learning assessment (ranked #1, with a rating of 4.51 and considered
“advanced”), Design instructional interventions (ranked #2, with a rating of 4.38 and
considered “essential”), and Organize instructional programs and/or products to be
designed, developed, and evaluated (ranked #3, with a rating of 4.34 and considered
“essential”). These three competencies encompass the major skills and tasks needed
by instructional designers. It should be noted that, although the competency of Design
learning assessment is considered advanced, some of the performance statements
within the competency are considered essential (such as identifying outcomes to be
measured and aligning measures to outcomes) and are highly ranked.
The next ranked competency was within the Professional Foundations domain:
Communicate effectively in written, oral, and visual form (ranked #4, with a rating of
4.33 and considered “essential”). This communication competency was ranked #1 in
the previous study of instructional designers (Richey et al., 2001), in the study of
training managers (Foxon, Richey, Roberts, & Spannaus, 2003), and in the study of
evaluators (Russ-Eft et al., 2008). This high ranking reflects the fact that these
professionals must depend upon clear and effective communications to accomplish their
work. Indeed, adding further
Table 7 Criticality Ratings of the ibstpi Instructional Designer Competencies
Competency
N (Average over
Performance Statements)
Mean Criticality Rating (1-5; 5 high)
S.D. Criticality
Rank Level of
Expertise
PROFESSIONAL FOUNDATIONS
1. Communicate effectively in visual, oral, and written form. 776 4.33 0.83 4 Essential
2. Apply research and theory to the discipline of instructional design. 771 4.12 0.92 12 Advanced
3. Update and improve knowledge, skills, and attitudes pertaining to the instructional design process and related fields. 769 4.02 0.95 18 Essential
4. Apply data collection and analysis skills in instructional design projects. 771 4.09 0.93 14 Advanced
5. Identify and respond to ethical, legal, and political implications of design in the workplace. 769 4.22 0.94 8 Essential
PLANNING AND ANALYSIS
6. Conduct a needs assessment in order to recommend appropriate design solutions and strategies. 706 4.23 0.91 7 Advanced
7. Identify and describe target population and environmental characteristics. 705 4.25 0.87 6 Essential
8. Select and use analysis techniques for determining instructional content. 703 4.26 0.84 5 Essential
9. Analyze the characteristics of existing and emerging technologies and their potential use. 704 4.09 0.89 13 Essential
DESIGN AND DEVELOPMENT
10. Use an instructional design and development process appropriate for a given project. 654 4.14 0.95 11 Essential
11. Organize instructional programs and/or products to be designed, developed, and evaluated. 651 4.34 0.82 3 Essential
12. Design instructional interventions. 652 4.38 0.82 2 Essential
13. Plan non-instructional interventions. 646 3.61 1.21 22 Advanced
14. Select or modify existing instructional materials. 650 3.98 0.99 19 Essential
15. Develop instructional materials. 652 4.19 0.95 9 Essential
16. Design learning assessment. . 654 4.51 0.79 1 Advanced
EVALUATION & IMPLEMENTATION
17. Evaluate instructional and non-instructional interventions. 634 4.02 1.03 17 Advanced
18. Revise instructional and non-instructional solutions based on data. 632 4.08 0.98 15 Essential
19. Implement, disseminate, and diffuse instructional and non-instructional interventions. 628 3.82 1.11 20 Advanced
MANAGEMENT
20. Apply business skills to managing the instructional design function. 619 3.77 1.23 21 Managerial
21. Manage partnerships and collaborative relationships. 620 4.05 1.06 16 Managerial
22. Plan and manage instructional design projects. . 621 4.15 1.02 10 Advanced
support to the importance of communication skills, the highest ranked performance
statement was that focused on writing and editing clear messages.
The three lowest ranked competencies were either considered “advanced” or
“managerial”: Plan non-instructional interventions (ranked #22, with a rating of 3.61 and
considered “advanced”), Apply business skills to managing the instructional design
function (ranked #21, with a rating of 3.77 and considered “managerial”), and
Implement, disseminate, and diffuse instructional and non-instructional interventions
(ranked #20, with a rating of 3.82 and considered “advanced”). All three of these
represent competencies that would be performance by more experienced and higher
level instructional designers.
Conclusions
The evolution of the ID as a discipline and the increased world-wide involvement
of these professionals in wider numbers and contexts demanded a re-examination of
the previous set of ibstpi competencies. Thus, the purpose of the study was to
determine and then validate competencies needed by instructional designers today.
The development and validation effort revealed the importance of competencies critical
for instructional designers working in different settings around the world.
The results of the ibstpi work on these instructional designer competencies will
have implications for ID and HRD researchers and for strategy development and
organizational learning. These competencies can be used in future studies of
instructional designers and HRD professionals to determine which competencies have
the greatest impact on the success of an instructional design effort, specifically in terms
of individual, group, and organizational learning. In addition, researchers can examine
whether or not certain competencies are more critical in specific organizational settings
and in specific cultural contexts. This initial work revealed certain demographic
differences emerging among respondents from North American and those from other
regions. Certainly, it will be important to undertake further studies to determine whether
the demographic differences lead to differences in ratings of competencies and what is
the nature of ID practice in different regions of the world..
From the practitioner standpoint, these competencies are critical for determining
what further professional development might be needed. Both instructional design
consultants individually and associations focused on instructional designers can use
these competencies to determine the types of professional development workshops and
training that are needed. In addition, universities and other educational institutions
providing professional development for instructional designers can use these
competencies in order to enhance the quality of their offerings. Certainly, it will be
important for instructional designers working in various regions of the world to determine
which of these competencies are of greatest importance within a particular region,
bearing in mind the expectations being placed on their counterparts in other parts of the
world. In addition, they can be used by organizations interested in hiring a staff member
or a consultant to undertake instructional design work within that organization. Given
the importance of these competencies and given the specific requirements of the
organization, selection efforts can be tailored to those competencies most needed.
These competencies and performance statements can also be used for individuals to
undertake some self-assessment to determine areas in which they may need to focus
for future professional development.
In summary, the work of ibstpi provides a framework for use by instructional
designers and those interested in instructional design work. This internationally-
validated framework can help to enhance both research and practice in instructional
design throughout the world. Ultimately, as with any set of standards of professional
practice, this set of competencies aspires to raise the standards of excellence among
practitioners and the impact these professionals have in the organizations they serve.
References
Dick, W., & Carey, L. (1978). The systematic design of instruction. New York, NY:
Harper-Collins.
Dick, W., & Reiser, R. A. (1989). Planning effective instruction. Englewood Cliffs, NJ:
Prentice-Hall.
Foxon, M., Richey, R. C., Roberts, R. C., & Spannaus, T. W. (2003). Training manager
competencies: The standards (3rd ed.). Syracuse, NY: ERIC Clearinghouse on
Information & Technology.
Gustafson, K. L., & Branch, R. M. (1997). Revisioning models of instructional
development. Educational Technology Research and Development, 45(3), 73-
89. doi: 10.1007/BF02299731
HR Guide to the Internet, (n.d.) Integrating competencies into HR programs. Retrieved
from: http://www.hr-guide.com/data/G480.htm
Knox, I. & Wilmott, D. (2008) Virtual teams: worlds apart. Proceedings ascilite
Melbourne 2008. 500-504. Retrieved from
http://www.ascilite.org.au/conferences/melbourne08/procs/knox.pdf.
Koszalka, T., & Wu, Y. (2010). Instructional Design Issues in a Distributed Collaborative
Engineering Design (CED) Instructional Environment. Quarterly Review of
Distance Education, 11(2), 105-125.
Marrelli, A. F. (1998). An introduction to competency analysis and modeling.
Performance Improvement, 37(5), 8-17.
McClelland, D. A. (1973). Testing for competence rather than for intelligence. American
Psychologist, 28(1), 1-14.
McLagan, P. A. (1997). Competencies: The next generation. Training and
Development, 51(5), 40-47. ERIC Document Reproduction Service No. EJ 543
935.
Morrison, A. J. (2000). Developing a global leadership model. Human Resource
Management, 39(2 & 3), 117–131
Patel, L. (2010, November). 2010 State of the Industry continued dedication. T+D,
64(11), 48-53.
Pieters, J. M. (1997). Training for human resource development in industrial and
professional organizations. In S. Dijkstra, N. Seel, F. Schott, & R. D. Tennyson
(Eds.). Instructional design: International perspectives. Theory, research, and
models (Vol 2). Mahwah, NJ: Erbaum.
Reiser, R. A. (2006). Florida State University Instructional Systems Program alumni
survey results summary. Unpublished manuscript. Department of Educational
Psychology and Learning Systems, Florida State University, Tallahassee, FL.
Richey, R. C. (1986). The theoretical and conceptual bases of instructional design.
London, UK/New York, NY: Kogan Page/Nichols.
Richey, R. C., Fields, D. C., & Foxon, M. J. (2001). Instructional design competencies:
The standards (3rd ed.). Syracuse, NY: ERIC Clearinghouse on Information and
Technology.
Richey, R. C., Klein, J. D., & Tracey, M. W. (2010). The instructional design knowledge
base: Theory, research, and practice. New York, NY: Routledge.
Rothwell, W. J., & Kazanas, H. C. (2008). Mastering the instructional design process: A
systematic approach (4th ed.). New York, NY: Wiley.
Russ-Eft, D., Bober, M. J., de la Teja, I., Foxon, M. J., & Koszalka, T. A. (2008).
Evaluator competencies: Standards for the practice of evaluation in
organizations. San Francisco, CA: Jossey-Bass.
Seels, B. (1989, May). The instructional design movement in educational technology.
Educational Technology, 29(5), 11-15.
Spencer, L. M., & Spencer, S. M. (1993). Competence at work: Models for superior
performance. New York, NY: Wiley.
Training Magazine’s Industry Report 1999. (1999, October). Training, 37.
