Upload
others
View
11
Download
1
Embed Size (px)
Citation preview
Development of Competency-Based
Web Learning Material and Effect
Evaluation of Self-Directed Learning
Aptitudes on Learning Achievements
Chi-Cheng Chang*National Taipei University of Technology, Taiwan ROC
This study aims to develop and evaluate competency-based web learning material (CBWLM) for
the college practicum Microprocessor Laboratory. After using the CBWLM for 8 weeks, this study
investigates CBWL’s learning effects and self-directed learning aptitudes (SDLAs) as well as
exploring the influence of SDLA on learning effects based on the sample of 38 students. The results
of this study indicate that over half of the students achieve the mastery level after using CBWLM.
SDLAs of the mid-CBWL and post-CBWL do not influence learning effects.
Research Background
Competency-based learning (CBL), a non-linear, systematic, and self-learning
process, allows learners to individually study each unit of the learning material
depending on their own competency. Furthermore, it allows learners to perform
repeated learning, remedial learning until they master each unit’s knowledge and skill
in order to achieve required competency standards. CBL possesses the nature of
systematic process, mastery learning, emphasizing individual differences and self-
directed learning. Such a learning method stressing competency standards is
specifically suitable for technical learning and training. Web learning is an inevitable
trend in the future. However, it has a weakness—there are limitations to the
presentation and learning methods for technical subjects. If we apply the concept of
CBL to web learning, both will complement each other and improve web technical
learning effects. As indicated by Lin and Liao (2000), the concept of competency-
based learning material (CBLM) can enhance the effect of web learning materials. To
sum up the argument mentioned above, it’s quite clear that competency-based web
learning (CBWL) or competency-based web learning material (CBWLM) ought to
*Institute of Technological and Vocational Education, National Taipei University of Technology,No. 1, Section 3, Chung-Hsiao E. Road, Taipei, Taiwan 106. Email: [email protected]
Interactive Learning Environments
Vol. 14, No. 3, December 2006, pp. 265 – 286
ISSN 1049-4820 (print)/ISSN 1744-5191 (online)/06/030265-22 � 2006 Taylor & Francis
DOI: 10.1080/10494820600954112
have a certain level of demand and feasibility. As there are few studies combining the
practices regarding CBL and web learning, CBWL or CBWLM becomes a topic
worthy of studying.
Westera and Sloep (1998) developed so-called ‘‘The Virtual Company’’, a
distributed distance learning environment built upon the notions of CBL and
constructivist learning, but the system seems not to bear supporting mechanisms of
CBL. Lin (2002) developed the web learning material of the competency-based
‘‘Digital Logic’’ course in the vocational senior high school by taking an ordinary web
learning system as a platform where CBLM was placed. However, there is no control
mechanism for CBL processes between each unit of the teaching material, which
seems not to conform to the nature of competency-based individual self-learning. Ho,
Hsu, and Tao (1999) referred to CBL in their web learning material, but the material
does not appear to match the concept and spirit of CBL. Most studies in the world
simply propose the idea of applying the competency-based concept in web learning
initially; no CBWLM’s substantial output is produced. Kirschner (2000), for
instance, discussed what instructional designers (ID) need to do when designing
electronic CBL environments. Woelk (2002) delivered a conception of a compe-
tency-based e-learning just-in-time system; Ritzen and Kosters (2002) also proposed
an idea of a competency-based web-based portfolio system. Kirschner (2000) further
argued that no full-fledged ID models are yet available to provide clear prescriptions
for how to design electronic CBL environments. Therefore, design and development
of electronic CBL environments may become highly cost-ineffective. For this reason,
there are some discussible issues: What on earth should a CBWLM consist of? And,
what are its learning effects? Theses study results can provide references for the
design and implementation of CBWLM, and will help enhance the effects of CBWL.
Elshout-Mohr, Oostdam, Snoek, and Dietze (2000) stated, CBL provides learners
with opportunities to regulate and manage their own learning. Wielenga (2000)
argued, competency-oriented learning should offer students an environment in which
they are given the responsibility to monitor their own learning processes. These
arguments imply self-directed learning is the characteristic of CBL. Hanna, Dudka,
and Runlee (2000) argued that students are required to possess self-directed learning
characteristics for successful web learning. This implies that web learning accentuates
active and self-directed learning. Richards, Dooley, and Lindner (2004) conceived
that web learning courses draw on the ability of learners to be self-directed, therefore,
web courses should focus on the self-directed nature of learners. Learners of
e-learning must experience self-directed learning (Ghaoui & Janvier, 2004; Ryan,
Scott, Freean, & Patel, 2000). Conrad and Donaldson (2004) argued that a learner
on the web must establish comfort with a higher level of self-direction than in a
traditional classroom. They further asserted that the learner would quit the online
course in frustration if this comfort level was not reached.
Self-directed learning aptitude (SDLA), derived from Guglielmino (1977), is a
learning behaviour allowing students to continue learning on their own initiative. The
SDLA consists of six factor aspects: Effective Learning, Fondness for Learning,
Learning Motivation, Active Learning, Independent Learning, and Creative
266 C.-C. Chang
Learning. Because web learning is becoming more prevalent in colleges, an issue thus
arises. What is the SDLA generated by college students through CBL on the web?
Moreover, Hanna et al. (2000) argued that assignments, projects, tests, and activities
might help students consider the important self-direction elements of online learning.
It means that assignments and activities may be the incitements to self-directed
learning behaviours. Thus, the other issue arising is that whether or not SDLA
reflects on the outcomes of assignments, projects, or tests. Namely, does SDLA
influence any learning effect? Are learning effects different among students with
varied SDLA? They are all valuable issues for further discussion.
Research Purposes
Based on the referred research background and motivation, this research aims to
develop a CBWLM for the practicum Microprocessor Laboratory of the Electronic
Department in college as well as to evaluate CBWL’s learning effects and SDLAs,
and to probe the influence of SDLA on learning effects with an empirical study.
Detailed research objectives associated with their corresponding research questions
and statistical methods are shown in Table 1.
The learning effect referred to in this study constitutes two aspects, the self-
perceived learning effect and the outcome of an achievement assessment, derived
Table 1. Research objectives, research questions and statistical method
Research objectives Research questions Statistical method
1. To evaluate learning
effects of CBWLM.
1.1. What are the self-perceived
learning effects after students
have used CBWLM?
Means
1.2. What are the outcomes of
achievement assessment after
students have used CBWLM?
2. To evaluate SDLAs for
different stages of CBWL.
2.1. What is the SDLA of students
before CBWL?
Means
2.2. What is the SDLA of students
during CBWL?
2.3. What is the SDLA of students
after CBWL?
3. To explore influences of
SDLA on learning effects
for CBWL.
3.1. Is there any statistically
significant difference in learning
effects among the students with
various SDLAs during CBWL?
One-way MANOVA
(dependent variables are
self-perceived learning
effect and outcome of
3.2. Is there any statistically
significant difference in learning
effects among the students with
various SDLAs after CBWL?
achievement assessment)
Competency-Based Web Learning Material 267
from the first and second level based on Kirkpatrick’s (1996) training effect
evaluation (Noe, 1999). The self-perceived learning effect captures average scores of
students’ comments through the questionnaire of a Likert 5-point rating scale; the
score of achievement assessment includes the laboratory assessment of four CBWLM
units (representing 50%) plus paper-pencil test (representing the other 50%). SDLA
consists of various factor aspects such as Effective Learning, Fondness for Learning,
Learning Motivation, and Active Learning, etc.
Literature Review
Competency-Based Learning Materials
Competency-based learning material is based upon a competency profile derived
from the competency analysis. As Westera and Sloep (1998) have stated, a CBL
environment should be based on authentic profiles that cover a set of
competencies. Woelk (2002) mentioned a competency gap analysis, identifying
what competencies the workers lack to effectively carry out their jobs, that is, based
on what the workers need to know and what they already know. In respect of
knowledge, skill, and attitude, CBLM stipulates the learning objectives and
organizes the units of the learning material with the substantial behaviour
objectives. Each unit’s learning steps and competency requirements should be
described in detail, and behaviour objectives of each competency should be
specified as well. These behaviour objectives not only explain the measurable
learning outcomes but also represent the standards to be achieved (Weng, 1996).
According to the guide of competency-based training material delivered by the
Division of Employee Training, Department of Labor (2005), Taiwan, the
structure of CBLM is composed of cover pages, learning guidelines, introduction,
definitions, learning objectives, learning activities, and testing. According to Tseng
(1999), this study prepares and induces mandatory contents and goals of CBLM as
shown in Table 2.
Furthermore, according to Woelk and Lefrere (2002) learner profile and
competency ontology may be included in the CBL or performance-based learning
environment. Learner profile refers to a description of the skills and of an individual.
The learner profile for a worker or learner may include, for instance, preferences,
experiences, and assessment of the worker’s competencies. Competency ontology is a
description including the required competencies for specified job tasks and
relationships among these competencies (Woelk & Lefrere, 2002, pp. 92, 95). It is
primarily to capture the competencies that a worker or learner must possess to solve
problems in his or her working processes. Competency ontology described by Woelk
and Lefrere may be regarded as so-called competency profile included in the
CBWLM developed by this study. Woelk and Lefrere further indicated that the
competency ontology might provide reasoning about competencies. Generally
speaking, the learner profile must be able to make reference to competency ontology,
learning objects, and working processes of employees.
268 C.-C. Chang
Self-Directed Learning Aptitude
Self-directed learning means learning something proactively, independently, and
patiently; being responsible to learn; learning which is a challenge; a self-training ability;
high curiosity; intense impetus to learning; self-assurance; enabling a fundamental
learning skill; scheduling time for learning; and planning the integral learning and
enjoying learning toward an objective (Teng, 1995). Self-directed learning may help
adapt to changing environment and enhance creativity (Ramsey & Couch, 1994).
Self-directed learning readiness (SDLR) questionnaire, developed by Guglielmino
(1977), measures learner or worker readiness for self-directed learning. The SDLR
questionnaire, completed several times through the Delphi approach, has been
verified to possess sufficient validity and reliability. It is based on a Likert 5-point
scale covering 58 items that yield a total score for identifying self-directed readiness.
After Guglielmino’s SDLR questionnaire was presented, there were some scholars
applying it to measuring SDLR or self-directed learning aptitude (SDLA) for various
students, and there were some scholars implementing studies regarding the reliability
and validity of the SDLR questionnaire (Long, 1987; Reynolds, 1986).
Table 2. Contents of CBLM and their goals
Contents Goals
1. Code and title of
learning material
1. The code and topic are disclosed on the cover page of each unit of the
learning material.
2. Introduction 2. Specify highlights and importance of the unit of the learning material to
inspire learning interests of students.
3. Learning objectives 3. List the fundamental occupation capability that students should provide
after completing the unit in a behaviour objective approach.
4. Pretest 4. Test students in their achievement level before learning; students
reaching the level are allowed to discard this unit of the learning
material.
5. Learning tasks 5. Based on the behaviour objectives, test results prior to learning, and
depending on the capability degree of students, teacher assigns students
the selective tasks such as learning activities, learning contents, and
assignments, etc.
6. Instructional media 6. To improve learning effects, it’s recommended to use teaching tools or
educational media that are favourable for students learning.
7. Learning activities 7. The learning activity can be divided into two parts: knowledge and skill,
and, occasionally, some tests.
8. References 8. Students may refer to relevant data if they find problems or intend to
have an in-depth discussion or further learning contents during the
learning process.
9. Posttest 9. Allow students to undertake the test after completing the unit of the
learning material for evaluation of learning effects.
Competency-Based Web Learning Material 269
Eight factor aspects were included in the SDLR questionnaire: openness to
learning opportunities, self-concept as an effective learner, initiative and indepen-
dence in learning, informed acceptance of responsibility for one’s own learning, love
of learning, creativity, positive orientation to the future, ability to use basic study skill
and problem-solving skills (Bonham, 1989; Guglielmino, 1977; Hung, 1995). These
eight factor aspects underlying the SDLR ascertain learner readiness for self-directed
learning and have been identified and further applied in many later studies (McCune
& Guglielmino, 1992; Siaw, 2002).
Guglielmino’s SDLR questionnaire has been translated into several languages around
the world (Guglielmino & Guglielmino, 1991), for instance, translated into Chinese by
Teng (1995) and revised by some experts in Taiwan. Consequently the number of the
questions was reduced from 58 to 55 and the original eight factor aspects were slashed
down to six aspects based on the consideration for cultural differences. The four factor
aspects of the Chinese-based SDLR, able to measure the degree of self-directed learning
aptitude are (1) Effective Learning, Fondness for Learning, (2) Learning Motivation,
Active Learning, (3) Independent Learning, and (4) Creative Learning.
Development of Competency-Based Web Learning Material
The CBWLM contents are based on the ‘‘competency profile’’ (required
competencies for a technician of microprocessor control, including 16 responsi-
bilities), announced by the Division of Employee Training, Department of Labor
(2005) in Taiwan, plus the proposal delivered by the instructor for the curriculum
progress. Finally, we decided to take the Task 1 (to understand basic architecture of
MCS-51 Microprocessor) and Task 4 (to familiarize with control and application of
MCS-51 Instruction) in Task 6 (Microprocessor Control) of the preceding 16
responsibilities as the CBWLM competency standard. Table 3 shows the competency
profile of a technician of microprocessor control (the sixth responsibility).
Based on the referred two competency standards (for Task 1 and Task 2), two
behaviour objectives in the CBWLM were formulated. They are, respectively, to
familiarize with attribute of output ports of MCS-51 and application of LED, and to
familiarize with attribute of output ports of MCS-51 and application of seven-section
displayer. Four learning units of the CBWLM were designed based on the two behaviour
objectives. The first behaviour covers three units of the CBWLM: ‘‘Practice of Single-
Lamp Right-Transferal’’, ‘‘Practice of Advertising Lamp’’, and ‘‘Practice of Thunder-
bolt Lamp’’. The second behaviour represents only one unit of the learning material,
‘‘Seven-Section Displayer of One Order’’. These four units of the learning material are
mainly designed to teach students the ‘‘Application of MCS-51 Output Port’’.
Contents and their descriptions of one unit of CBWLM based on the practicum
Microprocessor Laboratory are as follows:
. Material title: Subject of the unit.
. Competency profile: A set of competency standards which can be achieved after
completing the CBWLM.
270 C.-C. Chang
. Learning objectives: Competency levels which should be reached after completing
the unit.
. Learning guideline: Teaching students how to use CBWLM.
. Content description: Describing contents of the unit.
. Pretest: Learning the content of the unit only if passing the test.
. Test feedback: Listing wrongly answered questions and correct answers of the
pretest.
. Remedial learning (1): Students failing in the pretest need to take remedial
learning (1). After having learned the content of remedial learning (1), they
should attend the pretest again, and learn the laboratory knowledge only if they
pass the test.
. Laboratory knowledge: Laboratory knowledge associated with skill
operations.
. Laboratory demonstration: Teaching students skill operations.
. Posttest: Entering into the next unit if passing the test.
. Test feedback: Listing wrongly answered questions and correct answers of the
posttest.
. Remedial learning (2): Students failing in the posttest need to take remedial
learning (2). After having learned the content of remedial learning (2), they
Table 3. Competency profile of a technician of microprocessor control
Competencies of each task for the 6th responsibility (microprocessor control)
*Task 1. Tounderstand basic
architecture ofMCS-51Microprocessor
Task 2. To understand
basic architecture of
EM-78
Microprocessor
Task 3. To understand
basic architecture of
PIC Microprocessor
*Task 4. Tofamiliarize with
control andapplication ofMCS-51 instruction
Task 5. To be able to
manipulate basic I/O
of PIC
Task 6. To be able to
manipulate
interruption control
of PIC
Task 7. To be able to
manipulate
timepiece control of
PIC
Task 8. To be able to
manipulate
electrical circuit
control of PIC
Task 9. To be able to
manipulate
timepiece control of
MCS-51
Task 10. To be able to
manipulate electrical
circuit of MCS-51
Task 11. To be able to
manipulate MCS-51
LED
Task 12. To be able to
manipulate PIC
Task 13. To be able to
use PIC to produce
product
Task 14. To be able to
manipulate EM78
signal ship
Task 15. To be able to
manipulate basic I/O
control of EM78
Task 16. To be able to
manipulate
interruption control
of EM78
Task 17. To be able to
manipulate
timepiece of EM78
Task 18. To be able to
manipulate electrical
circuit of EM78
Task 19. To be able to
use EM78 to
produce product
Note: *Learning contents of Task 1 and Task 4 are included in the CBWLM developed by this
study.
Competency-Based Web Learning Material 271
should attend the posttest again, and enter into the next unit only if they pass the
test.
. Learning log: Recording the students’ online behaviours of using CBWLM.
Methods
Subjects
The subject of this study consists of 38 undergraduate students in the practicum
Microprocessor Laboratory, a sophomore course of the Department of Electronic
Engineering at a college located in northern Taiwan. This course, elective for one
semester, is a practicum that requires more learning by doing. After implementation
of the CBWLM for 8 weeks, we evaluate students’ learning effect through the
‘‘questionnaire for self-perceived learning effect’’ survey. We investigate the SDLA
before, during, and after the use of the CBWLM and further explore influences of
SDLA on learning effects.
After completing the course, these students are supposed to achieve some technical
standards. This is really correspondent to the rationale of CBL and thus becomes a
content of CBWL. Since the students are majoring in electronic or computer fields
they have a high level of computer use and access to the Internet.
Process
The study lasts 8 weeks. Before the CBWL course begins, the students complete the
pretest of the ‘‘SDLA questionnaire’’. After the CBWL course ends, the students
complete the posttest of the ‘‘SDLA questionnaire’’ and the ‘‘questionnaire for self-
perceived learning effects’’. The instructor first teaches students how to use CBWLM
before this study. The process of the CBWLM implemented in classroom teaching is
that the instructor, after part of the theory teaching, asks the students to access the
CBWLM website and practice writing in MCS-51 Microprocessor language;
the students, following the required procedure of CBWL, systematically learn the
CBWLM unit by unit. The procedure of CBWL followed by students is based on the
procedure of CBL proposed by Kuo (2000). In the procedure of CBWL, shown
in Figure 1, the students are to:
. Select a unit to self-learn and read learning objectives and laboratory contents.
. Take the pretest.
. Take remedial learning (1) if failing to pass the test.
. Learn laboratory knowledge and watch demonstration of the unit if passing the
pretest.
. Take the posttest after completing the learning of laboratory knowledge and
demonstration.
. Take remedial learning (2) until he or her succeeds if failing in the posttest.
. Enter into the next learning unit.
272 C.-C. Chang
The relationships among the research variables in correspondence to the research
questions proposed above are listed as follows and shown in Figure 2.
. Learning effects, a dependent variable, is composed of two aspects, namely, the
self-perceived learning effect and the outcome of the achievement assessment.
. Self-directed learning aptitude (SDLA), an independent variable, is divided into
three stages (before, during, and after CBWL). The SDLA consists of four factor
aspects: Effective Learning, Fondness for Learning, Learning Motivation, and
Active Learning.
Descriptive statistics is used to analyse the self-perceived learning effect of CBWL
and investigate SDLAs for three stages of CBWL. One-way MANOVA is used is to
determine if there is any statistically significant difference among learning effects for
various SDLAs during CBWL (and also after CBWL).
Figure 1. Procedure of competency-based web learning
Competency-Based Web Learning Material 273
Fig
ure
2.
Rel
atio
nsh
ips
amo
ng
rese
arch
vari
able
s
274 C.-C. Chang
Methods of Data Gathering
The study uses the questionnaire survey in the following stages to collect relevant
data.
Scoring of achievement assessment. According to Elshout-Mohr et al. (2000),
assessment of CBL cannot only be focused on the use of standard tests and fixed
criteria. There should also be an emphasis on the fairness of an assessment that allows
students to demonstrate relevant competencies at the end of a longer learning period.
Westera and Sloep (1998) claimed that students’ performances should be assessed
within practical, life-like contexts. Thus, grading of CBL achievement might integrate
tests, work, the learning process, and progress. In this study, the score of achievement
assessment is the summation of the laboratory assessment (representing 50%) and a
paper-pencil test (representing 50%); both are graded by the teacher. The laboratory
assessment consists of five checking aspects in terms of accuracy of laboratory
outcomes, amplitude of laboratory outcomes, creativity of laboratory outcomes,
correction of laboratory procedure, and attitude toward doing the laboratory. The
total score of the laboratory assessment is 100 points, that is, 20 points for each
checking aspect. The paper-pencil test includes 25 questions in fill-in-the-blank
format based on a scale of 100 points. The questions on the paper-pencil test cover
knowledge in four CBWLM units. Reliabilities of the laboratory assessment and
paper-pencil test are acceptable since they are all retrieved from assessment bases in
the teaching guide for instructors.
Evaluation of self-perceived learning effects. On finishing CBWL, the study uses the
questionnaire for self-perceived learning effects to detect perceptive learning effects
for students after using CBWLM. Key contents in the questionnaire are used to
assess whether the functions of CBWLM will enhance students’ ‘‘MCS-51
Microprocessor’’ knowledge and practical ability. The questionnaire contains 10
questions of a Likert 5-point rating scale and one opening question. The data
acquired is deemed a self-perceived learning effect.
Examination of SDLAs before, during, and after CBWL. The Chinese version of the
SDLA questionnaire, with some minor amendments by Teng (1995), is used in this
study to measure the extent to which individuals perceive themselves as processing
abilities and attitudes frequently associated with self-directed aptitudes in CBWL
experiences. A week prior to CBWL, during CBWL, and a week after CBWL we
implement the pretest, midtest and posttest of the SDLA questionnaire for the
students, so as to evaluate students’ SDLAs before, during, and after using CBWLM.
Validity and Reliability of Instruments
Validity and reliability analysis of SDLA questionnaire. Since the SDLA questionnaire,
modified by Teng (1995), is intended to be used for the measurement of web
Competency-Based Web Learning Material 275
learning students, the ulterior questions need to be moderately modified in
accordance with web learning nature. We modified the presentation style of the
questions in the SDLA questionnaire and sent them to a web-learning scholar, a CBL
expert, and an instructor for review. Since the purpose of the initial SDLA
questionnaire was more applicable to workers and is now used for college students in
this study, some terms used in the questions were modified. Since the Chinese-based
SDLA questionnaire, initially derived from Guglielmino (1977) and later modified by
Teng, has been revised by this study through expert review to achieve so-called expert
validity, it is really not necessary to redo the validity analysis by a pilot test.
After removing 17 questions in two factor aspects for inadequate Cronbach’s avalues (see the next section for details), 37 items in the SDLA questionnaire were
analysed by using a Likert 5-point rating scale that yielded a total score for self-
directed learning aptitude. As shown on the left-hand side in Table 4, five levels of
the SDLR constructed by Guglielmino and Guglielmino (1991) can be identified
according to their overall or mean scores. Based on the same rate of overall score or
mean scores, five levels and their ranges of scores for the SDLA in this study are
shown on the right-hand side in Table 4.
The study uses Cronbach’s a coefficient to test the reliability of the SDLA
questionnaire. Table 5 listed the internal reliability of the six factor aspects, where acoefficients lie between 0.6 and 0.9. It’s obvious that, besides Creative Learning and
Independent Learning, a values of the remaining factor aspects are all higher than
0.7. As a result, those two factor aspects are removed.
Validity and reliability analysis of learning effect questionnaire. We randomly select 30
students from the 38 students participating in the study to undertake the pilot test of
the questionnaire for self-perceived learning effects. We have received 28 effective
questionnaires in total. These students have properly checked the answers to the
questions. We have also modified ambiguous questions. Originally this study would
plan to use the method of factor analysis for building the validity. This method,
however, is discarded, because a small number of samples do not lead to genuine
validity (to build the validity via factor analysis usually requires over a hundred
samples).
Table 4. Five levels of the SDLA for students based on overall and mean scores
Guglielmino’s study This study
Scale by
overall scores
Scale by
mean scores
Term of
SDLA level
Scale by
overall scores
Scale by
mean scores
Term of
SDLA level
58 – 176 1 – 3.04 low 37 – 112 1 – 3.03 very low
177 – 201 3.05 – 3.47 below average 113 – 128 3.04 – 3.46 low
202 – 226 3.48 – 3.90 average 129 – 144 3.47 – 3.90 average
227 – 251 3.91 – 4.33 above average 145 – 160 3.91 – 4.32 high
252 – 290 4.34 – 5 high 161 – 185 4.33 – 5 very high
276 C.-C. Chang
Tab
le5
.R
elia
bilit
yo
fp
ilo
tte
sto
fS
DL
Aq
ues
tio
nn
aire
(n¼
28
)
Rel
iab
ilit
y
Eff
ecti
ve
lear
nin
g
(11
qu
esti
on
s)
Fo
nd
nes
sfo
r
lear
nin
g
(9q
ues
tio
ns)
Lea
rnin
g
mo
tiva
tio
n
(7q
ues
tio
ns)
Act
ive
lear
nin
g
(10
qu
esti
on
s)
Ind
epen
den
t
lear
nin
g
(13
qu
esti
on
s)
Cre
ativ
e
lear
nin
g
(4q
ues
tio
ns)
Cro
nb
ach
’sa
0.9
10
.73
0.7
20
.89
0.6
70
.63
Competency-Based Web Learning Material 277
For the pilot test, we gain a high value of Cronbach’s a coefficient that is equal to
0.92. It implies a sufficient reliability for the learning effect questionnaire.
Results and Discussions
Perceptive Learning Effects
After finishing competency-based web learning, we use the questionnaire for self-
perceived learning effects to implement measurement for the students and receive 38
effective questionnaires, no void questionnaires. The results are shown in Table 6.
In Table 6, the overall students’ learning effect (3.55 points) is classified as a
competent level. This indicates that the students are mostly satisfied with CBWLM.
In terms of the points, each question scores over 3 points. Question 5, ‘‘benefits of
feedbacks of the pretest and posttest’’, scores the highest (3.71 points), and Question
8, ‘‘online discussion is helpful to resolve curriculum problems’’ scores the lowest
(3.18 points), which indicates the highest effect of the ‘‘feedback of test’’ while the
effect of online discussion requires enhancement.
Table 6. Self-perceived learning effects (n¼ 38)
Self-perceived learning effects Mean SD
1. Does the mechanism, ‘‘pretest’’ and ‘‘remedial learning (1)’’, enable you
to consolidate your prerequisite knowledge before learning the CBWLM?
3.66 0.67
2. Does the mechanism, ‘‘posttest’’ and ‘‘remedial learning (2)’’, enable you
to master the laboratory knowledge of the learning material?
3.55 0.76
3. Does the visualized simulation of ‘‘Laboratory Knowledge’’ allow you to
have clear understanding of the executive outcomes of MCS-51
Microprocessor?
3.68 0.66
4. Does the streaming media material of ‘‘Laboratory Demonstration’’ allow
you to have more understanding of the practical product of MCS-51
Microprocessor?
3.66 0.71
5. Does the feedback from the pretest and posttest allow you to have clear
understanding of laboratory knowledge?
3.71 0.98
6. Is the procedure of CBWL [Pretest ! Remedial learning (1) ! Posttest
! Remedial learning (2)] helpful to you in learning MCS-51
Microprocessor?
3.66 0.75
7. Generally, are contents of CBWLM able to enhance your knowledge and
capability of MCS-51 Microprocessor?
3.66 0.67
8. Is online discussion forum with classmates helpful to you while you are
encountering problems in learning MCS-51 Microprocessor?
3.18 0.87
9. Are website-linking resources helpful to you in completing the
MCS-51 Microprocessor project?
3.3 0.87
10. Are online-handout resources helpful to you in completing the
MCS-51 Microprocessor project?
3.42 0.86
Overall 3.55 0.54
Note: The scoring is based on a 5-point rating scale with a total of 50 points.
278 C.-C. Chang
Scores of Achievement Assessment
After totalling the scores of the laboratory assessment (50%) and paper-pencil
test (50%), we obtain the scores of achievement assessment. In Figure 3, it is
clear that all of the students’ scores of achievement pass 60 points. If we take 80
points (100 points based) as the mastery learning standard, over a half of the
students (20) achieve certain competency standards (over 80 points), but there
are less than half of the students (18) failing to reach mastery standards (below
80 points). Finally, the students achieve mastery standards (a mean of 81.1
points).
Self-Directed Learning Aptitude
At a time before, during (a month later), and a week after CBWL, the study
implements measurements for the students by using the SDLA questionnaire and
receives 38 questionnaires without any void questionnaires.
Pretest of SDLA questionnaire. In Table 7, before the CBWL intervention, the overall
students’ SDLA (mean¼ 3.50 points) is classified as an average level. Factor aspect of
Fondness for Learning, classified as an average level, scores the highest (3.75 points),
and Effective Learning, classified as a low level, scores the lowest (3.27 points). As
regards the deviation between each factor aspect and its mean value, Fondness for
Learning has the largest deviation (standard deviation¼ 0.75) while Effective
Learning has the smallest one (standard deviation¼ 0.52).
Figure 3. Scores of achievement assessment (n¼ 38, M¼ 81.1, SD¼ 7.05)
Competency-Based Web Learning Material 279
Midtest of SDLA questionnaire. In Table 8, during the CBWL intervention, the overall
students’ SDLA (mean¼ 3.53) is classified as an average level. Factor aspect of
Fondness for Learning, classified as an average level, scores the highest (mean¼ 3.86
points) while Effective Learning, classified as a low level, scores the lowest
(mean¼ 3.27 points). As regards the deviation between each factor aspect and its
mean, Fondness for Learning has the largest deviation (standard deviation¼ 0.75)
while Effective Learning has the smallest one (standard deviation¼ 0.63).
Posttest of SDLA questionnaire. In Table 9, after the CBWL intervention, the overall
students’ SDLA (mean¼ 3.47 points) is classified as an average level. Factor aspect of
Fondness for Learning, classified as an average level, scores the highest (mean¼ 3.57
points) while Active Learning, classified as a low level, scores the lowest (mean¼ 3.27
points). As regards the deviation between each factor aspect and its mean, Fondness
for Learning has the largest deviation (standard deviation¼ 0.65) while Active
Learning has the smallest one (standard deviation¼ 0.52).
Concerning the SDLAs measured before, during, and after CBWL, the midtest of
SDLA has the highest result, the pretest of SDLA follows, and the posttest of SDLA
Table 7. Pretest of SDLA questionnaire (n¼ 38)
Factor aspects Minimum Maximum Mean SD Level of SDLA
Fondness for Learning (9 questions) 21 45 3.75 0.75 Average
Effective Learning (11 questions) 24 48 3.27 0.52 Low
Learning Motivation (7 questions) 17 34 3.62 0.69 Average
Active Learning (10 questions) 23 45 3.43 0.53 Low
Overall (37 questions) 85 160 3.50 0.54 Average
Note: The scoring is based on a 5-point rating scale with a total of 185 points.
The qualitative interpretation of the SDLA levels is based on the mean scores (m): 1�m� 3.03
(very low), 3.04�m� 3.46 (low), 3.47�m� 3.90 (average), 3.91�m� 4.32 (high), and
4.33�m� (very high).
Table 8. Midtest of SDLA questionnaire (n¼ 38)
Factor aspects Minimum Maximum Mean SD Level of SDLA
Fondness for Learning (9 questions) 23 45 3.86 0.75 Average
Effective Learning (11 questions) 23 53 3.27 0.63 Low
Learning Motivation (7 questions) 17 35 3.51 0.70 Average
Active Learning (10 questions) 20 49 3.49 0.69 Average
Overall (37 questions) 83 179 3.53 0.61 Average
Note: The scoring is based on a 5-point rating scale with a total of 185 points.The qualitative interpretation of the SDLA levels is based on the mean scores (m): 1�m� 3.03
(very low), 3.04�m� 3.46 (low), 3.47�m� 3.90 (average), 3.91�m� 4.32 (high), and
4.33�m�(very high).
280 C.-C. Chang
generates the lowest outcome by small differences. If comparing by level norm, there
is no changing level among the three tests of SDLA.
Influences of SDLAs of Mid-CBWL and Post-CBWL on Learning Effects
Based on the quartile, we divide ‘‘SDLA of mid-CBWL’’ into ‘‘low’’ (first 25%),
‘‘middle’’ (50%, middle), and ‘‘high’’ group (last 25%), and perform one-way
MANOVA together with learning effects (self-perceived learning effect and outcome
of achievement assessment). From Table 10 it is clear that there is no significant
difference in learning effects among the students of the three groups of ‘‘SDLA
during CBWL’’, showing the students’ ‘‘SDLA of mid-CBWL’’ will not influence
their learning effects.
From Table 11 it is clear that there is no significant difference in learning effects
among the students of the three groups of ‘‘SDLA after CBWL’’, showing the
students’ ‘‘SDLA of post-CBWL’’ will not influence their learning effects.
Conclusions and Implications
After CBWL has been used in classroom teaching and after-class review for the past 8
weeks, each student’s score of the achievement assessment is higher than 60 points
(100 points based). If we take 80 points as the mastery level, over a half of the
students (20 students) obtain a certain competency standard, less than half of them
(18 students) fail to pass the mastery level. For this reason, CBWL is generally
capable of enabling the students to achieve the mastery learning level and may further
identify ‘‘CBL is mastery learning’’ (Kang, 1997; Yang, 1998).
Comparison among SDLAs of pre-CBWL, mid-CBWL, and post-CBWL, SDLA
of mid-CBWL scores the highest, SDLA of pre-CBWL follows, and SDLA of
post-CBWL scores the lowest by slight differences. As regards factor aspects of
SDLA, Fondness for Learning is the highest for the three stages of CBWL, Effective
Learning is the lowest for pre-CBWL and mid-CBWL, and Active Learning is the
Table 9. Posttest of SDLA questionnaire (n¼ 38)
Factor aspects Minimum Maximum Mean SD Level of SDLA
Fondness for Learning (9 questions) 21 45 3.57 0.65 Average
Effective Learning (11 questions) 22 50 3.48 0.63 Average
Learning Motivation (7 questions) 15 32 3.50 0.61 Average
Active Learning (10 questions) 23 44 3.40 0.52 Low
Overall (37 questions) 87 159 3.47 0.53 Average
Note: The scoring is based on a 5-point rating scale with a total of 185 points.The qualitative interpretation of the SDLA levels is based on the mean scores (m): 1�m� 3.03
(very low), 3.04�m� 3.46 (low), 3.47�m� 3.90 (average), 3.91�m� 4.32 (high), and
4.33�m� (very high).
Competency-Based Web Learning Material 281
Tab
le1
0.
On
e-w
ayM
AN
OV
Ao
nle
arn
ing
effe
cts
for
vari
ou
sgro
up
so
f‘‘
SD
LA
of
mid
-CB
WL
’’(n¼
38
)
So
urc
eso
f
vari
atio
nD
egre
e
Wilk’s
�va
lue
(sig
nifi
can
tle
vel)
Lea
rnin
g
effe
cts
Deg
ree
Su
mo
f
squ
are
(SS
)
Mea
no
f
SS
(MS
)F
valu
e
Sig
nifi
can
t
leve
l
Bet
wee
ngro
up
s(S
DL
A)
10
.95
(0.4
1)
Sel
f-p
erce
ived
lear
nin
gef
fect
22
3.8
61
1.9
30
.38
0.6
9
Ach
ieve
men
tas
sess
men
t2
2.6
11
.30
50
.03
0.9
8
Wit
hin
gro
up
s(r
esid
ual
)3
5S
elf-
per
ceiv
edle
arn
ing
effe
ct3
51
09
1.6
53
1.1
9
Ach
ieve
men
tas
sess
men
t3
51
83
4.2
85
2.4
1
To
tal
37
Sel
f-p
erce
ived
lear
nin
gef
fect
37
11
15
.51
Ach
ieve
men
tas
sess
men
t3
71
83
6.8
9
282 C.-C. Chang
Tab
le1
1.
On
e-w
ayM
AN
OV
Ao
nle
arn
ing
effe
cts
for
vari
ou
sgro
up
so
f‘‘
SD
LA
of
po
st-C
BW
L’’
(n¼
38
)
So
urc
eso
fva
riat
ion
Deg
ree
Wilk’s
�va
lue
(sig
nifi
can
tle
vel)
Lea
rnin
gef
fect
sD
egre
e
Su
mo
f
squ
are
(SS
)
Mea
no
f
SS
(MS
)F
valu
e
Sig
nifi
can
t
leve
l
Bet
wee
ngro
up
s(S
DL
A)
10
.93
(0.3
8)
Sel
f-p
erce
ived
lear
nin
gef
fect
26
3.0
71
1.9
31
.05
0.3
6
Ach
ieve
men
tas
sess
men
t2
26
.83
1.3
05
0.2
60
.77
Wit
hin
gro
up
s(r
esid
ual
)3
5S
elf-
per
ceiv
edle
arn
ing
effe
ct3
51
05
2.4
43
0.0
7
Ach
ieve
men
tas
sess
men
t3
51
81
0.0
75
1.7
1
To
tal
37
Sel
f-p
erce
ived
lear
nin
gef
fect
37
11
15
.51
Ach
ieve
men
tas
sess
men
t3
71
83
6.8
9
Competency-Based Web Learning Material 283
lowest for post-CBWL. After using further inferential statistics, we find that CBWL
will enhance the factor, Fondness for Learning, in SDLA.
As indicated in the study result of Chung (2001), the students’ learning effects were
correlative with SDLA after web learning. But we find, in this study, that there is no
significant difference in learning effects among the students of various groups of
SDLA during CBWL, likewise having the same result for ‘‘after CBWL’’. These
imply that SDLAs of the mid-CBWL and post-CBWL do not influence learning
effects. The discrepancy between this study result and that of Chung needs a further
examination.
Subject to the instructor’s limitations, this study is targeted at a class with
few students, instead of two classes (or groups), for the contrastive experimental
study. It is recommended to perform the comparative study based on experimental
and controlled groups in the future, and to eliminate some variables that may
affect the study potentially, for a better understanding of the difference between
learning effects of CBWL and traditional CBL, CBWL and traditional classroom
learning.
References
Bonham, L. A. (1989). Self-directed learning: Emerging theory & practice. In H. Long (Ed.),
Self-directed orientation toward learning: A learning style? (pp. 55 – 68). Norman, OK:
Oklahoma Research Center for Continuing Professional and Higher Education, University
of Oklahoma.
Chung, Y. (2001). Influences of self-directed learning aptitude on asynchronous distance learning.
Unpublished master’s thesis, National Kaoshiung Normal University, Taiwan.
Conrad, R., & Donaldson, J. (2004). Engaging the online learner: Activities and resources for creative
instruction. San Francisco: Jossey-Bass.
Division of Employee and Training. (2005). Competency-based training. Retrieved November 30,
2005, from http://cbt.evta.gov.tw/
Elshout-Mohr, M., Oostdam, R., Snoek, M., & Dietze, A. (2000). Assessment in a competency-
oriented dynamic curriculum. Retrieved November 30, 2005, from http://www.efa.nl/
publicaties/publiexplo/english/index.html
Ghaoui, C., & Janvier, W. (2004). Smart proflexlearn: An intuitive approach to virtual learning
environment. In C. Ghaoui (Ed.), E-education applications: Human factors and innovative
approach (pp. 66 – 83). Hershey, PA: Information Science Publishing.
Guglielmino, L. (1977). Development of self-directed learning readiness scale. Dissertation Abstracts
International, 38, 6467A.
Guglielmino, L., & Guglielmino, P. (1991). Learning preference assessment facilitator guide. King of
Prussia, PA: Organizational Design and Development, Inc.
Hanna, D. E., Dudka, M. G., & Runlee, S. C. (2000). 147 practical tips for teaching online groups:
Essentials of web-based education. Madison, WI: Atwood.
Ho, C., Hsu, P., & Tao, C. (1999). Distance teaching for vocational distance and Internet training
material. Employee and Training, 17(4), 11 – 23.
Hung, S. (1995). Relationships between self-directed learning aptitude and learning achievement.
Unpublished master’s thesis, National Taiwan Normal University, Taipei, Taiwan.
Kang, T. (1997). Theory and curriculum development of competency-based vocational training.
Employee and Training, 15(6), 3 – 9.
284 C.-C. Chang
Kirkpatrick, D. (1996). Evaluation. In R. Craig (Ed.), The ASTD training and development handbook.
New York: McGraw-Hill.
Kirschner, P. A. (2000). Design strategies for competency based learning. Heerlen, The Netherlands:
Open University of The Netherlands.
Kuo, C. (2000). Procedure and management of CBT. Journal of Human Resources Development, 4,
28 – 33.
Lin, C., & Liao, M. (2000). Competency-based training of distance instruction. In Proceedings
of Conference on Application of Internet (pp. 392 – 399). Tainan, Taiwan: Cheng-Kung
University.
Lin, F. (2002). Planning and construction of competency-based instructional website: A case of ‘‘Digital
Logic’’ course in vocational technical high school. Unpublished master’s thesis, National Chang-
Hua University of Education, Taiwan.
Long, H. (1987). Item analysis of Guglielmino’s self-directed learning readiness scale. International
Journal of Lifelong Education, 6, 331 – 336.
McCune, S., & Guglielmino, L. (1992, January). Factor analytic study of the self-directed learning
readiness scale. Paper presented at the Sixth International Symposium on Adult Self-Directed
Learning, West Palm Beach, FL.
Noe, R. (1999). Employee training and development. New York: McGraw-Hill.
Ramsey, V., & Couch, P. (1994). Beyond self-directed learning: A partnership model of teaching
and learning. Journal of Management Education, 18(2), 15 – 20.
Reynolds, M. (1986). The directedness and motivational orientations of adult part-time students at a
community college. Unpublished doctoral dissertation, Syracuse University, NY.
Richards, L., Dooley, K., & Lindner, J. (2004). Online course design principles. In C. Howard,
K. Schenk, & R. Discenza (Eds.), Distance learning and university effectiveness: Changing educational
paradigms for online learning (pp. 99 – 118). Hershey, PA: Information Science Publishing.
Ritzen, M., & Kosters, J. (2002). Combining different aims in a portfolio system: A web-based
portfolio and the various ways in which it can serve the student. In C. Montgomerie & J. Viteli
(Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommu-
nications (pp. 1552 – 1557). Norfolk, VA: AACE.
Ryan, S., Scott, B., Freean, H., & Patel, D. (2000). The virtual university: The internet and resource-
based learning. London: Kogan Page.
Siaw, S. C. (2002). Fostering self-directed learning readiness by way of PBL intervention in
business education. In Online Proceedings of 2nd Asia-Pacific Conference on Problem-
Based Learning, Singapore. Retrieved November 30, 2005, from http://www.tp.edu.sg/
pblconference/
Teng, Y. (1995). Adult instruction and self-directed learning. Taipei, Taiwan: Wu-Nan Publication.
Ting, T. (1996). Relationships between self-directed learning readiness, computer attitude and computer
achievement for students of open university. Unpublished master’s thesis, National Chung-Cheng
University, Taiwan.
Tseng, K. (1999). Development of instructional material. In W. Chiang (Ed.), Introduction of
vocational technical education. Taipei, Taiwan: NTNU Publication.
Weng, S. (1996). Development of vocational competency-based training material: A case of industrial
electronic curriculum. Unpublished doctoral dissertation, National Taiwan Normal University,
Taipei, Taiwan.
Westera, W., & Sloep, P. (1998). The virtual company: Toward a self-directed, competence-based
learning environment in distance education. Educational Technology, 38(1), 32 – 36.
Wielenga, D. (2000). Proving competency: Integrative assessment and web-based portfolio system
in a dynamic curriculum. Retrieved November 30, 2005, from http://www.efa.nl/publicaties/
ass-eng.html
Woelk, D. (2002). E-learning, semantic web services and competency ontologies. In P. Barker &
S. Rebelsky (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and
Telecommunications (pp. 2077 – 2078). Norfolk, VA: AACE.
Competency-Based Web Learning Material 285
Woelk, D., & Lefrere, P. (2002). Technology for performance-based lifelong learning. In
R. Kinshuk et al. (Eds.), Proceedings of International Conference on Computers in Education
(pp. 92 – 96). Los, Alamitos: IEEE Computer Society.
Yang, C. (1998). Theory and practice of vocational and technical education. Taipei, Taiwan: Fu-Hsing
Publication.
286 C.-C. Chang