The moderating effect of instructional conceptions on the effect of powerful learning environments

The moderating effect of instructional conceptionson the effect of powerful learning environments

Frederick Kwaku Sarfo Æ Jan Elen

Received: 10 January 2006 / Accepted: 14 March 2007 / Published online: 18 April 2007� Springer Science+Business Media B.V. 2007

Abstract This study aimed at experimentally investigating the moderating role of

instructional conceptions on the effectiveness of powerful learning environments (PLE)

designed in line with the four-component instructional design model (4C/ID-model). The

study also investigated the influence of learning in a 4C/ID PLE on students’ instructional

conceptions. To achieve its goal, a study with a one by one by two pre-test post-test quasi-

experimental design was done. Three functionally equivalent classes of students from three

similar (secondary technical) schools were randomly exposed to three different treatments.

The participants consisted of 129 (47, 41, 41) students. The treatments were one control

group with a regular method of teaching, and two experimental groups: a 4C/ID PLE with

ICT, and a 4C/ID PLE without ICT. The instructional conceptions questionnaire was

administered both in the pre-and the post-test to assess students’ instructional conceptions.

Pre- and post-tests contain retention and transfer items. Technical teachers were trained to

implement the interventions. In contrast to expectation, findings show no moderating

effects of students’ instructional conceptions on the learning environments. Finally, the

results indicate that students’ instructional conceptions positively change after imple-

mentation of the three interventions. The theoretical, research, and practical implications of

the results for the instructional design and technology community as well as educational

practice are discussed.

Keywords Instructional conceptions � Conceived functionality � Powerful learning

environments � 4C/ID-model � Technical expertise � Moderating variable

F. K. Sarfo (&)University of Education of Winneba, P.O. Box 1277, Kumasi Campus, Kumasi, Ghanae-mail: [email protected]

J. ElenCenter for Instructional Psychology and Technology, Katholieke Universiteit Leuven, Vasaliusstraat 2,3000 Leuven, Belgiume-mail: [email protected]

123

Instr Sci (2008) 36:137–153DOI 10.1007/s11251-007-9023-8

Introduction

Recently, powerful learning environments (PLE) have been acknowledged in instructional

psychology literature as effective for the acquisition of (technical) expertise (De Corte

2003). Powerful learning environments are learning environments, rooted in cognitive and

social constructivist learning theories (Collins et al. 1989), that aim at fostering the

productive use of knowledge and skills. Powerful learning environments promote active

and constructive learning processes in students. More specifically, in the literature of

instructional design and technology, PLEs designed in accordance with the specifications

of the four components instructional design model (4C/ID PLE) of van Merrienboer (1997)

have been argued to be an interesting approach for complex learning, in other words for

acquisition of integrated sets of knowledge and skills (De Corte 2003; Merrill 2002; van

Merrienboer 1997; van Merrienboer et al. 2002; van Merrienboer and Paas 2003). Typical

for a 4C/ID PLE is that it is task-centered (Merrill 2002, 2006) and promotes the

simultaneous acquisition of multiple sets of goals.

Integrating information and communication technology (ICT) in PLEs has been argued

to further enhance the development of expertise (Lehtinen 2003; Pieters et al. 2003,

Romiszowski 1997). Even though a group of researchers (e.g., Clark 1994, 2001; Russell

1999) argue that there is nothing inherent in ICT that can bring significant difference in

learning gains, advocates of ICT in teaching and learning (e.g., Seel and Winn 1997;

Kozma 1991, 1994; Kozma and McGhee 2003) assert that taking into consideration their

processing capabilities, ICT can deliver certain methods better than other media.

Although 4C/ID learning environments attract a lot of positive attention, recently,

questions have been raised about their ubiquitous positive effects. An important principle

of 4C/ID PLE relates to task-centered learning (Merrill 2002, 2006). The basic assumption

of task-centered learning is that learners actively construct the learning experience based

on their own experiences and conceptions. Learners’ conceptions in this respect, rather

than the intentions of the designer, determine how learners interact with the learning

environments (Konings et al. 2005; Lowyck et al. 2004). Paris and Winograd (1990)

argued that students’ judgments or conceptions about aspects of the learning situation are

forerunners of their actions. However, the moderating effects of instructional conceptions

on the impact of 4C/ID PLEs in the traditional classrooms have not yet been

experimentally studied. This quasi-experimental study examined the role of instructional

conceptions with respect to the effectiveness of 4C/ID PLE with and without ICT for

acquisition of integrated set of knowledge and skills in the traditional classrooms.

Definition of instructional conceptions

Teaching may influence students in different ways, depending on their pattern of abilities,

motivation, attitudes, and prior knowledge (Entwistle and Peterson 2004). The assertion

that learning is a cumulative process stresses the important role of prior knowledge in

learning. Prior knowledge has been proven to be one of the major variables influencing the

ability of learners to acquire new knowledge (Elen 1995; Glaser 1984). It is one of the most

important conditions for and/or impediments to learning (Elen 1995). Dochy (1994,

p. 4699), described the main features of prior knowledge as ‘‘the whole of a person’s actual

knowledge that: (a) is available before a certain learning task, (b) is structured in schemata,

(c) is declarative and procedural, (d) is partly explicit and partly tacit, and (e) is dynamic in

nature and stored in the knowledge base.’’

138 F. K. Sarfo, J. Elen

123

Prior knowledge is argued to be either related to a specific domain or to be metacog-

nitive in nature (Flavell 1979). Domain related prior knowledge refers to knowledge in

specific subject-matter domains and can be either general or topical. Metacognitive

knowledge refers to knowledge about learning (Flavell 1979). According to Elen and

Lowyck (1999), metacognitive knowledge may pertain to three closely related and partly

overlapping ‘fields’ of knowledge: (1) learning (conceptions about cognitive strategies,

control strategies, and motivational strategies) and the learner (conceptions about the self

with respect to learning, (2) tasks (conceptions about the learning task, elements, and

requirements), and (3) the environment (instructional or other) in which learning occurs

(conceptions about key elements, demands and affordances). Students interpret a learning

environment in accordance with their conceptions about learning, learning tasks, and

learning environments (Elen and Lowyck 1999).

Instructional metacognitive knowledge or instructional conceptions is regarded to be

one kind of metacognitive environmental knowledge (Elen and Lowyck 1999, 2000).

Lowyck et al. (2004) define instructional conceptions as all ideas, concepts and theories

that an individual learner holds about (components of) the learning environments.

Instructional conceptions in this context—as being indicated by Lowyck et al. (2004)—are

not conceptions of scientific phenomena to be influenced by instruction. Neither are

instructional conceptions identical to conceptions of learning (e.g., Saljo 1979, Kember

2001) as they directly focus on students’ conceptions hold about instruction itself (Lowyck

et al. 2004; Elen and Lowyck 1999). In light of the above descriptions of instructional

conceptions (Lowyck et al. 2004; Elen and Lowyck 1999), instructional conceptions can be

defined as students’ general ideas about specific instructional interventions and computers

as instructional media.

Instructional conceptions and the design of instructional interventions

Lowyck et al. (2004) indicate that specific teaching and classroom behaviors determine

learning results through the filter of students’ behaviors and cognitions, and hence, these

behaviors and cognitions influence the effectiveness of instruction. This is linked to

Rothkopf (1968) who found that students do not react to the objective or nominal

instructional stimulus as constructed by the designer or teacher but to the stimulus as

transformed by the students themselves. Similarly, Winne (1985) asserts that students use

instructional interventions according to the functions they attribute to these interventions.

Students’ attributed functions are based on their instructional conceptions (Lowyck et al.

2004). In addition, Konings et al. (2005) indicate that the designers and teachers do not

have a direct influence on student learning. In contrast, students’ and teachers’ perceptions

and conceptions of a learning environment do influence student learning and the quality of

the learning outcomes (Konings et al. 2005). Instructional interventions seem effective

only if learners are ‘‘calibrated’’ to the intentions of instructional designer, and make use

of these interventions (Winne and Marx 1982). In order to avoid a possible lack of cali-

bration, Lowyck and Elen (1994) argued in favor of more explicitly considering students’

ideas about specific instructional interventions, in particular when designing learning

environments. To proceed, the next section reveals current/available research findings on

instructional conceptions and instructional interventions, based on the literature on

instruction and learning.

The moderating effect of instructional conceptions 139

123

Research findings on instructional conceptions

A critical analysis of the literature on instruction and learning reveals various research

findings indicating that students have instructional conceptions. For instance, in Clarke’s

(1994) study, the results indicate that students like clearly structured practical applications,

well-structured materials, techniques that increase their interest and promote informal

learning, and techniques that promote consolidation and integration of knowledge.

Similarly, research findings by Kember (2001) revealed that novice students view

instruction as a didactic process of transmitting knowledge while experienced students

view teaching as the process of facilitating learning.

Furthermore, to investigate the content and structure of metacognitive instructional

knowledge, Elen and Lowyck (1999) used 219 students as subjects in a study. The main

instrument was a questionnaire with essay-type questions. The results indicated that (1) at

the center of instruction stands an instructional agent that assumes most responsibility for

instructional processes, (2) instructional settings can be described in terms of the context, the

availability and the nature of goals, information and support, and in terms of structural

elements of the instruction that make available both information and support, and (3) stu-

dents have different opinions about the relative usefulness of 3 different delivery systems.

These results (Elen and Lowyck 1999) partly support Elen and Lowyck (1999) studies,

Clarke (1994), Hativa and Birenbaum’s (2000) and Kember’s (2001) studies that students

like structured teaching/instruction and dislike interventions that stress on the learning

processes. However, with regard to learning (Elen and Lowyck 1999), students also ask for

opportunities to be active and constructive and to participate in discussions. Similarly, in

Kember’s study the experienced students view teaching as process of facilitating learning.Innovative learning environments may bring about changes in learners’ instructional

conceptions and in the long run alter students’ conceptions and approaches towards more

advanced conceptions in line with the prevailing educational approach at their institutions

(Lowyck et al. 2004). There are research findings (e.g., Clarebout and Elen 2001; Kember

2001) that support the assertion that students’ instructional conceptions may change as a

result of interacting with learning environments.

There is much evidence in the above literature to indicate that students have instruc-

tional conceptions about specific interventions (e.g., traditional methods of teaching, PLEs)

and instructional media (e.g., computer). These conceptions may affect students’ inter-

pretation of instructional environments and therefore moderate the effectiveness of

instructional interventions (Elen and Lowyck 1999) and also for the development of

technical expertise. Figure 1, the framework of this study, highlights on this. However, in

the literature, different approaches and different assessment instruments are used. For

instance Clarke (1994) used semi-structured and open-ended questionnaire, Kember (2001)

used semi-structured face-to-face interviews, and Elen and Lowyck used questionnaire

with easy-type questions. There is no consistent measurement instrument to assess

students’ instructional conceptions; and more importantly there is no explicit empirical

evidence to support the claim that students’ instructional conceptions moderate the

effectiveness of interventions.

Instructional conceptions as a moderating variable

Generally speaking, a moderating variable can be a qualitative or a quantitative variable

that affects the direction and/or strength of the relation between an independent or


123

predictor variable (e.g., 4C/ID PLE) and a dependent or criterion variable (e.g., develop-

ment of technical expertise) (Baron and Kenny 1986). A moderator (e.g., instructional

conceptions) as a qualitative variable refers to a variable characterized by categorical

measurement (e.g., sex, race, class, group). A moderator as a quantitative variable refers to

a variable characterized by a continuous measurement (e.g., levels of instructional con-

ceptions, levels of reward). In testing a moderating hypothesis (Baron and Kenny 1986),

statistical analysis must measure and test the differential effect of the independent variable

on the dependent variable as a function of the moderator. How to measure and test the

differential effects depend in part on the level of measurement of the independent variable

and the moderator variable. For instance, if both the moderator and the independent var-

iable are categorical variables, analysis of variance (ANOVA) is the appropriate statistical

test, and moderation is indicated by an interaction effect. A moderator–interaction effect

would be said to occur if the relation is substantially reduced instead of being reversed. In

this contribution, it is argued that instructional conceptions moderate the effect of inter-

ventions on the achievement of learning outcomes. This indicates that as a result of

students’ instructional conceptions, a 4C/ID PLE (with and without ICT) might not pro-

mote the development of technical expertise as it would be expected. More specifically,

with a 4C/ID PLE intervention, students whose instructional conceptions are in accordance

with the principles of 4C/ID PLE (4C/ID PLE compliant) would perform better than those

whose instructional conceptions are related to the principles of a regular method of

teaching (regular method compliant). Contrary to this, with regular method of teaching,

regular method compliant students would perform better than the 4C/ID PLE compliant

students. Figure 2 highlights this.

The purpose of the study

As indicated in various research studies (e.g., Elen and Lowyck 1999; Konings et al. 2005;

Lowyck et al. 2004), instructional conceptions, being regarded as a moderating variable,

are considered as an important variable in designing interventions for effective learning.

This implies that the concept ‘‘instructional conceptions’’ is well explained and under-

stood, and design decisions can be based on sufficient scientific evidence. Nonetheless, in

Fig. 1 The framework for the study


123

the instructional design and technology literature, there is little or no empirical evidence to

support the proposition that instructional conceptions moderate the effect of interventions.

In this contribution, an attempt is made to construct a reliable instrument to assess stu-

dents’ instructional conceptions. More importantly, based on the instrument, an attempt is

made to find experimental evidence to support the argument that instructional conceptions

moderate the effect of interventions on learning outcomes. In addition, given that

instructional conceptions are assumed to develop through the interaction with specific

learning environments, attention is paid to the development of instructional conceptions as

a result of experienced interventions. Therefore the main research hypotheses are:

• Instructional conceptions of students moderate the effects of 4C/ID PLE with and

without ICT for the development of technical expertise in secondary technical students.

• After the implementation of the innovation, the students’ instructional conceptions

positively change.

Experimental design

Considering the purpose and the context of the study, the experiment (see Table 1) had a

one by one by two pre-test post-test quasi-experimental design (Campbell and Stanley

1963; Krathwohl 1993). Three classes of students from three schools were randomly

exposed to three different treatments. The treatments were (1) a regular method of teaching

for the control group, and (2) a 4C/ID PLE with and without ICT for the experimental

groups. All treatments were designed and validated by expert instructional designers and a

subject matter expert (SME) during the pilot study. Both the control group and the

experimental group received a pre-test and a post-test. The pre-test was administered prior

to the delivery of the three treatments. The main purposes of the pre-test were to determine

Fig. 2 Instructional conceptions as a moderating variable

Table 1 Design of the study

Pre-test

Control group Experimental group 1 Experimental group 2

Treatment Regular method 4C/ID PLE without computers 4C/ID PLE with computer

Post-test


123

the ability level of the three groups in relation to the terminal objective; to establish

groups’ equivalence; and to measure students’ instructional conceptions. The post-test was

administered after the three groups had experienced the treatments. It was to assess the

students’ achievement of the complex technical skill; and also to measure students’

instructional conceptions.

Participants

The group of participants consisted of 129 students selected from six Secondary

(Technical) Schools (47, 41, and 41—mean age = 18.1, and SD = 1.3).

Research materials

The terminal objective was selected from a secondary technical building drawing syllabus

in Ghana. Research materials included (1) the instructional conceptions questionnaire, (2)

materials and teacher guidelines for the three treatments: (a) 4C/ID PLE with ICT, (b) 4C/

ID PLE without ICT, and (c) regular method of teaching; and 3) assessment tasks identical

for all three treatments.

The instructional conceptions questionnaire was designed based on the assumption that

the use of specific interventions depends on the functionalities ascribed to the intervention

by the student (Elen and Lowyck 1999; Lowyck et al. 2004; Winne 1985). Therefore the

questionnaire addresses the contribution of components of a learning environment, for

instance, media, methods, or strategies to learning in a specific context. Indeed, it is

assumed that conceptions may be context-specific. It might be, for example, that for

technical education, working on a realistic task may be considered to be highly functional,

while for university education working on a realistic task might be considered to be

dysfunctional (e.g., Elen et al. 1998). A context specific questionnaire, then, is designed by

(a) specifying the educational context, (b) identifying the major active instructional

ingredients, and (c) specifying the items. For this study, the context clearly is technical

education and major ingredients were (1) lectures, (2) computers, (3) examples, (4) inte-

grated project activity, and (5) gaming. Eight items (see Table 2) were constructed to

measure the conceived functionalities of each specific intervention. For each item a 6-point

Likert-type scale was constructed from 1 ‘‘totally disagree’’ to 6 ‘‘totally agree.’’

The instructional conceptions questionnaire was pre-tested on 10 secondary technical

students in Ghana; and 205 college students (84 first year teacher education students and

Table 2 Items for measuring specific instructional conceptions

‘Specific intervention’ encourages students to start to work.

‘Specific intervention’ helps students to better understand the question.

‘Specific intervention’ helps students to determine what and how to learn.

‘Specific intervention’ allows students to make exercises.

‘Specific intervention’ directs students’ attention to relevant aspects.

‘Specific intervention’ helps students to think critically.

‘Specific intervention’ makes student learning faster.

‘Specific intervention’ makes students learning easier.


123

121 third year industrial engineering students) in Belgium. The results presented evidence

that most questions were understood as intended. The questionnaire with only 8 items and

205 subjects was subjected to factor analysis. It yielded a 1 or 2 factor solution for each of

the different components. In view of a parsimonious solution, the Cronbach alpha for the 1

factor solution was checked, including all 8 items in one scale. This resulted in Cronbach

alphas between .73 and .88. Considering these alpha-values, it was concluded that each

scale consisting of 8 items is reliable and measuring one underlying latent variable, namely

specific instructional conceptions or more precisely conceived functionality.

Based on the outcomes of the pre-testing, the questionnaire was used in three different

settings: with university students in South Africa and Belgium and with secondary tech-

nical students in Ghana. The South African study used a larger group of participants and as

such it was considered to be the reference study. Three hundred and ten undergraduate

students from the Department of Education of a public South Africa University were used.

After factor analysis, 3 scales were constructed and Cronbach alphas were calculated in

order to assess internal consistencies. Each scale related to the conceived functionality of

one particular adjunct aid: examples, questions and figures. The three scales registered

Cronbach alphas between .88 and .92 revealing high internal consistencies.

The aim of the Ghana study was to check whether the scales constructed in the South

Africa study gave sufficiently high internal consistency values and also, more importantly,

to validate the questionnaire and use it for the present study. Eighty-eight students from

three secondary technical schools in Ghana participated in the study. Cronbach alphas

registered between .62 and .87 for each specific instructional conception scales (see

Table 3). Even though the alpha-values for computer and integrated project work were

sufficient, they seemed to indicate that students were not very clear about these 2 scales.

Therefore, based on these results and considering the aim of the present study, the

instructional conception scales were updated into students’ conceptions about the func-

tionalities of (1) lectures, (2) doing exercises with computer, (3) examples, (4) working on

realistic task, and (5) group work as specific interventions in the context of technical

education (building drawing); these were used for the main study. Refer to Table 4 for the

reliability coefficients for the main study. The alpha coefficients (.70–.87) for the scales

provide sufficient statistical validity for the instructional conceptions questionnaire.

As previously mentioned, the three treatments constituted in this study are (a) the

control group treatment (regular method of teaching), and (b) the experimental group

treatments (4C/ID PLE with ICT and 4C/ID PLE without ICT). All three treatments were

designed to support the learners to achieve the same terminal objective ‘‘designing a single

building plan based on the local conditions.’’ The regular method of teaching was specially

designed based on the classical principles of instructional design (e.g., Gagne 1985; Gagne

and Briggs 1979; Jonassen et al. 1989; Leshin et al. 1992) and directions from the syllabus.

The 4C/ID PLE treatments were designed based on the principles of 4C/ID-model (van

Table 3 Cronbach alphas for the instructional conceptions scales

2. Specific instructional conceptions

a. Lecture .74

b. Computer .62

c. Integrated project activity .68

d. Example .71

c. Gaming .87


123

Merrienboer 1997; van Merrienboer and Paas 2003). The contents for the three treatments

were also identical, and they were based on 13 topics selected from the syllabus. The

information on the topics was selected from the required textbook (Greeno 2002) and some

were provided by the SME. Each treatment consisted of four lessons, and the instructional

time for each lesson was 90 min (this is the normal time for 2 teaching periods in

secondary technical schools in Ghana). The treatments varied with regard to the following

elements of the instructional approach: (1) instructional strategies (teaching methods), (2)

support from the teacher, and (3) the use of instructional media.

The assessment tasks consisted of pre-and post-retention and -transfer items (Mayer

2002). The tests consisted of 26 pre-test assessment items (13 retention test items and 13

transfer test items) and 26 post-test assessment items (13 retention test items and 13

transfer test items). Four questions (2 retention and 2 transfer) were constructed based on

each of the 13 topics selected from the syllabus towards the achievement of the terminal

objective.

The retention test items were designed to assess if the learners had mastered the

recurrent constituent skills of the complex technical skills (technical expertise). The

retention test items were very similar to the procedural and part-task information provided

on the achievement of the terminal objective. The transfer test items assessed if the learners

had mastered the non-recurrent constituent skills of the complex technical skills (technical

expertise). They were very dissimilar to the practice tasks.

Procedure

Technical teachers were trained to master very well how to deliver a specific treatment as

intended to ensure treatment fidelity in the ecological (classroom) setting (Krathwohl

1993). They were apologetically instructed to teach according to how they were trained.

The main researcher monitored the implementation of the treatments. The treatments for

the study were randomly assigned to three schools. The schools did not know which

treatment they belonged to. Each treatment consisted of six sessions. Each session took

90 min. The lessons took place in the regular classroom of each group. During the first

session, the instructional conceptions questionnaire and the assessment task (pre-test) were

administered by the researcher. The administration of the instructional conception

questionnaire took about 40 min, and the administration of the assessment test also took

about 40 min.

Table 4 Reliability coefficients for the scales of the research instruments

Reliability coefficients

Research material Pre-test Post-test

A. Specific Instructional conception Scale

1. Lectures .70 .73

2. Exercises with computers .83 .87

3. Examples .74 .75

4. Group work .80 .80

5. Working on a realistic task .79 .81

B. Assessment tasks .67 .68


123

The teacher(s) (for the control treatment) presented the regular method of teaching

systematically as designed, and in accordance with the order of sequence of the specific

instructional objectives in the lessons. The learners received full support from the teacher

throughout the lesson. The teacher talked for approximately 70 min. He used to answer

most of the students questions during question time. Similar strategies were used to teach

lesson 2, 3, and 4 under the control treatment.

Teachers (for the experimental treatments) also presented the experimental treatments

systematically as designed. Unlike the regular method of teaching, teacher used about

40 min to support the learners and the learners used approximately 50 min to work on the

learning tasks. Similar strategies were used for all the lessons in the experimental treat-

ments. However, learners received additional information on part-task practice during the

performance of learning tasks 1 and 2 of the second lesson and learning task 1 of the fourth

lesson of the experimental treatments. In 4C/ID PLE without ICT group, flashcard,

worksheet and chalkboard were used in drill-and-practice to present the part-task-practice

of the learning tasks. In 4C/ID PLE with ICT group, ICT was used for drill-and-practice to

present the part-task practice of the learning tasks.

The researcher administered the post-test (the instructional conceptions questionnaire

and the assessment test) during the sixth session of each treatment. The administration of

the instructional conceptions questionnaire took about 40 min and the administration of the

assessment test also took approximately 40 min. The pre- and post-assessment tests were

submitted to a naive SME (who did not know which school belonged to which treatment)

for blind marking. The main SME re-marked the tests and differences were discussed with

the researcher.

Data analysis

In order to recheck the quality of the instruments and also to choose appropriate statistical

tests to analyze the data for the main study, reliability, correlation, descriptive and

frequency statistical tests were performed. First and foremost, reliability analyses were

performed on the specific instructional conceptions items and the assessment tasks. Table 4

shows the overview of the Cronbach alphas for the main study. The reliability coefficients

for the specific instructional conception scales were between .70 and .87. The reliability

coefficients for pre and post-test assessment tasks were .67 and .68, respectively.

Moreover, to determine the nature of (linear) relationships among the different (specific

instructional conceptions or conceived functionality) scales: (1) lectures, (2) exercising

with computers, (3) examples, (4) working on realistic tasks, and (5) group work, the

correlation matrix was analyzed. Significant correlations were retrieved (see Table 5). This

result suggests that very similar answers are given on the items of the different specific

instructional conceptions scales. An encompassing ‘‘conceived functionality scale’’ was

constructed with high internal consistency, Cronbach alphas of .90 and .93 for pre and

post-tests respectively. Because students gave similar answers on the items of the different

specific instructional conceptions scales, the encompassing ‘‘conceived functionality

scale’’ was used for the main analysis and discussion.

Furthermore, in order to employ analysis of variance (ANOVA) to analyze the data, the

conceived functionality scale was changed from interval to categorical. To achieve this, a

descriptive statistic was executed to determine the minimum and the maximum values

(Table 6). It was found that it would be statistically convenient to divide the entire scores

into three categories. A frequency statistic was performed at 33 and 66 percentiles


123

(Table 7). Based on this, the conceived functionality scale was recoded into three

categories: Low (those who score between 2.98 and 4.56), Moderate (those who scored

between 4.56 and 5), and High (those who scored above 5). Pearson correlation analysis

was used to check if the strength of the conceived functionality scale has not been

underestimated by categorizing it into three groups (Moore and McCabe 2006). Alpha was

set at .05 for these statistical tests.

Results

The dependent variables were (1) learning gains (post-test scores minus pre-test scores; this

was to accommodate the differences, even not significant, between the groups), and (2)

students’ instructional conceptions (conceived functionality) after having experienced the

interventions. The independent variables were the three treatment conditions and students’

conceived functionality before the experience of the interventions (the three treatments).

One hundred and twenty-nine subjects (47 students for 4C/ID PLE with ICT treatment, 41

students for 4C/ID PLE without ICT treatment, and 41 students with regular method of

teaching treatment condition) were used for the main analyses.

It was not practical and feasible to randomly assign individual students to the treatment

groups. However, the assumption was that as the students were selected from three similar

class groups (pursuing the same course—building drawing) from three schools they were

supposed to be functionally equivalent with respect to the achievement of the terminal

objective. An ANOVA on the pre-test revealed no differences between the three groups of

students. Table 8 shows the overview of the mean scores.

Table 5 Correlation matrix of instructional conceptions scales (N = 141)

Lecture Doing exercises withcomputer

Examples Groupwork

Working onrealistic task

Lecture 1 .23(**) .31(**) .34(**) .31(**)

Doing exercises withcomputer

1 .39(**) .44(**) .44(**)

Examples 1 .55(**) .54(**)

Group work 1 .79(**)

Working on realistic task 1

** Correlation is significant at the .01 level (2-tailed)

Table 6 Descriptive statistics on the conceived functionality scale (N = 129)

Minimum Maximum Mean SD

Conceived functionality scale 2.98 5.95 4.79 .52

Table 7 Frequency table on conceived functionality scale (N = 129)

Percentile Conceived functionality scale

33 4.76

66 5.00


123

Learning gains and the moderating effect of conceived functionality

Analysis of variance (ANOVA) of the learning gains revealed a main effect of the treat-

ment (F(2,120) = 18.58, P = .000, g2 = .24). The LSD multiple comparison test revealed

that:

(a) students in the 4C/ID PLE with ICT group attained more learning gains (M = 10.06)

than students in the control group (M = 5.44),

(b) students in the 4C/ID PLE without ICT group attained more learning gains (M = 8.84)

than students in the control group (M = 5.44) (see Table 9), but

(c) there is no significant difference between the two experimental conditions in terms of

their learning gains.

The 4C/ID PLE with ICT group and the 4C/ID PLE without ICT performed equally

better.

An ANOVA indicated no interaction effect of conceived functionality of the learning

environments and the treatment conditions on the learning gains among the three groups

(Fig. 3). In order not to underestimate the strength of the conceived functionality scale by

categorizing it into three, Pearson correlation was performed on the relationship between

the conceived functionality scale (on interval level) and the learning gains. The Pearson

correlation analysis (r = .118) yielded no statistical significant relationship between the

conceived functionality scale and the learning gains.

Pre-test and post-test (conceived functionality)

Moreover, repeated measurement analysis revealed a significant difference between pre-

test and post-test conceived functionality (F(1,127) = 9.933, P = .002, g2 = .07) but no

difference between the post-test conceived functionality. Table 10 shows the overview of

mean scores on the pre-test and the post-test of students’ conceived functionality.

Discussion and conclusion

This study investigated the acquisition of a coordinated and integrated set of knowledge

and skills as a function of students’ instructional conceptions and the designed interven-

tions (4C/ID PLE with and without ICT and the regular method of teaching); and the effect

of the designed interventions on students’ instructional conceptions. To achieve the goal of

the study the following research hypotheses were formulated:

• Instructional conceptions of students moderate the effects of PLEs with and without

ICT for the development of technical expertise.

Table 8 Overview of mean scores of the pre-test

Conditions N Mean SD

1. Regular method of teaching 42 6.68 3.16

2. 4C/ID PLE without ICT 45 5.61 3.38

3. 4C/ID PLE with ICT 51 6.15 2.98

Total 138 6.13 3.18


123

• After the implementation of the innovation, the students’ instructional conceptions will

change towards the principles/strategies used in the different treatments.

The study was carried out in real classroom settings and the results are discussed in turn.

The result of the study revealed no interaction effect of learners’ instructional con-

ceptions (conceived functionality) on any of the treatments for promoting the development

of technical expertise. The study suggests that students’ instructional conceptions do not

moderate the effect of 4C/ID PLEs for promoting the development of technical expertise.

This result indicates that (in Ghana) secondary technical students’ ideas about specific

aspects of instruction (e.g., lectures, computers, realistic tasks, examples, and group work)

do not influence their interpretation and use of the instructional interventions. This result is

consistent with the findings (Elen and Lowyck 1999) that students have specific

Table 9 Overview of mean scores of the learning gains as a function of the three treatment conditions andthe conceived functionality pre-test

Conceived functionality in three categories Condition of treatment N Mean SD

Low Regular method of teaching 19 4.82 2.98

4C/ID PLE without ICT 14 8.60 2.42

4C/ID PLE with ICT 8 10.56 4.96

Total 41 7.23 3.98

Moderate Regular method of teaching 10 7.15 3.79



Total 43 8.86 3.65

High Regular method of teaching 12 5.00 3.69



Total 45 8.47 3.78

Total Regular method of teaching 41 5.44 3.46



Total 129 8.20 3.83

Fig. 3 No moderating effect of conceived functionality on the three treatments


123

instructional conceptions but it contradicts the assertion of (e.g., Elen 1995; Winne 1985;

Konings et al. 2005) that instructional conceptions affect the impact of interventions on

learning outcomes. In line with the current literature on instructional conceptions (e.g.,

Elen 1995; Elen and Lowyck 1999; Lowyck et al. 2004; Konings et al. 2005) the finding

that instructional conceptions of secondary technical students do not moderate the effect of

interventions for the acquisition of technical expertise is surprising but new, and therefore

significant. The finding suggests that while students have explicit conceptions about the

functionality of the learning environments, these conceptions do not influence their cog-

nitive activities (e.g., schema construction and schema automation). While this finding is a

new and theoretically challenging finding, there are various methodological issues that

need to be addressed.

First, research on instructional conceptions (e.g., Elen 1995; Elen and Lowyck 1999;

Winne 1985) has mainly used higher education students. In this study, secondary technical

students were used. It might be argued that conceptions of secondary technical students

have only a limited impact because they are not yet sufficiently developed. In other words,

it might be argued that in order to have a moderating effect, instructional conceptions need

to be sufficiently sophisticated.

Second, students reacted positively and rather homogeneously to the different inter-

ventions. The poor diversity of the student population with respect to instructional con-

ceptions may be the reason why a moderating effect could not be established. This calls for

a replication study with a more heterogeneous student population.

Third, it is also important to note that in spite of the fact that the conceived functionality

scale had a high internal consistency and can be considered to be very reliable, the results

question its criterion validity. Clearly, students do not react to the interventions in

accordance with the functions they ascribed to them. It is suggested to initiate a specific

study using qualitative methods to further investigate this validity problem.

Although there might be reasons to treat the result related to conceived functionality

with caution, this result still points to a theoretical outcome that needs to be explained. The

direct effect of PLEs on the acquisition of complex technical skills override the assertion

that students interpret the learning environments (interventions) based on their instruc-

tional conceptions. This makes the work of instructional designers/teachers easier. Since

there is no similar finding in the literature at the moment, the result calls for additional

empirical studies to validate or challenge the finding.

Furthermore, the results of the study revealed that students’ conceived functionality

changed after the implementation of 4C/ID PLE with and without ICT and regular method

of teaching. Since the change is positive it is argued that there is evolutional growth of

Table 10 Overview of mean scores on pre-test and post-test of conceived functionality

Conceived functionality Condition N Mean SD

Pre-test Regular method of teaching 41 4.69 .58

4C/ID PLE without ICT 41 4.73 .53

4C/ID PLE with ICT 47 4.97 .41

Total 129 4.80 .52

Post-test Regular method of teaching 41 4.79 .51

4C/ID PLE without ICT 41 4.85 .57

4C/ID PLE with ICT 47 5.16 .36

Total 129 4.94 .51


123

students conceived functionality. This finding supports the theoretical understanding of the

effect of interventions on students’ instructional conceptions (e.g., Elen and Lowyck 1999;

Lowyck et al. 2004). In addition, the finding adds new insights to existing theoretical

findings. First, based on the present finding, it can be proposed that students like a par-

ticular teaching technique because the teacher has consistently used it. This is consistent

with the old theory by Zajonc (1968) and research evidence by Sarfo (1994) that the more a

person is exposed to an object or a person, the more he or she responds positively to that

object or a person. Second, based on the result, it can also be proposed that students like a

certain teaching method or medium even if the teacher does not use it. This might be the

case when the method or medium (e.g., ICT) is trendy. This implies that students’ con-

ceived functionality of a particular teaching method or medium not necessarily results from

interactions in the learning environments. To have deeper insight about the effect of

interventions on the development of students’ instructional conceptions further study is

recommended to investigate and validate the proposition that (1) students like a certain

teaching method or medium because it has been consistently used by the teacher, and (2)

students may like a certain teaching method or medium even if it has not being used by the

teacher but when it is within a specific context somehow appealing. The positive change of

students’ conceived functionality after the interventions could be attributed to testing effect

(Krathwohl 1993). Since the same questionnaire (instructional conceptions questionnaire)

was administered as pre-test and post-test students might be familiar with the questionnaire

during the post test. Being familiar with the questionnaire might have influenced the

students to answer it more positively during the post-test.

This experimental study aimed at investigating the moderating effect of instructional

conceptions on the effect of innovative interventions (e.g., the 4C/ID PLE with and without

ICT) for development of technical expertise in traditional classrooms. The results are not

always as expected. On the one hand this creates new theoretical challenges and raises new

research questions. On the other hand, the finding that secondary technical students react to

the interventions in accordance with the intentions of the designer obliterates the assertion

that students interpret learning environments based on their instructional conceptions. This

implies that secondary technical students (in Ghana) are ‘‘compliant learners.’’ Thus, the

learners’ behaviors are under control of the designers or educational practitioners. This

contradicts the assertion by Goodyear (2000) that instructional designers or educational

practitioners cannot and should not assume that learners’ behavior is under their control.

The findings of this study suggest that the notion of ‘‘the compliant learner’’ might context

specific.

References

Baron, R. M., & Kenny, D. A. (1986). The moderation–mediator variable distinction in social psychologicalresearch: Conceptual, strategic, and statistical conditions. Journal of Personality and SocialPsychology, 51(6), 1173–1182.

Campbell, D. T., & Stanley, J. C. (1963). Experimental designs for research on teaching. In N. L. Gage(Ed.), Handbook of research on teaching (pp. 171–246). Chicago: Rand-McNally.

Clarebout, G., & Elen, J. (2001). The ParlEuNet-project: Problems with the validation of socio-construc-tivist design principles in ecological settings. Computers in Human Behaviour, 17, 453–464.

Clark, R. E. (1994). Media will never influence learning. Educational Technology Research andDevelopment, 42(2), 21–29.

Clark, R. E. (2001). What is next in the media, methods debate? In R. E. Clark (Ed.), Learning from media:Argument, analysis, and evidence (pp. 327–337). Greenwich, CT: Information Age Publishing.


123

Clarke, J. A. (1994). Tertiary students’ perceptions of important events that occur in their learning envi-ronments. Paper presented at the annual meeting of the American Educational Research Association,New Orleans, LA.

Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading,writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning and instruction: Essays in honorof Robert Glaser (pp. 453–494). Hillsdale, NJ: Lawrence Erlbaum.

De Corte, E. (2003). Designing learning environments that foster the productive use of acquired knowledgeand skills. In E. De Corte, L. Verschaffel, N. Entwistle, & J. J. G. van Merrienboer (Eds.), Powerfullearning environments: Unravelling basic components and dimensions (pp. 21–35). Oxford: ElsevierScience.

Dochy, F. J. R. C. (1994). Prior knowledge and learning. In T. Husen & T.N. Postlethwait (Eds.), Inter-national encyclopedia of education (2nd ed., pp. 4698–4702). Oxford: Pergamon Press.

Elen, J. (1995). Blocks on the road to instructional design prescriptions: A methodology for I.D. researchexemplified. Leuven: Leuven University Press.

Elen, J., & Lowyck, J. (1999). Metacognitive instructional knowledge: Cognitive mediation and instruc-tional design. Journal of Structural Learning and Intelligent Systems, 13(3–4), 145–169.

Elen, J., & Lowyck, J. (2000). Instructional metacognitive knowledge: A qualitative study on conceptions offreshmen about instruction. Journal of Curriculum Studies, 32(3), 421–444.

Elen, J., Lowyck, J., & Bamps, H. (1998). The relation between students’ perception and problem-basedlearning style components. In J. van Merrienboer & G. Moerkerke (Eds.), Instructional design forproblem-based learning. Proceedings of the 3rd workshop of the EARLI-SIG Instructional Design (pp.281–290). Maastricht: University of Maastricht.

Entwistle, N., & Peterson, E. (2004). Conceptions of learning and knowledge in higher education: rela-tionship with study behaviour and influences of learning. International Journal of Educational Re-search, 9(6), 407–428.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive developmentalinquiry. American Psychologist, 34, 906–911.

Gagne, R. M. (1985). The conditions of learning (4th ed.). New York: Holt, Rinehart, & Winston.Gagne, R. M., & Briggs, L. J. (1979). Principles of instructional design (2nd ed.). New York: Holt, Rinehalt,

& Winston.Glaser, R. (1984). Education and thinking. The role of knowledge. American Psychologist, 39, 93–104.Goodyear, P. (2000). Environment for lifelong learning: ergonomics, architectural, and educational design.

In M. J. Spector & M. T. Anderson (Eds.), Integrated and holistic perspectives on learning, instructionand technology (pp. 1–18). Dordrecht, NL: Kluwer.

Greeno, R. (2002). Principles of construction (2nd ed.). London, UK: Pearson.Hativa, N., & Birenbaum, M. (2000). Who prefers what? Disciplinary differences in students’ preferred

approaches to teaching and learning styles. Research in Higher Education, 41(2), 209–236.Jonassen, D. H., Hannum, W. H., & Tessmer, M. (1989). Handbook of task analysis procedures. New York:

Praeger.Kember, D. (2001). Beliefs about knowledge and the process of teaching and learning as a factor in

adjusting to study in higher education. Studies in Higher Education, 26(2), 205–221.Konings, K., Brand-Gruwel, S., & van Merrienboer, J. J. G. (2005). Powerful learning environments and

combining the perspectives of designers, teachers and students. British Journal of Educational Psy-chology, 75(4), 645–660.

Kozma, R. B. (1991). Learning with media. Review of Educational Research, 61, 179–211.Kozma, R. B. (1994). Will media influence learning: Reframing the debate. Educational Technology Re-

search and Development, 42(2), 7–19.Kozma, R. B., & McGhee, R. (2003). ITC and classroom practice. In R. B. Kozma (Ed.), Technology,

innovation, and educational change: A global perspective (model 2) (pp. 43–80). Eugene, OR: ISTE.Krathwohl, D. R. (1993). Methods of educational and social science research: An integrated approach.

White Plains, NY: Longmans.Lehtinen, E. (2003). Computer-supported collaborative learning: An approach to powerful learning envi-

ronment. In E. De Corte, L. Verschaffel, N. Entwistle & J. J. G. van Merrienboer (Eds.), Powerfullearning environments: Unravelling basic components and dimensions (pp. 35–54). Oxford: ElsevierScience.

Leshin, C. B., Pollock, J., & Reigeluth, C. M. (1992). Instructional design strategies and tactics. EnglewoodCliffs, NJ: Educational Technology Publications.

Lowyck, J., & Elen, J. (1994). Students’ instructional metacognition in learning environments (SIMILE).Leuven: C.I.P.& T., K.U. Leuven.


123

Lowyck, J., Elen, J., & Clarebout, G. (2004). Instructional conceptions: A prospective analysis. Interna-tional Journal of Educational Research, 41(6), 429–444.

Mayer, R. E. (2002). Multimedia learning. Cambridge, UK: University Press.Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development,

50(3), 43–59.Merrill, M. D. (2006). Hypothesized performance on complex tasks as a function of scaled instructional

strategies. In J. Elen & R. E. Clark (Eds.), Handling complexity in learning environments: Theory andresearch (pp. 265–281). Amsterdam: Elsevier.

Moore, D. S., & McCabe, G. P. (2006). Introduction to the practice of statistics (5th ed.). New York, NY:Freeman.

Paris, G. S., & Winograd, P. (1990). How metacognition can promote academic learning and instruction. InB. F. Jones & L. Idol (Eds.), Dimensions of thinking and cognitive instruction (pp. 15–44). Hillsdale,NJ: Lawrence Erlbaum.

Pieters, J. M, Limbach, R., & de Jong, T. (2003). Information support in designing powerful learningenvironment. In E. De Corte, L. Verschaffel, N. Entwistle & J. J. G. van Merrienboer (Eds.), Powerfullearning environments: Unravelling basic components and dimensions (pp. 89–106). Oxford: ElsevierScience.

Romiszowski, A. J. (1997). Web-based distance learning and teaching: Revolutionary invention or reactionto necessity. In B. H. Khan (Ed.), Web-based instruction (pp. 25–37). Englewood Cliffs, NJ: Educa-tional Technology Publications.

Rothkopf, E. Z. (1968). Two scientific approaches to management of instruction. In R. M. Gagne & W. J.Gephart (Eds.), Learning research and school subjects (pp. 107–132). Itasca, III: Peacock.

Russell, T. (1999). The no significant difference phenomenon. Ralleigh, NC: North Carolina StateUniversity.

Sarfo, K. F. (1994). People’s attitudes towards the mentally ill. Unpublished master’s thesis, University ofGhana, Ghana.

Saljo, R. (1979). Learning in the learner’s perspective. I-some common sense conceptions. Reports from theInstitute of Education, University of Gothenburg, N. 77.

Seel, N. M., & Winn, W. D. (1997). Research on media and learning: Distributed cognition and semiotics. InR. D. Tennyson, F. Schott, N. Seel & S. Dijkstra (Eds.), Solving instructional design problems,instructional design: International perspective (Vol. 1, pp. 293–326). Mahwah, NJ: LawrenceErlbaum.

van Merrienboer, J. J. G. (1997). Training complex cognitive skills: A four-component instructional designmodel for technical training. Englewood Cliffs, NJ: Educational Technology Publications.

van Merrienboer, J. J. G., Clark, R. E., & de Croock, M. B. M. (2002). Blueprints for complex learning: The4C/ID-Model. Educational Technology Research and Development, 50(3), 39–64.

van Merrienboer, J. J. G., & Paas, F. (2003). Powerful learning and the many faces of instructional design:Towards a framework for the design of powerful learning environments. In E. De Corte, L.Verschaffel, N. Entwistle, & J. J. G. van Merrienboer (Eds.), Powerful learning environments:Unravelling basic components and dimensions (pp. 3–20). Oxford: Elsevier Science.

Winne, P. H. (1985). Steps toward promoting cognitive achievements. The Elementary School Journal, 85,673–693.

Winne, P. H., & Marx, R. W. (1982). Students’ and teachers’ thinking processes for classroom learning. TheElementary School Journal, 82, 493–518.

Zajonc, R. B. (1968). The attitudinal effects of mere exposure. Journal of Personality and SocialPsychology, 9(2), 1–27.


123

Documents

The moderating effect of instructional conceptions on the effect of powerful learning environments