Computers in Human Behavior 25 (2009) 804–810

Contents lists available at ScienceDirect

Computers in Human Behavior

journal homepage: www.elsevier .com/locate /comphumbeh

Benefits of inserting support devices in electronic learning environments

Geraldine Clarebout *, Jan ElenCenter for Instructional Psychology and Technology, Katholieke Universiteit Leuven, Belgium

a r t i c l e i n f o

Article history:Available online 23 August 2008

0747-5632/$ - see front matter � 2008 Elsevier Ltd. Adoi:10.1016/j.chb.2008.07.006

* Corresponding author.E-mail address: geraldine.clarebout@ped.kuleuven

a b s t r a c t

Research on support device-usage reveals that support devices are seldom used, and if used often in aninadequate way such that it is no longer a learning opportunity [Aleven, V., Stahl, E., Schworm, S., Fischer,F., & Wallace, R. (2003). Help seeking and help design in interactive learning environments. Review of Edu-cational Research, 73, 277–320; Clarebout, G., & Elen, J. (2006). Tool use in computer-based learning envi-ronments: Towards a research framework. Computers in Human Behavior, 22, 389–411.]. In view ofestablishing a solid research agenda on the optimization of the use of instructional interventions and sup-port devices, this article discusses three experimental studies, each dealing with different aspects of sup-port device use. In a first study, the impact on support device use of different types and numbers ofadjunct aids was investigated. In a second study, the influence of advice on support device use in an openlearning environment is studied, while also considering various learner related variables. A third studyaddresses the use of support devices in a text-based environment.

The results of the three studies reveal that the amount of support device usage is limited and that evenadvice on the use of the support device cannot always enhance this use. Studies 1 and 2 revealed that thetype of support devices influenced the amount of usage. With respect to learner characteristics, studies 2and 3 revealed no significant effect of self regulation. Students’ mastery orientation on the other hand didinfluence the support device usage. Reasons for the low usage of support devices are addressed in thediscussion.

� 2008 Elsevier Ltd. All rights reserved.

1. Introduction

Gagné, Briggs, and Wager (1988) stated that ‘‘Instruction is a hu-man undertaking whose purpose is to help people learn. Althoughlearning may happen without any instruction, the effects of instruc-tion on learning are often beneficial and easy to observe” (p. 3).

Instruction assumes that learning is enhanced through means ofwell-targeted support. This is illustrated by different review stud-ies, stipulating the conditions in terms of the learner, the task, andthe context under which a specific kind of support can be calledfunctional. Dillon and Gabbard (1998), for instance, specified theconditions under which hypermedia can be beneficial for learnercomprehension; de Jong and van Joolingen (1998) analyzed thepower of computer simulations for science learning and Atkinson,Derry, Renkl, and Wortham (2000) derived principles to useworked-out examples in instructional settings. In all these casesan instructional intervention was regarded to be functional whenthe adequate use of the intervention resulted in more or more effi-cient learning.

These instructional interventions, more specifically support de-vices, can target different aspects of the learning process. The de-

ll rights reserved.

.be (G. Clarebout).

vices can compensate for domain specific knowledge (e.g.,dictionary and information lists); induce cognitive processes (e.g.,navigation map and concept maps) or metacognitive processes(e.g. reflection sheets). The latter two are also often referred to asscaffolds (Brush & Saye, 2001; de Jong, 2006; Hannafin, Land, & Oli-ver, 1999).

However, Clark and Estes (2002) reveal that not all support in alearning environment is necessarily functional, and hence beneficialor helpful for learners’ learning process. This may be due to a poordesign of the support device, or the learner’s use the support device.Support can only be functional when adequately used by the learn-ers (e.g., Rothkopf, 1971; Winne, 1982). Review studies provide evi-dence that learners often use support inadequately or not at all, andin many cases not as intended by the instructional designers (e.g.,Aleven, Stahl, Schworm, Fischer, & Wallace, 2003; Clarebout & Elen,2006). Although these reviews do show that the use of support de-vices is often problematic, the link with specific learner characteris-tics such as prior knowledge (Martens, Valcke & Portiers, 1997) orlearning style (Lee & Lehman, 1993) remains unclear. Additionally,research on learner control also does not provide evidence for anunivocal positive effect of learner control on learning. These studiessuggest that learners do not use or sub-optimally use support(Friend & Cole, 1990; Goforth, 1994; Williams, 1996). Adding sup-port devices to a learning environment, and leaving the decision

G. Clarebout, J. Elen / Computers in Human Behavior 25 (2009) 804–810 805

on the use with the learners rests on the assumption that learnersare good judges of their own learning process. This may constitutethe core of the problem. Researchers seem to agree that learnersmay often lack the capacity to make adequate learning decisions(Clark, 1990; Hill & Hannafin, 2001; Horz, Winter, & Fries, 2009).Learners, when confronted with support alternatives, regularlymake poor choices (Clark, 1990). Similarly, Perkins (1985) pointedout that learners who are provided with learning opportunities inlearning environments do not always grasp them. He suggests threeconditions that may increase the probability that opportunities aretaken, or that support devices will be used: (1) the opportunity isthere; (2) learners recognize the opportunity, and (3) learners aremotivated to use the support devices.

1. The opportunity is there: the support devices provided are func-tional to students’ learning process; their use results in betterlearning. As already mentioned, Clark and Estes (2002) indi-cated that this is not always the case. Typical examples of stud-ies in which the functionality of a support device can bequestioned are, studies that ask students to use a supportdevice to communicate while sitting in the same room (e.g.,Janssens, Erkens, & Kanselaar, 2007; Munneke, Andriessen,Kanselaar, & Kirschner, 2007).

2. Learners recognize the opportunity: the support devices pro-vided to the learners are recognized as functional. Studentsknow the relation between the support devices and their learn-ing, and know how to handle the support devices to foster theirlearning (Elen, Lowyck, & Proost, 1996). This knowledgeabilityentails, therefore, both students’ knowledge about the supportdevices and their self-regulating skills. Learners’ self-regulatingskills refer to the extent, learners are capable of monitoring andadjusting their learning process (Clark, 1990). High self-regu-lated learners are typically more capable to determine when aspecific support device may be beneficial for their learningcompared to low self-regulators. Problems may also occur whenstudents lack knowledge about the support devices and, hencemay misinterpret the intentions behind the support devices(Winne, 1985, 2004). Winne (1985, 2004) refers to this problemas students not being calibrated to the learning environment.There is a mismatch between a learners’ judgment of a (featureof) a task, and the externally determined measures of (the fea-ture of) the task This was for instance, the case in a study intropical medicine (Clarebout, Elen, Lowyck, Van den Ende, &Van den Enden, 2004). Thinking aloud protocols revealed thatstudents did not use the support devices because they thoughtusing the devices would imply cheating. The program develop-ers, however, purposely included those support devices to fos-ter the acquisition of diagnostic skills. Similarly, Marek,Griggs, and Christopher (1999) revealed that students’ knowl-edge of a specific adjunct aid moderated the effect of the learn-ing environment. Respondents in this study indicated a weakinclination to use adjunct aids since, according to their knowl-edge, these aids required more elaborate study activities.

3. Learners are sufficiently motivated to use the support devices:support devices will only be used when learners are willing toinvest effort in the task (Salomon, 1984). In addition to the will-ingness to invest effort, students’ goal orientation may also playa role. In Ryan and Pintrich’s study (1997) for instance, masterygoal oriented students – students who are motivated to developcompetencies, to master the task – believed that using supportdevices would be beneficial for their well-being, and thereforethey were more likely to use them. Performance goal orienta-tion – with a focus on demonstrating competencies comparedto others – seemed to be linked to suboptimal use of supportdevices and a negative attitude towards the support devices(e.g., perceiving help-seeking support devices as a threat to

one’s self-worth). Furthermore, mastery goal orientation seemsto increase the probability of requesting help, whereas a perfor-mance goal orientation seems to be linked to asking for theright answer (Newman, 1998; Ryan, Pintrich, & Midgley,2001). Since using support devices can be seen as a form of helpseeking, one may assume that goal orientation will also affectthe use of support devices.

4. While Perkins (1985) focuses mainly on the functionality of thesupport devices and learner related variables, some studieshave been performed on the enhancement of the use of supportdevices (e.g., Carrier, Davidson, Williams, & Kalweit, 1986; Lee &Lehman, 1993). In these studies, students who received instruc-tional cues or encouragement to use a specific support device,used the support device more often compared to studentswho did not receive these cues or encouragement. In theirstudy, Lee and Lehman (1993) found an interaction effect withlearning style. Learners were characterized as having an activelearning style when they exhibited curiosity; initiative and awide focus when selecting information. Learners were charac-terized as having a passive learning style when they onlyselected the information offered, showed indifference and hada narrow focus. The encouragements seem to affect those stu-dents only that fell in between these extremes. Only for the‘neutral’ learners encouragements had positive effects.

In view of establishing a solid research agenda on the optimiza-tion of the use of support devices, and hence the benefits of inte-grating support devices in learning environments, thiscontribution discusses three studies, each dealing with a differentaspect of support device usage. In a first study, the TechnologyManagement (TM)-study, the impact of the type and number of ad-junct aids on their use was investigated. A second study, the Drink-ing Cup study, studied the influence of advice on the use of supportdevices, while also considering metacognition as part of students’self-regulation skills. While in these two studies Perkins (1985)conditions were considered in the design, a third study, the Obesitystudy, explicitly addresses them in the research questions. Thethree studies all report on the use of support devices in electroniclearning environments, with non-embedded support devices. Ineach case, learners decide about the use of the devices. The overallresults of these studies together with suggestions for further re-search are described.

2. Study 1: Technology Management study

The Technology Management study (TM)-study focuses on theuse of different types of adjunct aids and specific learner relatedvariables. First, the study addresses the use of different types of ad-junct aids. Adjuncts aids are defined as instructional interventionsinserted in (electronic) text books in view of supporting learners’information processing. Adjunct aids have already been intensivelystudied (see for instance, Grabowski, 2004; Rothkopf, 1996). Re-views (André, 1979; Mayer, 1979) generally reveal positive results.However, positive results can only be broad about when the ad-junct aids actually induce cognitive processes (André, 1979), andare needed by the target group (Schnotz & Bannert, 2003). In mostof these studies the effects on learning of one particular type of ad-junct aid have been studied (e.g., André, 1979; Mayer, 1979). How-ever, designers and developers of instructional text typically insertmultiple and different types of adjunct aids in their material.Hence, the study described here is an attempt to gain more insightin the use of multiple and different types of adjunct aids.

The main research question in this study was: What is the com-bined impact of the number and nature of the aids on the use ofadjunct aids?

Table 2Frequency and proportional time spent (Percentage) on adjunct aids

Type of adjunct aid Frequency Proportional time spent

Mean SD Mean SD

Examples 1.78 2.40 2.15 3.34Questions 1.61 2.63 1.43 2.20Figures 2.23 3.15 1.07 1.60

Total 5.63 5.32 4.65 4.51

Total time on Text 360020 0 9040 0

Total time on adjunct aids 10430 0 10480 0

806 G. Clarebout, J. Elen / Computers in Human Behavior 25 (2009) 804–810

2.1. Method

2.1.1. ParticipantsParticipants were 203 students (98% female) attending under-

graduate classes at a public university in central South Africa. Allstudents were majoring in education and volunteered to partici-pate in the study. Students could gain a limited number of extrapoints by participating in the study.

2.1.2. Design, instruments, and procedureAn experimental design was used with amount of aids (five, ten

or fifteen) and type of aids (questions, examples or figures) as inde-pendent variables (see Table 1), and use of adjunct aids as depen-dent variable (expressed in frequency of access and proportionaltime spent on adjunct aids). Additionally, learning gains werelooked at by comparing the scores on a pre-test with the scoreson a post-test.

The instructional material was based on a chapter on technol-ogy management by Newby, Stepich, Lehman, and Russell (1996).The text was presented on a computer, distributed over 21 pages.Access to adjunct aids – except for the control group – was gainedby clicking on labeled buttons on the computer screen. Students’use of and time spent on using adjunct aids were registeredthrough log files (Access database).

The 203 students participated in sessions, each with up to 30students. When entering the computer room, first the pre-testwas administered, next students had to enter their name on thecomputer (for logging purposes), and could start studying the texton computer in one of eight conditions. Participants were ran-domly assigned to one of the conditions. After reading the text apost-test was administered. This session took about 50 min.

2.2. Results

Descriptive statistics (Table 2) show that on average, the ad-junct aids were accessed almost six times. From the average totaltime spent on reading the text, 4.65% was spent on consultingthe adjunct aids.

A MANOVA reveals an overall effect of condition (Wilk’s lamb-da = .520; F(14,356) = 9.83; p 6 .05; eta2 = .28). The test of betweensubject effects shows that this effect of condition is significant forboth dependent variables, namely frequency of access(F(7,202) = 16.42; p 6 .05; eta2 = .39) and proportional time spenton adjunct aids (F(7,202) = 16,42; p 6 .05; eta2 = .36).

To gain more insight in how the different conditions differedfrom each other, a Waller Duncan post-hoc test was performed,which resulted in different subsets (Table 3). For the impact of con-dition on frequency of use of the adjunct aids, the control groupand the condition with five examples belongs to a first group withlimited frequency of use. All conditions with five adjunct aids be-long to a second, partly overlapping, group with low to mediumfrequency of access. The third group contains two conditions with10 adjunct aids (condition with 5 questions and 5 examples, andcondition with 5 questions and 5 figures) as well as the condition

Table 1Conditions of the technology management study

Condition Number and type of Adjunct aids

1 No adjunct aids2 5 Examples3 5 Questions4 5 Figures5678

with 15 adjunct aids. A final subset consists of a partly overlappinggroup of three conditions with high frequency of access of the ad-junct aids. All conditions in which figures are combined with oneor two other types of adjunct aids belong to this group.

A somewhat different picture emerges when the proportion ofstudy time spent on adjunct aids is examined. A Waller Duncanpost-hoc test reveals four homogeneous subsets. The control groupbelongs to a first group. The conditions with five aids belong to asecond group. The third group contains conditions in which a med-ium to high proportion of study time is spent studying the adjunctaids, including the condition with questions and figures; the condi-tion with questions, figures, and examples, and the condition withquestions and examples. The group with the highest relative pro-portion of time devoted (from 7.2% to 8.34%) to the adjuncts aidsincludes the conditions where examples are combined with oneor two other adjunct aids.

With respect to learning outcomes, it could be seen that overallstudents did learn by studying the text. The average score on thepre-test was 4. 52 (SD = 2.35) and on the post-test 7.98(SD = 2.98), which is a significant difference (t = 17.33, p < .05).However, no overall effect was found on the learning outcomesfor the specific conditions.

3. Study 2: Drinking Cup study

In the Drinking Cup study, the influence of advice on supportdevice-usage was studied. Advice aimed at making the studentsmore knowledgeable about the support devices (cf. Condition 2of Perkins). The Drinking Cup-study investigates whether these po-sitive results can also be retrieved in open-learning environments.Open-learning environments confront students with an ill-struc-tured problem (Jacobson & Spiro, 1995; Jonassen, 1997). Theseenvironments are further characterized by the inclusion of a num-ber of support devices, and ample learner control (Hannafin, Hall,Land, & Hill, 1994).

This study investigates, whether advice adapted to the learners’problem-solving steps leads to more support device usage thannon-adapted advice (at fixed moments). The rationale for thisquestion is the suggestion by Clark (1990) that both too much ortoo less support may be detrimental. Consequently, it can be ex-pected that adapted advice will lead to more (adequate) use ofthe support devices, and hence to better performance.

5 Examples + 5 Questions5 Examples + 5 Figures5 Questions + 5 Figures

5 Examples + 5 Questions + 5 Figures

Table 3Homogeneous groups with respect to frequency of access and proportion of time spent on adjunct aids as a result of Waller Duncan tests

Frequency of access of adjunct aids

Condition N Subset 1 Subset 2 Subset 3 Subset 4

No adjunct aids 27 0.005 examples 27 2.22 2.225 questions 27 3.675 figures 26 4.315 examples and 5 questions 26 7.275 questions and 5 figures 28 8.68 8.685 examples, 5 questions, and 5 figures 23 9.13 9.135 examples and 5 figures 19 10.47

ProportionNo adjunct aids 27 0.005 figures 26 2.335 examples 27 3.135 questions 27 3.185 questions and 5 figures 28 5.575 examples, 5 questions, and 5 figures 23 7.21 7.215 examples and 5 questions 26 7.58 7.585 examples and 5 figures 19 8.34

G. Clarebout, J. Elen / Computers in Human Behavior 25 (2009) 804–810 807

Given the assumption, that students’ self-regulation skills influ-ence students’ support device usage, the relationship between stu-dents’ use of support devices, and their self-regulation skills isstudied.

Summarized, the research questions in the Drinking Cup-studywere: (1) Does advice encourage tool use?; (2) Does adapted ad-vice lead to more tool use then fixed advice?; and (3) Does stu-dents’ self-regulation influence tool use?

3.1. Method

3.1.1. ParticipantsOne hundred and eighty five first year educational sciences stu-

dents (97% female) of a Belgian university participated. Participa-tion in this study was part of the requirements of a specificcourse in their program.

3.1.2. Design, instruments, and procedureA 3 � 2 experimental design was used with condition (no ad-

vice, adapted advice, and fixed advice), and self regulation (high–low) as independent variables and the use of support devices (ex-pressed in frequency and proportional time) as dependent vari-ables. Also learning outcomes were studied as dependent variables.

The learning environment consisted of an open electronic learn-ing environment. Students had to suggest and give arguments forthe best possible alternative for plastic one-way drinking cups ona music festival, by looking at the problem from a financial, ecolog-ical and safety perspective. In other words, the environment con-fronted students with an ill-structured realistic problem. To solvethis problem, students got access to different support devices:information tools (official documents; videos of stakeholders),knowledge monitoring tools (problems solving script, reportingscript), and performance support tools (calculator, note pad). Inthe fixed advice condition every 7 min a support device was ran-domly selected and explained to the students, by pointing out itsfunction, and why and when this support device would be helpful.In the adapted advice condition, an evaluation was made of thestudents’ previous activities every 7 min. Based on those activitiesthe student was advised to use a specific support device.

Students’ self-regulation were measured by means of the meta-cognition scale of Vermunt’s Learning Style Inventory (1992, Cron-bach’s alpha = .80).

In the first session, LSI-questionnaires were administered. In thesecond session, students solved the drinking cup problem on acomputer. This session took place in groups of five students, each

working individually on a portable computer. Students could workon the problem for 1 h. Log files (Access database) were kept totrack students’ use of support devices. Students’ solution prior toconsulting any information was considered as a pre-test, whiletheir solution after working in the environments was used as apost-test measurement.

3.2. Results

In absence of a correlation between frequency of support deviceusage and proportional time spent on the support devices (Pearsonr = .02; p > .05), two separate ANOVA’s were done. None of themrevealed a significant main or interaction effect of condition andself-regulation.

Follow-up analyses with the frequency of and time spent on theuse of the different support devices as dependent variables did re-veal that for the information tools significant effects were found forthe proportion of time spent on this device, Wilk’s k = .765; F(14,314) = 3.21; p 6 .05; eta2 = .14. The students in the condition withadapted advice consulted more frequently the official documents,while students in the fixed advice condition spent most of the timeon the subjective information, i.e. video clips of stakeholdersexpressing their opinion on the subject.

Table 4 also reveals that students spent most of the time on theinformation tools (Info, videos) and only a small proportion of theirtime was spent on the other support devices.

With respect to the learning outcomes, no differences werefound between the different conditions.

The analysis of whether students followed the provided adviceresulted in an interesting additional finding. An ANOVA yielded asignificant effect of condition, F(1,114) = 8.36; p 6 .05; eta2 = .07.Participants who received adapted advice followed significantlymore the advice of the agent (Mean = .26; SD = .22) than did stu-dents who received fixed advice (Mean = .16; SD = .13). However,the results also revealed that most of the time students did not fol-low the advice. The descriptive statistics indicated that on average20.61% (SD = 18.18%) of the advice was followed. From the 124 par-ticipants in the experimental groups, 30 of them (24.2%) did neverfollow any advice.

4. Study 3: Obesity study

This Obesity study addressed systematically, Perkins’ three con-ditions. First, the support devices’ functionality was studied by

Table 4Proportional time spent on support devices

Support device Adapted advice Fixed advice No advice

Mean SD Mean SD Mean SD

Calc. .03 .09 .04 .08 .03 .05Assign. .01 .01 .01 .01 .01 .02Info .50 .10 .46 .15 .47 .17PO-c 04 .07 .05 .07 .04 .05R-c .02 .03 .02 .03 .04 .09Expl. .04 .02 .04 .03 .03 .02Videos .11 .08 .15 .07 .18 .11

Calc. = calculator; Assign. = access to the assignment; Info = access to objectiveinformation; PO-c = problem-solving checklist (access to worked-out example); R-c = reporting checklist; Expl. = access to explanation on functionalities of differenttools; videos = access to subjective information (stakeholders).

808 G. Clarebout, J. Elen / Computers in Human Behavior 25 (2009) 804–810

comparing the performance of a group of students with and with-out access to support devices. Second, in view of considering stu-dents’ instructional knowledgeability, students’ self-regulationwas measured. One condition contained advice to further increasestudents’ instructional knowledgeability. Prior to reading the text,all support devices were explained (in written) to the students inthis group, and suggestions were provided on when to use thesedevices during studying the text. The advice-group was expectedto use more frequently the available tools. Third, students’ goal ori-entation was measured.

Summarized the following research questions were addressed:(1) Are the support devices functional to students’ learning?; (2)How do learner related variables (self-regulation and goal orienta-tion) influence students’ use of support devices; and (3) Does ad-vice contribute to the use of support devices?

4.1. Method

4.1.1. ParticipantsParticipants were 216 first year educational sciences bachelor

students from a Belgian University (98% female). Participation inthe study was part of a specific course in their study program.

4.1.2. Design, instruments, and procedureAn one factorial experimental design was used with condition

(Support device + advice, support devices only, no support devices)as independent variables. Students’ metacognition and goal orien-tation were considered as predictors. The dependent variableswere, performance frequency of the use of support devices andtime spent on the support devices.

In the first session, students’ goal orientation and self-regula-tion skills were measured. For measuring students’ self-regulationthe metacognition scale of the LSI of (Vermunt 1992; Cronbach’salpha = .77) was used, and for goal orientation, a translation of El-liot’s instrument, consisting of a performance orientation and mas-tery orientation scale (1999) (performance orientation: Cronbach’salpha = .81; mastery orientation: Cronbach’s alpha = .77). In thesecond session, students were asked to read a computer-based texton obesity (Nederkoorn, Guerrieri, & Jansen, 2006). They were in-formed that a test with knowledge, insight and application ques-tions would be administered immediately after studying the text.

Table 5Performance Score, Frequency and Time Spent on Support Devices

Support devices + Advice

Mean SD

Performance score 0.63 0.16Frequency of support device usage 5.68 3.91Time spent on support devices 11.10 8.32

In two conditions, students could access different support devices,namely a dictionary, instructional goals, example questions, andhelp with interpreting figures and text. In one experimental group,students also received advice on the relevance and use of the sup-port devices. Log files (Access database) were kept to track stu-dents’ use of the support devices. Students received 25 min toread the text after which they ware asked to complete a knowledgetest.

4.2. Results

With respect to the research question addressing the first con-dition of Perkins, it was found that the two experimental groupsoutperformed the control group on the post-test, F(2,175) = 6.79;p 6 .05; eta2 = .072. This was seen as an indication that the supportdevices were functional. Comparing the two experimental groups(Table 5), the group with advice seems to have used more supportdevices, both in terms of frequency and time spent on the supportdevices, than the group without advice, but this difference is notsignificant. Interpreting the descriptive statistics for students’self-regulation, reveals that students’ are average self regulators(mean = 3.55 on a scale of 1–6; SD = .54). The mean scores for goalorientation are 3.84 (SD = .80) for performance orientation, and4.32 (SD = .67) for mastery orientation. Students’ self-regulationdid not influence tool use, but goal orientation, more specificallymastery orientation did (R2 = .15; beta = �.384; t = �2.528;p 6 .05). The more students are mastery oriented, the less supportdevices are used.

5. Discussion and conclusion

Overall in all three studies it is striking that the amount of use ofsupport devices is limited and it seems that even with advice thatthis remains low. The finding in the DC-study that by providingadapted advice, more objective information was consulted and lesssubjective information reveals the subtle interactions.

The three studies presented here all addressed specific issueswith respect to the use of support devices The TM-study and theDC-study suggest that the nature of the support device is relatedto its use. In the TM-study, more time was spent on the adjunctaids when examples were presented, while in the DC-study, theinformation tools were far more often consulted than the othertools. These tools might have an immediate relevance to the learn-ers, perhaps because learners are more acquainted with them andhence more knowledgeable. It even could be argued that in the DC-study, the information devices can hardly be called a support de-vice, since they provide the content of the learning material.

With respect to self-regulation, no influence was found in theDC- and Obesity-study for self-regulation, while there is a cleartheoretical ground for assuming this is an important variable withrespect to the use of support devices. This may be due to method-ological reasons such as:

1. The instrument: A part of the LSI was used, given its availabilityin Dutch and the reliabilities found in previous research, but itonly measures part of students’ self regulation. It even can be

Support devices Control

Mean SD Mean SD

0.67 0.16 0.50 0.193.98 3.04 – –7.73 5.70

G. Clarebout, J. Elen / Computers in Human Behavior 25 (2009) 804–810 809

argued that a survey instrument may not measure students’self-regulation skills (see also Veenman (2005)), but rather theirintentions; and hence it may be that what students express in asurvey differentiates from their actual behavior in a learningenvironment.

2. The sample: Looking at the means and the standard deviationsin the Drinking Cup and Obesity studies it seems that in bothstudies a rather homogeneous group was used. The distinctionbetween high and low regulators in the DC-study may conse-quently be called very arbitrary. Different results may be foundwhen a more heterogeneous population is studied.

The Obesity-study additionally addressed goal orientation. Aneffect was found, although opposite to results in studies on help-seeking (Ryan et al., 2001). One possible explanation may be thatstudents behave differently towards ‘human’ support devices andelectronic support devices, and that those students who are moreinclined to request help from a human instructional agent, maybe less inclined to use electronic support devices.

The findings of these three studies do raise some issues:

1. The issue of learner control: Clear positive effects of learner con-trol on performance have not yet been found. An argument thatunderlies the idea of learner control is that through learner con-trol learning environments gets adapted to the learner: a lear-ner can use exactly that amount of support that is needed byaccessing the right support devices at the right moment. How-ever, as already indicated, this presupposes learners to be capa-ble of making adequate decisions with respect to their learningprocess. This seems not to be the case. Hence, from an instruc-tional design perspective, one may wonder, whether providinglearners with non-embedded support devices is the best solu-tion. A doubt recently raised by Mayer (2004) and by Kirschner,Sweller, and Clark (2006), who all question the effectiveness ofdiscovery or inquiry-based learning environments given thatthese environments typically involve a large amount of learnercontrol.

2. The user aimed at: If an instructional designer has sufficient evi-dence about the functionality for a specific learner of a supportdevice, than it seems indicated to embed this support device forthis learner. This may not be indicated though in case the envi-ronment focuses on a high self-regulating learner. Such a lear-ner probably makes adequate selections. In case of poor orlimited evidence about the device’s functionality, a differentdecision seems indicated. Including the device may make thetask more demanding. An additional task namely processingthe information of the support device is added. However, alearning environment is typically not designed for one particu-lar type of learner; it targets high as well as low self regulatorylearners. In such a case, the support device may be beneficial foronly part of the group of learners. For high self regulators it isthen best to non-embed the support device so that they candecide themselves whether or not to use it. Leaving them incontrol of the decision process causes congruence or even con-structive friction (Vermunt & Verloop, 1999). Embedding thesupport devices for these learners would mean that an activityis taken over that they can perform themselves, which couldcause destructive friction (Vermunt & Verloop, 1999). On theother hand, for low self regulators who need the support offeredthrough the support device it is best to embed it so that theycannot but use it. Further research is needed to address the ben-efits and pitfalls of embedding or non-embedding supportdevices for different types of learners.

3. Adequate use of support devices: Even if a support device is 100%functional and embedded, adequate use is still not guaranteed(Greene & Land, 2000). Students need to be knowledgeable of

and willing to properly use the support device. Hence, and inline with Perkins’ second condition, the question becomeshow to make students (instructionally) knowledgeable withrespect to the learning opportunities offered through supportdevices. One possibility may be training students in using sup-port devices (e.g., Gräsel, Fischer, & Mandl, 2000). Such trainingmay precede the actual confrontation with the learning envi-ronment or may be integrated in it through direct advice onthe use of particular interventions or feedback on the use ofthe interventions. Goodyear (2000) proposes to integrateinstruction on the use in the support devices themselves. Good-year seems to argue that the phenomenon of inadequate ornon-use is an artifact of irrelevant, inauthentic learning tasks.He argues that learners have to be trained to use the interven-tions and that authentic tasks may motivate learners to use theinterventions. Another option is to design support devices insuch way that they cannot but be used optimally.

4. Learner related variables: The unexpected finding of the influ-ence of mastery orientation on tool use – compared to the find-ings of Ryan et al. 2001) – calls for further research on therelationship between goal orientation and tool use. Highly mas-tery oriented learners may specifically focus on the mastery ofthe content of the text, rather than on ‘additional’ informationprovided by the support devices. One may also wonder whatother learner variables need to be considered. Within the fieldof management information system research perceptual vari-ables have been identified as important antecedent variablesof human behavior (Davis, Bagozzi, & Warshow, 1989). In Daviset al.’s technology acceptance (TA) model it is stated that auser’s behavior in an environment is determined by the user’sattitude towards this environment. These attitudes in turn,are influenced by the user’s perceived usefulness and the per-ceived ease of use of a technology-based environment. Simi-larly, Salomon (1984) addresses that learners’ expectanciesand skills with regards to an instructional medium will influ-ence the amount of invested effort, and consequently the depthof the learners’ processing of the learning material. One couldargue that, specified to support devices, learners’ perceived use-fulness and expectancies will influence learners’ intention touse the support device. This will increase the probability ofactual using the support device and the mental effort investedto process the information from the support device, henceresulting in better learning.

The overview of the three studies allows to compare results andto make stronger claims with respect to the impact of specific vari-ables on the use of support devices. The studies reveal that thephenomenon of non-use or inadequate use of instructional inter-vention is highly complex (see also, Elen and Clarebout (2005)).Looking at the three studies it is clear that overall the use of sup-port devices was low, except for the information support devicesin the DC-study. Evidently, the question on the benefits of insertingsupport devices in electronic learning environments cannot be eas-ily answered and requires further research that considers task,learner and support device characteristics. A starting point couldbe the TA-model of Davis et al. (1989) as also suggested by Gerjetsand Hesse (2004). This would mean that learners’ attitudes are putcentral in studies on the use of support devices, and that it is as-sumed that these attitudes will determine how learners’ will reactto the support devices and how they will use them.

References

Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003). Help seeking andhelp design in interactive learning environments. Review of EducationalResearch, 73, 277–320.

810 G. Clarebout, J. Elen / Computers in Human Behavior 25 (2009) 804–810

André, T. (1979). Does answering higher-level questions while reading facilitateproductive learning? Review of Educational Research, 49, 280–319.

Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning fromexamples: Instructional principles from the worked examples research. Reviewof Educational Research, 70, 181–214.

Brush, T., & Saye, J. (2001). The use of embedded scaffolds with hypermedia-supported student-centered learning. Journal of Educational Multimedia andHypermedia, 10, 333–356.

Carrier, C., Davidson, G., Williams, M., & Kalweit, C. M. (1986). Instructional optionsand encouragement effects in a micro-computer concept lesson. Journal ofEducational Research, 79, 222–229.

Clarebout, G., & Elen, J. (2006). use in computer-based learning environments:Towards a research framework. Computers in Human Behavior, 22, 389–411.

Clarebout, G., Elen, J., Lowyck, J., Van den Ende, J., & Van den Enden, E. (2004).KABISA: Evaluation of an open learning environment. In A.-M. Armstrong (Ed.),Instructional design in the real world. A view from the trenches (pp. 119–135).Hershey: Information Science Publishing.

Clark, R. E. (1990). When teaching kills learning: Research on mathemathantics. InH. Mandl, E. De Corte, N. Bennett, & H. F. Friedrich (Eds.), European research in aninternational context: Volume 2. Learning and instruction (pp. 1–22). Oxford, NY:Pergamon Press.

Clark, R. E., & Estes, F. (2002). Turning research into results. A guide to selecting theright performance solutions. Atlanta, GA: CEP Press.

Davis, F. D., Bagozzi, R. P., & Warshow (1989). User acceptance of computer technology.A comparison of two theoretical models management science, 35, 982–1003.

de Jong, T. (2006). Scaffolds for computer simulation based scientific discoverylearning. In J. Elen & D. Clark (Eds.), Handling complexity in learningenvironments: Research and theory (pp. 107–128). London: Elsevier Science.

de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning withcomputer simulations of conceptual domains. Review of Educational Research,68, 179–201.

Dillon, A., & Gabbard, R. (1998). Hypermedia as an educational technology: A reviewof quantitative research literature on learner comprehension, control, and style.Review of Educational Research, 68, 322–349.

Elen, J., & Clarebout, G. (2005). Support in learning environments: Touching thelimits of instructional design. Educational Technology, 45(5), 44–47.

Elen, J., Lowyck, J., & Proost, K. (1996). Design of telematic learning environments: Acognitive meditational view. Educational Research and Evaluation: AnInternational Journal on Theory and Practice, 2, 213–230.

Elliot, A. J. (1999). Approach and avoidance motivation and achievement goals.Educational Psychologist, 34, 169–189.

Friend, C. L., & Cole, C. L. (1990). Learner control in computer-based instruction: Acurrent literature review. Educational Technology, 30(11), 47–49.

Gagné, R. M., Briggs, L. J., & Wager, W. W. (1988). Principles of instructional design(3rd ed.). New York: Holt, Rinehart and Winston.

Gerjets, P., & Hesse, F. W. (2004). When are powerful learning environmentseffective? The role of learner activities and students’ conceptions of educationaltechnology. International Journal of Educational Research, 41, 445–465.

Goforth, D. (1994). Learner control = decision making + information: A model andmeta-analysis. Journal of Educational Computing Research, 11, 1–26.

Goodyear, P. (2000). Environments for lifelong learning. Ergonomics, architectureand educational design. In J. M. Spector & T. M. Anderson (Eds.), Integrated andholistic perspectives on learning, instruction and technology (pp. 1–18). Dordrecht,NL: Kluwer Academic Publishers.

Grabowski, B. L. (2004). Generative learning contributions to the design ofinstruction and learning. In D. H. Jonassen (Ed.), Handbook of research oneducational communications and technology (2nd ed., pp. 719–743). Mahwah, NJ:Lawrence Erlbaum.

Gräsel, C., Fischer, F., & Mandl, H. (2000). The use of additional information in problem-oriented learning environments. Learning Environment Research, 3, 287–325.

Greene, B. A., & Land, S. M. (2000). A qualitative analysis of scaffolding use in aresource-based learning environment involving the world wide web. Journal ofEducational Computing Research, 23, 151–179.

Hannafin, M. J., Hall, C., Land, S., & Hill, J. (1994). Learning in open-ended learningenvironments: Assumptions, methods and implications. Educational Technology,34(10), 48–55.

Hannafin, M., Land, S., & Oliver, K. (1999). Open learning environments:Foundations, methods and models. In C. M. Reigeluth (Ed.). Instructionaldesign theories and models. A new paradigm of instructional theory (Vol. 2,pp. 115–140). Mahwah, NJ: Erlbaum.

Hill, J. R., & Hannafin, M. J. (2001). Teaching and learning in digital environments:The resurgence of resource-based learning. Educational Technology Research andDevelopment, 45(1), 65–94.

Horz, H., Winter, C., & Fries, S. (2009). Differential effects of situated prompts.Computers in Human Behavior, 25, 818–828.

Jacobson, M. J., & Spiro, R. J. (1995). Hypertext learning environments, cognitiveflexibility and the transfer of complex knowledge. Journal of EducationalComputing Research, 12, 301–333.

Janssen, J., Erkens, G., & Kanselaar, G. (2007). Visualization of agreement anddiscussion processes during computer-supported collaborative learning.Computers in Human Behavior, 23, 1105–1125.

Jonassen, D. H. (1997). Instructional design models for well-structured andillstructured problem-solving learning outcomes. Educational TechnologyResearch and Development, 45(1), 65–91.

Kirschner, P., Sweller, J., & Clark, R. E. (2006). Why minimal guidance duringinstruction does not work: An analysis of the failure of constructivist, discovery,problem-based, experiential, and inquiry-based teaching. EducationalPsychologist, 41, 75–86.

Lee, Y. B., & Lehman, J. D. (1993). Instructional cueing in hypermedia: A study withactive and passive learners. Journal of Educational Multimedia and Hypermedia, 2,25–37.

Marek, P., Griggs, R. A., & Christopher, A. N. (1999). Pedagogical aids in textbooks:Do students’ perceptions justify their prevalence? Teaching of Psychology, 26,11–19.

Martens, R. L., Valcke, M. M., & Portier, S. J. (1997). Interactive learningenvironments to support independent learning: The impact of discernabilityof embedded support devices. Computers in Education, 28, 185–197.

Mayer, R. E. (1979). years of research on advance organizers: Assimilation theory isstill the best predictor of results. Instructional Science, 8, 133–167.

Mayer, R. E. (2004). Should there be a three-strikes rule against pure discoverylearning? The case for guide methods of instruction. American Psychologist,59(1), 14–19.

Munneke, L., Andriessen, J., Kanselaar, G., & Kirschner, P. (2007). Supportinginteractive argumentation: Influence of representational tools on discussing awicked problem. Computers in Human Behavior, 23, 1072–1088.

Nederkoorn, C., Guerrieri, R., & Jansen, A. (2006). Leven in Luilekkerland [Living inwonderland]. De Psycholoog, 41(1), 10–16.

Newby, T. J., Stepich, D. A., Lehman, J. D., & Russell, J. D. (1996). Instructionaltechnology for teaching and learning. Designing instructionintegrating computersand using media. Englewood Cliffs, NJ: Merrill.

Newman, R. S. (1998). Students’ help seeking during problem solving: Influences ofpersonal and contextual achievement goals. Journal of Educational Psychology,90, 644–658.

Perkins, D. N. (1985). The fingertip effect: How information-processing technologyshapes thinking. Educational Researcher, 14, 11–17.

Rothkopf, E. Z. (1971). The concept of mathemagentic activities. Review ofEducational Research, 40, 325–336.

Rothkopf, E. Z. (1996). Controlled mathemagenic activities. In D. H. Jonassen (Ed.),Handbook of research for educational communications and technology(pp. 879–896). New York: Macmillan Library.

Ryan, A. M., & Pintrich, P. R. (1997). ‘‘Should I ask for help?” The role of motivationand attitudes in adolescents’ help seeking in math class. Journal of EducationalPsychology, 89, 329–341.

Ryan, A. M., Pintrich, P. R., & Midgley, C. (2001). Avoiding seeking help in theclassroom: Who and why? Educational Psychology Review, 13, 93–113.

Salomon, G. (1984). Television is ‘‘easy” and print is ‘‘tough”. The differentialinvestment of mental effort in learning as a function of perceptions andattribution. Journal of Educational Psychology, 76, 647–658.

Schnotz, W., & Bannert, M. (2003). Construction and interference in learning frommultiple representations. Learning and Instruction, 12, 141–156.

Veenman, M. V. (2005). The assessment of metacognitive skills: What can belearnerd from multi-method designs? In B. Moschner & C. Artelt (Eds.),Lernstrategien und Metakognition: Implikationen für Forschung und Praxis[Learning strategies and metacognition: Implications for research and practice](pp. 77–99). Berlin: Waxman.

Vermunt, J. (1992). Leerstijlen en sturen van leerprocessen in het hoger onderwijs: Naarprocesgerichte instructie en zelfstandig denken [Learning style and coachinglearning processes in Higher Education]. Lisse, NL: Swets & Zeitlinger.

Vermunt, J. D., & Verloop, N. (1999). Congruence and friction between learning andteaching. Learning and Instruction, 9, 257–280.

Williams, M. D. (1996). Learner-control and instructional technology. In D. H.Jonassen (Ed.), Handbook of research for educational communications andtechnology (pp. 957–983). New York: Macmillan Library.

Winne, P. H. (1982). Minimizing the black box problem to enhance the validity oftheories about instructional effects. Instructional Science, 11, 13–28.

Winne, P. H. (1985). Steps towards cognitive achievements. Elementary SchoolJournal, 85, 673–693.

Winne, P. H. (2004). Students’ calibration of knowledge and learning process:Implications for designing powerful learning environments. InternationalJournal of Educational Research, 41, 466–488.