Benefits of inserting support devices in electronic learning environments

  • Published on
    04-Sep-2016

  • View
    215

  • Download
    3

Transcript

  • le

    ium

    e-ust isp se

    ronments: Towards a research framework. Computers in Human Behavior, 22, 389411.]. In view of

    tated to help

    Derry, Renkl, and Wortham (2000) derived principles to useworked-out examples in instructional settings. In all these casesan instructional intervention was regarded to be functional whenthe adequate use of the intervention resulted in more or more ef-cient learning.

    These instructional interventions, more specically support de-vices, can target different aspects of the learning process. The de-

    Aleven, Stahl, Schworm, Fischer, &Wallace, 2003; Clarebout & Elen,2006). Although these reviews do show that the use of support de-vices is often problematic, the link with specic learner characteris-tics such as prior knowledge (Martens, Valcke & Portiers, 1997) orlearning style (Lee & Lehman, 1993) remains unclear. Additionally,research on learner control also does not provide evidence for anunivocal positive effect of learner control on learning. These studiessuggest that learners do not use or sub-optimally use support(Friend & Cole, 1990; Goforth, 1994; Williams, 1996). Adding sup-port devices to a learning environment, and leaving the decision* Corresponding author.

    Computers in Human Behavior 25 (2009) 804810

    Contents lists availab

    u

    eviE-mail address: geraldine.clarebout@ped.kuleuven.be (G. Clarebout).Instruction assumes that learning is enhanced throughmeans ofwell-targeted support. This is illustrated by different review stud-ies, stipulating the conditions in terms of the learner, the task, andthe context under which a specic kind of support can be calledfunctional. Dillon and Gabbard (1998), for instance, specied theconditions under which hypermedia can be benecial for learnercomprehension; de Jong and van Joolingen (1998) analyzed thepower of computer simulations for science learning and Atkinson,

    However, Clark and Estes (2002) reveal that not all support in alearning environment is necessarily functional, and hence benecialor helpful for learners learning process. This may be due to a poordesign of the support device, or the learners use the support device.Support can only be functional when adequately used by the learn-ers (e.g., Rothkopf, 1971;Winne, 1982). Review studies provide evi-dence that learners often use support inadequately or not at all, andin many cases not as intended by the instructional designers (e.g.,learningmay happenwithout any instruction, the effects of instruc-tion on learning are often benecial and easy to observe (p. 3).1. Introduction

    Gagn, Briggs, andWager (1988) sman undertaking whose purpose is t0747-5632/$ - see front matter 2008 Elsevier Ltd. Adoi:10.1016/j.chb.2008.07.006establishing a solid research agenda on the optimization of the use of instructional interventions and sup-port devices, this article discusses three experimental studies, each dealing with different aspects of sup-port device use. In a rst study, the impact on support device use of different types and numbers ofadjunct aids was investigated. In a second study, the inuence of advice on support device use in an openlearning environment is studied, while also considering various learner related variables. A third studyaddresses the use of support devices in a text-based environment.The results of the three studies reveal that the amount of support device usage is limited and that even

    advice on the use of the support device cannot always enhance this use. Studies 1 and 2 revealed that thetype of support devices inuenced the amount of usage. With respect to learner characteristics, studies 2and 3 revealed no signicant effect of self regulation. Students mastery orientation on the other hand didinuence the support device usage. Reasons for the low usage of support devices are addressed in thediscussion.

    2008 Elsevier Ltd. All rights reserved.

    hat Instruction is a hu-people learn. Although

    vices can compensate for domain specic knowledge (e.g.,dictionary and information lists); induce cognitive processes (e.g.,navigation map and concept maps) or metacognitive processes(e.g. reection sheets). The latter two are also often referred to asscaffolds (Brush & Saye, 2001; de Jong, 2006; Hannan, Land, & Oli-ver, 1999).cational Research, 73, 277320; Clarebout, G., & Elen, J. (2006). Tool use in computer-based learning envi-Benets of inserting support devices in e

    Geraldine Clarebout *, Jan ElenCenter for Instructional Psychology and Technology, Katholieke Universiteit Leuven, Belg

    a r t i c l e i n f o

    Article history:Available online 23 August 2008

    a b s t r a c t

    Research on support devicinadequate way such that iF., & Wallace, R. (2003). Hel

    Computers in H

    journal homepage: www.elsll rights reserved.ctronic learning environments

    age reveals that support devices are seldom used, and if used often in anno longer a learning opportunity [Aleven, V., Stahl, E., Schworm, S., Fischer,eking and help design in interactive learning environments. Review of Edu-

    le at ScienceDirect

    man Behavior

    er .com/locate /comphumbeh

  • terized as having a passive learning style when they onlyselected the information offered, showed indifference and had

    Humon the use with the learners rests on the assumption that learnersare good judges of their own learning process. This may constitutethe core of the problem. Researchers seem to agree that learnersmay often lack the capacity to make adequate learning decisions(Clark, 1990; Hill & Hannan, 2001; Horz, Winter, & Fries, 2009).Learners, when confronted with support alternatives, regularlymake poor choices (Clark, 1990). Similarly, Perkins (1985) pointedout that learners who are provided with learning opportunities inlearning environments do not always grasp them. He suggests threeconditions that may increase the probability that opportunities aretaken, or that support devices will be used: (1) the opportunity isthere; (2) learners recognize the opportunity, and (3) learners aremotivated to use the support devices.

    1. The opportunity is there: the support devices provided are func-tional to students learning process; their use results in betterlearning. As already mentioned, Clark and Estes (2002) indi-cated that this is not always the case. Typical examples of stud-ies in which the functionality of a support device can bequestioned are, studies that ask students to use a supportdevice to communicate while sitting in the same room (e.g.,Janssens, Erkens, & Kanselaar, 2007; Munneke, Andriessen,Kanselaar, & Kirschner, 2007).

    2. Learners recognize the opportunity: the support devices pro-vided to the learners are recognized as functional. Studentsknow the relation between the support devices and their learn-ing, and know how to handle the support devices to foster theirlearning (Elen, Lowyck, & Proost, 1996). This knowledgeabilityentails, therefore, both students knowledge about the supportdevices and their self-regulating skills. Learners self-regulatingskills refer to the extent, learners are capable of monitoring andadjusting their learning process (Clark, 1990). High self-regu-lated learners are typically more capable to determine when aspecic support device may be benecial for their learningcompared to low self-regulators. Problemsmay also occur whenstudents lack knowledge about the support devices and, hencemay misinterpret the intentions behind the support devices(Winne, 1985, 2004). Winne (1985, 2004) refers to this problemas students not being calibrated to the learning environment.There is a mismatch between a learners judgment of a (featureof) a task, and the externally determined measures of (the fea-ture of) the task This was for instance, the case in a study intropical medicine (Clarebout, Elen, Lowyck, Van den Ende, &Van den Enden, 2004). Thinking aloud protocols revealed thatstudents did not use the support devices because they thoughtusing the devices would imply cheating. The program develop-ers, however, purposely included those support devices to fos-ter the acquisition of diagnostic skills. Similarly, Marek,Griggs, and Christopher (1999) revealed that students knowl-edge of a specic adjunct aid moderated the effect of the learn-ing environment. Respondents in this study indicated a weakinclination to use adjunct aids since, according to their knowl-edge, these aids required more elaborate study activities.

    3. Learners are sufciently motivated to use the support devices:support devices will only be used when learners are willing toinvest effort in the task (Salomon, 1984). In addition to the will-ingness to invest effort, students goal orientation may also playa role. In Ryan and Pintrichs study (1997) for instance, masterygoal oriented students students who are motivated to developcompetencies, to master the task believed that using supportdevices would be benecial for their well-being, and thereforethey were more likely to use them. Performance goal orienta-tion with a focus on demonstrating competencies compared

    G. Clarebout, J. Elen / Computers into others seemed to be linked to suboptimal use of supportdevices and a negative attitude towards the support devices(e.g., perceiving help-seeking support devices as a threat toa narrow focus. The encouragements seem to affect those stu-dents only that fell in between these extremes. Only for theneutral learners encouragements had positive effects.

    In view of establishing a solid research agenda on the optimiza-tion of the use of support devices, and hence the benets of inte-grating support devices in learning environments, thiscontribution discusses three studies, each dealing with a differentaspect of support device usage. In a rst study, the TechnologyManagement (TM)-study, the impact of the type and number of ad-junct aids on their use was investigated. A second study, the Drink-ing Cup study, studied the inuence of advice on the use of supportdevices, while also considering metacognition as part of studentsself-regulation skills. While in these two studies Perkins (1985)conditions were considered in the design, a third study, the Obesitystudy, explicitly addresses them in the research questions. Thethree studies all report on the use of support devices in electroniclearning environments, with non-embedded support devices. Ineach case, learners decide about the use of the devices. The overallresults of these studies together with suggestions for further re-search are described.

    2. Study 1: Technology Management study

    The Technology Management study (TM)-study focuses on theuse of different types of adjunct aids and specic learner relatedvariables. First, the study addresses the use of different types of ad-junct aids. Adjuncts aids are dened as instructional interventionsinserted in (electronic) text books in view of supporting learnersinformation processing. Adjunct aids have already been intensivelystudied (see for instance, Grabowski, 2004; Rothkopf, 1996). Re-views (Andr, 1979; Mayer, 1979) generally reveal positive results.However, positive results can only be broad about when the ad-junct aids actually induce cognitive processes (Andr, 1979), andare needed by the target group (Schnotz & Bannert, 2003). In mostof these studies the effects on learning of one particular type of ad-junct aid have been studied (e.g., Andr, 1979; Mayer, 1979). How-ever, designers and developers of instructional text typically insertmultiple and different types of adjunct aids in their material.Hence, the study described here is an attempt to gain more insightin the use of multiple and different types of adjunct aids.ones self-worth). Furthermore, mastery goal orientation seemsto increase the probability of requesting help, whereas a perfor-mance goal orientation seems to be linked to asking for theright answer (Newman, 1998; Ryan, Pintrich, & Midgley,2001). Since using support devices can be seen as a form of helpseeking, one may assume that goal orientation will also affectthe use of support devices.

    4. While Perkins (1985) focuses mainly on the functionality of thesupport devices and learner related variables, some studieshave been performed on the enhancement of the use of supportdevices (e.g., Carrier, Davidson, Williams, & Kalweit, 1986; Lee &Lehman, 1993). In these studies, students who received instruc-tional cues or encouragement to use a specic support device,used the support device more often compared to studentswho did not receive these cues or encouragement. In theirstudy, Lee and Lehman (1993) found an interaction effect withlearning style. Learners were characterized as having an activelearning style when they exhibited curiosity; initiative and awide focus when selecting information. Learners were charac-

    an Behavior 25 (2009) 804810 805The main research question in this study was: What is the com-bined impact of the number and nature of the aids on the use ofadjunct aids?

  • 2.1. Method

    2.1.1. ParticipantsParticipants were 203 students (98% female) attending under-

    graduate classes at a public university in central South Africa. Allstudents were majoring in education and volunteered to partici-pate in the study. Students could gain a limited number of extrapoints by participating in the study.

    pected that adapted advice will lead to more (adequate) use ofthe support devices, and hence to better performance.

    5 Examples + 5 Questions

    Table 2Frequency and proportional time spent (Percentage) on adjunct aids

    Type of adjunct aid Frequency Proportional time spent

    Mean SD Mean SD

    Examples 1.78 2.40 2.15 3.34Questions 1.61 2.63 1.43 2.20Figures 2.23 3.15 1.07 1.60

    Total 5.63 5.32 4.65 4.51

    806 G. Clarebout, J. Elen / Computers in Human Behavior 25 (2009) 8048102.1.2. Design, instruments, and procedureAn experimental design was used with amount of aids (ve, ten

    or fteen) and type of aids (questions, examples or gures) as inde-pendent variables (see Table 1), and use of adjunct aids as depen-dent variable (expressed in frequency of access and proportionaltime spent on adjunct aids). Additionally, learning gains werelooked at by comparing the scores on a pre-test with the scoreson a post-test.

    The instructional material was based on a chapter on technol-ogy management by Newby, Stepich, Lehman, and Russell (1996).The text was presented on a computer, distributed over 21 pages.Access to adjunct aids except for the control group was gainedby clicking on labeled buttons on the computer screen. Studentsuse of and time spent on using adjunct aids were registeredthrough log les (Access database).

    The 203 students participated in sessions, each with up to 30students. When entering the computer room, rst the pre-testwas administered, next students had to enter their name on thecomputer (for logging purposes), and could start studying the texton computer in one of eight conditions. Participants were ran-domly assigned to one of the conditions. After reading the text apost-test was administered. This session took about 50 min.

    2.2. Results

    Descriptive statistics (Table 2) show that on average, the ad-junct aids were accessed almost six times. From the average totaltime spent on reading the text, 4.65% was spent on consultingthe adjunct aids.

    A MANOVA reveals an overall effect of condition (Wilks lamb-da = .520; F(14,356) = 9.83; p 6 .05; eta2 = .28). The test of betweensubject effects shows that this effect of condition is signicant forboth dependent variables, namely frequency of access(F(7,202) = 16.42; p 6 .05; eta2 = .39) and proportional time spenton adjunct aids (F(7,202) = 16,42; p 6 .05; eta2 = .36).

    To gain more insight in how the different conditions differedfrom each other, a Waller Duncan post-hoc test was performed,which resulted in different subsets (Table 3). For the impact of con-dition on frequency of use of the adjunct aids, the control groupand the condition with ve examples belongs to a rst group withlimited frequency of use. All conditions with ve adjunct aids be-long to a second, partly overlapping, group with low to mediumfrequency of access. The third group contains two conditions with10 adjunct aids (condition with 5 questions and 5 examples, andcondition with 5 questions and 5 gures) as well as the condition

    Table 1Conditions of the technology management study

    Condition Number and type of Adjunct aids

    1 No adjunct aids2 5 Examples3 5 Questions4 5 Figures5

    678with 15 adjunct aids. A nal subset consists of a partly overlappinggroup of three conditions with high frequency of access of the ad-junct aids. All conditions in which gures are combined with oneor two other types of adjunct aids belong to this group.

    A somewhat different picture emerges when the proportion ofstudy time spent on adjunct aids is examined. A Waller Duncanpost-hoc test reveals four homogeneous subsets. The control groupbelongs to a rst group. The conditions with ve aids belong to asecond group. The third group contains conditions in which a med-ium to high proportion of study time is spent studying the adjunctaids, including the condition with questions and gures; the condi-tion with questions, gures, and examples, and the condition withquestions and examples. The group with the highest relative pro-portion of time devoted (from 7.2% to 8.34%) to the adjuncts aidsincludes the conditions where examples are combined with oneor two other adjunct aids.

    With respect to learning outcomes, it could be seen that overallstudents did learn by studying the text. The average score on thepre-test was 4. 52 (SD = 2.35) and on the post-test 7.98(SD = 2.98), which is a signicant difference (t = 17.33, p < .05).However, no overall effect was found on the learning outcomesfor the specic conditions.

    3. Study 2: Drinking Cup study

    In the Drinking Cup study, the inuence of advice on supportdevice-usage was studied. Advice aimed at making the studentsmore knowledgeable about the support devices (cf. Condition 2of Perkins). The Drinking Cup-study investigates whether these po-sitive results can also be retrieved in open-learning environments.Open-learning environments confront students with an ill-struc-tured problem (Jacobson & Spiro, 1995; Jonassen, 1997). Theseenvironments are further characterized by the inclusion of a num-ber of support devices, and ample learner control (Hannan, Hall,Land, & Hill, 1994).

    This study investigates, whether advice adapted to the learnersproblem-solving steps leads to more support device usage thannon-adapted advice (at xed moments). The rationale for thisquestion is the suggestion by Clark (1990) that both too much ortoo less support may be detrimental. Consequently, it can be ex-

    Total time on Text 360020 0 9040 0

    Total time on adjunct aids 10430 0 10480 05 Examples + 5 Figures5 Questions + 5 Figures

    5 Examples + 5 Questions + 5 Figures

  • nt on

    et 1

    HumGiven the assumption, that students self-regulation skills inu-ence students support device usage, the relationship between stu-dents use of support devices, and their self-regulation skills isstudied.

    Summarized, the research questions in the Drinking Cup-studywere: (1) Does advice encourage tool use?; (2) Does adapted ad-vice lead to more tool use then xed advice?; and (3) Does stu-dents self-regulation inuence tool use?

    3.1. Method

    3.1.1. ParticipantsOne hundred and eighty ve rst year educational sciences stu-

    dents (97% female) of a Belgian university participated. Participa-tion in this study was part of the requirements of a speciccourse in their program.

    3.1.2. Design, instruments, and procedureA 3 2 experimental design was used with condition (no ad-

    vice, adapted advice, and xed advice), and self regulation (highlow) as independent variables and the use of support devices (ex-

    Table 3Homogeneous groups with respect to frequency of access and proportion of time spe

    Frequency of access of adjunct aids

    Condition N Subs

    No adjunct aids 27 0.005 examples 27 2.225 questions 275 gures 265 examples and 5 questions 265 questions and 5 gures 285 examples, 5 questions, and 5 gures 235 examples and 5 gures 19

    ProportionNo adjunct aids 27 0.005 gures 265 examples 275 questions 275 questions and 5 gures 285 examples, 5 questions, and 5 gures 235 examples and 5 questions 265 examples and 5 gures 19

    G. Clarebout, J. Elen / Computers inpressed in frequency and proportional time) as dependent vari-ables. Also learning outcomes were studied as dependent variables.

    The learning environment consisted of an open electronic learn-ing environment. Students had to suggest and give arguments forthe best possible alternative for plastic one-way drinking cups ona music festival, by looking at the problem from a nancial, ecolog-ical and safety perspective. In other words, the environment con-fronted students with an ill-structured realistic problem. To solvethis problem, students got access to different support devices:information tools (ofcial documents; videos of stakeholders),knowledge monitoring tools (problems solving script, reportingscript), and performance support tools (calculator, note pad). Inthe xed advice condition every 7 min a support device was ran-domly selected and explained to the students, by pointing out itsfunction, and why and when this support device would be helpful.In the adapted advice condition, an evaluation was made of thestudents previous activities every 7 min. Based on those activitiesthe student was advised to use a specic support device.

    Students self-regulation were measured by means of the meta-cognition scale of Vermunts Learning Style Inventory (1992, Cron-bachs alpha = .80).

    In the rst session, LSI-questionnaires were administered. In thesecond session, students solved the drinking cup problem on acomputer. This session took place in groups of ve students, eachworking individually on a portable computer. Students could workon the problem for 1 h. Log les (Access database) were kept totrack students use of support devices. Students solution prior toconsulting any information was considered as a pre-test, whiletheir solution after working in the environments was used as apost-test measurement.

    3.2. Results

    In absence of a correlation between frequency of support deviceusage and proportional time spent on the support devices (Pearsonr = .02; p > .05), two separate ANOVAs were done. None of themrevealed a signicant main or interaction effect of condition andself-regulation.

    Follow-up analyses with the frequency of and time spent on theuse of the different support devices as dependent variables did re-veal that for the information tools signicant effects were found forthe proportion of time spent on this device, Wilks k = .765; F(14,314) = 3.21; p 6 .05; eta2 = .14. The students in the condition withadapted advice consulted more frequently the ofcial documents,while students in the xed advice condition spent most of the time

    Subset 2 Subset 3 Subset 4

    2.223.674.31

    7.278.68 8.689.13 9.13

    10.47

    2.333.133.18

    5.577.21 7.217.58 7.58

    8.34adjunct aids as a result of Waller Duncan tests

    an Behavior 25 (2009) 804810 807on the subjective information, i.e. video clips of stakeholdersexpressing their opinion on the subject.

    Table 4 also reveals that students spent most of the time on theinformation tools (Info, videos) and only a small proportion of theirtime was spent on the other support devices.

    With respect to the learning outcomes, no differences werefound between the different conditions.

    The analysis of whether students followed the provided adviceresulted in an interesting additional nding. An ANOVA yielded asignicant effect of condition, F(1,114) = 8.36; p 6 .05; eta2 = .07.Participants who received adapted advice followed signicantlymore the advice of the agent (Mean = .26; SD = .22) than did stu-dents who received xed advice (Mean = .16; SD = .13). However,the results also revealed that most of the time students did not fol-low the advice. The descriptive statistics indicated that on average20.61% (SD = 18.18%) of the advice was followed. From the 124 par-ticipants in the experimental groups, 30 of them (24.2%) did neverfollow any advice.

    4. Study 3: Obesity study

    This Obesity study addressed systematically, Perkins three con-ditions. First, the support devices functionality was studied by

  • In two conditions, students could access different support devices,namely a dictionary, instructional goals, example questions, andhelp with interpreting gures and text. In one experimental group,students also received advice on the relevance and use of the sup-port devices. Log les (Access database) were kept to track stu-dents use of the support devices. Students received 25 min toread the text after which they ware asked to complete a knowledge

    Table 4Proportional time spent on support devices

    Support device Adapted advice Fixed advice No advice

    Mean SD Mean SD Mean SD

    Calc. .03 .09 .04 .08 .03 .05Assign. .01 .01 .01 .01 .01 .02

    808 G. Clarebout, J. Elen / Computers in Human Behavior 25 (2009) 804810comparing the performance of a group of students with and with-out access to support devices. Second, in view of considering stu-dents instructional knowledgeability, students self-regulationwas measured. One condition contained advice to further increasestudents instructional knowledgeability. Prior to reading the text,all support devices were explained (in written) to the students inthis group, and suggestions were provided on when to use thesedevices during studying the text. The advice-group was expectedto use more frequently the available tools. Third, students goal ori-entation was measured.

    Summarized the following research questions were addressed:(1) Are the support devices functional to students learning?; (2)How do learner related variables (self-regulation and goal orienta-tion) inuence students use of support devices; and (3) Does ad-vice contribute to the use of support devices?

    4.1. Method

    4.1.1. ParticipantsParticipants were 216 rst year educational sciences bachelor

    students from a Belgian University (98% female). Participation inthe study was part of a specic course in their study program.

    4.1.2. Design, instruments, and procedureAn one factorial experimental design was used with condition

    (Support device + advice, support devices only, no support devices)as independent variables. Students metacognition and goal orien-tation were considered as predictors. The dependent variableswere, performance frequency of the use of support devices andtime spent on the support devices.

    In the rst session, students goal orientation and self-regula-tion skills were measured. For measuring students self-regulationthe metacognition scale of the LSI of (Vermunt 1992; Cronbachs

    Info .50 .10 .46 .15 .47 .17PO-c 04 .07 .05 .07 .04 .05R-c .02 .03 .02 .03 .04 .09Expl. .04 .02 .04 .03 .03 .02Videos .11 .08 .15 .07 .18 .11

    Calc. = calculator; Assign. = access to the assignment; Info = access to objectiveinformation; PO-c = problem-solving checklist (access to worked-out example); R-c = reporting checklist; Expl. = access to explanation on functionalities of differenttools; videos = access to subjective information (stakeholders).alpha = .77) was used, and for goal orientation, a translation of El-liots instrument, consisting of a performance orientation and mas-tery orientation scale (1999) (performance orientation: Cronbachsalpha = .81; mastery orientation: Cronbachs alpha = .77). In thesecond session, students were asked to read a computer-based texton obesity (Nederkoorn, Guerrieri, & Jansen, 2006). They were in-formed that a test with knowledge, insight and application ques-tions would be administered immediately after studying the text.

    Table 5Performance Score, Frequency and Time Spent on Support Devices

    Support devices + Advice

    Mean SD

    Performance score 0.63 0.16Frequency of support device usage 5.68 3.91Time spent on support devices 11.10 8.32test.

    4.2. Results

    With respect to the research question addressing the rst con-dition of Perkins, it was found that the two experimental groupsoutperformed the control group on the post-test, F(2,175) = 6.79;p 6 .05; eta2 = .072. This was seen as an indication that the supportdevices were functional. Comparing the two experimental groups(Table 5), the group with advice seems to have used more supportdevices, both in terms of frequency and time spent on the supportdevices, than the group without advice, but this difference is notsignicant. Interpreting the descriptive statistics for studentsself-regulation, reveals that students are average self regulators(mean = 3.55 on a scale of 16; SD = .54). The mean scores for goalorientation are 3.84 (SD = .80) for performance orientation, and4.32 (SD = .67) for mastery orientation. Students self-regulationdid not inuence tool use, but goal orientation, more specicallymastery orientation did (R2 = .15; beta = .384; t = 2.528;p 6 .05). The more students are mastery oriented, the less supportdevices are used.

    5. Discussion and conclusion

    Overall in all three studies it is striking that the amount of use ofsupport devices is limited and it seems that even with advice thatthis remains low. The nding in the DC-study that by providingadapted advice, more objective information was consulted and lesssubjective information reveals the subtle interactions.

    The three studies presented here all addressed specic issueswith respect to the use of support devices The TM-study and theDC-study suggest that the nature of the support device is relatedto its use. In the TM-study, more time was spent on the adjunctaids when examples were presented, while in the DC-study, theinformation tools were far more often consulted than the othertools. These tools might have an immediate relevance to the learn-ers, perhaps because learners are more acquainted with them andhence more knowledgeable. It even could be argued that in the DC-study, the information devices can hardly be called a support de-vice, since they provide the content of the learning material.

    With respect to self-regulation, no inuence was found in theDC- and Obesity-study for self-regulation, while there is a cleartheoretical ground for assuming this is an important variable withrespect to the use of support devices. This may be due to method-ological reasons such as:

    1. The instrument: A part of the LSI was used, given its availabilityin Dutch and the reliabilities found in previous research, but itonly measures part of students self regulation. It even can be

    Support devices Control

    Mean SD Mean SD

    0.67 0.16 0.50 0.19

    3.98 3.04 7.73 5.70

  • how to make students (instructionally) knowledgeable with

    Humargued that a survey instrument may not measure studentsself-regulation skills (see also Veenman (2005)), but rather theirintentions; and hence it may be that what students express in asurvey differentiates from their actual behavior in a learningenvironment.

    2. The sample: Looking at the means and the standard deviationsin the Drinking Cup and Obesity studies it seems that in bothstudies a rather homogeneous group was used. The distinctionbetween high and low regulators in the DC-study may conse-quently be called very arbitrary. Different results may be foundwhen a more heterogeneous population is studied.

    The Obesity-study additionally addressed goal orientation. Aneffect was found, although opposite to results in studies on help-seeking (Ryan et al., 2001). One possible explanation may be thatstudents behave differently towards human support devices andelectronic support devices, and that those students who are moreinclined to request help from a human instructional agent, maybe less inclined to use electronic support devices.

    The ndings of these three studies do raise some issues:

    1. The issue of learner control: Clear positive effects of learner con-trol on performance have not yet been found. An argument thatunderlies the idea of learner control is that through learner con-trol learning environments gets adapted to the learner: a lear-ner can use exactly that amount of support that is needed byaccessing the right support devices at the right moment. How-ever, as already indicated, this presupposes learners to be capa-ble of making adequate decisions with respect to their learningprocess. This seems not to be the case. Hence, from an instruc-tional design perspective, one may wonder, whether providinglearners with non-embedded support devices is the best solu-tion. A doubt recently raised by Mayer (2004) and by Kirschner,Sweller, and Clark (2006), who all question the effectiveness ofdiscovery or inquiry-based learning environments given thatthese environments typically involve a large amount of learnercontrol.

    2. The user aimed at: If an instructional designer has sufcient evi-dence about the functionality for a specic learner of a supportdevice, than it seems indicated to embed this support device forthis learner. This may not be indicated though in case the envi-ronment focuses on a high self-regulating learner. Such a lear-ner probably makes adequate selections. In case of poor orlimited evidence about the devices functionality, a differentdecision seems indicated. Including the device may make thetask more demanding. An additional task namely processingthe information of the support device is added. However, alearning environment is typically not designed for one particu-lar type of learner; it targets high as well as low self regulatorylearners. In such a case, the support device may be benecial foronly part of the group of learners. For high self regulators it isthen best to non-embed the support device so that they candecide themselves whether or not to use it. Leaving them incontrol of the decision process causes congruence or even con-structive friction (Vermunt & Verloop, 1999). Embedding thesupport devices for these learners would mean that an activityis taken over that they can perform themselves, which couldcause destructive friction (Vermunt & Verloop, 1999). On theother hand, for low self regulators who need the support offeredthrough the support device it is best to embed it so that theycannot but use it. Further research is needed to address the ben-ets and pitfalls of embedding or non-embedding supportdevices for different types of learners.

    G. Clarebout, J. Elen / Computers in3. Adequate use of support devices: Even if a support device is 100%functional and embedded, adequate use is still not guaranteed(Greene & Land, 2000). Students need to be knowledgeable ofrespect to the learning opportunities offered through supportdevices. One possibility may be training students in using sup-port devices (e.g., Grsel, Fischer, & Mandl, 2000). Such trainingmay precede the actual confrontation with the learning envi-ronment or may be integrated in it through direct advice onthe use of particular interventions or feedback on the use ofthe interventions. Goodyear (2000) proposes to integrateinstruction on the use in the support devices themselves. Good-year seems to argue that the phenomenon of inadequate ornon-use is an artifact of irrelevant, inauthentic learning tasks.He argues that learners have to be trained to use the interven-tions and that authentic tasks may motivate learners to use theinterventions. Another option is to design support devices insuch way that they cannot but be used optimally.

    4. Learner related variables: The unexpected nding of the inu-ence of mastery orientation on tool use compared to the nd-ings of Ryan et al. 2001) calls for further research on therelationship between goal orientation and tool use. Highly mas-tery oriented learners may specically focus on the mastery ofthe content of the text, rather than on additional informationprovided by the support devices. One may also wonder whatother learner variables need to be considered. Within the eldof management information system research perceptual vari-ables have been identied as important antecedent variablesof human behavior (Davis, Bagozzi, & Warshow, 1989). In Daviset al.s technology acceptance (TA) model it is stated that ausers behavior in an environment is determined by the usersattitude towards this environment. These attitudes in turn,are inuenced by the users perceived usefulness and the per-ceived ease of use of a technology-based environment. Simi-larly, Salomon (1984) addresses that learners expectanciesand skills with regards to an instructional medium will inu-ence the amount of invested effort, and consequently the depthof the learners processing of the learning material. One couldargue that, specied to support devices, learners perceived use-fulness and expectancies will inuence learners intention touse the support device. This will increase the probability ofactual using the support device and the mental effort investedto process the information from the support device, henceresulting in better learning.

    The overview of the three studies allows to compare results andto make stronger claims with respect to the impact of specic vari-ables on the use of support devices. The studies reveal that thephenomenon of non-use or inadequate use of instructional inter-vention is highly complex (see also, Elen and Clarebout (2005)).Looking at the three studies it is clear that overall the use of sup-port devices was low, except for the information support devicesin the DC-study. Evidently, the question on the benets of insertingsupport devices in electronic learning environments cannot be eas-ily answered and requires further research that considers task,learner and support device characteristics. A starting point couldbe the TA-model of Davis et al. (1989) as also suggested by Gerjetsand Hesse (2004). This would mean that learners attitudes are putcentral in studies on the use of support devices, and that it is as-sumed that these attitudes will determine how learners will reactto the support devices and how they will use them.

    Referencesand willing to properly use the support device. Hence, and inline with Perkins second condition, the question becomes

    an Behavior 25 (2009) 804810 809Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003). Help seeking andhelp design in interactive learning environments. Review of EducationalResearch, 73, 277320.

  • Andr, T. (1979). Does answering higher-level questions while reading facilitateproductive learning? Review of Educational Research, 49, 280319.

    Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning fromexamples: Instructional principles from the worked examples research. Reviewof Educational Research, 70, 181214.

    Brush, T., & Saye, J. (2001). The use of embedded scaffolds with hypermedia-supported student-centered learning. Journal of Educational Multimedia and

    Jacobson, M. J., & Spiro, R. J. (1995). Hypertext learning environments, cognitiveexibility and the transfer of complex knowledge. Journal of EducationalComputing Research, 12, 301333.

    Janssen, J., Erkens, G., & Kanselaar, G. (2007). Visualization of agreement anddiscussion processes during computer-supported collaborative learning.Computers in Human Behavior, 23, 11051125.

    810 G. Clarebout, J. Elen / Computers in Human Behavior 25 (2009) 804810Hypermedia, 10, 333356.Carrier, C., Davidson, G., Williams, M., & Kalweit, C. M. (1986). Instructional options

    and encouragement effects in a micro-computer concept lesson. Journal ofEducational Research, 79, 222229.

    Clarebout, G., & Elen, J. (2006). use in computer-based learning environments:Towards a research framework. Computers in Human Behavior, 22, 389411.

    Clarebout, G., Elen, J., Lowyck, J., Van den Ende, J., & Van den Enden, E. (2004).KABISA: Evaluation of an open learning environment. In A.-M. Armstrong (Ed.),Instructional design in the real world. A view from the trenches (pp. 119135).Hershey: Information Science Publishing.

    Clark, R. E. (1990). When teaching kills learning: Research on mathemathantics. InH. Mandl, E. De Corte, N. Bennett, & H. F. Friedrich (Eds.), European research in aninternational context: Volume 2. Learning and instruction (pp. 122). Oxford, NY:Pergamon Press.

    Clark, R. E., & Estes, F. (2002). Turning research into results. A guide to selecting theright performance solutions. Atlanta, GA: CEP Press.

    Davis, F. D., Bagozzi, R. P., &Warshow (1989). User acceptance of computer technology.A comparison of two theoretical models management science, 35, 9821003.

    de Jong, T. (2006). Scaffolds for computer simulation based scientic discoverylearning. In J. Elen & D. Clark (Eds.), Handling complexity in learningenvironments: Research and theory (pp. 107128). London: Elsevier Science.

    de Jong, T., & van Joolingen, W. R. (1998). Scientic discovery learning withcomputer simulations of conceptual domains. Review of Educational Research,68, 179201.

    Dillon, A., & Gabbard, R. (1998). Hypermedia as an educational technology: A reviewof quantitative research literature on learner comprehension, control, and style.Review of Educational Research, 68, 322349.

    Elen, J., & Clarebout, G. (2005). Support in learning environments: Touching thelimits of instructional design. Educational Technology, 45(5), 4447.

    Elen, J., Lowyck, J., & Proost, K. (1996). Design of telematic learning environments: Acognitive meditational view. Educational Research and Evaluation: AnInternational Journal on Theory and Practice, 2, 213230.

    Elliot, A. J. (1999). Approach and avoidance motivation and achievement goals.Educational Psychologist, 34, 169189.

    Friend, C. L., & Cole, C. L. (1990). Learner control in computer-based instruction: Acurrent literature review. Educational Technology, 30(11), 4749.

    Gagn, R. M., Briggs, L. J., & Wager, W. W. (1988). Principles of instructional design(3rd ed.). New York: Holt, Rinehart and Winston.

    Gerjets, P., & Hesse, F. W. (2004). When are powerful learning environmentseffective? The role of learner activities and students conceptions of educationaltechnology. International Journal of Educational Research, 41, 445465.

    Goforth, D. (1994). Learner control = decision making + information: A model andmeta-analysis. Journal of Educational Computing Research, 11, 126.

    Goodyear, P. (2000). Environments for lifelong learning. Ergonomics, architectureand educational design. In J. M. Spector & T. M. Anderson (Eds.), Integrated andholistic perspectives on learning, instruction and technology (pp. 118). Dordrecht,NL: Kluwer Academic Publishers.

    Grabowski, B. L. (2004). Generative learning contributions to the design ofinstruction and learning. In D. H. Jonassen (Ed.), Handbook of research oneducational communications and technology (2nd ed., pp. 719743). Mahwah, NJ:Lawrence Erlbaum.

    Grsel, C., Fischer, F., &Mandl, H. (2000). Theuse of additional information in problem-oriented learning environments. Learning Environment Research, 3, 287325.

    Greene, B. A., & Land, S. M. (2000). A qualitative analysis of scaffolding use in aresource-based learning environment involving the world wide web. Journal ofEducational Computing Research, 23, 151179.

    Hannan, M. J., Hall, C., Land, S., & Hill, J. (1994). Learning in open-ended learningenvironments: Assumptions, methods and implications. Educational Technology,34(10), 4855.

    Hannan, M., Land, S., & Oliver, K. (1999). Open learning environments:Foundations, methods and models. In C. M. Reigeluth (Ed.). Instructionaldesign theories and models. A new paradigm of instructional theory (Vol. 2,pp. 115140). Mahwah, NJ: Erlbaum.

    Hill, J. R., & Hannan, M. J. (2001). Teaching and learning in digital environments:The resurgence of resource-based learning. Educational Technology Research andDevelopment, 45(1), 6594.

    Horz, H., Winter, C., & Fries, S. (2009). Differential effects of situated prompts.Computers in Human Behavior, 25, 818828.Jonassen, D. H. (1997). Instructional design models for well-structured andillstructured problem-solving learning outcomes. Educational TechnologyResearch and Development, 45(1), 6591.

    Kirschner, P., Sweller, J., & Clark, R. E. (2006). Why minimal guidance duringinstruction does not work: An analysis of the failure of constructivist, discovery,problem-based, experiential, and inquiry-based teaching. EducationalPsychologist, 41, 7586.

    Lee, Y. B., & Lehman, J. D. (1993). Instructional cueing in hypermedia: A study withactive and passive learners. Journal of Educational Multimedia and Hypermedia, 2,2537.

    Marek, P., Griggs, R. A., & Christopher, A. N. (1999). Pedagogical aids in textbooks:Do students perceptions justify their prevalence? Teaching of Psychology, 26,1119.

    Martens, R. L., Valcke, M. M., & Portier, S. J. (1997). Interactive learningenvironments to support independent learning: The impact of discernabilityof embedded support devices. Computers in Education, 28, 185197.

    Mayer, R. E. (1979). years of research on advance organizers: Assimilation theory isstill the best predictor of results. Instructional Science, 8, 133167.

    Mayer, R. E. (2004). Should there be a three-strikes rule against pure discoverylearning? The case for guide methods of instruction. American Psychologist,59(1), 1419.

    Munneke, L., Andriessen, J., Kanselaar, G., & Kirschner, P. (2007). Supportinginteractive argumentation: Inuence of representational tools on discussing awicked problem. Computers in Human Behavior, 23, 10721088.

    Nederkoorn, C., Guerrieri, R., & Jansen, A. (2006). Leven in Luilekkerland [Living inwonderland]. De Psycholoog, 41(1), 1016.

    Newby, T. J., Stepich, D. A., Lehman, J. D., & Russell, J. D. (1996). Instructionaltechnology for teaching and learning. Designing instructionintegrating computersand using media. Englewood Cliffs, NJ: Merrill.

    Newman, R. S. (1998). Students help seeking during problem solving: Inuences ofpersonal and contextual achievement goals. Journal of Educational Psychology,90, 644658.

    Perkins, D. N. (1985). The ngertip effect: How information-processing technologyshapes thinking. Educational Researcher, 14, 1117.

    Rothkopf, E. Z. (1971). The concept of mathemagentic activities. Review ofEducational Research, 40, 325336.

    Rothkopf, E. Z. (1996). Controlled mathemagenic activities. In D. H. Jonassen (Ed.),Handbook of research for educational communications and technology(pp. 879896). New York: Macmillan Library.

    Ryan, A. M., & Pintrich, P. R. (1997). Should I ask for help? The role of motivationand attitudes in adolescents help seeking in math class. Journal of EducationalPsychology, 89, 329341.

    Ryan, A. M., Pintrich, P. R., & Midgley, C. (2001). Avoiding seeking help in theclassroom: Who and why? Educational Psychology Review, 13, 93113.

    Salomon, G. (1984). Television is easy and print is tough. The differentialinvestment of mental effort in learning as a function of perceptions andattribution. Journal of Educational Psychology, 76, 647658.

    Schnotz, W., & Bannert, M. (2003). Construction and interference in learning frommultiple representations. Learning and Instruction, 12, 141156.

    Veenman, M. V. (2005). The assessment of metacognitive skills: What can belearnerd from multi-method designs? In B. Moschner & C. Artelt (Eds.),Lernstrategien und Metakognition: Implikationen fr Forschung und Praxis[Learning strategies and metacognition: Implications for research and practice](pp. 7799). Berlin: Waxman.

    Vermunt, J. (1992). Leerstijlen en sturen van leerprocessen in het hoger onderwijs: Naarprocesgerichte instructie en zelfstandig denken [Learning style and coachinglearning processes in Higher Education]. Lisse, NL: Swets & Zeitlinger.

    Vermunt, J. D., & Verloop, N. (1999). Congruence and friction between learning andteaching. Learning and Instruction, 9, 257280.

    Williams, M. D. (1996). Learner-control and instructional technology. In D. H.Jonassen (Ed.), Handbook of research for educational communications andtechnology (pp. 957983). New York: Macmillan Library.

    Winne, P. H. (1982). Minimizing the black box problem to enhance the validity oftheories about instructional effects. Instructional Science, 11, 1328.

    Winne, P. H. (1985). Steps towards cognitive achievements. Elementary SchoolJournal, 85, 673693.

    Winne, P. H. (2004). Students calibration of knowledge and learning process:Implications for designing powerful learning environments. InternationalJournal of Educational Research, 41, 466488.

    Benefits of inserting support devices in electronic learning environmentsIntroductionStudy 1: Technology Management studyMethodParticipantsDesign, instruments, and procedure

    Results

    Study 2: Drinking Cup studyMethodParticipantsDesign, instruments, and procedure

    Results

    Study 3: Obesity studyMethodParticipantsDesign, instruments, and procedure

    Results

    Discussion and conclusionReferences

Recommended

View more >