REVIEW

The Role of Working Memory in Multimedia Instruction:Is Working Memory Working During Learningfrom Text and Pictures?

Anne Schüler & Katharina Scheiter &

Erlijn van Genuchten

Published online: 27 May 2011# Springer Science+Business Media, LLC 2011

Abstract A lot of research has focused on the beneficial effects of using multimedia, thatis, text and pictures, for learning. Theories of multimedia learning are based on Baddeley’sworking memory model (Baddeley 1999). Despite this theoretical foundation, there is onlylittle research that aims at empirically testing whether and more importantly how workingmemory contributes to learning from text and pictures; however, a more thoroughunderstanding of how working memory limitations affect learning may help instructionaldesigners to optimize multimedia instruction. Therefore, the goal of this review is tostimulate such empirical research by (1) providing an overview of the methodologies thatcan be applied to gain insights in working memory involvement during multimedialearning, (2) reviewing studies that have used these methodologies in multimedia researchalready, and (3) discussing methodological and theoretical challenges of such an approachas well as the usefulness of working memory to explain learning with multimedia.

Keywords Multimedia learning .Working memory . Cognitive Theory ofMultimediaLearning . Cognitive Load Theory . Dual-task methodology .Working memory capacity

In the last two decades, a lot of educational research has focused on the beneficial effects ofusing multimedia for learning by referring to the Cognitive Theory of Multimedia Learning(Mayer 2009) and/or the Cognitive Load Theory (Sweller et al. 1998). Both theoriessuggest that multimedia learning can be best explained by paying close attention to howinformation is processed and stored in the human mind. In particular, working memory isassumed to be crucial in multimedia learning, because all information needs to be processed

Educ Psychol Rev (2011) 23:389–411DOI 10.1007/s10648-011-9168-5

A. Schüler (*) : K. Scheiter : E. van GenuchtenKnowledge Media Research Center, Konrad-Adenauer-Strasse 40, 72072 Tübingen, Germanye-mail: a.schueler@iwm-kmrc.de

K. Scheitere-mail: k.scheiter@iwm-kmrc.de

E. van Genuchtene-mail: e.genuchten@iwm-kmrc.de

in working memory before it can be stored in long-term memory. Both, the CognitiveTheory of Multimedia Learning and Cognitive Load Theory refer to Baddeley’s workingmemory model (e.g., Baddeley 1999) as their theoretical foundation.

Despite the assumed importance of working memory for multimedia learning, only fewstudies have empirically tested how it contributes to learning from text and pictures. Weagree with Tardieu and Gyselinck (2003) that such research “is necessary in order tovalidate a theory based on load in working memory” (p. 18). Therefore, the goal of thispaper is to stimulate such empirical research by (1) providing an overview of themethodologies that allow investigating working memory components involvement duringmultimedia learning, (2) reviewing studies that have used these methodologies inmultimedia research, and (3) discussing the challenges and usefulness of such an approach.

To better understand how working memory may play a crucial role in multimedialearning, Baddeley’s model is first outlined before discussing how it has been applied tomultimedia learning.

Baddeley’s Working Memory Model

According to Baddeley (1999), working memory is composed of multiple subsystems,namely, the phonological loop, the visuo-spatial sketchpad, and the central executive. Morerecently, the episodic buffer was added (e.g., Baddeley 2000). Each subsystem has its ownlimited capacity, which enables the subsystems to act relatively independent from eachother. This implies that tasks involving different subsystems can be performed togetherequally well as separately. Conversely, there will be interference between two tasksinvolving the same subsystem. In addition, brain research has shown that the subsystemsare associated with different brain regions (for overviews, see Baddeley 1998; Smith andJonides 1997). Together, these findings imply a functional as well as neuropsychologicaldissociation between the subsystems.

The phonological loop (PL) is the subsystem that processes verbal information. Spokenwords enter the PL’s passive storage unit (i.e., the phonological store) directly, whereaswritten words have to be first converted from a visual code into an articulatory code. Thisconversion is done by the PL’s subvocal rehearsal process before the words are transferredto the phonological store.

The visuo-spatial sketchpad (VSSP) is the subsystem that processes visual and spatialinformation. The VSSP’s visual component deals with visual characteristics of objects (e.g.,shape or color; Logie 1995), whereas its spatial component deals with relational or spatialinformation and the control of movements, for example, arm or eye movements (Lawrenceet al. 2001; Logie 1995; Logie and Marchetti 1991). The VSSP is likely to be responsiblefor picture processing.

The central executive (CE) is the subsystem that is responsible for (a) monitoring andcoordinating the operation of the subsystems and linking them to long-term memory, (b)switching attention between tasks and allocating attention to stimuli, (c) assigninginformation to one of the subsystems, (d) updating and regulating working memorycontents, and (e) coding representations for their time and place of appearance (Baddeley1996; Smith and Jonides 1999).

Prior to 2000, Baddeley assumed that coordination between the two subsystems washandled by the CE; however, the CE lacked storage capacity for retaining information indifferent codes. Therefore, the episodic buffer was added to the model, which allowstemporary storage of multimodal information, and combines information from the PL and

390 Educ Psychol Rev (2011) 23:389–411

VSSP with each other and with prior knowledge (Baddeley 2000). Like the PL and theVSSP, the episodic buffer is controlled by the CE.

With respect to multimedia learning, there are a couple of implications that can bederived from Baddeley’s model. First, limitations of working memory resources shouldhave adverse effects on multimedia learning. Second, these adverse effects should dependon which subsystem is affected by the resource limitations. This implies that limitations inthe PL negatively affect the processing of verbal information, whereas limitations in theVSSP negatively affect the processing of pictorial information. Finally, limitations in theCE and episodic buffer should negatively affect the integration of verbal and pictorialinformation.

The Role of Working Memory in Theories of Multimedia Learning

Two assumptions of Baddeley’s model have been incorporated into the Cognitive Theory ofMultimedia Learning and Cognitive Load Theory; however, these theories differ in howheavily they rely on these assumptions. The distinction between a verbal and a visuo-spatial working memory subsystem is reflected in Cognitive Theory of MultimediaLearning’s assumption that humans possess an auditory/verbal channel and a visual/pictorial channel for processing multimedia materials (Mayer 2005). These channels aredistinguished on the basis of the sensory mode of representations (visual vs. auditory) andon their presentation code (verbal vs. pictorial). Mayer (2005) argues that the distinctionaccording to the sensory mode of representations is “most consistent with Baddeley’s […]distinction between the visual-spatial sketchpad and the phonological (or articulatory) loop”(p. 34). It is important to note that this interpretation is not necessarily in line withBaddeley’s model, since Baddeley distinguishes the PL and the VSSP according to theinformation’s representational code (verbal vs. pictorial) and not according to their mode, assuggested by Mayer (cf. Rummer et al. 2010). We will come back to this issue whendiscussing the explanation underlying one of the most prominent multimedia design effects,that is, the modality effect.

A second assumption based on Baddeley’s (1999) model is that the working memorysubsystems have limited resources for processing information in parallel. Accordingly,Cognitive Theory of Multimedia Learning assumes that each channel can only handle alimited amount of information simultaneously. These limitations in processing capacityprovide the main argument for how instruction can be improved by optimizing thecognitive demands that arise during studying multimedia materials.

Whereas Cognitive Theory of Multimedia Learning places equal weight on bothassumptions, Cognitive Load Theory emphasizes the limited-capacity assumption at theexpense of the dual-channel assumption. In the Cognitive Load Theory, cognitive resourcesare seen as a unitary construct that are not tied to a specific code of information, that is,verbal or visuo-spatial; instead, a distinction is made between cognitive load resulting fromthe domain’s complexity in interaction with a learner’s prior knowledge (intrinsic load) andload resulting from either harmful (extraneous load) or helpful cognitive processes(germane load; Sweller et al. 1998). According to the Cognitive Load Theory, cognitiveoverload occurs when the sum of these load types exceeds available working memorycapacity. However, it is unclear how Cognitive Load Theory’s assumptions map ontoBaddeley’s model, in which each subsystem has available its own resources and cognitiveoverload occurs only within a subsystem. Only for the modality effect, Cognitive LoadTheory explicitly distinguishes between two working memory subsystems. This distinction

Educ Psychol Rev (2011) 23:389–411 391

resembles Cognitive Theory of Multimedia Learning’s distinction and will later bediscussed in the context of the modality effect.

Neither Cognitive Theory of Multimedia Learning nor Cognitive Load Theory compriseassumptions about the role of the CE (or the episodic buffer) in multimedia learning,despite the fact that at least the Cognitive Theory of Multimedia Learning places emphasison the fact that meaningful learning occurs only when verbal and pictorial information areintegrated with each other and with prior knowledge. According to Baddeley’s model, thecoordination between the subsystems and long-term memory as well as storage ofmultimodal information is accomplished by the CE in cooperation with the episodic buffer(cf. Gyselinck et al. 2008).

To conclude, theories of multimedia learning assign a pivotal role to working memory inlearning from text and pictures; however, the link between these theories and the Baddeleymodel is loose and appears at least partly inconsistent. Therefore, it is worthwhile toinvestigate empirically how working memory is involved during multimedia learning. Todo so, reliable methods have been developed in cognitive psychology allowing to assessPL, VSSP, and CE contribution in any information-processing task. These methods areoutlined next. No established methods exist yet to examine episodic buffer involvementduring learning.

Assessing the Contribution of Working Memory Resources to Learning

According to the capacity approach to assessing working memory involvement, thecapacities of the subsystems are gauged and linked to learning outcomes to test whetherinter-individual differences in working memory capacity explain variance in learningoutcomes (Andrade 2001). If differences in the capacity of one of the subsystems areassociated with different learning outcomes, it can be deduced that this subsystem wasinvolved during learning.

In the dual-task approach, participants perform a dual task in addition to learning. Thisdual task requires information processing in one of the subsystems. If there is interferencebetween the primary learning task and the dual task—as, for instance, indicated by a drop inlearning outcomes compared with a control condition without dual task—it can be deducedthat this subsystem was involved during learning.

The Capacity Approach

Individual working memory capacity is gauged by measuring how much information can beprocessed by the subsystem(s) in question, which is denoted as the span of that subsystem.In the following, we make a distinction between simple span and complex span tasks(Carretti et al. 2009).

Simple span tasks are designed to gage the PL and VSSP capacity by measuring theamount of information that can be stored over a short period of time. They requireparticipants to recall sequences of stimuli, such as numbers or spatial configurations.They start with a short sequence (e.g., three stimuli) with three sequences of an equalnumber of stimuli (i.e., a set). After a sequence has been presented, the participant hasto recall the stimuli in the correct order. When recall performance of a set is above acertain threshold, the instructor presents the next set, with each sequence beingextended by one stimulus. When recall of a set is below a certain criterion, the task is

392 Educ Psychol Rev (2011) 23:389–411

terminated. The sequences’ length of the last correctly recalled set reflects the capacityof the subsystem.

Complex span tasks are designed to assess CE capacity. Participants are asked to storeinformation in the PL or VSSP while additionally engaging in other cognitive activities (e.g.,Daneman and Carpenter 1980). The CE is assumed to be involved during these tasks, becauseaccomplishing them requires executive functions to monitor and coordinate between multiplecognitive tasks. To gauge CE capacity, at least two complex span tasks have to beadministered, one involving the CE and PL and the other one involving the CE and VSSP.Then, latent variable analysis can be used to determine CE contribution, based on thevariance in performance shared by the two tasks. The variance unique to each task isattributed to the contribution of the two subsystems (cf. Conway et al. 2005). Table 1 givesan overview of the tests used to measure individual differences in working memory capacity.

Phonological loop One of the most frequently used simple span tasks to measure PLcapacity is the digit span task (Wechsler 1958). In this task, participants have to recallsequences consisting of randomly ordered digits between zero and nine.

Visuo-spatial sketchpad The most common simple span tasks to measure VSSP capacityare the Corsi block task for the spatial component and the Visual Pattern Test for the visualcomponent. In the Corsi block task (Milner 1971; Vandierendonck et al. 2004), participantshave to remember spatial sequences. The instructor taps cubes that are arranged irregularlyon a board in fixed sequences. After presentation, the participant has to recall the sequencesby tapping the same cubes in the same order.

Table 1 Tasks to measure working memory capacity

Task Working memorysubsystem

Reliability Validity Time(min)

References

Digit span PL .80 to .91 .46 to .47 15 Wechsler (1958), Spreen andStrauss (1998), and Watersand Caplan (1996)

Corsi block VSSP .38 .27 to .35 15 Milner (1971), Spinnler andTognoni (1987), and DellaSala et al. (1999)

VPT VSSP .73 to .75 .27 to .35 15 Della Sala et al. (1997, 1999)

Reading spantask

CEPL .70 to .80 .23 to .47 15 Daneman and Carpenter (1980),Conway et al. (2005), Kaneet al. (2004), and Waters andCaplan (1996)

Listening span CEPL .77 .52 to .77 20 Daneman and Carpenter (1980)and Lehman and Tompkins (1998)

Operationalspan

CEPL .80 .40 to .73 15 Turner and Engle (1989), Kaneet al. (2004), and Conwayet al. (2005)

Countingspan task

CEPL .77 .47 –a Engle et al. (1999) and Kaneet al. (2004)

Spatial spantask

CEVSSP .72 to .83 .83 –a Shah and Miyake (1996) andWechsler (1997)

a No information available

Educ Psychol Rev (2011) 23:389–411 393

The Visual Pattern Test, developed by Della Sala et al. (1997), gauges the capacity of thevisual part of the VSSP by measuring recall performance of abstract visual patterns. Thesepatterns are presented in two-dimensional matrices, in which half of the cells are filledblack. After presentation, participants have to recall the pattern by marking the black cellsin an empty version of the same matrix.

The correlations between the Corsi block task and the Visual Pattern Test are low (Logieand Pearson 1997). Similarly, Della Sala et al. (1999) showed interference between theVisual Pattern Test and a visual dual task but little interference between the Visual PatternTest and a spatial dual task. They found the opposite pattern for the Corsi block task. Thesefindings indicate that the Visual Pattern Test and the Corsi block task gauge the capacitiesof independent visual and spatial components of the VSSP.

Central executive Examples of complex memory tasks that involve the CE and the PL(CEPL) are the Reading Span Task (with written stimuli) and the Listening Span Task (withspoken stimuli) constructed by Daneman and Carpenter (1980). In these tasks, participantsverify unrelated sentences while remembering the last word of each sentence in the set. TheCEPL capacity corresponds to the maximum number of correctly recalled final words whilevalidating sentences. Both tasks show similar correlations with reading and listeningcomprehension, indicating that general language processes instead of specific reading orlistening processes are addressed by these tasks.

A similar CEPL measure is the Operational Span Task (Turner and Engle 1989). TheOperational Span Task requires participants to solve mathematical operations whileremembering words presented after each equation. At the end of a set, participants recallthe presented words.

Finally, CEPL capacity can be gauged by the Counting Span Task. This task involvescounting shapes or objects (e.g., blue circles) on a display while ignoring other objects (e.g.,blue squares) and storing the total number for later recall. After the presentation of a set,participants have to repeat the stored numbers in the correct order (Engle et al. 1999).

The Spatial Span Task (Shah and Miyake 1996) is the only complex span task thatinvolves the CE and the VSSP (CEVSSP). In this task, participants have to mentally rotateletters to decide whether or not these letters are mirrored. In addition to this verificationtask, participants have to recall the angles of rotation (e.g., 0°, 45°, 90°, etc.) at the end ofeach set of letters. However, it can be argued that the recall of angles, which are numbers,also involves the PL.

The Dual-Task Approach

Three kinds of dual tasks can be differentiated: secondary tasks, interference tasks, andpreload tasks (Andrade 2001; Cocchini et al. 2002). The differentiation into three types oftasks is based on (a) the requirements imposed onto the participant, that is, active versuspassive processing of information related to the dual task and (b) the point in time the dualtasks are presented, that is, before or during the main task.

The most common dual-task paradigm involves the use of a secondary task in additionto a primary task (e.g., multimedia learning). The secondary task is specifically designed totap the resources of one working memory subsystem (Andrade 2001). If both tasks rely onthe same subsystem, their simultaneous performance will yield interference and as aconsequence, performance on either one or both tasks will decrease compared with a

394 Educ Psychol Rev (2011) 23:389–411

control condition, in which participants perform the two tasks separately. If there is nodecrease in performance, it can be concluded that the primary task is processed in adifferent subsystem than the secondary task. Less common variants of the dual-taskparadigm include the use of interference tasks and preload tasks. Interference tasks are tasksin which participants are exposed to irrelevant stimuli that are assumed to get access toworking memory automatically. Therefore, presenting these stimuli interferes with theprimary task and hence reduces performance (Andrade 2001). Preload tasks requireparticipants to keep information that is presented before the primary task in workingmemory while trying to encode the information of the primary task (Cocchini et al. 2002).Thus, when performance in the primary task and/or preload task performance are impairedcompared with a control condition, it can be concluded that the primary task and thepreload task require the same resources.

Whereas secondary tasks and preload tasks require active rehearsal of information by theparticipant, irrelevant stimuli are assumed to get access automatically to the respectiveworking memory subcomponent. Active rehearsal and automatic access are both assumedto load the respective working memory subsystem.

Phonological loop Articulatory suppression, irrelevant speech, or verbal preload tasks canbe used to examine PL involvement. Articulatory suppression is a secondary task to loadthe PL by requesting participants to articulate syllables, words, or numbers (Murray 1967).The task disturbs subvocal rehearsal and recoding of written information that may berequired for the primary task (Gathercole and Baddeley 1993).

To examine PL involvement with an interference task, irrelevant speech can be presented(e.g., Colle and Welsh 1976). According to Baddeley (1999), effects of irrelevant speech onverbal tasks cannot be attributed to distraction, because presenting music without wordsdoes not affect performance in the primary task in the same way. To examine PLinvolvement with a verbal preload task a verbal stimulus, for example, a list of words, ispresented before the primary task and has to be recalled after having worked on the primarytask (e.g., Cocchini et al. 2002).

Visuo-spatial sketchpad Dual tasks such as spatial tapping, interference tasks, and visuo-spatial preload tasks can be used to measure VSSP involvement. Spatial tapping is asecondary task that is assumed to address the spatial component of the VSSP. In this task,participants have to continuously conduct specific movements, which are known to becontrolled by the spatial component of the VSSP (e.g., pressing buttons on a hiddenkeyboard: Della Sala et al. 1999; foot tapping: Miyake et al. 2004). Spatial tapping hasbeen shown to interfere with spatial tasks (e.g., Farmer et al. 1986), but not with visualtasks (Logie and Marchetti 1991). Therefore, when spatial tapping interferes with theprimary task, it can be concluded that this task is processed in the spatial component of theVSSP. To measure visual VSSP involvement, no secondary task exists to our knowledgebut only interference or preload tasks.

To examine visual VSSP involvement with an interference task, irrelevant visual stimuliare presented (e.g., McConnell and Quinn 2000). Quinn and McConnell (1996) introducedthe Dynamic Visual Noise technique, in which they showed learners dots, which changedcontinuously and randomly between black and white. They demonstrated that thistechnique interfered with visual mnemonics, but not with a spatial task. Static VisualNoise did not show these interference effects (see McConnell and Quinn, experiment 1). To

Educ Psychol Rev (2011) 23:389–411 395

examine VSSP involvement with the use of a visuo-spatial preload task, a visuo-spatialstimulus, for example a visual pattern, is presented before the primary task and has to berecalled after the primary task (e.g., Kruley et al. 1994).

Central executive To examine CE involvement, random generation tasks are used. Randomgeneration tasks are secondary tasks, in which participants generate random sequences bynaming letters or numbers or by tapping in a random order (e.g., Baddeley et al. 1998).Attentional processes of the CE are required in this task, because participants have todeliberately avoid stereotypical sequences, such as 1–2–3 or A–B–C. However, as randomgeneration tasks also load the VSSP or the PL (e.g., because letters have to be verbalized),impaired performance can also indicate interference in one of these subsystems. Therefore,similar to the measurement of individual working memory capacity, at least two randomgeneration tasks should be used, loading either the PL or the VSSP. CE contribution canthen be determined by means of latent variable analysis. Finally, no established interferencetasks or preload tasks for the CE exist to our knowledge.

Advantages and Disadvantages of Capacity and Dual-Task Approach

Implementation The capacity approach is convenient to assess working memory sub-components involvement because capacity measures are easy to administer before or afterthe actual experiment. However, gauging capacity within the same time slot as theexperiment may prolong the experiment substantially, thereby causing fatigue and lack ofmotivation in the participants. This may become problematic especially when assessing CEinvolvement, because two complex span measures are required for unambiguousinterpretation of the data (i.e., a CEPL and a CEVSSP task).

The dual-task approach is more challenging to implement. The first challenge is thatbaseline performance of the primary task should be gauged (i.e., the learning task withoutthe dual task). In many basic cognitive psychology experiments this can be accomplished ina within-subjects design. However, with multimedia learning materials, usually nosufficiently comparable and therefore suitable stimuli exist that can be sued for a baselineassessment. Hence, a between-subjects design has to be used, in which for everyexperimental condition investigated in the study two versions exist, one with and onewithout dual task. Not only does a between-subjects design increase error variance becauseof individual differences in responding to both, the primary and dual task, but also thenumber of participants increases in such a design.

The second challenge is that an adequate dual task should be selected. It should bepossible to perform the primary task and the secondary task at the same time. For instance,a spatial finger-tapping task cannot be used when the hands are required to provide inputduring learning. In these cases, interference or preload tasks are better options.

Participant sample There are no specific requirements about sample size arising from thecapacity approach unless the aim is to determine CE capacity based on latent variableanalyses. In this case, large sample sizes are required (cf. Conway et al. 2005). However,the capacity approach requires a sufficient amount of inter-individual variance with regardto participants’ working memory capacities. If this is not the case, it will be unlikely that arelationship between working memory capacity and learning outcomes is found. Especially

396 Educ Psychol Rev (2011) 23:389–411

with rather homogenous samples of, for example, university (psychology) students thisproblem is likely to occur. The dual-task approach, on the other hand, makes no specificpresumptions about sample characteristics; however, it may lead to a considerable increasein sample size necessary to determine baseline performance.

Analysis and Interpretation Data resulting from the capacity approach are easy to analyze,thereby providing a readily estimate of working memory components involvement.However, it is important to note that these data are correlative in nature. This implies thatit is possible that a third variable (e.g., intelligence) underlies the observed relationship.Therefore, ideally more than one capacity measure should be used. If correlationsbetween the capacity measures and learning outcomes are found (i.e., pointing towarddissociations between, for instance, PL and VSSP), an involvement of the underlyingworking memory component will be more likely, because if results are due to a thirdfactor, it should influence all correlations between individual working memory measuresand learning outcomes in a similar way (cf. Conway et al. 2003; Salthouse and Pink2008).

Moreover, the capacity approach relies on the assumption that working memory capacityconstitutes a relatively stable construct (i.e., a personal trait) that has an effect on processingirrespective of the situation. Hence, it presupposes that inter-individual capacity differencesare large enough to affect learning outcomes and that effects are not overridden by othervariables (e.g., prior knowledge). However, Baddeley (1999, p. 54ff.) acknowledges that,for instance, differences in PL capacity play hardly any role in accounting for differences inreading comprehension of adult readers, even though all information has to pass the PL instandard reading tasks. However, the majority of adult readers has much experience ofreading so that lower-level skills and capabilities are unlikely to pose any obstacles; instead,higher-level skills such as the ability to draw inferences based on prior knowledge willbecome more influential.

The dual-task approach is a more direct approach to measuring working memorycomponents involvement than the capacity approach, because the dual-task approachassesses which aspects of the experimental variations interfere with processes located inspecific subsystems. Thus, decreases in task performance can be unambiguously tracedback to interference in working memory and hence be interpreted in a causal manner.Moreover, as the dual-task approach focuses on how working memory handlessituational demands depending on how many resources are already claimed by otherprocesses, it may also be more sensitive and allows for a more fine-grainedmeasurement of working memory components involvement. In particular, the exacttime course of working memory components involvement can be determined over timeand as a function of the processes conducted to accomplish the primary task. Therefore,the dual-task approach is tied more closely to the cognitive processes occurring, forinstance, during multimedia learning.

To conclude, there are important differences between the validity of the two approaches.Whereas findings from the dual-task approach can be attributed unambiguously to theunderlying working memory system, findings from measuring individual working memorycapacity have to be interpreted with care, especially if only the capacity of one subsystemhas been gauged. On the other hand, individual difference measures are easy to apply;hence, despite their potential drawbacks they are often used in studies in which workingmemory components involvement is not the main research question.

Educ Psychol Rev (2011) 23:389–411 397

Assessing Working Memory Involvement During Multimedia Learning

In this section, we address our second goal of this review by discussing studies thatexamined the general contribution of working memory to multimedia learning. Moreover,we review studies that examined whether working memory components are involved inspecific multimedia design effects, namely, the modality effect, to see whether theirempirical evidence is in line with what to be expected from multimedia theories.

The literature search was conducted by searching PsychInfo and ERIC for the keywords“working memory,” “multimedia,” “(learning with) text and pictures,” “working memorycapacity,” “dual task,” and “individual differences” and examinations of references ofindividual journal articles. Only studies that used at least one of the before-mentioned,established measures were included into the review, whereas studies investigating workingmemory components involvement using other types of measurements were excluded (withthe exception of one study by Brünken et al. 2002, see below).

General Contribution of Working Memory to Multimedia Learning

Evidence from Studies Using the Capacity Approach One of the few researchers, who haveexamined the influence of working memory capacity on learning using simple spanmeasures, are Gyselinck and colleagues (Gyselinck et al. 2002; Gyselinck et al. 2000).Gyselinck et al. (2000) presented written texts about physics to learners (e.g., staticelectricity) and varied between subjects whether or not a static picture accompanied thetext. They divided learners into groups with either high or low VSSP capacity based ontheir performance on the Corsi block task. To ensure that PL capacity was comparableacross groups, the digit span task was also administered. Learners with high VSSP capacitybenefited from static pictures whereas learners with low VSSP capacity did not. Thisindicates that the VSSP is involved during static picture processing and that sufficientVSSP capacity is required to learn from pictorial representations.

In a follow-up study, Gyselinck et al. (2002, experiment 1) again varied between subjectswhether or not written texts were accompanied by static pictures. Again, learners with highVSSP capacity benefited more from static pictures than learners with low VSSP capacity.Furthermore, learners with high VSSP capacity were strongly disturbed by a spatial tappingtask performed during learning compared with low-capacity learners. A possibleexplanation is that learners with high VSSP capacity relied more on pictures than learnerswith low VSSP capacity and, therefore, were more disturbed by the spatial tapping task,leading to impaired learning outcomes.

Gyselinck et al. (2002, experiment 2) investigated PL involvement during learning witheither only written text or a (labeled) static picture. They compared learners with either highor low PL capacity but equal VSSP capacity. Learners with high PL capacity performedbetter than learners with low PL capacity in the text-only condition, whereas there were nocorresponding differences in the picture-only condition. This indicates that as expected, thePL is involved during text processing but not in picture processing. Furthermore, anarticulatory suppression task disturbed text processing of learners with high PL capacity toa large extent, but had no effect on learners with low PL capacity. Again, a possibleexplanation is that learners with high PL capacity relied more on the text than learners withlow PL capacity and, therefore, had less PL capacity available to perform the articulatorytask, leading to impaired performance. On the other hand, a spatial tapping task had no

398 Educ Psychol Rev (2011) 23:389–411

influence on text processing in both the high and low PL capacity groups, indicating thatthe VSSP was not involved during text processing.

A study using both simple and complex span measures was conducted by Pazzaglia etal. (2008). They asked Italian middle school students to learn about Germany’s geographyfrom a hypermedia system. After learning, the authors assessed the students’ visuo-spatialmental representation of the hypermedia structure (map recognition test) and tested whetherinformation was memorized and correctly integrated (semantic test). The presentationconsisted of spoken text, written text, and pictures. Participants performed two simple spantasks, the digit span and Corsi block task, and two complex span tasks, the Listening Spanand Dot Matrix Task. In the Dot Matrix Task, participants first verified a visual matrixequation and then viewed a 5×5 matrix containing a dot. After a series of these equationsand matrices, participants marked all dots in a single answer 5×5 matrix. This task hadbeen designed by the authors for this study and was assumed to assess the CEVSSP. Theresults showed that Listening Span Task performance correlated with semantic knowledgeacquisition, whereas Dot Matrix Task performance did not. Instead, the role of the CLVSSPclearly emerged in the map recognition test assessing the visuo-spatial mental represen-tation of the hypermedia structure. Neither digit span nor Corsi block task performance wasrelated to map and semantic test performance, which might be because the dependentvariables were not pure verbal or visuo-spatial memory measures, but required participantsto connect information units into a coherent, global structure. This might explain why onlyCEVSSP and CEPL capacity correlated significantly with learning outcomes.

Only one complex span task was used in studies by Austin (2009, experiment 2) andDoolittle et al. (2009). Participants were assigned to one of three groups: animation andwritten text, animation and spoken text, or animation, spoken, and written text. Participantslearned about lightning (Austin) or the working of a pump (Doolittle et al. 2009) and wereafterwards tested using a recall and transfer test. In both studies, CEPL capacity was gaugedusing the Operational Span Task. In Austin’s study the CEPL predicted a significantproportion of the variance in transfer test scores. Doolittle et al. (2009) used an extremegroup design. Participants with high CEPL capacity outperformed low-capacity participantson both tests. These results imply that the CEPL contributed to multimedia learning.Interestingly, there were no interactions between CEPL span and the three multimediaconditions, indicating that the CEPL was involved to the same extent in all three conditions(for similar results, see Doolittle and Mariano 2008; Lusk et al. 2009).

Sanchez and Wiley (2006) used a Reading Span and an Operational Span Task toinvestigate whether CEPL capacity affects the effectiveness of adding a picture to improvetext comprehension. They assigned participants to a text-only group, a group with text andconceptually relevant pictures, or a group with text and irrelevant (i.e., seductive) pictures.The authors computed a composite score from the two CEPL tasks and used this score toform extreme groups. Only text comprehension of learners with low CEPL capacity sufferedfrom presenting seductive pictures. A possible explanation is that high-capacity learners arebetter able to control their attention and ignore irrelevant pictures. This was confirmed byeye-tracking data showing that learners with high CEPL capacity spent less time looking atirrelevant pictures.

Dutke and Rinck (2006) used the Reading Span and the Spatial Span Task to investigatehow individual differences affect learning of spatial arrangements, which consisted of fiveobjects (five words or five icons) located at five positions in a 2×3 matrix. For example,one spatial arrangement consisted of the five words strawberries, apples, banana, pear, andpineapple, which were located on five different positions within the 2×3 matrix.Importantly, the complete spatial arrangement was never shown to participants but had to

Educ Psychol Rev (2011) 23:389–411 399

be inferred from four adjacent pairs of objects presented on their specific positions withinthe matrix. As dependent variable, the authors used the time to verify whether completearrangements or object pairs depicted spatial relations that had been indirectly shown. Theydivided participants into four groups according to their CEPL and CEVSSP capacity (i.e., lowCEPL–low CEVSSP, high CEPL–low CEVSSP, etc.). The results showed that integratingobjects (i.e., words or icons) into a complete arrangement was most difficult for participantswith low CEPL and low CEVSSP indicating that the CE plays a role in (mentally) integratinginformation, regardless of the involved subsystem. Furthermore, the authors expected that itwould take longer to verify spatial arrangements containing words than verifying spatialarrangements containing icons. The results showed that low CEVSSP capacity led toimpaired performance in both cases, probably because of the spatial arrangement of theicons and words; however, low CEPL capacity impaired performance only with spatialarrangements containing words. Thus, the processing of spatial arrangements irrespective ofwhether they contain words or icons seems to rely on the CE and the VSSP, whereas theprocessing of spatial arrangements with words seems to additionally rely on the PL.

To summarize, CEPL as well as CEVSSP measures indicate CE involvement duringconnecting and integrating information, but show diverse results about the processing ofverbal and pictorial information. In accordance with the theoretical assumptions CEPL

measures are involved during verbal information processing, whereas CEVSSP measures areinvolved during pictorial information processing (and not vice versa). These specificcorrelations indicate that the relationships between measures and learning outcome canlikely be traced back to the underlying subsystem and not to a third variable. The role of thePL and VSSP capacity as measured by simple span tasks is less clear, because only a fewstudies have been conducted that addressed this issue. Whereas Gyselinck et al. (2000;2002) showed specific correlations between PL and VSSP capacity and text and pictureprocessing, respectively, Pazzaglia et al. (2008) observed no relationships, which might,however, also be explained by the dependent variables they used. Table 2 gives anoverview over the empirical evidence for the contribution of working memory inmultimedia learning by measuring its capacity.

Evidence from studies using the dual-task approach One of the first studies using the dual-task methodology in multimedia learning was conducted by Kruley et al. (1994). Theyaddressed the question whether the VSSP and the PL are involved during understandingmultimedia material using a preload design. In two experiments, they varied within subjectswhether the participants had to keep visuo-spatial or verbal information in working memoryduring learning (preload condition) or not (control condition). Furthermore, in bothexperiments they varied within subjects whether or not static pictures accompanied auditorydescriptions about causal systems. To investigate the VSSP involvement in multimedialearning (experiment 1), participants in the preload condition had to remember the locationof dots presented in 2×2 matrices in working memory during learning. A matrix waspresented before each sentence. Learning (i.e., listening to a sentence) was interrupted aftereach sentence and a second matrix was presented. Participants had to decide whether thismatrix matched the matrix shown earlier. In the control condition, participants saw anempty matrix before each sentence; after each sentence, they saw a matrix and decidedwhether the dots in the matrix were above the centerline of the grid. Thus, no maintenanceof visuo-spatial information was required during learning. The authors expectedinterference between processing of pictures and simultaneously retaining the visuo-spatialmatrix, because they assumed that both tasks would load the VSSP. This hypothesis wasconfirmed for performance in the visuo-spatial matrix task. When no pictures but only

400 Educ Psychol Rev (2011) 23:389–411

spoken texts were presented, there was no interference. This indicates that both pictures andmatrices were processed in the VSSP, but not text.

To investigate PL involvement during multimedia learning (experiment 4), participantsretained digits during learning in the preload condition. In the preload condition, four digitswere presented before each sentence. After the presentation of a sentence, participantsverified whether the order of two test digits was the same as in the original sequence. In thecontrol condition, the test digits and the original sequence were shown between thesentences, so that participants were not required to retain the original sequence. Asexpected, the verbal preload task impaired text comprehension and performance in the digittask in the text only as well as in the text-picture condition, indicating that the text and theverbal preload task were processed in the PL. Furthermore, as the presentation of a picturehad no specific influence on the performance in the digit task, it can be concluded thatpictures were not processed in the PL.

Gyselinck et al. (2000) conducted a similar study with written instead of spoken text. Inaddition to measuring individual working memory capacity and varying picturepresentation between subjects (see prior section), they varied within subjects whetherparticipants had to keep visuo-spatial information or verbal information (preloadconditions) or both (control condition) in working memory during learning. To load theVSSP, participants retained matrices during learning, whereas to load the PL theymemorized nonwords. In the control condition, participants retained the position of threenonwords within a 2×2 matrix during learning. In this study, PL and VSSP involvement inmultimedia learning using the dual-task methodology was not confirmed. A likely

Table 2 Empirical evidence for the contribution of working memory in multimedia learning by measuringworking memory capacity

Verbal information Pictorial information Connecting/integratinginformation

PL Involvement in processingwritten and spoken texts(Gyselinck et al. 2002)

No involvement in processingstatic pictorial information(Gyselinck et al. 2002)

–

VSSP No involvement in processingwritten and spoken texts(Gyselinck et al. 2000;Gyselinck et al. 2002)

Involvement in processingstatic pictorial information(Gyselinck et al. 2000;Gyselinck et al. 2002)

–

CEPL Involvement in spoken andwritten text processing(Austin 2009; Doolittle et al.2009; Dutke and Rinck 2006;Pazzaglia et al. 2008)

No involvement in theprocessing of static pictorialinformation (Dutke and Rinck2006)

Involvement in connecting andintegrating information(Austin 2009; Doolittle et al.2009; Dutke and Rinck 2006;Pazzaglia et al. 2008)Involvement in controllingattention (Sanchez and Wiley2006)

CEVSSP No involvement in written textprocessing (Dutke and Rinck2006; Pazzaglia et al. 2008)

Involvement in the processingof static pictorial information(Dutke and Rinck 2006) TheCEVSSP predictedperformance in a maprecognition test (Pazzaglia etal. 2008)

Involvement in connecting andintegrating information(Dutke and Rinck 2006)

Educ Psychol Rev (2011) 23:389–411 401

explanation is that both subsystems were heavily loaded in the control condition, whichrequired information processing in both the PL and VSSP.

Gyselinck et al. (2002) used secondary tasks instead of preload tasks. In addition to thebetween-subject picture presentation (see prior section), they varied within subjects whetherparticipants performed a spatial tapping task, an articulatory suppression task, or nosecondary task during learning. The results confirmed the hypothesis that picture processinginvolves the VSSP and text processing the PL. Specifically, the spatial tapping taskimpaired comprehension only when texts were accompanied by pictures, but not when onlytexts were presented. On the other hand, the articulatory suppression task impaired learningwithin the text-only and the text-picture conditions to a similar extent, without diminishingthe advantage of picture presentation.

Whereas the above mentioned studies concentrated on causal tasks (e.g., the functioningof a volcano), Brunyé et al. (2006) examined whether working memory components arealso involved during learning procedures (e.g., assembling Kinder Egg™ toys). They variedpresentation format between subjects, with a written text only, a static picture only, and awritten text-static picture group. Furthermore, the authors manipulated between groupswhether participants performed a spatial tapping task (VSSP), an articulatory suppressiontask (PL), a random generation articulatory task (CEPL), or a random generation tappingtask (CEVSSP) during learning. In the control group, participants learned without secondarytask. As in Gyselinck et al. (2002), there was interference between spatial tapping andpicture processing as well as between articulatory suppression and text processing,indicating that pictures were processed in the VSSP and text was processed in the PL.Additionally, learners studying text and picture suffered more from the random generationtasks than the text-only or picture-only groups, indicating that the CE plays an importantrole in multimedia learning, probably because CE resources are needed to integrate textsand pictures.

So far, only dual-task studies have been reported that examined working memorycomponents involvement during learning with text and static pictures. Nam and Pujari(2005) investigated whether the VSSP is also involved when dynamic pictures arepresented by varying whether animations, static pictures, or no pictures accompanied awritten text describing technical systems (e.g., the functioning of a fridge). To examineVSSP involvement, learners conducted a finger-tapping task during learning. No secondarytask was performed in the control groups. The interaction between presentation type andsecondary task showed that learning was more impaired when combining spatial tappingwith either dynamic or static pictures than without pictures. This implies that both static anddynamic pictures are processed in the VSSP. It must be noted, however, that spatial tappingalso impaired performance in the text-only condition, indicating that text is also processedin the VSSP under specific circumstances (see for example, De Beni et al. 2005).

To summarize, the studies using the dual-task methodology confirm the assumption thatthe PL and VSSP are involved during multimedia learning: The PL is involved duringwritten and spoken text processing, but not in the processing of static and dynamic pictures;the VSSP is involved during static and dynamic picture processing, but not during theprocessing of written and spoken text. Only one study by Brunyé et al. (2006) investigatedCE involvement during multimedia learning using the dual-task methodology. The resultsconfirm the assumption that the CE is involved during connecting and integratingmultimedia information (see Table 3 for an overview).

Accordingly, both types of methodologies confirm the assumption that the differentworking memory subcomponents have specific responsibilities during multimedia learning.This indicates that information-processing theories, such as Baddeley’s working memory

402 Educ Psychol Rev (2011) 23:389–411

model (Baddeley 1999), can be used to derive hypotheses about the processing ofmultimedia materials as has been suggested by Cognitive Theory of Multimedia Learningand Cognitive Load Theory. However, as discussed earlier these theories appear to be onlyloosely linked to Baddeley’s model. Therefore, it is unclear whether their explanations thatare based on working memory processes also hold when looking at more specificmultimedia design effects. One effect for which assumptions about its cause refer toworking memory processes, is the modality effect, that is, the finding that combiningspoken text with pictures yields better learning outcomes than combining written text withpictures (for an overview, see Ginns 2005).

Contribution of Working Memory to the Modality Effect

According to Cognitive Theory of Multimedia Learning and Cognitive Load Theory, themodality effect can be explained by assuming that in the initial processing stages inworking memory, written texts and pictures compete for the same resources in the visualpart of the visual/pictorial channel (i.e., the VSSP), because both are presented visually.However, with spoken texts, texts are processed in the auditory part of the auditory/verbalchannel (i.e., the PL), whereas pictures are processed in the visual part of the visual/pictorial channel. As discussed before, this assumption is not consistent with Baddeley’smodel (Baddeley 1999), which suggests that both spoken and written texts are solelyprocessed in the PL (i.e., unless they contain visuo-spatial information, cf. de Beni et al.2005). The aforementioned methodologies make it possible to shed light on the workingmemory explanation of the modality effect: according to Baddeley (1999) the degree to

Table 3 Empirical evidence for working memory involvement during multimedia learning by applying dual-task methodology

Verbal information Pictorial information Connecting/integratinginformation

PL Involvement in processing writtenand spoken texts (Brunyé et al.2006; Gyselinck et al. 2002;Gyselinck et al. 2008; Kruley etal. 1994)

No involvement in processingstatic pictorial information(Brunyé et al. 2006; Gyselincket al. 2002; Kruley et al. 1994)

No involvement inconnecting andintegrating information(Brunyé et al. 2006)

VSSP No involvement in processingwritten and spoken texts (Brunyéet al. 2006; Gyselinck et al.2002; Gyselinck et al. 2008;Kruley et al. 1994) Involvementin processing written and spokentexts (Nam and Pujari 2005)

Involvement in processing staticand dynamic pictorialinformation (Brunyé et al. 2006;Gyselinck et al. 2002; Gyselincket al. 2008; Kruley et al. 1994;Nam and Pujari 2005)

No involvement inconnecting andintegrating information(Brunyé et al. 2006)

CEPL – – Involvement inconnecting andintegrating information(Brunyé et al. 2006)

CEVSSP – – Involvement inconnecting andintegrating information(Brunyé et al. 2006)

Educ Psychol Rev (2011) 23:389–411 403

which the VSSP is involved during multimedia learning should be the same for spoken andfor written text, whereas according to Cognitive Theory of Multimedia Learning andCognitive Load Theory the VSSP should be involved more strongly in multimediapresentations containing written rather than spoken text. Thus, there should be strongerinterference between dual tasks and multimedia learning in case of written than in spokentext.

To test this assumption, Brünken et al. (2002) asked students to learn from either awritten or a spoken multimedia presentation, while responding as fast as possible whenevera letter presented at the top of the computer screen changed color (dual task). Unfortunately,this dual task is likely to induce higher demands related to early (sensory) visual attentionprocesses, because it requires visually monitoring the letter, and hence does not allowinvestigating working memory involvement in an unambiguous manner. Accordingly andunsurprisingly, this dual task interfered with multimedia learning especially in conditionswith written instead of spoken text, because with written text, visual attention is requirednot only for processing the pictorial but also the verbal input (cf. Cierniak et al. 2009, for adiscussion of this problem). Furthermore, this kind of dual task does not allow fordistinguishing between the load within either the PL and VSSP, which is, however,necessary to validate the specific assumptions made by Cognitive Theory of MultimediaLearning and Cognitive Load Theory. The same is true for studies conducted by Brünken etal. (2004) and Seufert et al. (2009), which is why we do not describe them in detail here.

A more adequate test of the working memory explanation of the modality effect comesfrom a study by Gyselinck et al. (2008). The authors presented static pictures to learnersand varied between subjects whether the pictures were accompanied by either spoken orwritten text. Within subjects, they varied whether learners performed a spatial finger-tapping task, an articulatory suppression task, or no secondary task during learning. Theresults showed that articulatory suppression impaired learning compared with the controlcondition, which indicates that both written and spoken texts are processed in the PL.Additionally, the spatial tapping task impaired learning with written and spoken multimediamaterials to the same extent, possibly because learners in both conditions processed staticpictures. Most importantly, the authors found no interference between written text and thespatial tapping task, which is not in line with the assumption that written text is initiallyprocessed in the visual/pictorial channel as proposed by the Cognitive Theory ofMultimedia Learning and Cognitive Load Theory. For alternative explanations of themodality effect that are based on sensory memory rather than working memory processes,the reader is referred to Rummer et al. (2010, 2011) and Schmidt-Weigand et al. (2010).

The modality effect provides an excellent example of how working memorymethodologies can be used to test specific assumptions about the causes underlyingmultimedia design effects. To do so, it is vital that these explanations are described at alevel that allows a clear prediction to be made about underlying working memorymechanisms, which, with the exception of the modality effect, however, hardly exist.

Discussion

Multimedia theories explain multimedia learning against the background of Baddeley’sworking memory model (Baddeley 1999). Our overview of studies that explicitlyinvestigated working memory contribution to multimedia learning suggests that there isconverging evidence that the PL and the VSSP have different responsibilities duringprocessing multimedia instruction, with the PL being responsible for processing verbal

404 Educ Psychol Rev (2011) 23:389–411

information—irrespective of the modality in which the text is presented—and with theVSSP being responsible for processing pictorial information such as static or dynamicvisualizations (e.g., Brunyé et al. 2006; Gyselinck et al. 2002, 2008; Kruley et al. 1994;Nam and Pujari 2005). Relatively little research has focused on how text and pictures areintegrated in working memory; however, the first results indicate that the CE plays a majorrole in this process (e.g., Austin 2009; Brunyé et al. 2006; Doolittle et al. 2009; Pazzaglia etal. 2008). Accordingly, we can conclude based on this review that working memory by andlarge is working during learning from text and pictures in the way one would expect itbased on Baddeley’s model.

There is relatively little research yet about working memory components involvementduring multimedia learning. One reason could be that the dual-task methodology as the—potentially better suited—way of investigating working memory contribution may bedifficult to implement, because the stimuli used in multimedia research are much morecomplex than stimuli in basic cognitive psychology experiments. The second reason couldbe that task features of experimental materials are more difficult to control systematically inmultimedia learning (e.g., the question of whether spatial information is also contained inthe text, which will possibly affect whether VSSP is involved during verbal informationprocessing or not; De Beni et al. 2005; Schmidt-Weigand and Scheiter 2011). Finally, it isdifficult to generate multiple instances of instructional materials that all share the samefeatures and that can be learned independently from each other, so that they can be used inwithin-subjects designs for the assessment of baseline performance. To sum up, especiallythe use of dual-task methodologies imposes certain requirements that may be difficult tomeet with multimedia materials, thereby explaining why until now there have been so fewstudies investigating working memory involvement during multimedia learning.

There are slightly more studies, in which working memory capacities were gauged andrelated to performance. Because of the correlative nature of this approach, the results have to beinterpreted with care. Also, in most studies we reviewed, participants were categorized intohigh and low-capacity groups on the basis of their performance in the respective tests. Theresulting group variable was then incorporated in statistical analyses as a nominal factor (e.g.,Gyselinck et al. 2000). This dichotomizing procedure often results in rather small group sizesassociated with a loss of power in the statistical analyses. Therefore, many authors havesuggested using continuous predictors in regression analysis instead (e.g., Irwin andMcClelland 2003; MacCallum et al. 2002), which allows testing for both main effects ofworking memory capacity on multimedia learning and interactions between working memorycapacity and multimedia design variables (cf., Aiken and West 1991). Accordingly, especiallyresearchers using the working memory capacity approach need to be aware of the limitationsin terms of interpreting results as being caused by capacity differences and analyze their databy using adequate statistical procedures.

Whereas the issues discussed until this point involve methodological aspects that need tobe considered in future research, more important conclusions can be drawn with respect torefining theories of multimedia learning. Even though the reviewed studies stronglysupported the theories’ assumption that working memory components are involved duringmultimedia learning, this review also reveals that there are inconsistencies betweenBaddeley’s working memory model (Baddeley 1999) and the way the model has been usedin multimedia research.

First, the theories of multimedia learning refer to Baddeley’s model in a rather superficialway, which leads to inconsistencies: Cognitive Load Theory (Sweller et al. 1998) promotesa notion of overall working memory load (with the exception of the modality-effectexplanation). This is much more in line with the short-term memory concept of Atkinson

Educ Psychol Rev (2011) 23:389–411 405

and Shiffrin (1968) than with the model of Baddeley, whose major theoretical achievementcompared with its theoretical predecessor has been the differentiation of working memoryinto various subsystems.

Second and more importantly, both Cognitive Theory of Multimedia Learning andCognitive Load Theory incorporate Baddeley’s assumptions about the distinction betweenworking memory subsystems in a way that is inconsistent with Baddeley’s model and thatcannot be empirically confirmed. In particular, dual-task studies do not provide any supportfor the assumption that written text is initially processed in the VSSP, because nointerference with written text processing could be demonstrated for secondary tasks loadingthe VSSP (e.g., Brunyé et al. 2006; Gyselinck et al. 2008). Thus, these empirical resultscall into question Cognitive Theory of Multimedia Learning’s and Cognitive Load Theory’sassumption that all visually presented materials (including written text) are initiallyprocessed in the VSSP, that is, the visual part of the visual/pictorial channel. This impliesthat the distinction of the two channels on the basis of sensory mode of representation (i.e.,auditory vs. visual) is questionable, unless it refers to sensory rather than working memory.In other words, a distinction of the two channels on the basis of presentation code (i.e.,verbal vs. pictorial) is more suitable to explain text and picture processing.

The third inconsistency is that based on Baddeley’s model, it can be assumed that the CE isimportant during multimedia learning, which was confirmed by the studies in this review. Thisfinding can be explained by the fact that the presentation of two information sources (i.e., textand picture) requires the allocation of attention to stimuli, while ignoring others, as well asmonitoring and coordinating the subsystems. However, the CE is not explicitly incorporatedinto Cognitive Theory of Multimedia Learning, but only mentioned briefly as a structure thatallocates, monitors, and adjusts limited cognitive resources (Mayer 2005). In earlier versions ofthe Cognitive Load Theory, the CE is not acknowledged at all. In more recent publicationsschemas stored in long-term memory are assumed to take over executive functions (e.g.,Sweller 2005). Accordingly, in the framework of Cognitive Load Theory executive control isdomain specific in that it depends on the schemas acquired so far. This view differs fromBaddeley’s notion of a domain-unspecific CE. Moreover, according to Baddeley (e.g.,Baddeley 1999), the CE is an active control structure that is similar to the SupervisoryAttentional System initially proposed by Norman and Shallice (1986) as a system fordeliberate action control for novel and complex tasks. Norman and Shallice propose anothercontrol mechanism that is, however, limited to the control of routine or semi-automatic, well-learned actions. This contention scheduling „acts through the activation and inhibition ofsupporting and conflicting schemas“ (Norman and Shallice 1986, p. 3) and may, thus, bemore alike to the way Sweller (2005) conceives executive control. Nevertheless, it isimportant to note that it is not identical to deliberate action control established by the CE orthe Supervisory Attentional System, respectively. In short, Sweller and Baddeley havedifferent understandings of the central executive, despite the fact that Baddeley’s workingmemory model builds one of the theoretical foundations of Cognitive Load Theory. Becauseof its empirically demonstrated importance during multimedia learning, we suggest to morestrongly emphasize the role of the CE as conceptualized in Baddeley’s working memorymodel by explicitly embedding it into multimedia theories. Moreover, the ongoing debateabout the existence and functioning of the episodic buffer (see Pearson 2006 for an overview)needs to be taken into account in future research, as from a theoretical point of view thissubsystem is also involved during multimedia learning (cf. Gyselinck et al. 2008). However,currently no established methodologies exist that allow investigating this assumption.

It is important to note that the inconsistencies mentioned here are not just part of anacademic debate but have important consequences for educational practice. These

406 Educ Psychol Rev (2011) 23:389–411

inconsistencies challenge some of the explanations for well-established empirical effects inmultimedia research. For instance, if written text is processed in the PL from the beginningon (similar to spoken text), then there is no reason to believe that using spoken rather thanwritten text would free cognitive resources of the VSSP. If other mechanisms than the onesinitially suggested were responsible for causing a multimedia design effect, then theremight be also other boundary conditions under which this effect can be observed. Thus,recommendations about the design of multimedia materials may need to be changed as afunction of modifying explanations at the theory level. Hence, conducting research with theaim of gaining a better understanding of the cognitive mechanisms underlying multimedialearning is worthwhile not only for theory building, but also for improving instructionalmaterials.

Nevertheless, even though we believe that theories of multimedia learning should begrounded in cognitive psychology, some general concerns remain when linking educationaltheories to basic information-processing theories such as Baddeley’s working memorymodel.

The first concern is that basic information-processing theories do not offer the mostuseful level of explanation, even if working memory’s subsystems are involved duringmultimedia learning. That is, when we discuss working memory contributions, we typicallyrefer to processes that take up to two seconds, while multimedia learning in real-worldeducational scenarios spans across much longer timeframes. With longer timeframes,higher-level variables such as motivation or the availability of cognitive and metacognitivelearning strategies may become more important to account for differences in learningoutcomes. Hence, even though multimedia learning is information processing for explain-ing differences in achieving educational objectives with multimedia materials, a broaderperspective is required. According to this view, working memory is necessary but notsufficient to explain multimedia learning.

The second concern is that pictures might serve different functions in the learningprocess (see Carney and Levin 2002), which in turn might induce different workingmemory demands on the learner. Currently, this aspect is rather neglected in multimediaresearch, however, it cannot be excluded the possibility that the multimedia effects—whichare based on assumptions about working memory demands—depend on the specificfunctions of the presented pictures.

The third concern is that any attempt to ground educational theories in cognitivepsychology will always be faced with the challenge to keep up with the insights generatedin cognitive psychology. For instance, alternatives to the Baddeley model have beenproposed such as Cowan’s Embedded Processing Model (see Miyake and Shah 1999, for anoverview). However, up to now none of these alternative models has become as widespreadacross different research communities as Baddeley’s model; hence, educational researchersmay still be well advised when referring to the latter. Novel insights may also come fromneuroscience research. In fact, there is a vast amount of neuroscience research thatcorroborates the assumptions about working memory (for an overview see Osaka et al.2007). However, so far there are hardly any studies that have utilized the methodologies ofneuroscience to examine working memory components involvement during multimedialearning, even though possible links between neuroscience and Cognitive Load Theoryhave been discussed (cf., for a review Antonenko et al. 2010). Finally, especially readingcomprehension research has made a theory shift by moving from amodal to modality-specific representations in long-term memory (e.g., Fischer and Zwaan 2008). All thesedevelopments may play a role in building comprehensive cognitive theories of multimedialearning in the future. However, accounting for these developments in multimedia research

Educ Psychol Rev (2011) 23:389–411 407

requires extensive knowledge of both educational as well as cognitive psychology research.At the moment, it seems safe to argue that using the current multimedia learning theories isappropriate for educational research as long as they explain the phenomena of interest.Nevertheless, we need to be aware that we might overlook things, stagnate on the currentstatus quo because new insights are missing, or move into wrong directions, because thetheories that we use shape the way we conduct research in the field.

References

Aiken, L., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park:Sage.

Andrade, J. (2001). Working memory in perspective. Hove: Psychology Press.Antonenko, P. D., Paas, F., Grabner, R., & Van Gog, T. (2010). Using electroencephalography to measure

cognitive load. Educational Psychology Review, 22, 425–438. doi:10.1007/s10648-010-9130-y.Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and its control processes. In

K. W. Spence & J. T. Spence (Eds.), The psychology of learning and motivation: Advances in researchand theory (pp. 89–195). New York: Academic.

Austin, K. A. (2009). Multimedia learning: Cognitive individual differences and display design techniquespredict transfer learning with multimedia learning modules. Computers & Education, 53, 1339–1354.doi:10.1016/j.compedu.2009.06.017.

Baddeley, A. D. (1996). Exploring the central executive. The Quarterly Journal of Experimental Psychology,49, 5–28. doi:10.1080/027249896392784.

Baddeley, A. D. (1998). Recent developments in working memory. Current Opinion in Neurobiology, 8,234–238. doi:10.1016/S0959-4388(98)80145-1.

Baddeley, A. D. (1999). Essentials of Human Memory. Hove: Psychology Press.Baddeley, A. D. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive

Sciences, 4, 417–423. doi:10.1016/S1364-6613(00)015382.Baddeley, A. D., Emslie, H., Kolodny, J., & Duncan, J. (1998). Random generation and the executive control

of working memory. Quarterly Journal of Experimental Psychology, 51, 819–852. doi:10.1080/027249898391413.

Brünken, R., Steinbacher, S., Plass, J. L., & Leutner, D. (2002). Assessment of cognitive load in multimedialearning using dual task methodology. Experimental Psychology, 49, 109–119. doi:10.1027//1618-3169.49.2.109.

Brünken, R., Plass, J. L., & Leutner, D. (2004). Assessment of Cognitive Load in Multimedia Learning withDual-Task Methodology: Auditory Load and Modality Effects. Instructional Science, 32, 115–132.doi:10.1023/B:TRUC.0000021812.96911.c5.

Brunyé, T. T., Taylor, H. A., Rapp, D. N., & Spiro, A. B. (2006). Learning procedures: The role of working memoryin multimedia learning experiences. Applied Cognitive Psychology, 20, 917–940. doi:10.1002/acp.1236.

Carney, R. N., & Levin, R. N. (2002). Pictorial illustrations still improve students’ learning from text.Educational Psychology Review, 14, 5–26. doi:10.1023/A:1013176309260.

Carretti, B., Borella, E., Cornoldi, C., & De Beni, R. (2009). Role of working memory in explaining theperformance of individuals with specific reading comprehension difficulties: A meta-analysis. Learningand Individual Differences, 19, 246–251. doi:10.1016/j.lindif.2008.10.002.

Cierniak, G., Scheiter, K., & Gerjets, P. (2009). Explaining the split-attention effect: Is the reduction ofextraneous cognitive load accompanied by an increase in germane cognitive load? Computers in HumanBehavior, 25(2), 315–324. doi:10.1016/j.chb.2008.12.020.

Cocchini, G., Logie, R. H., Della Sala, S., MacPherson, S. E., & Baddeley, A. D. (2002). Concurrentperformance of two memory tasks: Evidence for domain-specific working memory systems. Memory &Cognition, 30, 1086–1095.

Colle, H. A., & Welsh, A. (1976). Acoustic masking in primary memory. Journal of Verbal Learning andVerbal Behaviour, 15, 17–31. doi:10.1016/S0022-5371(76)90003-7.

Conway, A. R. A., Kane, M. J., & Engle, R. W. (2003). Working memory capacity and its relation to generalintelligence. Trends in Cognitive Science, 7, 547–552. doi:10.1016/j.tics.2003.10.005.

Conway, A. R. A., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., & Engle, R. W. (2005).Working memory span tasks: A methodological review and user’s guide. Psychonomic Bulletin &Review, 12, 769–786.

408 Educ Psychol Rev (2011) 23:389–411

Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal ofLearning and Verbal Behavior, 19, 450–466. doi:10.1016/S0022-5371(80)90312-6.

De Beni, R., Pazzaglia, F., Gyselinck, V., & Meneghetti, C. (2005). Visuospatial working memory andmental representation of spatial description. European Journal of Cognitive Psychology, 17, 77–95.doi:10.1080/09541440340000529.

Della Sala, S., Gray, C., Baddeley, A. D., & Wilson, L. (1997). Visual Pattern Test: A test of short-termvisual recall. London: Harcourt Assessment.

Della Sala, S., Gray, C., Baddeley, A., Allamano, N., & Wilson, L. (1999). Pattern spans: A tool for unweldingvisuo-spatial memory. Neuropsychologia, 37, 1189–1199. doi:10.1016/S0028-3932(98)00159-6.

Doolittle, P. E., & Mariano, G. J. (2008). Working memory capacity and mobile multimedia learningenvironments: Individual differences in learning while mobile. Journal of Educational Multimedia andHypermedia, 17, 511–530.

Doolittle, P. E., Terry, K. P., & Mariano, G. J. (2009). Multimedia learning and working memory capacity. InR. Zheng (Ed.), Cognitive effects of multimedia learning (pp. 17–33). London: Premier ReferenceSource.

Dutke, S., & Rinck, M. (2006). Multimedia learning: Working memory and the learning of word and picturediagrams. Learning and Instruction, 16, 526–537. doi:10.1016/j.learninstruc.2006.10.002.

Engle, R. W., Tuholski, S. W., Laughlin, J. E., & Conway, A. R. A. (1999). Working memory, short-termmemory, and general fluid intelligence: A latent-variable approach. Journal of Experimental Psychology,128, 309–331. doi:10.1037/0096-3445.128.3.309.

Farmer, E. W., Berman, J. V. F., & Fletcher, Y. L. (1986). Evidence for a visuo-spatial scratch-pad in workingmemory. The Quarterly Journal of Experimental Psychology, 38(4-A), 675–688.

Fischer, M. H., & Zwaan, R. A. (2008). Embodied language: A review of the role of the motor system inlanguage comprehension. Quarterly Journal of Experimental Psychology, 61, 825–850. doi:10.1080/17470210701623605.

Gathercole, S., & Baddeley, A. (1993). Working memory and language processing. Hove: Erlbaum.Ginns, P. (2005). Meta-analysis of the modality effect. Learning and Instruction, 15, 313–331. doi:10.1016/

j.learninstruc.2005.07.001.Gyselinck, V., Ehrlich, M.-F., Cornoldi, C., de Beni, R., & Dubois, V. (2000). Visuospatial working memory

in learning from multimedia systems. Journal of Computer Assisted Learning, 16, 166–176.doi:10.1046/j.1365-2729.2000.00128.x.

Gyselinck, V., Cornoldi, C., Dubois, V., de Beni, R., & Ehrlich, M.-F. (2002). Visuospatial memory andphonological loop in learning from multimedia. Applied Cognitive Psychology, 16, 665–685.doi:10.1002/acp.823.

Gyselinck, V., Jamet, E., & Dubois, V. (2008). The role of working memory components in multimediacomprehension. Applied Cognitive Psychology, 22, 353–374. doi:10.1002/acp.1411.

Irwin, J. R., & McClelland, G. H. (2003). Negative consequences of dichotomizing continuous predictorvariables. Journal of Market Research, 40, 366–371. doi:10.1509/jmkr.40.3.366.19237.

Kane, M. J., Hambrick, D. Z., Tuholski, S. W., Wilhelm, O., Payne, T. W., & Engle, R. W. (2004). Thegenerality of working memory capacity: A latent-variable approach to verbal and visospatial memoryspan and reasoning. Journal of Experimental Psychology: General, 133, 189–217. doi:10.1037/0096-3445.133.2.189.

Kruley, P., Sciama, S. C., & Glenberg, A. M. (1994). On-line processing of textual illustrations in thevisuospatial sketchpad: Evidence from dual-task studies. Memory & Cognition, 22, 261–272.

Lawrence, B. M., Myerson, J., Oonk, H. M., & Abrams, R. A. (2001). The effect of eye and limb movementson working memory. Memory, 9, 433–444. doi:10.1080/09658210143000047.

Lehman, M. T., & Tompkins, C. A. (1998). Reliability and validity of and auditory working memorymeasure: Data from elderly and right-hemisphere damaged adults. Aphasiology, 12, 771–785.doi:10.1080/02687039808249572.

Logie, R. H. (1995). Visuo-spatial working memory. Aberdeen: Lawrence Erlbaum Associates.Logie, R. H., & Marchetti, C. (1991). Visuo-spatial working memory: Visual, spatial or central executive? In

R. H. Logie & M. Denis (Eds.), Mental images in human cognition (pp. 105–115). Amsterdam: North-Holland Press.

Logie, R. H., & Pearson, D. G. (1997). The inner eye and the inner scribe of visuo-spatial working memory:Evidence from developmental fractionation. European Journal of Cognitive Psychology, 9, 241–257.doi:10.1080/713752559.

Lusk, D. L., Evans, A. D., Jeffrey, T. R., Palmer, K. R., Wikstrom, C. S., & Doolittle, P. F. (2009).Multimedia learning and individual differences: Mediating the effects of working memory capacity withsegmentation. British Journal of Developmental Psychology, 40(4), 636–651. doi:10.1111/j.1467-8535.2008.00848.x.

Educ Psychol Rev (2011) 23:389–411 409

MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization ofquantitative variables. Psychological Methods, 7, 19–40. doi:10.1037//1082-989X.7.1.19.

Mayer, R. E. (2005). Cognitive Theory of Multimedia Learning. In R. E. Mayer (Ed.), The Cambridgehandbook of multimedia learning (pp. 31–48). New York: Cambridge University Press.

Mayer, R. E. (2009). Multimedia learning (2nd ed.). Cambridge: Cambridge University Press.McConnell, J., & Quinn, J. G. (2000). Interference in visual working memory. The Quarterly Journal of

Experimental Psychology, 53, 53–67. doi:10.1080/027249800390664.Milner, B. (1971). Interhemispheric differences in the localization of psychological processes in man. British

Medical Bulletin, 27(3), 272–277.Miyake, A., & Shah, P. (1999). Models of working memory. Mechanisms of active maintenance and

executive control. Cambridge: Cambridge University Press.Miyake, A., Emerson, M. J., Padilla, F., & Ahn, J. (2004). Inner speech as a retrieval aid for task goals: The

effects of cue type and articulatory suppression in the random task cuing paradigm. Acta Psychologica,115, 123–142. doi:10.1016/j.actpsy.2003.12.004.

Murray, D. J. (1967). The role of speech responses in short-term memory. Canadian Journal of Psychology,21, 263–276. doi:10.1037/h0082978.

Nam, C. S., & Pujari, A. (2005). The role of working memory in multimedia learning. In Proceedings of the11th International Conference on Human-Computer Interaction (HCII ‘05). Las Vegas: Mira DigitalPublishing. CD-ROM

Norman, D., & Shallice, T. (1986). Attention to action: Willed and automatic control of behavior. In R.Davidson, G. Schwartz, & D. Shapiro (Eds.), Consciousness and self regulation: Advances in researchand theory (Vol. 4, pp. 1–18). New York: Plenum.

Osaka, N., Logie, R. H., & D’Esposito, M. (2007). The cognitive neuroscience of working memory. Oxford:Oxford University Press.

Pazzaglia, F., Toso, C., & Cacciamani, S. (2008). The specific involvement of verbal and visuospatialworking memory in hypermedia learning. British Journal of Educational Technology, 39(1), 110–124.

Pearson, D. G. (2006). The episodic buffer. Implications and connections with visuo- spatial research. In T.Vecchi & G. Bottini (Eds.), Imagery and spatial cognition (pp. 139–153). Amsterdam: John Benjamins.

Quinn, J. G., & McConnell, J. (1996). Irrelevant pictures in visual working memory. The Quarterly Journalof Experimental Psychology, 49, 200–215. doi:10.1080/027249896392865.

Rummer, R., Schweppe, J., Fürstenberg, A., Seufert, T., & Brünken, R. (2010). Working memoryinterference during processing texts and pictures: Implications for the explanation of the modality effect.Applied Cognitive Psychology, 24, 164–176. doi:10.1002/acp.1546.

Rummer, R., Schweppe, J., Fürstenberg, A., Scheiter, K., & Zindler, A. (2011). The perceptual basis of themodality effect in multimedia learning. Journal of Experimental Psychology: Applied (in press).

Salthouse, T. A., & Pink, J. E. (2008). Why is working memory related to fluid intelligence? PsychonomicBulletin & Review, 15, 364–371. doi:10.3758/PBR.15.2.364.

Sanchez, C. A., & Wiley, J. (2006). An examination of the seductive details effect in terms of workingmemory capacity. Memory & Cognition, 34, 344–355.

Schmidt-Weigand, F., & Scheiter, K. (2011). The role of spatial descriptions in learning from multimedia.Computers in Human Behavior, 27, 22–28. doi:10.1016/j.chb.2010.05.007.

Schmidt-Weigand, F., Kohnert, A., & Glowalla, U. (2010). Explaining the modality and contiguity effects:New insights from investigating students’ viewing behavior. Applied Cognitive Psychology, 24, 226–237. doi:10.1002/acp.1554.

Seufert, T., Schütze, M., & Brünken, R. (2009). Memory characteristics and modality in multimedialearning: An aptitude-treatment-interaction study. Learning and Instruction, 19, 28–42. doi:10.1016/j.learninstruc.2008.01.002.

Shah, P., & Miyake, A. (1996). The separability of working memory resources for spatial thinking andlanguage processing: An individual differences approach. Journal of Experimental Psychology: General,125, 4–27. doi:10.1037/0096-3445.125.1.4.

Smith, E. E., & Jonides, J. (1997). Working memory: A view from neuroimaging. Cognitive Psychology, 33,5–42. doi:10.1006/cogp.1997.0658.

Smith, E. E., & Jonides, J. (1999). Storage and executive processes in the frontal lobes. Science, 283, 1657–1661. doi:10.1126/science.283.5408.1657.

Spinnler, H., & Tognoni, G. (1987). Italian standardization and classification of neuropsychological tests.Italian Journal of Neurological Science, 6, 1–120.

Spreen, O., & Strauss, E. (1998). A Compendium of Neuropsychological Tests. New York: Oxford UniversityPress.

Sweller, J. (2005). Implications of cognitive load theory for multimedia learning. In R. E. Mayer (Ed.), TheCambridge handbook of multimedia learning (pp. 19–30). New York: Cambridge University Press.

410 Educ Psychol Rev (2011) 23:389–411

Sweller, J., van Merriëboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructionaldesign. Educational Psychology Review, 10, 251–285. doi:10.1023/A:1022193728205.

Tardieu, H., & Gyselinck, V. (2003). Working memory constraints in the integration and comprehension ofinformation in a multimedia context. In H. van Oostendorp (Ed.), Cognition in a Digital World (pp. 3–24).London: Lawrence Erlbaum Associates.

Turner, M. L., & Engle, R. W. (1989). Is working memory capacity task dependent? Journal of Memory andLanguage, 28, 127–154. doi:10.1016/0749-596X(89)90040-5.

Vandierendonck, A., Kemps, E., Fastame, M. C., & Szmalec, A. (2004). Working memory components ofthe Corsi blocks task. British Journal of Psychology, 95, 57–79. doi:10.1348/000712604322779460.

Waters, G. S., & Caplan, D. (1996). The measurement of verbal working memory capacity and its relation toreading comprehension. The Quarterly Journal of Experimental Psychology, 49A, 51–79. doi:10.1080/027249896392801.

Wechsler, D. (1958). The measurement and appraisal of adult intelligence. Baltimore: Williams & Wilkins.Wechsler, D. (1997). WAIS-III and working memoryS-III: Technical manual. San Antonio: Harcourt Brace &

Company.

Educ Psychol Rev (2011) 23:389–411 411