Effects of advance organizers, mental models and abilities on task and recall performance using a mobile phone network

Effects of Advance Organizers, Mental Models andAbilities on Task and Recall Performance Using

a Mobile Phone Network

JANICE LANGAN-FOX1*, CHRIS PLATANIA-PHUNG2 andJENNIFER WAYCOTT2

1Swinburne University of Technology, Australia2The University of Melbourne, Australia

SUMMARY

Mobile phone usage is now at saturation point in most Western countries. The current researchinvestigated the usability of services provided by a mobile phone network, specifically whether twodifferent forms (text, graphic) of an advance organizer (AO) assisted novice users in applyinginformation supplied in a manual. It was hypothesized that a graphic AO would facilitate thedevelopment of coherent mental models of the network to enhance task performance, and that lowerability groups in particular would benefit from AOs. Contrary to prediction, the text AO groupoutperformed both the graphic AO and control groups. Lower ability groups also benefited more froma text AO than a graphic AO. Copyright # 2006 John Wiley & Sons, Ltd.

ADVANCE ORGANIZERS AND NEW TECHNOLOGY

In the Information Age it is increasingly important for the layperson to acquire knowledge

of new technology quickly and efficiently. One such technology is mobile phone networks,

with subscription growing rapidly worldwide (Australian Mobile Telecommunications

Association [AMTA], 2004). In Australia, approximately 16 million people subscribe to

mobile phone networks, representing almost 80% of the population (AMTA, 2004b). Thus,

a challenging everyday learning task is familiarization with ‘hardware’ that is the mobile

phone and its associated service network. Manuals have been the standard source of

information but can be challenging to use, especially for novices (Allwood & Kalen, 1997;

Weiss, 1995). One approach to ease learning could be to include an advance organizer

(AO), an overview or summary of information that is organized ‘in advance’ that is

presented before the learner has approached the task. AOs appear to facilitate learning of

scientific systems (Mayer, 1983, 1987, 1989) and technological devices (Mayer &

Bromage, 1980), and have the potential to act as an aid in using new technologies and

comprehending manuals (Langan-Fox, Waycott, & Albert, 2000a).

APPLIED COGNITIVE PSYCHOLOGYAppl. Cognit. Psychol. 20: 1143–1165 (2006)

Published online 6 July 2006 in Wiley InterScience(www.interscience.wiley.com). DOI: 10.1002/acp.1258

*Correspondence to: Janice Langan-Fox, Australian Graduate School of Entrepreneurship, Faculty of Businessand Enterprise, Mail H25, Swinburne University of Technology, P.O. Box 218, Hawthorn, Victoria 3122,Australia. E-mail: [email protected]

Contract/grant sponsor: Ericsson (Australia) Pty Ltd.

Copyright # 2006 John Wiley & Sons, Ltd.

AIMS OF THE INVESTIGATION

Currently, there is a lack of understanding regarding how and why AOs are effective and an

absence of AO research in the context of human-system interaction. Furthermore, the

possible role of an AO-by-cognitive ability interaction in predicting performance is

unclear. The current research advanced the literature by (1) integrating the AO literature

with the mental models literature (learning differences due to AOs can be tied to

differences in users’ mental models thus explaining more precisely how and why AOs are

useful); (2) applying AO research, typically conducted in a classroom setting, to the

learning of an everyday technological device (i.e. a mobile phone and its network); and (3)

investigating whether different types of AOs are more efficacious for low versus high

ability users in a controlled, experimental setting with inexperienced or ‘novice’ users. The

need for research on AOs in this context is paramount as mobile phones continue to become

a necessity for everyday functioning (AMTA, 2004a,b), and as they become more complex

due to customization and increased access to multiple networks. Thus, the current research

made a strong applied contribution as well as a theoretical one. Specific aims of the

research were to identify (1) the types of learning facilitated by AOs (task vs. recall

performance); (2) the most effective AO type (text vs. graphic) in relation to performance;

(3) the effect of AOs on users’ mental models of a mobile phone network; (4) the impact of

mental models on user-network interaction and performance; and (5) the effect of ability on

AO efficacy. Below, we review the literature on different types of AOs and their effect on

performance as well as relevant research on mental models. Hypotheses are stated and

numbered in the text throughout the literature review.

THE FACILITATIVE EFFECT OF ADVANCE ORGANIZERS

FOR LEARNING

AOs emergedmore than 40 years ago as a result of Ausubel’s (1968) subsumption theory of

meaningful learning. Ausubel (1968) argued that in order for material to be meaningfully

learned, it must be subsumed by higher order, more abstract concepts. Traditionally AOs

have been regarded as a popular pedagogical learning aid (Ausubel, 1960), presented

before the target learning material with the function of linking prior knowledge to new

concepts. However, contemporary AOs do not necessarily activate the learner’s prior

knowledge but rather provide an organizing framework that the learner might not otherwise

possess (Mayer, 2003). For example, Mayer and Bromage (1980, p. 211) defined an AO as

‘a stimulus that is presented prior to learning, and contains a system for logically

organizing the incoming information into a unified structure’. Thus, AOs do not necessarily

require the learner to hold prior knowledge relevant to the target learningmaterial. AOs that

activate prior knowledge (so that target learning material is actively linked to them) are

referred to as Ausbulean AOs. Such AOs were not a focus for the current research given that

it focused on novice users who had no prior knowledge of mobile phone networks (see

‘Participants’ below for more detail).

In the only available research on AOs and new technology, Mayer and colleagues tested

the efficacy of various AOs on college students who were learning computer programming

modelled on FORTRAN, BASIC and SEQUEL (for a review see Mayer, 1981). In one

study, Mayer and Bromage (1980) examined the text recall of 108 undergraduates in

relation to a new computer programming language with an AO given either before or after

Copyright # 2006 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. 20: 1143–1165 (2006)

1144 J. Langan-Fox et al.

reading instructional material. The ‘before’ group scored higher on recall of conceptual

idea units, produced more appropriate intrusions, and made more novel inferences, while

the ‘after’ group scored higher on recall of technical idea units and produced more

inappropriate intrusions, connectives and vague summaries.

ADVANCE ORGANIZER EFFICACY AND THE DEVELOPMENTOF MENTAL MODELS

There is an absence of AO research on mobile phone usage and a lack of understanding about

the relationship between cognition and AO effectiveness. For instance, a frequently cited

problem in the literature is inconsistent findings regarding the efficacy of AOs (Ausubel,

1978; Barnes & Clawson, 1975; Luiten, Ames, & Ackerson, 1980; McEneany, 1990).

Recently, Mayer (2003) developed Assimilation Encoding Theory (AET) and proposed that

AOs improve performance through facilitating the acquisition of optimal knowledge

representations or ‘mental models’, allowing the learner to organize the material into a

coherent structure. Indeed, Sutherland, Pipe, Schick, Murray, and Gobbo (2003) found that

providing anAO about an upcoming event to children facilitated integration of the experience

into a general knowledge representation. According to AET, the pre-organization of

information into a coherent structure within the learner is more likely to impact on

‘meaningful learning’ than ‘less important’ types of learning such as passive recall of

information. Thus, task performance should benefit more from AOs than recall performance.

In linewithAET,Mayer (1983) found that students whowere providedwith anAOperformed

better on creative problem-solving but worse on verbatim retention (recognition and recall)

when compared to a control group. AOs may hinder verbatim retention because learners re-

organize the material and translate it into their own words. Another explanation is that AOs

encourage elaborate processing: AOs can be designed to have a different macrostructure to

that of the ‘to-be-learned’ material (Kintsch, 1994; Mannes & Kintsch, 1987).

THE PRESENT INVESTIGATION

Advance organizers, mental models and task and recall performance

Consistent with AET, participants who receive AOs should demonstrate better learning

than those who do not receive AOs. We anticipated that AOs would facilitate the

development of superior mental models of how network services are related to each other,

thus enabling a better understanding of instructions and better task performance during

exposure to the manual. However, we did not expect differences in mental models to

translate into better recall performance.

H1: AO groups will perform better than a Control group on task performance.

H2: AO groups and the Control group will not differ on recall performance.

Text versus graphic AOs

One aspect of AOs that may influence their efficacy is how they are presented, for instance

in Text or Graphic form (Langan-Fox et al., 2000a). Both types of AOs may facilitate

construction of an initial mental model (Mayer, Dyck, & Cook, 1984; Schnotz, 1993)


Advance organizers, abilities, mental models 1145

enabling the novice to effectively comprehend and implement instructional material.

However, Graphic AOs may be more effective than Text AOs. Text displays represent a

continuous (usually vertical) linear flow of information and are effective in conveying

‘within-concept relations’ while graphical displays such as matrices better convey ‘cross-

concept relations’, making it easier for a learner to engage in integrative learning (Larkin &

Simon, 1987; Robinson & Kiewra, 1995; Waller, 1981). Graphic AOs also have the

potential for ‘dual-mode’ or verbal and spatial cognition (Mayer & Moreno, 2002; Paivio

1986). When information is processed both verbally and spatially, there is an ‘additive’

effect on retention (Paivio, 1986; Quealy & Langan-Fox, 1998).

H3: The Graphic AO group will show higher task performance than the Text AO group.

Network usage, learning and mental models

A mobile phone user must interact with a network of services such as call connection,

voicemail messages and directory services. When interacting with a network (or system)

for the first time, the novice user will normally encounter verbal or visual information in the

form of instructional material; learning will occur if he or she is able to understand network

concepts and their relationships, for instance, the steps involved in making a call and the

relationships between those steps. People internalize knowledge in the form of mental

models (Hanisch, Kramer, & Hulin, 1991; Johnson-Laird, 1983; Wilson & Rutherford,

1989) which form part of short-term or long-term memory (Craik, 1943; Langan-Fox,

Anglim, & Wilson, 2004). Rouse and Morris (1986, p. 351) suggested that mental models

help to describe and explain systems: ’[mental models are] the mechanisms whereby

humans are able to generate descriptions of system purpose and form, explanations of

system functioning and observed system states, and predictions of future system states’.

Thus, a mental model can be seen as a cognitive representation of system functioning,

which is utilized to guide system usage (Sebrechts, Marsh, & Furstenburg, 1990). Research

has identified differences between experts’ and novices’ mental models (Hanisch et al.,

1991; Redding & Cannon, 1992) as well as mental model change or development over time

(Langfield-Smith & Wirth, 1992). Accordingly, system learning and usability can be

assessed in relation to users’ mental models. For instance, if users’ mental models are

inconsistent with the system, it could be that there is a problem with the instructional

material or the way in which ‘intuitive understanding’ is gleaned from the system.

In the current research, participants’ mental models of a mobile phone network were

evaluated using Pathfinder (Schvaneveldt, Durso, & Dearholt, 1989). Mental models were

viewed as a set of connections between concepts that would help facilitate learning of the

mobile phone network (Rouse & Morris, 1986). Pair-wise (proximity) ratings, matrices

representing perceived relationships between concepts, were transformed into diagram-

matic representations in which concepts were represented by nodes and relationships

between concepts were represented by links (see ‘Construction of Mental Models’ below

for more detail).

Three criteria can be applied to assess users’ mental models with reference to using a

device or system: (1) ‘coherence’—the extent to which knowledge is logically organized

within an integrated structure; (2) ‘validity’—the correspondence between a user’s

representation of a system and the output of that system; and (3) ‘integration’— the extent

to which the user’s representation is integrated with other knowledge (Greeno & Simon,

1984). Based on these criteria, we examined a number of ‘extensions’ of users’ mental



models: the relationships among network concepts derived from the pair-wise ratings;

ability to control the network in a series of tasks; and ability to understand the network as

indicated in tests of declarative and procedural knowledge. In addition, users’ mental

models were assessed before and after exposure to AOs, to gauge change in knowledge

structure.

H4: The Graphic AO group will demonstrate (1) higher mental model coherence and (2)

greater structural change in mental models from Session 1 (no exposure to AO) to

Session 2 (exposure to AO) than the Text AO and Control groups (see ‘Method’ for more

detail about Sessions 1 and 2).

Learner characteristics: cognitive ability

Mayer (1979) suggested that AOs may have a discriminating effect, with learning

conditional on factors such as ability level, prior knowledge and the complexity of the

material. Sutherland et al. (2003) suggested that AO efficacy is likely to depend on content,

organization and overlap with experience of the event itself, as well as the interaction

between these variables. While the efficacy of AOs for learning scientific and technical

information has been supported (Mannes & Kintsch, 1987; Mayer, 1981; Mayer &

Bromage, 1980), we suggest that efficacy might depend on cognitive ability. A long-

standing argument is that AOs are more beneficial to people with lower ability (Mayer,

1978). The assumption is that AOs provide a coherent cognitive structure that the low

ability learner would not otherwise possess or create (Brooks, Simultis, & O’Neil, 1985;

Mayer, 1980; Tyler, Delaney, & Kinnucan, 1983). Yet, a meta-analysis of the literature

(Stone, 1983) found that those with medium ability benefited most from AOs while those

with low and high ability benefited less. Despite these results, in theory, AOs should reduce

any deficit associated with slower word encoding by providing a schema to facilitate better

organization for learning and retention (cf. Tyler et al., 1983). Consistent with this idea,

AOs have been beneficial to the learning impaired (Darch & Gerston, 1986) and to people

with low ability who attend to the organizer (Lenz, Alley, & Schumaker, 1987; Mayer,

1978). A study of computer programme learning (Mayer, 1975) found that low

mathematical ability participants benefited more from an AO than did high ability

participants. With the exception of work by Mayer (1975; 1980; 1983), no research has

examined the interactive effect of AOs and cognitive ability on performance when using a

technological device and system.

H5: Low ability participants will benefit more from AOs in relation to task performance

than high ability participants.

H6: The superiority of Graphic AOs over Text AOs will be greater for low ability

participants.

METHOD

Participants

Participants were 94 undergraduate students (32 men, 62 women) enrolled at a major

university in the State of Victoria, Australia. The mean age was 20 years (SD¼ 1.78).



Volunteers were screened via a phone interview to ensure that they were ‘novices’,

specifically that they: (1) had never owned a mobile phone; (2) had no experience with

mobile phone networks; and (3) were novice users of the network. It was important to

engage novices so that the impact of AO exposure on learning and mental model

development could be determined. Moreover, engaging novices was more practical than

assessing level of experience and its interaction with learning. Seventy participants spoke

English as a first language while 24 participants spoke English as a second language (ESL).

This variation in the sample enabled us to examine whether ESL participants had greater

difficulty in using the mobile phone network.

Design

Participants were randomly assigned to one of three experimental conditions: (1) Text AO

group, 34% of participants (10 men, 22 women); (2) Graphic AO group, 33% of

participants (10 men, 21 women); or (3) Control group, 33% of participants (12 men, 19

women).

Materials

Background questionnaire

Four background variables were assessed: (1) attitude towards mobile phones (‘like’,

‘dislike’, ‘no opinion’); (2) intention to buy a mobile phone (‘buy’, ‘not buy’, ‘no

opinion’); (3) general technical experience (‘very little’, ‘moderate’, ‘none’); and (4)

attitude towards technology (‘like very much’, ‘like sometimes’, ‘dislike’).

Advance organizers (AOs)

Two AO groups (Text, Graphic) were compared to a Control group. The AOs did not have

an analogical relationship to the manual but rather had a direct relationship in terms of

content (i.e. service labels, for example CALLforward, were mentioned in both the AOs

and the manual). Participants were required to learn how to access and use network

services. The AOs, described in detail below, were presented on an A3 size sheet (11.69

inches by 16.54 inches) and gave definitions of each network service. Figures 1 and 2

present the Text and Graphic AOs, respectively. The definitions were derived from the

standard instruction manual. Subsuming concepts (e.g. CALLforwarding) were

represented by subheadings in bold font.

The text AO consisted of an A3 sheet which provided an outline displaying descriptions

of each service under the appropriate heading, to be read down the page. No information

was numbered and descriptions of each service were in full sentences. The Graphic AOwas

presented on an A3 sheet as a tree diagram (Langan-Fox et al., 2000a), with the same

descriptions of each service under the appropriate heading but with connecting lines

showing the relationships between services. The diagram was hierarchically arranged,

which required reading each boxed-item individually, both across and down the page. The

top level denoted the most general subsuming concept ‘INtouch services’; the second level

distinguished between message services and other services (also subsuming concepts); the

third and fourth levels included the services and their descriptions; and the bottom level

indicated common attributes among the services. The aim was to promote within-concept

and cross-concept processing by the learner. The AOs were equivalent in terms of content

and word count (252 words) so that any AO group differences in learning could only be



attributed to differences in format (Larkin & Simon, 1987). The Control group received an

A3 size ’General Information Sheet’ which provided general information about network

services. It was identical in word length to the AOs.

Task performance

Participants were given 13 tasks to complete using the mobile phone network (see

Appendix A). For each task, participants were presented with a brief scenario and a task

relating to that scenario for example ‘You urgently need to contact your boss, but you know

Figure 1. Text advance organizer



Figure

2.Graphic

advance

organizer



that he/she will be in a meeting with his/her mobile phone turned off. However, you also

know your boss’s phone is connected to the CALLalert message service, so you decide to

call his/her number and leave a message to ring you back urgently on your mobile phone.

Task: Call your boss’s phone and leave a message’. Phone numbers were provided with the

instructions.

Recall performance

Two recall tests were designed to measure knowledge necessary to complete tasks using

the mobile phone network and were administered 5 minutes after completion of the 13

tasks described above.

Declarative Knowledge Test. Participants completed 20 items which involved choosing

the appropriate procedure to complete a given task using the network for example ‘You

want to activate the CALLwait and CALLhold services. Which one of the following four

responses (a–d) is the most appropriate action to take? (a) Press � 4 3 # send; (b) Press # 4 3# send; (c) Press # # [CALLforwarding code] # send; (d) Enter your four-digit security

code’.

Procedural Knowledge Test. Participants were presented with steps necessary to complete

19 questions using the network and were told to indicate the order in which the given steps

should be completed. This measure was constructed along the lines of ‘If. . ., then. . .‘scenarios, consistent with Anderson’s (1996) conceptualization of procedural knowledge

for example ‘‘You want to change your CALLback security code. Using the numbers 1 to 4,

please indicate the order of steps you would take: ‘Press 9 8’, ‘Press send’, ‘Press 1 2 1’,

‘Follow verbal instructions’’’.

Cognitive ability tests

Four cognitive ability tests from the extensively used Ekstrom, French, and Price (1976)

test battery were used.1 The tests were chosen on the basis that they tapped abilities that

were important for learning and performance in the present context: (a) Calendar Test:

accuracy in following directions; (b) Inference Test: logical reasoning; (c) Object-Number

Test: associative memory; and (d) Advanced Vocabulary Test: knowledge of the English

language. The relationship between cognitive ability and task performance was used to

determine whether AOs were more useful for lower-ability learners.

Mental models: pair-wise ratings method

A key issue in mental models research is defining the boundaries of mental models. Using a

pre-determined, common pool of concepts to elicit mental models facilitates the

comparison of individual mental models (Roth & Roychoundhury, 1993). Although it is

acknowledged that a single mental model could not possibly represent all of the concepts

within a domain (Novak &Gowin, 1984), such models provide a ‘workable representation’

of domain knowledge.

It is common in mental models research for relevant concepts to be compared by asking

participants to rate the relatedness of all concepts to each other (Langan-Fox, Code, &

Langfield-Smith, 2000). In the pair-wise ratings method, similarity or relatedness

(proximity) ratings are collected for each possible pair of concepts in the total concept

1Details of tests supplied on request.



pool. These data are transformed into a matrix which is then used as the input for data

analysis. The optimal number of concepts to sustain a person’s concentration in the pair-

wise ratings method is around 15 to 20 (Royer, Cisero, & Carlo, 1993). The mental model is

not articulated, but rather inferred through statistical analysis (see Langan-Fox et al., 2004;

Langan-Fox et al., 2000; and Langan-Fox, Wirth, Code, Langfield-Smith, & Wirth, 2001

for more detail). In the current research, participants rated the relatedness of pairs of

network services (in terms of function) on a 7-point scale (1¼ not at all related, 7¼ highly

related). Fifteen concepts or network services, derived from the manual, were used for

example ‘In considering the function of CALLback—an answering service that enables

callers to leave voice messages when you are unavailable—how related are: CALLback

(takes voice messages) and CALLbarring (restricts calls)?’

Apparatus

A hands-free Nokia mobile phone was connected to an IBM-compatible computer with a

15-inch colour monitor that presented instructions for each of the 13 tasks (see ‘Task

Performance’ above). A software programme was written and installed on the computer to

enable the recording of Dual Tone Multi-Frequency (DTMF) tones elicited by the mobile

phone each time a button was pressed.

Procedure

The experiment was conducted over two sessions. Session 1 involved completing the

Background Questionnaire, Cognitive Ability Tests and Pair-wise Ratings, which were

used to derivemental models of the mobile phone network. Session 2 was conducted within

3 days of Session 1 and involved completing of the 13 tasks designed to measure Task

Performance; the Declarative and Procedural Knowledge Tests designed to measure Recall

Performance; and the same Pair-wise Ratings completed during Session 1.

At the start of Session 2, participants were randomly allocated to one of the three

experimental conditions. The Text and Graphic AO groups were given 2minutes to read the

AO and the Control group was given 2 minutes to read the General Information Sheet. Two

minutes was chosen as an appropriate time interval as it allowed enough time to read the

AOs, while preventing any effect due to simply spending more time studying the

instructions. Three sets of instructional material were provided to all participants,

regardless of experimental condition: (1) a tabulated guide to using the mobile phone

(‘Quick Reference Guide’); (2) a manual containing instructions concerning the mobile

phone network; and (3) computerized instructions for each of the 13 tasks. Once the AO (or

the General Information Sheet) was removed, all experimental groups were given 10

minutes to read the manual, to which they could refer throughout the 13 tasks. They then

completed the 13 tasks, the recall tests and the pair-wise ratings. Participants did not have

access to the manual when completing the recall tests.

RESULTS

Preliminary Analyses

Randomization checks

Multiple Kruskall Wallace Tests revealed that there were no experimental group

differences on attitudes towards mobile phones, x2(2)¼ 1.55, p> 0.05; intention to buy a



mobile phone, x2(2)¼ 3.07, p> 0.05; general technical experience, x2(2)¼ 1.27,

p> 0.05; or attitude towards technology, x2(2)¼ 0.52, p> 0.05 (responses were in the

form of rankings). Multiple Pearson Chi-Square Tests indicated that there were no

experimental group differences on gender, x2(2)¼ 0.45, p> 0.05 or the distribution of

native and non-native English participants, x2(2)¼ 1.36, p> 0.05. Finally, Univariate

ANOVAs indicated that there were no experimental group differences in Verbal Reasoning,

F(2, 91)¼ 0.61, p> 0.05 or Associative Memory, F(2, 91)¼ 3.00, p> 0.05. These results

confirm that the randomization of participants to experimental groups was successful,

thereby allowing any differences in performance to be attributed solely to experimental

condition.

Construction of mental models

Within each experimental group, the raw data from the pair-wise ratingswere combined to obtain

the ‘average’ ratings at each session. This resulted in six separate mental models: Control

Session 1, Text AOSession 1, Graphic AOSession 1, Control Session 2, Text AOSession 2 and

Graphic AO Session 2. Averaged data were used as it was not expedient to analyse and

compare 188 mental models (i.e. 94 participants assessed at Session 1 and 2). Average ratings

were calculated by finding the group means for each of the 105 pair-wise ratings.

Group differences in mental models were examined using Pathfinder (Schvaneveldt

et al., 1989), a computerized, networking technique which is used to derive associative

networks. Pathfinder networks consist of nodes (concepts) and links (pair-wise

relationships between concepts). Networks are created by finding the shortest path

between any two nodes in the network, and eliminating paths that violate triangle

inequality. Thus, the primary source of information in a network is the presence or absence

of a link. Concepts that are highly related in a participant’s mental model are separated by

fewer links and appear closer together in the network.

The Pathfinder network similarity (NETSIM) function is used to reveal differences in the

way knowledge is structured in two networks. NETSIM is equal to the number of common

links in two networks divided by the number of links that are in either of the networks.

NETSIM is essentially a reflection of the degree of association or correspondence between

two networks, and can be evaluated similarly to a correlation coefficient. Thus, two

identical networks would yield a NETSIM of 1, and two networks that share no links would

yield a NETSIM of 0. Although Pathfinder has been widely used in the mental models

literature (see Langan-Fox et al., 2000b), there are currently insufficient data to develop

norms regarding the ‘optimal’ degree of association between two networks. A disadvantage

of Pathfinder is that the layout of items in a network is arbitrary, representing associative

but not semantic information about relationships.

To evaluate knowledge re-structuring (from Session 1 to Session 2) as a result of AO

exposure, ‘coherence’ scores were computed using the Pathfinder scaling algorithm

(Schvaneveldt, 1990). Coherence is a measure of the internal consistency of an individual’s

data obtained from the pair-wise ratings and is based on the assumption that relatedness

between a pair of concepts can be predicted by the relatedness of the concepts to other

concepts in the set. For instance, given that ‘A’ and ‘B’ are related, and that ‘B’ and ‘C’ are

related, it can be assumed that ‘A’ and ‘C’ are related. The coherence score falls between 0

and 1, with higher scores reflecting higher coherence. Although the Pathfinder algorithm

does not provide any reliability information for coherence, research has shown that it can

successfully distinguish experts and novices (see e.g. Goldsmith, Johnson, & Acton, 1991;

Gonzalvo, Canas, & Bajo, 1994).



Construction of task performance variables

The following variables were constructed using the performance data obtained from the 13

tasks: (a) Inefficiency (how many extra buttons were pressed in comparison to the

minimum number necessary); (b) Proportion Correct (the proportion of tasks completed

successfully out of all tasks attempted); and (c) Total Error (the number of times errors

were made during the 13 tasks). Several ‘Total Error’ subcategories, representing poorer

operationalization of network features, were developed in collaboration with network

engineers: (a) dialled wrong number, (b) incorrect CALLforwarding code, (c) did not enter

correct command at appropriate time, (d) dialled ‘Voicemail Box Number’ instead of 121,

(e) failed to press ‘End’, (f) failed to complete some aspect of task, and (g) ‘Other’.

Construction of cognitive ability variables

Two factors were derived from the four cognitive ability tests using Principal Components

Analysis (PCA) with Varimax rotation. The first factor, ‘Verbal Reasoning’, was

interpreted as comprehension of the English language and the ability to understand and

follow written instructions. This factor included the Advanced Vocabulary, Calendar and

Inference tests with component loadings of 0.85, 0.81 and 0.80, respectively. The second

factor, ‘Associative Memory’, was interpreted as the ability to recall items that are

associated with one another. This was measured by the Object-Number test and had a

component loading of 0.98.

Main analyses

Descriptive statistics

Table 1 shows the means and standard deviations for Task Performance, Recall

Performance and Cognitive Ability by experimental group. Correlations between the

variables are shown in Table 2.

Table 1. Means and standard deviations for task performance, recall performance, and cognitiveability overall and by experimental group (control, text AO, graphic AO)

Variable n Experimental group

Overall

Mean (SD)

Control(n¼ 31)

Mean (SD)

Text AO(n¼ 31)

Mean (SD)

Graphic AO(n¼ 31)

Mean (SD)

Task performancea

Inefficiency 93 105.00 (66.92) 118.65 (74.24) 95.35 (63.14) 101.00 (62.68)Total error 93 14.59 (6.80) 16.61 (7.35) 13.03 (5.83) 14.13 (6.83)Proportion correct 93 0.73 (0.21) 0.68 (0.22) 0.78 (0.18) 0.72 (0.22)

Recall performancDeclarative knowledge 94 2.16 (0.90) 2.21 (0.82) 2.03 (0.87) 2.24 (1.01)Procedural knowledge 94 0.88 (0.32) 0.91 (0.31) 0.88 (0.33) 0.84 (0.33)

Cognitive abilityVerbal reasoningb 94 0.00 (0.10) �0.15 (1.07) 0.12 (1.02) 0.02 (0.90)Associative memory 94 16.22 (7.00) 14.42 (6.82) 15.69 (6.31) 18.58 (7.42)

aData were missing for one participant for the Task Performance variables.bBased on factor scores.



Experimental group differences in task performance (H1, H3)

H1 predicted that the AO groups would demonstrate superior task performance to the

Control group, and H3 further predicted that the Graphic AO group would demonstrate

superior task performance to the Text AO group. A MANOVA revealed that there were no

multivariate effects for the Task Performance variables, Wilk’s Lambda F(6,176)¼ 0.96,

p> 0.05. However, contrasts revealed that there were some significant univariate group

differences. In partial support of H1, the Control group scored significantly higher on Total

Error than the Text AO group, t(2, 90)¼ 2.10, p< 0.05 and significantly lower on

Proportion Correct than the Text AO group, t(2, 90)¼�1.95, p< 0.05. Thus, there were

significant differences between the Control group and the Text AO group for two of the

three Task Performance variables. Contrary to prediction however, there were no

significant differences between the Control group and the Graphic AO group on any of

the task performance variables. H3 was unsupported: the Graphic AO group did

not significantly outperform the Text AO group on any of the task performance

variables.

Experimental group differences in recall performance (H2)

H2 predicted that the AO groups and the Control group would not differ on recall

performance. In full support of H2, a MANOVA comparing group differences in Recall

Performance revealed that there were no significant multivariate effects, Wilk’s Lambda,

F(4, 180)¼ 0.56, p> 0.05, and no significant univariate effects for Declarative Knowledge,

F(2, 91)¼ 0.51, p> 0.05 or Procedural Knowledge, F(2, 91)¼ 0.35, p> 0.05.

Mental model coherence and structural change from session 1 to 2 (H4)

H4(a) predicted that the Graphic AO group would demonstrate higher mental model

coherence than the Text AO and Control groups. Coherence values were computed for the

three experimental groups and were as follows for Sessions 1 and 2, respectively: Control

group 0.26 and 0.31; Text AO group 0.24 and 0.34; and Graphic AO group 0.25 and 0.28.

Table 2. Relationships between task performance, recall performance, and cognitive ability

Task performance Recall performance Cognitive ability

Inefficiency Totalerror

Proportioncorrect

Declarativeknowledge

Proceduralknowledge

VerbalReasoning

AssociativeMemory

Inefficiency —Total error 0.65�� —Proportioncorrect

�0.13 �0.68�� —

Declarativeknowledge

�0.02 0.16 �0.21� —

Proceduralknowledge

�0.01 0.17 �0.30�� 0.43�� —

Verbalreasoning

�0.18 �0.38�� 0.47�� 0.18 �0.43�� —

Associativememory

�0.01 �0.21� 0.37�� 0.18 �0.19 0.17 —

�p< 0.05; ��p< 0.01; ��p< 0.001.



Coherence values were higher at Session 2 than at Session 1 for all three experimental

groups. However, differences in Coherence from Session 1 to Session 2 were significant for

the Text AO group only, indicating that mental model coherence improved for this group

but not the other groups. Furthermore, contrary to H4(a), it was the Text AO group rather

than the Graphic AO group that obtained the highest coherence value at Session 2,

suggesting that participants in this group developed more coherent mental models of the

network than did participants in other groups.

H4(b) predicted that the Graphic AO group would demonstrate greater structural change

in their mental models from Session 1 (no exposure to AO) to Session 2 (exposure to AO)

than the Text AO and Control groups. To establish whether the structure of mental models

changed from Session 1 to Session 2, we calculated the network similarity (NETSIM) of

the mental models for Session 1 and 2 within each group (see Table 3). The mental models

for the Control and Graphic AO groups did not show much structural change from Session

1 to Session 2 as reflected by their high NETSIM values (0.81 and 0.82, respectively). That

is, these two groups showed highly similar mental models from Session 1 to Session 2. In

comparison, the NETSIM of the Text AO group is substantially lower (0.64) suggesting

that, contrary to H4(b) the mental models of the participants in this group (rather than the

Graphic AO group) changed considerably from Session 1 to Session 2 as a consequence of

exposure to the AO.

Graphical representations of the Pathfinder networks for the three experimental groups

were generated. SMS Text, Voicemail and CALLhold were central concepts in all six

networks (i.e. Control Session 1, Text AO Session 1, Graphic AO Session 1, Control

Session 2, Text AO Session 2, Graphic AO Session 2). Figures 3 and 4 show the results of

the Pathfinder analyses for the experimental group that changed most from Session 1 to

Session 2: the Text AO group.2 In line with the findings for coherence, these figures show

that from Session 1 to Session 2, the Text AO group’s mental model becamemore logical in

terms of a clearer association between the functions and the procedural (chain) steps

involved in those functions. The mental model for Session 1 had two major organizing

‘wheels’ with two major concepts at the ‘hub’(SMS text and CALLhold—somewhat

unrelated concepts) and two ‘singleton’ concepts (CALLfw and CALLbar). In contrast, at

Session 2, the mental model had changed so that SMS text alone became the ‘hub’ concept

with three sequentially structured concept chains and singleton concepts feeding directly

into the triad chains. The change represents a more refined and simplified mental model of

the structure of the network and its functions.

Table 3. Similarities (NETSIM) of pathfinder networks

Control

Session 1

Control

Session 2

Graphic AO

Session 1

Graphic AO

Session 2

Text AO

Session 1

Text AO

Session 2

Control Session 1 —Control Session 2 0.81 —Graphic AO Session 1 0.71 0.87 —Graphic AO Session 2 0.87 0.80 0.82 —Text AO Session 1 0.81 0.64 0.75 0.81 —Text AO Session 2 0.70 0.75 0.65 0.71 0.64 —

Note: All values are significantly more similar than would be expected by chance.

2Mental models for Control and Graphic AO groups for Sessions 1 and 2 supplied on request.



Effect of AOs for low ability groups (H5, H6)

H5 predicted that low ability learners would benefit more from AOs in regard to task

performance than high ability learners. To examine experimental condition by cognitive

ability interactions, ability variables derived from the PCA (see ‘Preliminary Analyses’

above) were divided into low and high ability groups using the median split method.

Subsequently, 3� 2 ANOVAs were conducted with Experimental Condition (Text AO

group, Graphic AO group, Control group) and Verbal Ability or Associative Memory (low

VoicemailIntVoice

CALLbk-

CALLscree

SMS text

CALL fw-

SMS

CALLwait

CALLbar

CALLhold

VoiceBox

CALLbk

CALLfw

VoiceBox

CALLalert

Figure 4. Mental model for the text AO group at Session 2

CALLbk-

SMS text

SMS

CALLwait

IntVoice

CALL fw-

Voicemail

VoiceBox

VoiceBox

CALLscree

CALLhold

CALLalert

CALLbar

CALLfw

CALLbk

Figure 3. Mental model for the text AO group at Session 1



vs. high) as between-subjects factors. Separate ANOVAs were conducted for each of the

Task Performance variables (i.e. Inefficiency, Proportion Correct, Total Error). Based on

these six analyses, two significant interactions were found. The first interaction was

between Experimental Group and Verbal Reasoning in the prediction of Inefficiency, F(2,

87)¼ 3.34, p< 0.05 (see Figure 5). The second interaction was between Experimental

Condition and Associative Memory in the prediction of Proportion Correct, F(2,

87)¼ 5.23, p< 0.01 (see Figure 6). Consistent with H5, AOs seemed to reduce ability-

based performance differences. In particular, the Text AO appeared to reduce any

advantage associated with high Verbal Reasoning and Associative Memory ability in

relation to Inefficiency and Proportion Correct, respectively. However, this result is

0

50

100

150

200

Text AOGraphic AOControl

Group

Inef

fici

ency

Low Verbal Ability

High Verbal Ability

Figure 5. Mean inefficiency scores as a function of experimental group and verbal reasoning ability

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00


Group

Pro

po

rtio

n c

orr

ect

Low Assoc. Memory Ability

High Assoc. Memory Ability

Figure 6. Mean proportion correct scores as a function of experimental group and associativememory ability



inconsistent with H6, which predicted that Graphic AOs would be more beneficial than

Text AOs for low ability learners. Additional analyses, examining the interaction between

Experimental Condition and English Nativity in the prediction of Inefficiency, are shown in

Appendix B. Briefly these analyses indicated that the Text AO also benefited those with a

less competent command of English.

DISCUSSION

This study investigated whether AOs assisted novices in learning services provided by a

mobile phone network. The efficacy of Text versus Graphic AOs was compared, including

their relation to mental models and their efficacy for different ability groups. A mental

models approach to AO research is beneficial in that learning differences due to AOs can be

tied to differences in users’ mental models thus explaining more precisely how and why

AOs are useful.

Results showed that the Text AO group made fewer errors than the Control group, with

the Text AO presumably providing users with an ideational scaffolding of the network. The

Text AO had a facilitative effect for two of the three Task Performance variables, which is

generally consistent with Assimilation Encoding Theory (AET) and previous research

demonstrating that AOs facilitate meaningful system learning (Mayer, 1979; 1981). Also,

as predicted by AET, and confirming past research showing a null or small effect of AOs on

recall (Barnes & Clawson, 1975), we found that AOs did not have a facilitative effect for

recall performance. Notably, the utility of Graphic AOs was not supported. Contrary to

prediction, participants in the Graphic AO group did not outperform the Text AO or Control

groups. In addition, the Text AO was superior to the Graphic AO in assisting lower ability

and ESL users.

Results showed that for the Text AO group, mental models changed as a function of

exposure to the AO and were different from the Control group, suggesting that mental

models are indeed sensitive to instructional intervention (AOs) designed to affect

knowledge re-organization. Although coherence for all three experimental groups was

relatively low for both Session 1 and Session 2, coherence for the Text AO group improved

from Session 1 to Session 2. Thus, rather than simply demonstrating that a Text AO had an

impact on recall and task performance, our research demonstrated why the AO was

effective in facilitating performance: exposure to the AO resulted in a change in users’

mental models of the network, in particular the development of more coherent mental

models. When people shift to new network services, some restructuring of knowledge must

take place. Current users of network services could be provided with a Text AO to highlight

how they need to restructure their knowledge in order to successfully adapt to a new

network. Novices in particular may benefit from the inclusion of Text AOs in manuals.

The absence of an advantage for the Graphic AO group is inconsistent with research that

has argued for the added benefit of graphic displays in conveying information (Robinson &

Kiewra, 1995) and dual-coding (Mayer & Moreno, 2002; Paivio, 1986; Schnotz, 1993).

However the findings do not represent a complete refutation of this theoretical orientation.

First, research (Anderson, 1996) has suggested that people are inclined to pay more

attention to text than to graphical displays and are more skilled and experienced at reading

text than graphical information. This may apply to students in particular, who were the

participants in our research. Kloster and Winne (1989) found that Graphic AOs were only

beneficial if students utilized them effectively. Thus, it is possible that the Graphic AO was



not fully exploited by the participants. The actual way in which participants used the AOs

was not examined in the current research. This issue could be addressed by comparing

participants in a condition in which instructions for AO use are provided versus a condition

in which no instructions are provided. Second, the effectiveness of Graphic AOs may

depend on the specific graphic used (and its relation to prior knowledge or existing mental

models). The relative efficacy of different Graphic AOs, and the interaction between

Graphic AOs and existing knowledge in the prediction of AO efficacy, should be evaluated

in future research. Third, while graphic displays may be more effective at presenting

relations between concepts, their efficient structure might cause shallow processing,

resulting in poorer performance (Robinson & Kiewra, 1995).

Future research is needed to resolve the issue of whether Text or Graphic AOs are

superior, and in which context (e.g. different samples or tasks) one type of AO is superior to

the other. Eye-gaze monitoring, which measures differences in attention allocation (i.e.

style and depth of AO use) and learning strategies, may be a useful methodology for such

research (see Peebles & Cheng, 2003).

AOs seemed to reduce differences in Inefficiency between high and low ability

participants. For instance, the Text AO reduced Inefficiency for lower verbal ability

participants and ESL participants and was more beneficial for lower ability participants

than for higher ability participants. This finding is consistent with AET (Mayer, 1979),

which suggests that AOs are especially efficacious for learners lacking in prerequisite

skills. Our findings are also consistent with the notion that AOs assist people in

constructing mental models that they would otherwise not be able to achieve (Brooks et al.,

1985; Lenz et al., 1987; Mayer, 1980; Tyler et al., 1983).

The current study showed that AOs do not always have a facilitative effect; their utility is

highly conditional. Thus, AO type and individual differences must be taken into account in

any future research or application of AOs. The challenge is to create AOs that are valuable

across a broad range of novice users of technological devices. In this regard, it may be

advantageous to adopt computer-assisted-instruction (Quealy & Langan-Fox, 1998) which

can match user ability to appropriate AOs.

In conclusion, the study provided preliminary support for the general proposal that Text

AOs can achieve a new role in facilitating the learning of commonly used technological

devices (Langan-Fox et al., 2000a). Our results also suggested that the utility of

habitually used linear Text AOs has been underestimated. Although Graphic AOs are

potentially superior to Text AOs in the sense of conveying cross-concept relations, it may

be that training in how to utilize Graphic AOs is required in order for users to implement

them effectively. Thus, Text AOs are likely to have a practical advantage over Graphic AOs

in that users do not have to invest time in learning strategies for implementing them

effectively. In addition, Text AOs are easier and faster to construct and are thus more

appealing from an instructional design point of view. Three major strengths of the study

were the use of an ecologically valid learning task; the integration of AOs with the mental

models literature enabling theoretical development; and tests of both task and recall

performance with high and low ability participants. The findings extended the utility of

AOs outside the classroom setting, to a technological device used by people from a diverse

range of backgrounds, including those who have low verbal ability or a less competent

command of English. Given that the current study focused on immediate learning, it is yet

to be seen whether the advantage of AO-facilitated mental models is maintained over time.

Nevertheless, as the learning demand on users is greatest in the initial stage of using a new

technology, based on the current results, we encourage the inclusion of AOs at the



beginning of user manuals. We hope the current study will stimulate more research on

improving novice learning of new technology through innovative cognitive instructional

design.

ACKNOWLEDGEMENTS

This research is part of a larger project on Mobile Telecommunications Network Usability

and was funded by Ericsson (Australia) Pty Ltd. The authors are grateful to anonymous

reviewers for their advice regarding the present article. The assistance of Sharon Grant (nee

Code) in refining the final drafts of this article is acknowledged.

REFERENCES

Allwood, C. M., & Kalen, T. (1997). Evaluating and improving the usability manual. Behavior andInformation Technology, 16, 43–57.

Anderson, J. R. (1996). The architecture of cognition. Boston: Harvard University Press.Australian Mobile Telecommunications Association. (2004a). Australian Mobile Telecommunica-tions Industry Executive Summary 2004. http://www.amta.org.au/amta/site/amta [22 February2005].

Australian Mobile Telecommunications Association. (2004b). The Impact of the Mobile Telephone inAustralia. Australian Mobile Telecommunications Association Conference, September, 2004.http:www.assa.edu.au [22 February 2005].

Ausubel, D. P. (1960). The use of advance organizers in the learning and retention of meaningfulverbal material. Journal of Educational Psychology, 51, 267–272.

Ausubel, D. P. (1968). Educational psychology: A cognitive view. New York: Holt, Rinehart andWinston.

Ausubel, D. P. (1978). In defense of advance organizers: A reply to the critics. Review of EducationalResearch, 48, 251–257.

Barnes, B. R., & Clawson, E. V. (1975). Do advance organizers facilitate learning? Recommen-dations for further research based on an analysis of 32 studies. Review of Educational Research, 45,637–659.

Brooks, L. W., Simultis, Z. M., & O’Neil, F. O. (1985). The role of individual differences in learningstrategies research. In R. F. Dillon (Ed.), Individual differences in cognition (pp. 32–47). NewYork: Academic Press.

Craik, K. (1943). The nature of explanation. Cambridge, England: Cambridge University Press.Darch, C., &Gerston, R. (1986). Direction setting activities in reading comprehension: A comparisonof two approaches. Learning Disability Quarterly, 9, 235–243.

Ekstrom, R., French, J., & Price, L. (1976).Kit of referenced tests for cognitive factors. Princeton, NJ:Education Testing Service.

Goldsmith, T. E. W., Johnson, P. J., &Acton, W. H. (1991). Assessing structural knowledge. Journalof Educational Psychology, 83, 88–96.

Gonzalvo, P., Canas, J., & Bajo, M. (1994). Structural representations in knowledge acquisition.Journal of Educational Psychology, 86, 601–616.

Greeno, J. G., & Simon, H. A. (1984). Problem solving and reasoning. In R. C. Atkinson,R. Hernstein., G. Lindzey, & R. D. Luce (Eds.), Stevens’ handbook of experimental psychology(pp. 214–228). New York: Wiley.

Hanisch, K. A., Kramer, A. F., & Hulin, C. L. (1991). Cognitive representations, control andunderstanding of complex systems: A field study focusing on components of users’ mental modelsand expert/novice differences. Ergonomics, 34, 1129–1145.

Johnson-Laird, P. N. (1983). Mental models. Cambridge, MA: Harvard University Press.Kintsch, W. (1994). Text comprehension, memory, and learning. American Psychologist, 49, 294–303.



Kloster, A. M., & Winne, P. H. (1989). The effects of different types of organizers on students’learning from text. Journal of Educational Psychology, 81, 9–15.

Langfield-Smith, K., &Wirth, A. (1992). Measuring differences between cognitive maps. Journal ofthe Operational Research Society, 43, 1135–1150.

Langan-Fox, J., Anglim, J., & Wilson, J. R. (2004). Mental models, team mental models andperformance: Process, development and future directions. Human Factors and Ergonomics inManufacturing, 14, 1–21. Hoboken, NJ: Wiley.

Langan-Fox, J., Code, S., & Langfield-Smith, K. (2000b). Teammental models: Methods, techniquesand applications. Human Factors, 42, 1–30.

Langan-Fox, J., Waycott, J. L., & Albert, K. (2000a). Linear and graphic advance organizers:Properties and processing. International Journal of Cognitive Ergonomics, 4, 19–34.

Langan-Fox, J., Wirth, A., Code, S., Langfield-Smith, K., & Wirth, A. (2001). Analyzing shared andteam mental models. International Journal of Industrial Ergonomics, 28, 99–112.

Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words.Cognitive Science, 11, 65–100.

Lenz, B. K, Alley, G. R., & Schumaker, J. B. (1987). Activating the inactive learner: Advanceorganizers in the secondary content classroom. Learning Disability Quarterly, 10, 53–67.

Luiten, J., Ames, W., & Ackerson, G. (1980). A meta-analysis of the effects of advance organizers onlearning and retention. American Educational Research Journal, 17, 211–218.

Mannes, S. M., & Kintsch, W. (1987). Knowledge organization and text organisation. Cognition andInstruction, 4, 91–115.

Mayer, R. E. (1975). Different problem solving competencies established in learning computerprogramming with and without meaningful models. Journal of Educational Psychology, 67, 725–734.

Mayer, R. E. (1978). Advance organizers that compensate for the organization of text. Journal ofEducational Psychology, 70, 880–886.

Mayer, R. E. (1979). Can advance organizers influence meaningful learning? Review of EducationalResearch, 49, 371–383.

Mayer, R. E. (1980). Elaboration techniques that increase the meaningfulness of technical text: Anexperimental test of the learning strategy hypothesis. Journal of Educational Psychology, 72, 770–784.

Mayer, R. E. (1981). The psychology of how novices learn computer programming. ComputingSurveys, 13, 121–141.

Mayer, R. E. (1983). Can you repeat that? Qualitative effects of repetition and advance organizers onlearning from science prose. Journal of Educational Psychology, 75, 40–49.

Mayer, R. E. (1987). Educational psychology: A cognitive approach. Boston: Little Brown.Mayer, R. E. (1989). Models for understanding. Review of Educational Research, 59, 43–64.Mayer, R. E. (2003). The promise of multimedia learning: Using the same instructional designmethods across different media. Learning and Instruction, 13, 125–139.

Mayer, R. E., & Bromage, B. K. (1980). Different recall protocols for technical texts due to advanceorganizers. Journal of Educational Psychology, 72, 209–225.

Mayer, R. E., Dyck, J. L., & Cook, L. K. (1984). Techniques that help readers build mental modelsfrom scientific text: Definitions pretraining and signalling. Journal of Educational Psychology, 76,1089–1105.

Mayer, R. E., & Moreno, R. (2002). Aids to computer-based multimedia learning. Learning andInstruction, 12, 107–119.

McEneany, J. E. (1990). Do advance organizers facilitate learning? A review of subsumption theory.Journal of Research and Development in Education, 23, 89–96.

Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. New York: Cambridge University Press.Paivio, A. (1986). Mental representations: A dual coding approach. New York: Oxford UniversityPress.

Peebles, D., & Cheng, P. C. H. (2003). Modeling the effect of task and graphical representation onresponse latency in a graph reading task. Human Factors, 45, 28–45.

Quealy, J., & Langan-Fox, J. (1998). Attributes of delivery media in computer-assisted instruction.Ergonomics, 41, 257–279.



Redding, R. E., & Cannon, J. R. (1992). Expertise in air traffic control (ATC): What is it, and how canwe train for it? InProceedings of the Human Factors Society 36th Annual Meeting (pp. 1326–1330). Santa Monica, CA: Human Factors and Ergonomics Society.

Robinson, D. H., & Kiewra, K. A. (1995). Visual argument: Graphic organizers are superior tooutlines in improving learning from text. Journal of Educational Psychology, 87, 455–467.

Roth, W. M., & Roychoundhury, A. (1993). The concept map as a tool for the collaborativeconstruction of knowledge: A micro-analysis of high school physics students. Journal of Researchin Science Teaching, 30, 503–534.

Rouse, W. B., & Morris, N. M. (1986). On looking into the black box: Prospects and limits in thesearch for mental models. Psychological Bulletin, 100, 349–363.

Royer, J. M., Cisero, C. A., & Carlo, M. S. (1993). Techniques and procedures for assessing cognitiveskills. Review of Educational Research, 63, 201–243.

Schvaneveldt, R. W. (1990). Pathfinder associative networks: Studies in knowledge organization.Norwood. NJ: Ablex.

Schvaneveldt, R. W., Durso, F. T., & Dearholt, D. W. (1989). Network structures in proximity data.In G. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory(Vol. 24, pp. 249–284). New York: Academic Press.

Schnotz, W. (1993). Some remarks on the commentary: On the relation of dual coding and mentalmodels in graphics comprehension. Learning and Instruction, 3, 247–249.

Sebrechts, M. M., Marsh, R. L., & Furstenburg, C. T. (1990). Integrative modeling: Changes inmental models during learning. In S. P. Robertson, &W.W. Zachary (Eds.), Cognition, computing,and cooperation (pp. 338–398). Westport, CT: Ablex Publishing.

Stone, C. L. (1983). A meta-analysis of advance organizer studies. Journal of ExperimentalEducation, 51, 194–199.

Sutherland, R., Pipe, M., Schick, K., Murray, J., & Gobbo, C. (2003). Knowing in advance: Theimpact of prior event information on memory and event knowledge. Journal of Experimental ChildPsychology, 84, 244–263.

Tyler, S.W., Delaney, H., & Kinnucan, M. (1983). Specifying the nature of reading ability differencesand advance organizer effects. Journal of Educational Psychology, 3, 359–373.

Waller, R. (1981). Understanding network diagrams. Paper presented at the Annual Meeting of theAmerican Educational Research Association, Los Angeles.

Weiss, E. H. (1995). The retreat from usability: User documentation in the post-usability era. AsteriskJournal of Computer Documentation, 19, 3–18.

Wilson, J. R., & Rutherford, A. (1989). Mental models: Theory and application in human factors.Human Factors, 31, 617–634.



APPENDIX

APPENDIX A: DESCRIPTION OF PERFORMANCE TASKS

APPENDIX B: EXPERIMENTAL CONDITION BY ENGLISH

NATIVITY INTERACTION

Native English speakers scored significantly higher than non-native English speakers on

Verbal Reasoning, t(92)¼ 5.99, p< 0.001, however there were no significant differences

between the groups on Associative Memory, t(92)¼ 1.92, p> 0.05. Analyses revealed that

native English speakers scored lower on Inefficiency than non-native English speakers,

t(91)¼�2.02, p< 0.05. Thus, it is possible that non-native English speakers were

disadvantaged by their lower Verbal Reasoning ability. We examined whether differences

in Inefficiency between the two groups varied as a function of experimental condition.

Figure A1 shows that there was a significant interaction between experimental group and

English nativity in the prediction of Inefficiency, F(2, 87)¼ 5.43, p< 0.05, with the Text

and Graphic AOs seemingly reducing Inefficiency scores among non-native English

speakers. The Verbal Reasoning scores for non-native English speakers did not

Task Participants were instructed to. . .

1 Call their ‘boss’ (the phone number was provided). The boss’s phone was connected to theCALLalert service. Participants were asked to leave a contact number and a message to‘call back urgently’.

2 Call their ‘boss’ (whose phone was connected to CALLalert), to leave a contact numberwith an extension, and a message saying ‘call back tomorrow’.

3 Check if CALLback was active or de-active for when the phone is unreachable; thenre-activate the service so that calls were diverted to the CALLback service when the phonewas unreachable.

4 Listen to a voicemail message and check the time and date that it had been left.5 Record a new CALLback greeting, then review and store it.6 Change the CALLback security code from the default setting of 3333 to 1122.7 Ensure that the time stamping option was selected for Victorian local time, that is, that the

timeand date of left messages was given in the local time.

8 De-activate the diversions to the CALLback service, then to activate an ‘all calls’diversion to CALLscreen.

9 De-activate the ‘all calls’ diversion to CALLscreen.10 Activate the CALLwait/CALLhold service.11 To call their ‘boss’s’ number, and then when the phone was answered (by the experimenter),

put that call on hold and dial their voicemail box. They then had to return to the first call.12 Participants were told to de-activate the CALLwait/CALLhold service, then activate

CALLforwarding for when the phone is engaged.13 Participants were told to de-activate CALLforwarding diversion.



significantly differ as a function of experimental condition, F(2, 23)¼ 0.46, p> 0.05. Thus,

the differences in inefficiency scores across the experimental groups can be attributed to

the AOs.

0

50

100

150

200


Group

Inef

fici

ency

English 1st Language

English 2nd Language

Figure A1. Mean inefficiency scores as a function of experimental condition and english nativity.



Documents

Effects of advance organizers, mental models and abilities on task and recall performance using a mobile phone network