Translating between multiple representations:

discussion

EARLI Symposium

Padua, August, 2003

Richard Cox

HCT Group, University of Sussex

richc@sussex.ac.uk

Overview

• Definitions, framing issues, the translation ‘problem space’

• How do learners come to understand relationships between representations?

• How we might discriminate between occasions when a learner is engaged in ER comprehension activity and when s/he is successfully reasoning with an ER ?

• Supporting users at various levels of expertise

Framing issues…

• 1. Translation “..convey..from one place to another..”, “..turn from one language into another retaining the meaning” (OED)

• 2. Consider dimensions in which translation possible between (multiple) ERs - extending Palmer (1978) to the 2 ER case ...

Represented world (Palmer, 78)

a b c d

Representing world (ER)

taller than --> longer thana b c d

Representing worlds

taller than--> longer thantaller than --> shorter than

a b c d a b c d

ER 1 ER 2

a b c da b c d

Both ERs represent same aspects of represented world - YES

Same aspect of representation doing the mapping - YES

a b c d a

b

c

d

Both ERs represent same aspects of represented world - YES

Same aspect of representation doing the mapping - NO

taller than -> longer than taller than --> points to

a b c d a b c d

Both ERs represent same aspects of represented world - NO

Same aspect of representation doing the mapping - YES

taller than -> longer than WIDER than -> longer than

a b c da

b

c

d

Both ERs represent same aspects of represented world - NO

Same aspect of representation doing the mapping - NO

WIDER than - -> LARGER than taller than --> points to

same aspect of world?

sameER

mapping?

YES NO

YES

NO

a 3rd dimension - multimodality...

GRAPHICAL

LINGUISTIC

and a 4th, 5th, 6th...• static vs dynamic ERs (interactive, animated...)

• not to mention other factors...– degree of abstraction of ER(s)

– type of graphical representation (picture, non-picture)

– number of ERs co-present in display

– 3D, VR ...

• combinatorial explosion of translation routes!

How do learners come to understand relationships between

representations? • Depends on relationships and translation ‘route’

• If same aspect of world represented - can explore ER-to-ER links (dynalinks), perhaps without reference to the world and maybe solely via other (constraining) ERs

• If different aspects of world represented - then ER-to-ER translation is usually via represented world

How do learners come to understand relationships between representations?

• Learning may not be so much `more’ or `less’ but different in mode– exploration of dynalinks, constraining ERs and

correlated displays ---> implicit learning (eg. Berry & Broadbent, 1984)

– need to consider learning outcomes ... how important is need for explicit (verbalizable) knowledge? if implicit knowledge acquired - assess differently?

– interaction of methodology with learning outcome - `think aloud’ likely to keep knowledge and reasoning explicit (with extra cognitive load perhaps)

`Staring at’ versus using - telling the difference..

• how can we discriminate between when a learner is engaged in a) ER comprehension activity and b) reasoning with the ER ?

• 1. correlate ER behaviour with performance (as many researchers do!) here rich process data very useful - innovative methodologies pay off ... – DEMIST: logging, dynalinking, analyser

– SDE: RFV, think aloud and logging

– SimQuest: think-aloud and cognitive load probes

`Staring at’ versus using - telling the difference..

2. Assess learner’s background knowledge of ERs beforehand

• examine mental organisation of ER knowledge - category organisation differs in poor versus better reasoners (Cox & Grawemeyer, 2003) and in people with different backgrounds (Lohse et al 1994)

Supporting users at various levels of expertise

• The `intermediate learner’ effect (du Boulay et al., Seufert)– some knowledge (of ERs, of domain) necessary to

benefit from support – support should segue into integration thru’ degrees of

compartmentalisation and complexity of domain k. and ERs

• need to know more about what `knowing an ER’ actually means - characterise partial knowledge and misconceptions (eg viewing graphs as pictures der Meij & de Tong) - look at different levels of cognitive processing system - ie. perceptual, semantic memory organisation and output levels

Supporting users at various levels of expertise

• by making aspects of world that are modelled explicit (very nice colour coding in der Meij & de Jong) assists learner to assess redundancy level of MERs in display, directs attention in dynalinking and exploration

• sometimes a tension between learner-centred design and traditional ‘ease of use’ HCI ... ask who is system for, should there be different versions for different users?

References p.1of 2

• Berry, D.C. & Broadbent, D.E. (1984) On the relationship between task performance and associated verbalizable knowledge. The Quarterly Journal of Experimental Psychology, 36a, 209-231.

• Cox, R. & Grawemeyer, B. (2003) The mental organisation of external representations. Proceeding of the European Cognitive Science conference (EuroCogSci03), Osnabruck, Sep.

References p. 2 of 2

• Lohse, G.L., Biolsi, K., Walker, N. & Rueter, H. (1994) A classification of visual representations. Communications of the ACM, 37(12), 36-49.

• Palmer, S.E. (1978) Fundamental aspects of cognitive representation. In E. Rosch & B.B.Lloyd (eds) Cognition and categorization. Hillsdale, NJ: Lawrence Erlbaum Associates.

• Other works cited were papers presented at this symposium