Multimodal reformulation during shared synchronous note-taking : potential pedagogical consequences for teachers and students Kristine Lund*, Gregory Dyke°,

Multimodal reformulation during shared synchronous note-taking :

potential pedagogical consequences for teachers and students

Kristine Lund*, Gregory Dyke°, Jean-Jacques Girardot°*ICAR Research Lab, CNRS, University of Lyon

°RIM Department, Ecole des Mines de St. Etienne

2

• Motivation Note-taking and multimodal reformulation

• The corpus A series of tool-supported meetings

– tutoring computer programming projects at l’Ecole des Mines de St. Etienne

• Tatiana (Trace Analysis Tool for Interaction ANAlysts) Synchronization of multi-source data, analyses and visualizations

• Results Form and content / pedagogical consequences ?

• Synthesis and perspectives

OverviewQuickTime™ et un

décompresseur TIFF (non compressé)sont requis pour visionner cette image.

3

Note-taking

• Listen, understand and assimilate, choose what to record, study (Suritsky & Hughes, 1991)

Modify definition for our context…(e.g. also reading, but no trace of studying)

• Look for patterns in note-taking and determine efficiency in relation to students’ results

The value of detailed notes rises with time between the time notes are taken and the test (Williams & Eggert, 2002) ;

A strong correlation between the quantity of notes taken and exam performance (Nye, et. al. 1984).

• Note-taking as a tool for transforming knowledge and not as data collection (Castello & Monereo, 2005)

We focus on the process of note-taking

4

Multimodal interactions (and reformulations)

• Modes Abstract resources “non-material” that help us to make sense

– touch, hearing, smell, taste (senses), – gesture, speech (means of expression)

• Media Specific material forms in which modes are expressed

– The human body, a mechanical body, computer (physical apparatus)

• Example of a mode Manipulate an object to write

• Example of four media

QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.

QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.QuickTime™ et un

décompresseur TIFF (non compressé)sont requis pour visionner cette image.

Kress & Van Leeuwen (2001) ; Bellik & Teil (1992)

5

Reformulation

• Collaborative writing / note-taking context• An oralo-graphic situation, articulates two different semantic modes

Apotheloz (2001) Cooperation in this task consists in “showing in a continuous way, each step of the way, the manner in which what is formulated articulates with what has already been formulated”

QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.

QuickTime™ et undécompresseur TIFF (non compressé)

sont requis pour visionner cette image.

QuickTime™ et undécompresseur sont requis pour visionner cette image.

6

A multimodal corpus (talk, writing)

• First year computer programming projects at Ecole des Mines in St. Etienne 33 tutoring sessions, lasting from 30 to 60 minutes a piece (PPF --> 12 transcribed) 9 groups 3 project types, each chosen by three groups. These 9 groups were tutored by 2 professors.

– Computer traces of note-taking and transcribed dialogue– The computer program / final product– Final reports– The presentations– Grades











7

Exploratory research questions

• What are the nature of the multimodal reformulations ?• How can they demonstrate conceptual transformations ?



?

8

Tatiana - Trace Analysis Tool for Interaction ANAlysts• Synchronize multi-source data

Computer traces of human interaction Videos and their corresponding transcription (dialogue,

etc.)

• Replay in order to understand activity• Define analysis units and categorize facets of

the interaction • Visualize the categorizations on a temporal

line• Regroup, re-categorize, re-visualize• Share



9

Synchronize the multi-source data, replay and think about units of analysis / Research Q

Transcription of talk

Log fileShared text editor where notes are taken (replayed)

Video

10

Create the note-taking writing units (done manually)

Log fileNote-taking units

11

Categorize the reformulated - reformulation pair (done manually)

Categorize the transcription of

talk

Categorization schema

Categorize the note-taking units

12

Create the links between the reformulated - reformulation pair (manual)

Categorized transcription

of talk

Categorized note-taking

units

Directional link between the reformulated and

the reformulation

13

Visualization (automatically created)

Transcription of talk

A’s note-taking B’s note-

takingThe reformulated The reformulation • Synchronize multi-source data

• Replay in order to understand activity

• Define analysis units and categorize facets of the interaction

• Visualize the categorizations on a temporal line

• Regroup, re-categorize, re-visualize• Share

14

Tatiana - example analysis of corpus

Log Data (1)

Writing units (2)

Transcription (3)

Tool replayer (5)

Video (6)

Visualization (4)

Highlights are synchronized

Remotecontrol (7)

Transcription

Writing units

Reformulation

15

Results - session 1

16

A writing conflict provokes a spatial organization

Results - session 2

QuickTime™ et undécompresseur


QuickTime™ et undécompresseur sont requis pour visionner cette image.

17

Chronological time-lag between monologue and note-taking

Results - session 2

18

Differences in the structure of reformulation

Results - session 2

19

Reformulation of tutor dialogue written and co-elaborated by two students

Results - session 2

20

Tutor reaction upon seeing note-taking

Results - session 2

21

Synthesis, interpretations and questions

• Multimodal reformulation, co-elaborative written reformulation of dialogue Not yet seen oral reformulation of co-elaborative note-taking

• From written to oral Response to a student question; Incite students to take note-taking further (conceptual deepening)

– Is this still reformulation ?

• From oral to written Anticipation of teacher talk ; do such students understand content better ? How to understand the choices students make when reformulating ? What is the linguistic nature of these reformulations ? How do the characteristics of the shared text editor influence note-taking and co-

elaboration? (technical affordances of resources)– Collaborative note-taking ?

For what type of content is there a time-lag between teacher talk and note-taking ?



22

Perspectives

• More closely study the nature of these multimodal reformulations• How can they illustrate knowledge transformation ?

We don’t look to measure student performance in terms of exam results as a result of note-taking.

Rather, we want to quality the transformation of conceptual knowledge through characterizing reformulation.

What are the pedagogical consequences ?

• How can the shared text editor characteristics influence note-taking ? More or less co-elaboration ?

• Can note-taking be understood as a sort of formative (vs. summative) evaluation ?• What about auto and hetero-regulation ? And vicarious learning?

23

Thanks to the PPF !

• Transcription of dialogue from videos• Communication between TATIANA and CARTE

Collection, activity Analysis and Regulation based on Traces Enriched (Christophe Courtin, Stéphane Talbot / Chambery)

– to facilitate automating certain analysis tasks (e.g. definition of note-taking units, categorization)

– to highlight relevant sequences of events in large numbers of TATIANA traces

– …

CARTE supplies the following software tools :– convert TATIANA traces CARTE traces and vice versa– import/export CARTE traces to and from a corpus of traces– Analyze CARTE traces by means of rules (pre-defined patterns)

24

You can download Tatiana

• http://code.google.com/p/tatiana/

[email protected]

[email protected]

[email protected]

[email protected]



Documents

Multimodal reformulation during shared synchronous note-taking : potential pedagogical consequences for teachers and students Kristine Lund*, Gregory Dyke°,