Data Analysis Workshop: Analysing gesture, body posture and action in digital learning environmentsM

OD

E m

ultim

odal

met

hodo

logi

esFO

R RE

SEAR

CHIN

G D

IGIT

AL D

ATA

AND

EN

VIRO

NM

ENTS

http://mode.ioe.ac.uk

MODE multimodal methodologiesFOR RESEARCHING DIGITAL DATA AND ENVIRONMENTS

To explore methods of analysing physical interaction in digital learning environments. In particular, the role of gesture, action with objects and body posture will be explored in relation to key themes including:1) Collaboration and communication2) Cognition and meaning making3) Digital literacyMethodological Issues of coding, sampling, the development of research hypothesis, and the application of concepts of embodiment will be discussed.

Aim and focus

Project 1: Digital technologies in the operating theatre

Project 1: Digital technologies in the operating theatre

Project 2: Embodied learning with emerging digital technologies

Project 2: Embodied learning with emerging digital technologies

PhD 1:Digital learning & trans-cription

PhD 1:Digital learning & trans-cription

Training and Capacity Building Activities:Public lectures, working papers, how-to guides, conferences, seminars, research

data workshops, research blogs, online forum, introductory and advanced training

Training and Capacity Building Activities:Public lectures, working papers, how-to guides, conferences, seminars, research

data workshops, research blogs, online forum, introductory and advanced training

PhD 2: Digital moving images

PhD 2: Digital moving images

PhD 3:Social Media

PhD 3:Social Media

TS1: Digital data capture

TS1: Digital data capture

TS2: Transcription dissemination

TS2: Transcription dissemination

TS3: Mixed methods

TS3: Mixed methods

TS4: Time, place & space

TS4: Time, place & space

TS5:EmbodimentTS5:Embodiment

PDFPDF

MODE multimodal methodologiesFOR RESEARCHING DIGITAL DATA AND ENVIRONMENTS

• Fine grained micro-analysis• Focus on meaning making as social and semiotic• Meaning is always multimodal• Language seen as one part of a multimodal ensemble• Each mode understood as realizing different ‘work’ • People orchestrate meaning via selection &

configuration• Emphasis on situated action: context and the sign

maker in shaping the resources available for meaning making and how these are selected and designed

• focus on people’s situated choice of resources rather than emphasizing the system of available resources

The focus of multimodality

MODE multimodal methodologiesFOR RESEARCHING DIGITAL DATA AND ENVIRONMENTS

Quick introduction to four key concepts for today:

• Mode• Semiotic resource• Modal affordance• Multimodal orchestration

MODE multimodal methodologiesFOR RESEARCHING DIGITAL DATA AND ENVIRONMENTS

MODE: Position, posture, movement, gesture, gaze

Semiotic resource

Ambient Wood, EQUATOR (Rogers et al., 2003/4)

MODE multimodal methodologiesFOR RESEARCHING DIGITAL DATA AND ENVIRONMENTS

MODAL AFFORDANCETangible LightTAble: Price, Roussos, Pontual Falcao, Sheridan (2009)

MODE multimodal methodologiesFOR RESEARCHING DIGITAL DATA AND ENVIRONMENTS

MULTIMODAL ORCHESTRATION: TRANSDUCTION/RESEMIOTIZATION

The Game-Catcher prototypeChildren’s Playground Games and Songs in the New Media Age - 2009-2011 Andrew Burn

mode@ioe.ac.uk

http://mode.ioe.ac.uk[Glossary resource]