DMW, IET, and ME, KMi, Open University, 2005

Exploring Pedagogy, Presence and Participation in Interactive

Formative Assessment

Denise Whitelock and Marc EisenstadtIET and KMi,

The Open University,e-mail: d.m.whitelock@open.ac.uk

DMW, IET, and ME, KMi, Open University, 2005

Presentation Outline

• What have we learnt about interactive learning?

• What does it mean to work in the presence of others?

• From scaffolded feedback to scaffolded assessment?

DMW, IET, and ME, KMi, Open University, 2005

Global Warming

DMW, Institute of Educational Technology, September 1997DMW, Institute of Educational Technology, September 1997

DMW, IET, and ME, KMi, Open University, 2005

Global Warming

DMW, IET, and ME, KMi, Open University, 2005

Interactivity and Feedback

• Interaction, Feedback loops• Tell, Explore, Check• Predict, Look and Explain

• Entering the discourse of a subject via audio feedback

• Scaffolded text feedback (Bruner & Woods)

• SHOW ME button

DMW, IET, and ME, KMi, Open University, 2005

Interactivity and Cognitive Change Scores

0

2

4

6

8

1 0

G a l a p a g o s O r g a n i cM o l e c u l e s

M e i o s i s &M i t o s i s

S e i s m i cW a v e s

I n t e r a c t i v i t yC o g . C h a n g e

DMW, IET, and ME, KMi, Open University, 2005

What do we know about Presence?

• Motivating• Engagement• Nature of task (Malone & Leper)

DMW, IET, and ME, KMi, Open University, 2005

Working in the presence of others is better than working alone?

• PuckLand simulation• Pairs matched for cognitive

dissonance• Worked completely alone and in the

presence of others• Significant difference between

working in presence of others and working alone (Whitelock et al, 1995)

DMW, IET, and ME, KMi, Open University, 2005

Presence

• No agreed definition • Immersion and Presence (Slater et

al, 1995)• Immersion is how much a person is

“in” the environment• Presence is a sense of “being there”

DMW, IET, and ME, KMi, Open University, 2005

Presence and Conceptual Learning: Whitelock et al’s Cube

• Presence, Immediacy of Control and Representational Fidelity are the variables and predict:

• High Presence and high degree of Immediacy of Control equals high implicit learning

• Low value of Immediacy of Control is more likely to be associated with explicit learning

• Representational Fidelity has a smaller affect on conceptual understanding than appropriately designed infidelities

• Desktop VR environments tested and support above hypotheses, see Jelfs & Whitelock, 2000; Brna, 1999; Whitelock & Jelfs, 2004

DMW, IET, and ME, KMi, Open University, 2005

The Role of Presence: Findings from BuddySpace

Instant Messaging with location info

Automatic group and map generation

Very scalable and customizable

Analysis of 1000 discussion forum messages among OU language students showed that at least 20% of those were location-centric

DMW, IET, and ME, KMi, Open University, 2005

Buddyspace

15 evaluation questionnaires from long term users (>6months)

Users were asked to rate ‘group belongingness’ engendered by 20 activities, situations, physical and digital artifacts.

DMW, IET, and ME, KMi, Open University, 2005

Buddyspace

• Automatically-generated groups and enhanced ‘state’ information are perceived as the most beneficial and most frequently-used feature of BuddySpace

• BuddySpace maps, personal rosters and group rosters ranked within top 5 items

BuddySpace endorses our long-term goal of fostering a sense of ‘group belongingness’

DMW, IET, and ME, KMi, Open University, 2005

From Interactivity and Presence findings to building Formative Assessment

Environments

• Formative assessment is a powerful means to improve student learning (Black & Williams, 2001)

• Effect size on standardised tests of between 0.4 and 0.7

• How can we support our students to become more reflective learners and engage in formative assessment tasks?

DMW, IET, and ME, KMi, Open University, 2005

OpenMark and BuddySpace

• Need a system to provide scaffolded feedback to students

• Check reasoning• Communicate with others• Find a partner with relevant

experience to support information sharing

DMW, IET, and ME, KMi, Open University, 2005

OpenMark

DMW, IET, and ME, KMi, Open University, 2005

OpenMark

DMW, IET, and ME, KMi, Open University, 2005

BuddySpace

• Automatic map construction to show locations

• Automatic roster (‘buddy list’) construction using OU login authentication… so it knows which workgroup or tutorial group I’m in

• Many views: map, timelines, etc.• Plug-ins for future enhancements

DMW, IET, and ME, KMi, Open University, 2005

BuddySpace

DMW, IET, and ME, KMi, Open University, 2005

• “State of mind”• goals, plans, intentions• devices• user profiling

DMW, IET, and ME, KMi, Open University, 2005

Smart inset chosen,depending onactual data

AutomaticMapGeneration

DMW, IET, and ME, KMi, Open University, 2005

AutomaticMapGeneration

Right-click to ‘drill down’into ‘cluster’ nodes(provides scaleability)

DMW, IET, and ME, KMi, Open University, 2005

BuddyFinder

•Who else is facing a similar problem?•Where can I find complementary approaches?•Has anyone found contrary evidence?

Generally: “Find me the person who can really help me now.”

More specifically:

DMW, IET, and ME, KMi, Open University, 2005

Arbitrary keywordsat user’s discretion

BuddyFinder

DMW, IET, and ME, KMi, Open University, 2005

Easysearch

DMW, IET, and ME, KMi, Open University, 2005

Matches‘light up’

DMW, IET, and ME, KMi, Open University, 2005

InvokesGoogle’s Web Servicebehind the scenes,if requested,and auto-highlightswithin user’s own map!

BuddyFinder

DMW, IET, and ME, KMi, Open University, 2005

Working with Course Teams

• H806 Learning in the Connected Economy – BuddyFinder

• Technology short course – OpenMark addressing misconceptions

• Combining BuddyFinder and a science problem solving environment

DMW, IET, and ME, KMi, Open University, 2005

Next: ‘Yoked’ apps via BuddySpace

Student A

Student B(‘yoked’, butwithout full

screen sharingrequired!)

DMW, IET, and ME, KMi, Open University, 2005

Interactive Formative Assessment

No longer learning about X but reflecting about X

• Students are able to find facts similar to X• Know how X might be disputed• Are able to make predictions about X• Know how to use X in an argument• Know how far X can be pushed• Supported with e-tools and pedagogical

strategies

DMW, IET, and ME, KMi, Open University, 2005

Thank you

• Chris Denham: BuddyFinder and OpenMark tools

• Jiri Komzak: BuddySpace• Martin Dzbor and Rod Norfor: Jabber

and BuddySpace servers• Jon Rosewell, Technology• Chris Pegler, H806• S103 Team