Confidence Level Explicitation in collaborative SG

Mireia Usart , Margarida Romero & Esteve AlmirallESADE Business SchoolDirection of Educational Innovation and Academic Quality (DIPQA).

Contact: [email protected]

Impact of the explicit Feeling of Knowledge in the learners’ interaction and

performance in collaborative Game Based Learning

TABLE OF CONTENTS1. Introduction

I. Collaborative LearningII. Collaborative Game Based Learning (GBL)III. Advantages and challenges in Collaborative GBLIV. Knowledge Group Awareness Support in Collaborative GBLV. Description of Collaborative KGA tools

2. MethodologyI. General Research designII. HypothesesIII. Game design

I. KGA tool designII. Panels design

IV. Game playV. Participants

3. Results4. Discussion

2

3

1. Introduction: theoretical background

4

1. Introduction: collaborative learning

Game Based Learning

Collaborative Learning

Collaborative GBL

Learning

Computer Supported Learning

Computer Supported

GBL

Computer Supported

CollaborativeLearning

Computer Supported

Collaborative GBL

5

1. Introduction: collaborative learning

Game Based Learning

Collaborative Learning

Collaborative GBL

Learning

Computer Supported Learning

Computer Supported

GBL

Computer Supported

CollaborativeLearning

Computer Supported

Collaborative GBL

Collaborative Learning:

A method to facilitate a knowledge basis and facilitate argument construction. According to Kim and Baylor (2006), collaboration within peers brings out activity and can stimulate motivation.

Collaborative GBL:

Learning environment that involves individual and group interpretations of given information. Peers can play together in order to construct new patterns and generate new problems (Jacques, 1995).

Games can enhance (Gee, 2003; Kirriemuir & McFarlane, 2004): Problem solving skills, decision making, knowledge

transfer and meta-analytic skills.

Collaborative Games can help in: Putting learning into a context (Leemkuil, de Jong, de

Hoog & Christoph, 2003). Giving students a friendly environment with specific

content and skills (Burgos, Tattersall & Koper, 2007).

Advantages of Collaborative Game Based Learning (GBL)

1. Introduction: collaborative GBL

A high level of engagement (Herz, 2001). Continuing motivation when designers incorporate feedback and collaboration

(Malouf, 1988).Transform knowledge into social capital (Herz, 2001).Players participate in the knowledge construction in a social context.

Roberts and McInnerney (2007) defends that some students have antipathy towards group work and the consequent lack of interest in this kind of educational activities.

Lack of effectiveness and poor learning results if no instructional measures or support are added in order to guide the learning process (Leemkuil et al., 2003; Kirschner, Sweller & Clark, 2006).

Need of regulation for both individual and collectiveactions in this kind of environments (Azevedo, 2008).

Challenges for Collaborative GBL

1. Introduction: collaborative GBL

It is quite possible that arguing does not lead to more understanding of the issue: people stick to their own

viewpoints, or peers do not present very strong arguments (Munneke et al., 2007).

Types of GA (Gutwin, C. & Greenberg, S. , 1995)

• Activity Awareness• Social Awareness• Action Awareness• Knowledge Awareness• …

“The understanding of the activities of others which provides a context for your own activity in a Collaborative learning or working situation” (Dourish and Belloti, 1992; p. 1).

1. Introduction: KGA support in Collaborative GBL

Knowledge Group Awareness (KGA)

Group Awareness (GA)

ST2: 3/10

ST. 2

ST1: 10/10

Being informed about partners’ knowledge and sharing this state (Dehler et al., 2010); a representation of other’s knowledge built in a collaborative environment in order to create a shared understanding of a task (Nickerson, 1999).

ST. 1

Types of GA (Gutwin, C. & Greenberg, S. , 1995)

• Activity Awareness• Social Awareness• Action Awareness• Knowledge Awareness• …

“The understanding of the activities of others which provides a context for your own activity in a Collaborative learning or working situation” (Dourish and Belloti, 1992; p. 1).


Knowledge Group Awareness (KGA)

Group Awareness (GA)

ST2: 3/10

ST. 2

ST1: 10/10

Being informed about partners’ knowledge and sharing this state (Dehler et al., 2010); a representation of other’s knowledge built in a collaborative environment in order to create a shared understanding of a task (Nickerson, 1999).

ST. 1

Gutwin and Greenberg (1995) found that GA is required to coordinate activity, manage shared resources, and

understand the overall state of an activity.”

Does my teammate have previous knowledge on the task?

Which were his individual answers to the collaborative task ?


Is he sure of his performance in the game?

Group Awareness Widgets (GAw) are tools or functionalities providing the learners with social information promoting group awareness (Kreijns & Kirschner, 2002; Jermann et al., 2001)

For promoting KGA, these widgets require players’ Knowledge Elicitation (KE). We consider 3 types of KE (pre, per and post gaming):

1. Previous knowledge level2. Knowledge as performance in the game3. Level of certainity or (confidence CL)

Does my teammate have previous knowledge on the task?

Which were his individual answers to the collaborative task ?


Is he sure of his performance in the game?

How can players share this information in a collaborative game environment?

Group Awareness Widgets (GAw) are tools or functionalities providing the learners with social information promoting group awareness (Kreijns & Kirschner, 2002; Jermann et al., 2001)

For promoting KGA, these widgets require players’ Knowledge Elicitation (KE). We consider 3 types of KE (pre, per and post gaming):

1. Previous knowledge level2. Knowledge as performance in the game3. Level of certainity or confidence (CL)

Metacognition

Feeling Of Knowledge (FOK) is a metacognitive feeling of how people determine what they know about a question before actually answering it operates whenever memory is required (Reder & Ritter, 1992).

1. Introduction: Description of Collaborative KGA tools

Judgment of learning (JOL): the retrieval after the process of learning (Efklides, 2005). It can present a positive (JOL+) or negative tendency (JOL-).

Learning

Accuracy of retrieval Certainty Level (CL) : a metacognitive process that

expresses how sure a person is about the correctness of his or her own performance, belief or knowledge state (Leclercq, D. & Poumay, M., 2008).

Metacognitive processes

Metacognition

Feeling Of Knowledge (FOK) is a metacognitive feeling of how people determine what they know about a question before actually answering it operates whenever memory is required (Reder & Ritter, 1992).

1. Introduction: Description of Collaborative KGA tools

Judgment of learning (JOL): the retrieval after the process of learning (Efklides, 2005). It can present a positive (JOL+) or negative tendency (JOL-).

Learning

Accuracy of retrieval Certainty Level (CL) : a metacognitive process that

expresses how sure a person is about the correctness of his or her own performance, belief or knowledge state (Leclercq, D. & Poumay, M., 2008).

Metacognitive processesI think I have learnt little this

last hour because I was

asleep

I’m absolutely sure Lisbon is the Capital of

Portugal.

I know the name of my

peer but I can’t recall it!

14

2. Methodology

(Meta)knowledge sharing among the members of a dyad

2. Methodology: General Research Design

• Students’ on-task interaction

Qua

si-e

xper

imen

tal d

esig

n Control group

Knowledge Elicitating

• Individual performance in Games

• Collaborative performance in Games

• Students’ on-task interaction

• Individual performance in Games

• Collaborative performance in Games

+

+

++

16

2. Methodology: hypotheses

Performance

Individual Correction Discussion Collaborative

H1: Individual CL expression leads

to better performance

(accuracy)

H2a: Sharing the CL with the dyad

peer leads to better confidence

accuracy.

H2b: Shared visualisation of CL leads to a better

collaborative performance.

H3: Shared visualisation of CL leads to more on-

task comments between peers.

H4: CL improvement

within the diferent stages should be

observed.

2. Methodology: Context

“Análisis y Planificación Financiera I”

eF-Game: 1st draft

Grades in BBA and Law

We quantify metacognitive processes and therefore study the KGA and its relation with game performance in a collaborative game by: the Confidence or Certainty Level (CL) elicitation in a 3-colour scale, our KGAw.

2. Methodology: The traffic Light tool design

Red “I’m not sure of my answer, I could have chosen either A or B.”

Amber “I’m quite sure of my answer.”

Green “I’m totally sure of my answer”

2. Methodology: the eF-Game

Our KGAwidget:

The eF-Gameside 1: instructions

2. Methodology: Game Play

21

2. Methodology: participants

Number of participants

Previous Experience in finance (5-point Likert Scale)

Previous Knowledge in finance(5-point Likert Scale)

Age

8 (4 dyads) M= 2.25SD= + 1,08

M= 2.13SD= + 1,24

M= 35.63 SD=+ 4,10

8 (4 dyads) M= 2.37SD= + 1,30

M= 2.37 SD= + 1,30

M = 39.63SD= + 6,19

Participants in the PMD 2011 – Valencia

22

3. Results

23

3. Results

Performance


M= 5,13SD= 0,83

M= 5,25 SD= 0,89

M= 0,75SD= 0,46

M= 5,38 SD= 0,74

M= 5,13 SD= 0,99

M= 5,50SD= 0,53

M= 1,75SD= 1,66

M= 5,50SD= 0,76

Confidence Level

M= 9,25 SD= 2,31

M= 9,38 SD= 1,60

M= 10,00SD= 1,93

24

3. Results

Performance


H1: Individual CL expression leads

to better performance

(accuracy)

H2a: Sharing the CL with the dyad

peer leads to better confidence

accuracy.

H2b: Shared visualisation of CL leads to a better

collaborative performance.

H3: Shared visualisation of CL leads to more fon-

task comments between peers.

H4: CL improvement

within the diferent stages should be

observed.

VX XX

X

25

4. Discussion

26

4. Discussion

Performance


H1: failed probably due to the size of the sample and the homogeneity of the student’s financial previous level.

H2a & b: CL Dyads performed slightly better, no significant differences were found, also due to size of the sample and because the face-to-face environment: control group could also infer also knowledge cues from partners.

H3: dyads using KGAw were focused in on-task discussions (Brennan and Williams, 1992). Further research needed.

H4: the little evolution of confidence elicitation through the game can be due to homogeneity and sample size.

4. Discussion: future research

eF-Game V1.1

We are working on a web based, multi-language SG to monitor online discussions and a 10-graded Confidence Level elicitation tool, that should help peers share metacognitive information (Padrós, Romero & Usart, 2011).

Virtual Players

Chat monitoring

Spanish and English version

Personalized avatars

Virtual teacher

Different scenarios

References

28

4. References

Azevedo, R. (2008). The role of self-regulation in learning about science with hypermedia. In D. Robinson & G. Schraw (Eds.), Recent innovations in educational technology that facilitate student learning. (p.127–156).

Brennan, S. E., & Williams, M. (1995). The Feeling of Another's Knowing: Prosody and Filled Pauses as Cues to Listeners about the Metacognitive States of Speakers. Journal of Memory

and Language, 34, 3, 383.

Burgos, D., Tattersall, C., & Koper, R. (2007). Re-purposing existing generic games and simulations for e- learning. Computers in Human Behavior, 23, 6, 26-56.

Dehler, J., Bodemer, D., Buder, J., & Hesse, F. W. (2011). Guiding knowledge communication in CSCL via group knowledge awareness. Computers in Human Behavior, 27, 3, 1068-1078.

Dourish,P. & Bellotti, V. (1992). Awareness and coordination in a shared workspace. In Proceedings of the ACM Conference on Computer-Supported Cooperative Work (CSCW ’92, Toronto, Canada, Oct. 31–Nov. 4), M. Mantel and R. Baecker, Eds. ACM Press, New York, pp. 107–114.

Efklides, A. (2005). Metacognition and affect: What can metacognitive experiences tell us about the learning process?. Educational Research Review, 1, 1, 3-14.

Gee, J. P. (2005). Learning by Design: Good Video Games as Learning Machines. E-learning, 2, 1, 5-16.

29

4. References

Gutwin, C., Greenberg, S. (1995). Support for Group Awareness in Real Time Desktop Conferences. In: Proceedings of The Second New Zealand Computer Science Research Students’ Conference, University of Waikato, Hamilton, New Zealand.

Gutwin, C., Roseman, M. & Greenberg, S. (1996). A Usability Study of Awareness Widgets in a Shared Workspace Groupware System.

Herz, C.J. (2001). Gaming the system: What higher education can learn from multiplayer online worlds. The Internet and the University, Educause Forum on the Future of Higher Education http://www.educause.edu/ir/library/pdf/ffpiu019.pdf [retrieved 21/02/2011]

Jacques, D. (1995). Games, simulations and case studies. A review. In D. Saunders (Ed.), The simulation and gaming yearbook. Vol. 3: Games and simulations for business. London: Kogan Page.

Jermann P., Soller A. & Muehlenbrock, M. (2001). From mirroring to guiding: A review of the state of art technology for supporting collaborative learning. Proceedings of the First European Conference on Computer-Supported Collaborative Learning, Maastricht, The Netherlands, 324-331.

Kim, Y., & Baylor, A. L. (2006). A Social-Cognitive Framework for Pedagogical Agents as Learning Companions. Educational Technology Research and Development, 54, 6, 569-596.

Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching. Educational Psychologist, 41, 2, 75-86.

Kirriemuir, J. & McFarlane, A. (2004). Literature review in games and learning: A report for NESTA Futurelab. Bristol: NESTA Futurelab.

Kreijns, K., & Kirschner, P. A. (2002). Group awareness widgets for enhancing social interaction in computer-supported collaborative learning environments: design and implementation. Proceedings - Frontiers in Education Conference, 1.

Leclercq, D. & Poumay, M. (2005). Three metacognitive indices for realism in self-assesment.LabSET, University of Liège. http://www.labset.net/media/prod/three_meta.pdf. [Retrieved 09/08/2011]

Leemkuil, H., de Jong, T., de Hoog, R., and Christoph, N. (2003). KM Quest: A Collaborative Internet-Based Simulation Game. Simulation & Gaming, 34, 1, 89-111. Malouf, D. B. (1988). The effect of instructional computer games on continuing student motivation. The Journal of Special Education, 21, 4, 27-3 8.

Munneke, L., Andriessen, J., Kanselaar, G., Kirschner, P. (2007): Supporting interactive argumentation: Influence of representational tools on discussing a wicked problem. Computers in Human Behavior 23, 3, 1072

Nickerson, R. (1999). How we know – and sometimes misjudge – what others know: Imputing one’s own knowledge to others. Psychological Bulletin, 125, 6, 737–759.

Reder, L. M., Ritter, F. E. (1992). What Determines Initial Feeling of Knowing? Familiarity with question terms, not with the answer. Journal of experimental Psychology: Learning, Memory and Cognition,

18, 3, 435-451.

Roberts, T.S., McInnerney, J.M. (2007): Seven Problems of Online Group Learning and Their Solutions. Educational Technology & Society 10, 4, 257–268

4. References

http://www.labset.net/media/prod/three_meta.pdf

Thank you very much for your attention!

Lisbon, 19/10/2011

Mireia Usart

ESADEDirection of Educational Innovation and Academic Quality (DIPQA)http://www.esade.edu

31

http://www.esade.edu/