Achieving productive cmc_slideshow_brant_knutzen_ver2

The Teaching and Learning CentreLingnan University

Brant Knutzen

Lingnan University

Social Constructivism

Transactivity

Study #1

Assessment methods

Participation marking scheme

Study #2

CMC : Computer-mediated Conferencing

Synchronous : “chat”, or instant messaging

Asychronous : “discussion forum”

The effective construction of knowledge is a product of the collaborative group

Efficacy has been found to be linked to the process that learners utilize in working on the task together (Fischer et al 2002)

Process: social negotiation of arguments and argument sequences (Leitão 2000; Voss & Dyke 2001)

John Biggs captured the educational value of discussion when he stated:

"Good dialogue elicits those activities that shape, elaborate, and deepen understanding“

(Biggs 1999 p. 5)

Transactivity: the method by which students build on the contributions of their fellow learners

(Berkowitz & Gibbs 1983)

Transactive communication:

Participants respond to and build on each other’s contributions

Peer exchange of information and ideas

Social negotiation of knowledge

Each participant brings their own experiences to apply to a common educational goal

A key theoretical construct for measuring collaboration

How can we describe it in easily grasped ways?

Quantitative

Qualitative

How can we formulate the instructional design conditions which consistently result in more productive and transactive learning activities?

Case study of one course (Knutzen)▪ International school in Hong Kong – secondary level

▪ 1-to-1 laptop blended learning environment

▪ Introduction to Psychology course

▪ Sample size = 24

Investigation of instructional design conditions to achieve a highly productive online discussion

At start of study, average student production in online discussions = 1 post

Four conditions to achieve productive online discussions: Teacher facilitated social formation of small groups

Class time to initiate online discussion interaction

Setting open-ended, challenging topic questions that encourage discussion and debate

Assessment system that reinforces production and peer interaction

At end of study, average studentproduction: over 10 posts per discussion!

Over the following three teaching years:

Extensive use of the online discussion design

Full-time instruction of secondary students

▪ 1-to-1 laptop environment

▪ IT classes

▪ Psychology

Part-time instruction of post-graduate students

Literally hundreds of online discussions

Design continues to result in good production

Traditional – teacher-assessed subjective marking

Review contributions by each student

Award mark based on:

▪ Participation – any contribution to discussion

▪ Interaction - responding and seeking feedback

▪ Transaction – sharing / exchanging useful information and resources

▪ Transformation - ideas and understanding clearly develop as a function

of interaction and transaction

Best method for summative assessment

A highly productive discussion can easily produce over 200 posts!

A teacher can become a victim of their own success

How much time can they devote to quantitativemarking?

How much time remains for qualitativefeedback?

Desired graduate attributes:

Critical thinking skills

Excellent cooperative skills

▪ Integrity

▪ Personal responsibility

Subjective peer-assessment can directly address the development of these attributes

Requires student training

Requires review and evaluation by teacher

Possible problems:

“Revenge grading” or 報復

▪ you gave me a low grade, I will give you a low grade

“Back-scratching” or 賄賂

▪ If you give me a high grade, I will give you a high grade

One solution: objective peer-rating based on participation

No judgment, just rating using a clear system:

Moodle can automatically average these grades!

Moodle averages the peer-awarded marks

Grades produced by participation:▪ One post = 6 -> D-

▪ Two posts = 8 -> B-

▪ Three posts = 8.6 -> B

▪ Four posts = 9 -> A-

▪ Five posts = 9.2 -> A-

▪ Six posts = 9.33 -> A

▪ Seven posts = 9.42 -> A

▪ Eight posts = 9.5 etc

More participation = higher grade

Students cannot mark own work, only others

Awareness of contributions by other students

Team-building incentive

Teacher has plenty of time to:

Monitor progress

Provide qualitative feedback

Name: the “6 / 10 / 10 / 10” peer-rated participation-based marking scheme

Knutzen & Kennedy – study conducted in 2009

Two versions of same course at HKU

Sample size: n = 53

Same lecture content, same 5 discussion topics

Different instructional design of discussions:

Teacher facilitated social formation of small groups

In-class time given to begin discussions

Open-ended topic questions which encourage discussion and debate

Peer-rated participation marking scheme (6 / 10 / 10 / 10)

Quantitative:▪ Production = Total number of discussion posts / n

▪ Interactivity = Total number of feedback posts / n

▪ Group Activity = Total number of discussion posts / # topics

▪ Transactivity = Production × Interactivity

Qualitative: a new type of graphical representation – the “BushGraph”

a non-transactive Moodle discussion forum:

The BushGraph of Class #1 – discussion #1

Each student creates their own discussion topic and posts once, but rarely reads or gives feedback to any other posts – a “lawn”

The same Moodle discussion, with improved instructional design:

The BushGraph of Class #2 - discussion #1

The BushGraph of Class #1 - discussion #2

Class #1 – Discussion #3

The one discussion in Class #1 worth any points!

5% of total grade, teacher assessed

The BushGraph of Class #1 - discussion #4

The BushGraph of Class #1 - discussion #5

Participation marking has high reliability

Participation marking has very high validity

Could there be a further relationship between the discussion grades <-> project grades and the descriptive statistics, such as interactivity?

Class #2 Discussion

Grades

Correlation

With

Final Project

GradesDiscussion # Production Interactivity Group Activity Transactivity

1 3.4 1.75 11.3 5.95 0.608

2 5 2.8 20 14.00 0.757

3 5.56 4.15 22.6 23.07 0.919

4 6.15 3.45 24.6 21.22 0.675

5 6.25 3.55 25 22.19 0.885

As interaction in a discussion goes up, the learning activity becomes more accurate in assessing student performance

We found evidence that the redesign of the instructional design variables to meet the conditions developed in study #1 have a significant positive effect on:

the production in the resulting online discussions

several measures of transactivity:

▪ Interactivity

▪ Group activity

▪ Transactivity (Interactivity × Group Activity)

Peer-marked participation-based scoring method was found to have:

High reliability

▪ Good consistency between discussion grades

Very high validity

▪ Strong correlations to teacher-assessed learning activities and the course total grade

Sample sizes were small

Almost exclusively Asian students

Replication and extension of this study with a larger sample size in other cultural settings would provide additional perspectives with greater predictive validity

Possible additional factor affecting results:

differences in instructor teaching style

emphasis on the importance of online discussions

Determine a causal direction:

is the highly interactive discussion environment conducive to the creation and demonstration of higher levels of understanding?

Or

are students with higher levels of understanding more capable and willing to create interactive discourse?

To achieve highly productive and transactive online discussions in a blended 1-to-1 teaching environment, instructors should incorporate a specific set of four design conditions:

Teacher facilitation of social grouping of students

Class time to initiate online discussion interaction

Setting open-ended, challenging topic questions that encourage discussion and debate

Assessment system that reinforces production and peer interaction

When these instructional design conditions

are successfully incorporated, the potential

benefits of social constructivism as an

instructional design paradigm can be realized

within a blended educational environment.

Berkowitz, M. and Gibbs, J. (1983) "Measuring the developmental features of moral discussion", Merrill-Palmer Quarterly, Vol. 29, pp 399-410.

Biggs, J. (1999) "What the Student Does: teaching for enhanced learning", Higher Education Research & Development, Vol. 18, No. 1, pp 57.

Fischer, F., Bruhn, J., Gräsel, C., and Mandl, H. (2002) "Fostering collaborative knowledge construction with visualization tools", Learning and Instruction, Vol.12, pp 213-232.

Leitão, S. (2000) "The potential of argument in knowledge building" Human Development, Vol. 43, pp 332-360.

Voss, J.F. and Van Dyke, J.A. (2001), "Argumentation in Psychology", Discourse Processes, Vol. 32, No. 2/3, pp 89-111.

Questions?

Send me an email to geta copy of this paper:

BKnutzen@LN.edu.hk

Thank you for coming!