Loureiro et al. (2011). E-assessment in collaborative blended learning
E-assessment in collaborative blended learning: evolving practices and students’
perceptions
Loureiro, MªJoão, Pombo, L. Balula, A. Moreira, A.
Loureiro et al. (2011). E-assessment in collaborative blended learning
Sumary
• problem/research questions• context• user generated content experience
– objectives– methodology
• results• final considerations
Loureiro et al. (2011). E-assessment in collaborative blended learning
interactioninteraction
how to improve?
practice research
• problem
policy/management
Loureiro et al. (2011). E-assessment in collaborative blended learning
bLearning•nature
•teaching strategies
•learning
•e-assessment
focus
impact
Online
context
practices (evolve, improve, innovate)
research
Loureiro et al. (2011). E-assessment in collaborative blended learning
• evolution/research questions
EES(MsD)
DE(PhD 2008/09)
DE(PhD 2010711)
evauation of teaching
(students’ perceptions)
evauation of PA activities
assessment of coll. work
(students’ perceptions)
assessment of coll. work
(students’ perceptions)
quality of peer assessment
evauation of PA activities
Loureiro et al. (2011). E-assessment in collaborative blended learning
– PhD program in Multimedia in Education module• Distance Education (DE) – two editions (2008/09 and 2010/11)
– blearning regime • face-to-face and at a distance• group work
– DE module aim - develop research competences – • search, select, systematize, synthesize and literature related with DE;• communication, collaborative work and assessment competences (self-
and peer- assessment) “good enough doctorate” (Wisker, 2010)
Loureiro et al. (2011). E-assessment in collaborative blended learning
– PA benefits:
• promote higher order thinking, developing students’ critical thinking, communication, lifelong learning, problem solving…
• increase the amount of feedback, from teachers and from peers• enhance students’ sense of ownership, responsibility and students’
motivation, since they find it useful, attractive and enjoyable• support active and autonomous learning • can avoid the involvement of free-riders in group work, i.e,
facilitate the identification of individual contributions
Boud and Falchikov (2007), Joordens et al. (2009), Li et al. (2009, 2010), Peng (2010), Rourke et al. (2008), Sluijsmans et al. (2004), Topping (2000, 2008, 2010),
Zundert et al. (2010)
Loureiro et al. (2011). E-assessment in collaborative blended learning
– PA pitfalls:
• students’ attitudes toward PA may hinder the above mentioned benefits – unconfidence towards PA –> resistant toward PA
• PA can be subjective – students might be too hard, critical or boasting, or not used to assess their peers -> increase students’ anxiety
• PA is time-consuming – training, preparation, and monitoring are needed
– PA advantages increase – formative feedback
Boud and Falchikov (2007), Joordens et al. (2009), Li et al. (2009, 2010), Peng (2010), Rourke et al. (2008), Sluijsmans et al. (2004), Topping (2000, 2008, 2010),
Zundert et al. (2010)
Loureiro et al. (2011). E-assessment in collaborative blended learning
• User Generated Content (UGC) experience and methodology
– case studies (blearning courses/user generated content – literature review ODL) / action research (reflexion on/after action).
– strategies - project work, problem solving, collaborative learning and assessment for learning -> quality in online learning.
– duration of the DE module, participants & tools:• 2008/09: five-week period – 24 students – Wiki (
http://ead0809.wetpaint.com), Google Docs, EndNote Web…;• 2010/11: four weeks) – 18 students – Ning (http://
labconteudosdigitais.ning.com), Google Docs, EndNote Web…
Loureiro et al. (2011). E-assessment in collaborative blended learning
• changes in 2010/11 edition:
– provided extra opportunities for shared PA and use of assessment criteria of the group work • it was used for formative e-assessment of three versions of
the ongoing work instead of one;
– assignment made by email -> confidentiality -> authenticity.
Loureiro et al. (2011). E-assessment in collaborative blended learning
• results (difficulty and relevance of PA activities)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Self and peer-assessment of the group work and of the develloped competences
Weekly workgroup reflexive analysis
Survey of perceptions about how to make a literature search…
2010/11 Academic Year
Difficulty Relevance
2008/09 20010/11
Loureiro et al. (2011). E-assessment in collaborative blended learning
• results (privacy, participants and components )
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%
Collabarative work should be assessed as a whole with the same mark for all elements
Collabarative work should beassessed with several components
Collabarative work should be assessed individually, by teachers and by colleagues
Collabarative work should be assessed only by teachers
Self and peer assessment of the group works should be published in the module's site
Self and peer assessment of developed competences (inside each group) should be published in the
module's site
Academic Year - 2010/11
completely disagree disagree agree completely agree don't know
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%
Collabarative work should be assessed as a whole with the same mark for all elements
Collabarative work should beassessed with several components
Collabarative work should be assessed individuall, by teachers and by colleagues
Collabarative work should be assessed only by teachers
Self and peer assessment of the group works should be published in the module's site
Self and peer assessment of developed competences (inside each group) should be published in the
module's site
Academic Year - 2008/09
completely disagree disagree agree completely agree don't know
2008/09 20010/11
Loureiro et al. (2011). E-assessment in collaborative blended learning
• final considerations - connections with UGC state of art1. definition of UGC
content created by postgraduate students (literature reviews); shared freely; content has not been evaluated through a process of formal peer
review (journals); students performed self-reflexive task concerning the groupwork
and the developed competencies.
2. quality in elearning– activities fit the students’ needs and enriched learning context,
active involvement in the production of a literature review (product and processes).
Loureiro et al. (2011). E-assessment in collaborative blended learning
• final considerations - relations with the CONCEDE framework
– CONCEDE framework for peer creation - peer-validation• writing process involved several activities: authoring, editing,
updating, revising, reflecting…; • DE module has been evaluated by different stakeholders:
– students;– university professor not involved in the module - peer-evaluation; – formal peer-assessment - submitted for publication (journals and
international conferences, the experience has been evaluated at the first and second level of the quality pyramid.
Loureiro et al. (2011). E-assessment in collaborative blended learning
• final considerations– limitations - personal experience, few contexts, number of
students/comparability, qualitative approach…– further work
• content analysis (PA, reflexive activities, products…), triangulation• share, dissemination -> transfer to other context
– old - improve students learning - meets new???• does the available ICT tools help, how, in what circumstances? • is its use sustainable?
– environment, economical, social development impact???– inclusion or exclusion??)
Loureiro et al. (2011). E-assessment in collaborative blended learning
questions and comments???
Loureiro et al. (2011). E-assessment in collaborative blended learning
%0 20 40 60 80 100
Use of the negotiated criteria
Adequacy of the chosen vocabulary
Provision of constructive feedback(criticisms)
Provision of constructive feedback(questions)
Provision of constructive feedback(suggestions for improvement)
Fairness of the score
Adequacy of the final considerations
• results (1th edition DE) – quality of PA
Loureiro et al. (2011). E-assessment in collaborative blended learning
modules organisation (DocProg)
• module organisation