Upload
niki-lambropoulos
View
562
Download
1
Embed Size (px)
DESCRIPTION
Quality In Computer Supported Collaborative eLearning by Lambropoulos & Romero in Madrid at the Quality eLearning Workshop coordinated by Dr. Margarida Romero http://www.expoelearning.com/
Citation preview
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learning
Jueves, 25 de febrero de 2010
Quality Assurance in Computer Supported Collaborative Learning
Environments
Euro-CAT Ergonomic Evaluation
Dr. Niki Lambropoulos * Dr. Margarida Romero
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
• What is Quality?
• Utility, usability, acceptability– Case study: Toll staff training (learning situation + ergonomic
criteria)
• CSCL ergonimic evaluation
– Case study: Euro-CAT Ergonomic Evaluation
• 5 quality criteria
Quality Assurance in Computer Supported Collaborative Learning Environments
Agenda
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
What is quality?
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
• Fit of purposeBergman and Klefsjö (1994):
“The quality of a product (article or service) is its ability to
satisfy
the needs and expectations of the customers”
• European Foundation for Quality in eLearning (EFQUEL)
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
1.
Utility, usability, acceptabilityCase study: Toll staff training (learning situation + ergonomic criteria)
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010Tooll staff training needs
• USA Toll Tariffs:No training is necessary, even
children can became toll staff ! • European Toll Tariffs:• Intensive training needs to achieve 2 main
learning objectives1. Vehicle category identification2. Angry management
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Where and when can a toll worker could follow an elearning course?
During the work hours?
During the coffee break?
At home?
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010As if the learning situation were not enough complex.... we will consider the usability criteria Is the
course useful?
Is the course easy to
use?
…mmmh, if the course is not useful, compatible to my workplace and out of the work life, reliable… and my colleagues don’t like the course, will I accept
the course?
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
• As if the learning situation was not complex enough.... we will consider the ergonomics criteria
– UTILITY– USABILITY – ACCESSIBILITY
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010As if the learning situation were not enough complex.... we will consider the usability criteria
UTILITY
Relevance and efficacy (Tricot, 2007) for the performance in academic settings or the workplace
Is the course
useful?
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010As if the learning situation were not enough complex.... we will consider the usability criteria
Is the course easy to
use?
USABILITY
Computer Learning Environments easy to learn and
easy to use (Preece et al, 1994)
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010As if the learning situation were not enough complex.... we will consider the usability criteria
ACCEPTABILITY Not only consider the usefulness and usability,
but also robustness, cost and reliability of ICT applications
…mmmh, if the course is not useful, compatible to my workplace and out of the work life, reliable… and my colleagues don’t like the course, will I accept
the course?
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010As if the learning situation were not enough complex.... we will consider the usability criteria
SOCIAL ACCEPTABILITY Not only consider the usefulness and usability,
but also robustness, cost and reliability of ICT applications
Joel says the course is nuts; for me, is quite
boring, and you?
No, Joel is nuts, the course is just
not enough challenging and
interactive
[Joel] The course don’t work in my computer… I needed to use my son computer to
follow the course…
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Nielsen’s Taxonomy of System Acceptability
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Toll staff training – Case study Conclusions
1. In addition to pedagogical criteria we need to consider the utility, usability and (social) acceptability criteria
2. All these criteria need to be considered for a successful elearning course
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
2.
CSCL ergonomic evaluationCase study: Euro-CAT Ergonomic Evaluation
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
HCI-Ed in 7 Steps
1.1. Context & Learning Values - HypothesisContext & Learning Values - Hypothesis2.2. (Iterative) Design – Requirements (Iterative) Design – Requirements 3.3. Evaluation with user groups/expertsEvaluation with user groups/experts4. Development5. Evaluation with user groups6. Re-Design & Development7. Study & Research - Tool Release
1CH 2DR 3E 4D 5E 6D 7RTR
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Research Problem
Distributed learning teams in team-based learning situations aim to achieve organisational convergence.
However, there are collaboration problems due to mediated distance & time flexibility.
1CH 2DR 3E 4D 5E 6D 7RTR
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
1. Key CSCeL concept: Collaboration Awareness Students’ difficulties Group Awareness Time convergence
2. Learners’ Intentions (learning goals): Task Convergence3. Users’ operations (actions/tasks): Pedagogical Usability
Step1a: Context & Learning Values
1CH 2DR 3E 4D 5E 6D 7RTR
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Being HumanCollaboration Awareness
Students’ difficulties: In addition to f2f Collaborative Learning difficulties, CSCeL situation adds Computer Mediated Communication (CMC) and ILE difficulties (Kirschner,
2009)
Group Awareness: understanding of the activities of others, which provides context for your own activity (Dourish &
Bellotti, 1992)
Group time awareness: Distributed learning students need to develop an awareness of their team-mates’ temporal patterns in order to regulate the Time-on-Task efficiently in the collective level (Romero, 2010)
1CH 2DR 3E 4D 5E 6D 7RTR
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
3C Model: Communication, Coordination & knowledgeConvergence (Ellis et al, 1991)
How?
1CH 2DR 3E 4D 5E 6D 7RTR
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
If group awareness is a prerequisite for convergence then increase of group awareness will result in increase of
organisational convergence and thus, collaboration awareness.
Step1b: Hypothesis
1CH 2DR 3E 4D 5E 6D 7RTR
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Euro-CAT…• Collaboration Awareness Tool • Task-independent• Supports 3C via
– Mirroring capabilities– Awareness cues
• Does not replace other 3C tools
Step2: (Iterative) Design - Requirements
1CH 2DR 3E 4D 5E 6D 7RTR
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Step2: (Iterative) Design - Requirements
The design can be defined as an individual and collective activity, finalized by a project to develop a physical and symbolic artefact. The artefact is rarely well defined in this step.
1CH 2DR 3E 4D 5E 6D 7RTR
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Key Concepts for Design
Group ProgressUsersUsers
Scheduling
Trail tracking
NowNow
Awareness Cues
1CH 2DR 3E 4D 5E 6D 7RTR
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Euro-CAT Screenshot
1CH 2DR 3E 4D 5E 6D 7RTR
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Step3: User/Expert Evaluation @ this workshop
Euro-CAT Ergonomic Evaluation – Pedagogical Utility/Usability/Accessibility Questionnaire &
System Usability Scale (SUS)– Experts/users evaluation– Think aloud protocol
1CH 2DR 3E 4D 5E 6D 7RTR
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Results 1a/3: Questionnaire
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Results 1b/3 Questionnaire
EuroCAT Ergonomics Mean Std. Deviation
Walk up Usability 4.4000 .54772
Ease of Use 4.0000 .70711
Functionality Integration 3.6000 .54772
Learnability 4.2000 .44721
Interface Attractiveness 2.4000 1.14018
Originality 3.6000 1.51658
Collaboration 3.0000 .70711
NavigationNavigation 4.8000 .44721
Task Regulation 3.8000 1.48324
Message Legibility 4.0000 1.22474
Observe Availabilities 4.2000 .83666
Facilitate Group Work 3.2000 .83666
Overall Satisfaction 3.8000 .83666
Overall Enjoyability 3.6000 1.14018
Overall Usability 3.0000 .70711
Overall Score 3.706/5.00
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Results 3a/3
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Results 3b/3
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Results 3c/3
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Solutions
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
• Ask the e-tutors/e-learners for solutions!• Direct fit between educational task and the method chosen
to pursue it• Iterative Design
– Engaging different user/expert groups• Next step: Evaluation in context
Conclusions & Insights
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
1. Utility
2. Usability
3. Acceptability
4. Learner/user-centred design (HCI-Ed)
5. Adaptive eLearning:
- Diverse Teaching/Learning Styles
- Learning Alone/Learning Together
- Learning anywhere/anytime
5 criterios de calidad en E-learning
Ergonomics
Virtualc@mpus Internacional 2010La calidad en contenidos e interacciones en e-learningJueves, 25 de febrero de 2010
Thank you!
Questions?
Dr. Niki Lambropoulos
Dr. Margarida Romero