Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Domain-driven competence assessment in virtual learningenviroments. Application to planning and time management

skills

Antonio Balderas, Alvaro Galan-Pinero

J. Antonio Caballero Hernandez, Gregorio Rodrıguez Gomez

Juan Manuel Dodero, Manuel Palomo-Duarte

Universidad de Cadiz

November, 2014

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Content

1 Context & issue

2 ReST web service

3 Model-Driven approach

4 Case study

5 Conclusions and future research

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Table of contents

1 Context & issue

2 ReST web service

3 Model-Driven approach

4 Case study

5 Conclusions and future research

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Higher education: focus has turned from knowledge to skills

Relevance on developing subject-specific knowledge

Development of generic competences

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Learning Managment Systems (LMS) provide teachers with facilities tomanage courses

How to assess generic skills?

Assessment through grading

Difficult to link learning activities with skills in curricula

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Can assessment tools support teachers to evaluate generic skills?

This work presents two alternative answers:

1 A ReST-based web app for (manual) assessment of skills

2 An Model-driven approach for (computer-assisted) assessment of skills

Two LMS-independent computer tools were built up and adapted forMoodle

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Table of contents

1 Context & issue

2 ReST web service

3 Model-Driven approach

4 Case study

5 Conclusions and future research

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

GesCompEval: ReST-based web service & app for learning assessment

Web interface to link skills and learning outcomes with learningactivities and vice versa

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Integrated in Moodle by the GesCompEval MD extension

Uses the EvalCOMIX web service for skill assessment

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Table of contents

1 Context & issue

2 ReST web service

3 Model-Driven approach

4 Case study

5 Conclusions and future research

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

EvalCourse design follows the MDA approach

Eclipse Modeling Framework (EMF) project based on a domainmeta-model to produce an Rich Client Platform (RCP) desktopapplication

Users are teachers with or without computer programming skills

EvalCourse outcomes are in different formats

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

EvalCourse SASQL: a Domain-Specific Language (DSL) designed toquery the LMS for online learning assessment

Evidence name of the indicator:get studentsshow milestones | participation | access | interactionin assignment | workshop | forum | campus | wiki [list ids]from course id | backup.

SASQL syntax

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Table of contents

1 Context & issue

2 ReST web service

3 Model-Driven approach

4 Case study

5 Conclusions and future research

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Case study developed in an course on Functional Programming

Computer Science Degree in the University of Cadiz during the2013-14 academic year

Students’ assessment of planning and time management skills

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

First, EvalCOMIX+GesCompEval used to provide explicit grades foreach skill

e.g. submission time is assessed as a single attribute with 4 values tomeasure time management skills

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Evidences of the skills are marked up in the activity (e.g. forum)

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Evidences and grades are reported

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Table: Grades obtained with GesCompEval

Time management Knowledge in practical situations

anon4 91.75 94

anon5 50.50 94

anon7 66.67 96

anon8 58.75 94

anon9 42.00 100

anon10 25.25 100

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Second, EvalCourse was used in the same course

1st aspect of analysis: checking if students delivered their assignmentson time, delayed, or had pending assignments

SASQL query:

Evidence milestone list:get studentsshow milestonesin assignment.

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Table: List of students’ assignments

Name Ratio Milestones In-time Pending Access

anon4 100 4 4 0 218

anon5 100 4 4 0 78

anon7 50 3 2 1 135

anon8 100 4 4 0 171

anon9 75 4 3 0 149

anon10 50 4 2 0 117

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

After, query issued to find out students’ participation in (e.g. forum)activity

SASQL query:

Evidence forum interaction:get studentsshow interactionin forum.

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Table: Students’ participation in forum activity

Name Debate starter Debate participation Total Access

anon4 1 2 3 64

anon5 3 1 4 36

anon7 1 3 4 75

anon8 0 0 0 48

anon9 0 2 3 28

anon10 3 1 4 41

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Comparison: in Gescompeval, anon4 has a higher grade, the resthave less than 67

Table: Grading obtained by Gescompeval

Plan and manage time Knowledge in practical situations

anon4 91.75 94

anon5 50.50 94

anon7 66.67 96

anon8 58.75 94

anon9 42 100

anon10 25.25 100

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Comparison: in EvalCourse, considering only the activitiesdelivered on time, anon4+anon5+anon8 achieved the max forthis skill

Table: Assignments of the students

Name Ratio Milestones In-time Pending Access

anon4 100 4 4 0 218

anon5 100 4 4 0 78

anon7 50 3 2 1 135

anon8 100 4 4 0 171

anon9 75 4 3 0 149

anon10 50 4 2 0 117

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Comparison: but considering the number of tasks submitted ontime and a number of accesses above the average, only anon4fullfills both indicators

Table: Assignments of the students

Name Ratio Milestones In-time Pending Access

anon4 100 4 4 0 218

anon5 100 4 4 0 78

anon7 50 3 2 1 135

anon8 100 4 4 0 171

anon9 75 4 3 0 149

anon10 50 4 2 0 117

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

These are just examples of (valid?) interpretations the teacher could dowith the reported indicators

The validity of applying certain indicators to assess a particular skill isoutside of scope of this work

Deeper research required, but it provides technical support to do it

As long as teachers manage to provide accurate indicators, they canperform ’objective’ assessment of skills in massive online courses (e.g.MOOCs) through EvalCourse

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Table of contents

1 Context & issue

2 ReST web service

3 Model-Driven approach

4 Case study

5 Conclusions and future research

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Generic skills enable succesful integration of students in authenticlearning environments

But in regular academic contexts, assessing skills is complex andsomewhat subjective

Two alternatives to perform skill assessment:A manual skill assessment approach + GesCompEval ReST-based webservice & app)A computer-assisted assessment approach + EvalCourse model-drivenapproach

GesCompEval + EvalCourse can be applied to assess skills of studentsin terms of their interaction with the LMS or othercomputer-supported learning environments

EvalCourse is aimed for courses with a great number of studentsand/or activities

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Ongoing and future work:

EvalChrome: a Chrome plugin that allows to assess Moodle activitiesthat are developed on external web pages (e.g. a wiki) and mark-upevidences of assessments for further feedback

Connector to link EvalCourse and Mediawiki under development

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Context & issue ReST web service Model-Driven approach Case study Conclusions and future research

Thanks for your attention

GesCompEval:https://www.assembla.com/spaces/inteweb-gescompeval

EvalCourse: https://www.assembla.com/spaces/evalcourse

Antonio Balderas et al.

Domain-driven competence assessment in virtual learning enviroments. Application to planning and time management skills

Aknowledgements: