Early Warning System for Identifying students at risk of

failingRoger Brown

Center for Educational TechnologyUniversity of Cape Townroger.brown@uct.ac.za

Overview

PART 1• Why we started looking at

EWS• The functional requirements

of the system• Institutional fit• Matching Vula (Sakai)

affordances to the functional requirements

• Developments that would allow Sakai to act as an EWS (well some of them anyway!)

• How UCT is moving forward

PART 2 - Discussion• Activity and course grade• Your ideas

12th Sakai Conference – Los Angeles, California – June 14-16 2

Part1: Introduction

• In 2009 Senate Re-admission Review Committee recommended that greater attention needed to be given to Faculty EWS.

• The SRRC report recommended that:• the SRRC monitors data from the Faculty EWS with the

aim of assessing and reporting on the impact mid-year exclusions have on throughput rates, and

• Investigate the EWS issue to assess the most effective approach to adopt a single system thereby providing consistency across faculties.

EWS – Functional requirements

• The ability to record a standard number of “in course”results/performance/grade

• The ability to create a current class list of all valid registered students in a course, populate this list with “in course results”, and load these to the students’ PeopleSoft record.

• Specification of an “at risk” threshold value for these results• Recording of comments against a student• Generation of communication to identified students• Retentions of a permanent record of these communications• Specified reporting of student performance

• For a student within a course across all courses• For a specified cohort of students

• Access and authorisations to entry grades, viewing of grades and running of reports, course conveners and/or mentors and/or other “intervention” managers

EWS – Institutional criteria

• Technical Implementation

• Technical Integration

• Usability and User Support Requirements

• Technical Support

• Security and Authorisations

• Student Access

• Overall Reporting Capacity

• Cost (of licensing, implementation and support)

• Vendor and Product Sustainability

EWS Functional requirements and Vula (Sakai 2.7) -Integration

IDvault

PeopleSoft HEDA - Higher Education Data Analyzer Demographic and K12 data (currently used by IPD)

Vula: Affordances vs Requirements

Groups: Easily created and populated

Configurable Roles: Accessand authority

Gradebook

Vula: Affordances vs Requirements

Communication: EmailInternal messageSMS

• Secure• Familiar• Widely accepted by admin/academic staff (2474 staff used Vula in 2010)

Vula: Affordances vs Requirements -Gradebook

1. Import marks from excel

2. Weighting and Categories

3. Integrates with Vula testing tools

Assessing students at risk?

Out of Gradebook Back to My Workspace

Vula: Limitations (currently)

• “At Risk” assessment would need to be done outside Vula

• Not all lecturers/course convenors use Vula

• Vula is not the authoritative source of marks

• Vula does not “push” data to PS

• Staffing

Vula: R&D for EWS application

• Automated Gradebook export to ETL platform or preferably an internal logic

• Internal or external algorithm development for risk analysis

• Auto grouping based on risk analysis

• Reporting communications, display, etc

• Predictive logic based on previous student performance

How UCT is moving forward

The task team recommended in phase 1

• EWS to utilise the functionality of PeopleSoft (some developments required)

• Improve the integration of Sakai and PS

• gradebook export to PS (it’s easier to get grades into GB than into PS )

• Samigo & Asn => GB => PS

12th Sakai Conference – Los Angeles, California – June 14-16 13

Part 2: Vula: Final grade vs all events (PSY1001W)

Your Ideas

12th Sakai Conference – Los Angeles, California – June 14-16 15

?

Predictive modelling

Learning analytics

GB in students’ “My workspace”

Data mining – automating and exposing

Using ETLCommunicating “failure”

A “read only” SU role in Sakai

More than grades only? – is attendance a predictor?