What is a successful learning process in a MOOC?Wil van der Aalst, Joos Buijs, Hanneke Duisterwinkel, Maikel Leemans

TU/e @ MOOC’s

• Partner of Coursera since 2013• Showing the excellence of TU/e • Emphasize niche• International brand awareness• Partnership with Eurotech• Experimenting with new way’s of teaching to

improve our on-campus education

PAGE 215-04-2023

15-04-2023

Projectteam MOOCs

Professor

Information Expertise Center

Communication Expertise Center Video production Educational

support ICT Services Data expertise

PHD Projectleader Support

PAGE 3

4

Statistics (first run)• 42.146 enrollments• 821 signature tracks• 178 different countries• 671.927 videos watched• 36.751 quizzes submitted• 4,415 forum post made

  

Statistics• 16.091 enrollments• 164 signature tracks• 153 different countries• 204.843 videos watched• 12.083 quizzes

submitted• 1.031 forum post made

 

New MOOC projects:

• Eurotech MOOC • One MOOC on Energy Efficient Building and

Communities developed by: DTU, TUM, EPFL and TU/e

 • EIT ICT First Year online master Embedded Systems

• Six specializations in total • Two specialization from TU/e

PAGE 515-04-2023

15-04-2023

What is Process Mining?

PAGE 6

15-04-2023

Process Mining on MOOCs

PAGE 7

15-04-2023

Our ‘process model’: structure!

Nov 12 Nov 19 Nov 26 Dec 3 Dec 10 Dec 17 Dec 24 Dec 31 Jan 7

Week 1

Week 2

Week 3

Week 4

Week 5

Week 6

Final Quiz

Tool Quiz

Peer Ass.

Peer Review

PAGE 8

A week: ~8 videos of ~15 min. each

+ a week quiz

For ‘distinction’

15-04-2023

General Statistics

PAGE 9

Registered Visited Watched a lecture Browsed forums Submitted exercise Obtained certificate

43218

29209

20868

5845 5798

1688

68% 71% 28%

29%

15-04-2023

Registration behavior

PAGE 10

15-04-2023

Annotate students

• Achievement:• Failed• Successful

− Normal− Distinction

• Signature track:• Yes• No

• Lectures watched• Week 1 only• Week 1&2 only• Week 1,2 and 3• All weeks• Any (e.g. at least 1)

PAGE 11

15-04-2023

Basic Group Statistics

Signature Track?

Certificate #Students #Watched Videos

Avg. per student

Yes Failed 226 8,069 36

Normal 223 15,367 69

Distinction 285 21,392 75

No Failed 13,465 143,721 11

Normal 702 41,467 59

Distinction 293 19,203 66

PAGE 12

Heterogeneous group:- 10,996 watched lecture from week 1

(3,451 week 1+2, 1,407 1+2+3)- Only 269 failed students watched a video

of each week

15-04-2023

Signature Track vs Non-Sign.

PAGE 13

Certificate Weeks Watched

#Students Avg. videos per student

Sign. Non-S Sign. Non-S

failed all weeks 39 269 79 69

failed any 226 13,465 36 11

successful all weeks 354 628 92 83

successful any 508 995 72 61

normal all weeks 158 427 87 82

normal any 223 702 69 59

distinction all weeks 196 201 97 86

distinction any 285 293 75 66

Signature track always watches

more videos than non-signature.

Failed students watch many, but less videos than

successful students

15-04-2023 PAGE 14Behavior Over Time

LateStarters

Christmas

Delay for‘yellow’

15-04-2023 PAGE 15

DistinctionNormal

Signature

Non-Signature

Failed

~PreviousSlide

Seem togive up

More intensewatching of weeks 5 & 6

Followschedule

15-04-2023

Do Successful Students Study Sequentially?

PAGE 17

Which group studies most ‘sequentially’?

Signature yes/no?

Grading failed/normal/distinction?

Watched weeks 1-2-3/all/any?

signature grading weeks % matchno failed all weeks 0.660    no normal all weeks 0.634    no distinction all weeks 0.624    yes normal all weeks 0.593    no distinction any 0.592    no normal any 0.591    yes failed all weeks 0.573    yes normal any 0.571    no failed week 1,2,3 0.548    yes distinction all weeks 0.547    yes distinction any 0.534    no failed week 1,2 0.466    

15-04-2023

Behavior of Non-Sign. Failed Students that Watched all Weeks

PAGE 18

Week 1

Week 6

Each lecture watched 10-20 times by 269 students

overall

15-04-2023

But how do Successful Students Watch?

PAGE 19(285 Students on signature track that got distinction)

Intro & Week 1

Week 2 & 3 Week 4, 5 & 6

15-04-2023 PAGE 20

15-04-2023 PAGE 21

Towards deadlines of final quiz and peer assignment watch behavior (far) less structured

Weeks 5 & 6 very unstructured

15-04-2023

Behavior Compared

PAGE 22

Normal Cert. Cert. with Distinction

Signature Track Sequence but every lecture can be skipped.Week 6 is unordered.(223 students)

Week 2-4 structured,Week 1 unorderedWeek 6 very unstructured and possible repetitive(285 students)

Non-Signature Track Sequence, even week 6.Every lecture can be skipped.(702 students)

Sequence, but every lecture can be skipped.Week 6 is very unstructured and repetitive(293 students)

15-04-2023

Behavior Compared

PAGE 23

Normal Cert. Cert. with Distinction

Signature Track Rather structured. Some lectures in different orders.Difficult lecture watched multiple times.Week 6 unordered.(158 students)

Sequence for some weeks without skips.Week 1 unordered.Difficult lectures repeated.Weeks 4 and 5 unstructured.(196 students)

Non-Signature Track Very sequential(427 students)

Very sequential(201 students)

For students that watched at least 1 Lecture of each week

15-04-2023

Overall Conclusion

1. Successful students do not necessarily study structurally

• Distinction and/or signature students have significant preliminary knowledge?!?!

2. Structured studying does not guarantee success

3. Signature track students aim for a certificate while non-signature track students aim for knowledge?

PAGE 24

15-04-2023

Future Analysis

• Include more information• Quizzes (and their results)• Analyse timing of watching and quiz making• Forum interaction

• Compare runs of the same MOOC• Who re-joined?• Is the 2nd group of students different? (gut feeling: yes!)

PAGE 25

http://www.edsa-project.eu/

15-04-2023 PAGE 26

15-04-2023 PAGE 27

15-04-2023

Data Structure

PAGE 28