April 9, 2023

Conceptualizing Interaction & Learning in MOOCs

Rebecca Eynon & Nabeel Gillani

Overall goal

Focus on interaction Communication via discussion forums

1. The development of profiles that reflect the different ways and reasons that people interact with one another in MOOCs

2. How these interaction profiles are related to learner characteristics and course outcomes

Research approach

Case study of one MOOC from Coursera with strong emphasis on encouraging interactions between learners

Use data from this MOOC to develop typology, then use data from future versions of the MOOC

Mixed methodology Visualisation of posts and views, social network analysis, in-depth

interviews, pre and post surveys and qualitative observations Moving between the quant and qual methods to keep refining the

model

Towards an understanding of interaction

Trends in interaction patterns Who? When? What?

Exploring the network What “counts” as interaction? Crowds or communities?

Towards a typology Next steps

The course

6 week course, March to May 2013 4-6 hours per week Assignments

Required: Weekly quizzes, final strategic analysis assignment (evaluated via peer-assessment)

Optional: Discuss business cases in the discussion forums

Multiple sub-forums: Final project, cases, lectures, readings, study groups, questions for

professor, technical feedback, course material feedback

Participation

Over 90,000 registered 49,682 used the lecture videos at least once 20,082 submitted at least one quiz 4,445 posted at least once in discussion forums 2,208 received >= 70%

6

Forum & course participation: location

Continent % Course participants

% Forum participants

North America 32% 32%

South America 7% 10%

Europe 28% 25%

Asia 26% 24%

Africa 5% 6%

Oceania 2% 2%

Forum & course participation: education

Highest attainment % Course participants

% Forum participants

Some high school 1% 1%

Completed high school 3% 4%

Some college 10% 11%

Bachelors 43% 42%

Masters 40% 39%

Doctorate 3% 4%

N=7337

Forum & course participation: score

Final Score % Course participants

% Forum participants

< 50% 97% 72%

50% <= score < 60% 0.1% 1%

60% <= score < 70% 0.1% 1%

70% <= score < 80% 0.14% 2%

80% <= score < 90% 0.21% 3%

90% <= score 2.2% 21%

Patterns of interaction

A framework for forum analysis Analyse communication trends and interactions according to sub-

forums – justified by low participation overlaps between sub-forums

Forum activity is “bursty”, with most activity occurring earlier in the course

Different sub-forums encourage different patterns of information access and contribution

Study Groups

Days since course began Readings

Final Projects

Cases

Different sub-forums encourage different patterns of information access and contribution

Exploring the network: what counts?

Two questions: What do we consider as a "tie" between two learners?  Do we trust the observed ties as meaningful?

Let's assume the observed network is a noise-corrupted version of the true underlying network (Psorakis et al. 2011)

Draw N samples of possible networks, based on thread co-participation Determine the significance of a particular tie in the observed learner-to-

learner network based on the sampled ones

This formulation helps us disregard ties that we attribute to chance (e.g., one-off interactions in a sea of other interactions)

)

Significant networks

Sub-forum (# nodes) # Edges in full network

# Edges in significant network

% Decline

Lectures (617) 12,644 3,988 68%

Readings (1,108) 35,728 11,259 68%

Cases (1,114) 102,171 57,490 44%

Final Projects (1,019) 23,244 12,557 46%

Study Groups (1,359) 41,819 11,609 72%

Qtns for Prof(284) 2,758 896 68%

Course Material Feedback (252)

2,752 729 74%

Tech Feedback (231) 3,087 339 89%

Study groups sub-forum

Study Groups - Full Study Groups - Significant

Crowds versus communities?

Forums mostly harbour crowds, not communities, of learners characterized by weak ties

How do people experience the forums?

Contribute to learning

“I like the forums, you learn a lot (…) people teach you – without necessarily telling you what the solution is – but they guide you.”

(Emengo, 40s, bachelor degree, Nigeria)

Not fit for purpose

“I personally dislike the forum dynamics. I don’t like it ……because it’s not in real-time. It’s not just about real-time, it’s also about feedback. In forums, most people, say, enter, say something, stay for maybe half an hour or so, and then they leave. And they tend not to come back to the same forum ever again. So, I really dislike that.” (Lucas, Spain, Masters, mid 20s)

Not for interaction but for information

“Very little. I mean (…) I really enjoy collaborating with people in my job (...) but I found it was easier to just read the materials on my own; I didn’t feel the need to leverage the community in order to complete the work.” (Oliver, Canada, early 30s, graduate )

“Having done a science degree, I’m very comfortable with researching (…) for me, I think for me to just be asking a question into the forum universe, you don’t really know if the person answering know what they’re talking about” (Julia, UK, 20s graduate)

Sub-forums are generally “vulnerable”, and some are more vulnerable than others

Forum disaggregation

Video link

Next steps

)

Continue to develop a qualitative set of learner profiles that incorporates a number of dimensions to explain if, how and why people use the forums

Helps to inform the development of the quantitative set of learner profiles that reflect the different ways and reasons that people interact with one another in MOOC

Link these profiles to learner characteristics and course outcomes

Test on a wider set of data

Acknowledgements

Project team Chris Davies, Isis Hjorth, Taha Yasseri

Professor Michael Lenox and Kristin Palmer, UVA

Coursera

Project sitehttp://www.oii.ox.ac.uk/research/projects/?id=121