Professional Learning through Massive Open Online Courses

Allison Littlejohn, Colin MilliganCaledonian Academy, Glasgow Caledonian University Glasgow, Scotland

Obiageli UkadikeHarvard Medical School – Harvard Catalyst, Boston MA

@allisonl @cdmilligan

Technology-enhanced Professional Learning

page 2

Littlejohn & Margaryan, 2013

Technology-enhanced Professional Learning

page 3

Continual

Self-regulated

Redefined relationships

Littlejohn & Margaryan, 2013

Personalised

Social

page 4

Social cognitive model of self- regulation (Zimmerman, 2000)Self-regulation is the ‘self-generated thoughts, feelings

and actions that are planned and cyclically adapted to the attainment of personal goals’

Forethought

Performance

Self-reflection

Forethought Performance Self-reflection

Task analysis Goal setting Strategic planningSelf-motivation belief Self-efficacy Task interest/value

Self-control Task strategies Elaboration Critical Thinking Help seeking Interest enhancement

Self-judgement Self-evaluationSelf-reaction Self-satisfaction/affect  

page 5

Learner

Learning strategy Learning behaviourLearning opportunitiesLearning tools

page 6

Tensions across autonomy- diversity, openess- connectedness in MOOCsMackness, Mak & Williams (2010)

Critical/ digital literacies in MOOCsFini et al (2009), Kop et al (2011)

Self-regulated learning behaviours in a MOOCMilligan, Littlejohn & Margaryan (2013)

What’s known about Professional Learning in MOOCs

Project provides new insight into:• Learning behaviours in MOOCs while self-regulating • MOOC environment design recommendations to

encourage the development and emergence of SRL behaviours.

Relevant for professional learning, though applicable to a range of settings.

Outputs aimed to impact future research and development

page 7

What’s known about Professional Learning in MOOCs

Research Questions

• RQ: What self-regulated learning strategies and behaviours do professionals adopt in MOOCs?

• RQ: How are Massive Open Online Courses currently designed to support self-regulated learning?

• RQ: How can MOOCs be designed to encourage professionals to self-regulate their learning?

Context:• MOOC for healthcare professionals (edX: HarvardX)• Findings and outputs tested in other health MOOCs (FutureLearn: Kings College London)

page 8

Research Context: edX ‘Fundamentals of Clinical Trials’

• Harvard Medical School, Harvard School of Public Health, Harvard Catalyst.

• Delivered via edX Oct 2013-Feb2014 https://www.edx.org/course/harvardx/harvardx-hsph-hms214x-fundamentals-941

• 24,000 registered, 400+ participants in study, contacted by course announcement in week 5 (of 14).

• Provides an introduction to the scientific, statistical, and ethical aspects of clinical trials research.

• Participants assumed to have a background in biostatistics and epidemiology.

page 9

Fundamentals of Clinical Trials

page 10

Video, with commentary...

Fundamentals of Clinical Trials

page 11

Follow the textbook link to read the ebook online…

Fundamentals of Clinical Trials

page 12

Discussion forum, encourages in-depth discussion

Fundamentals of Clinical Trials

page 13

MCQ assessment items.

Limited number of attempts

Scores are recorded in the progress tab.

RQ: What self-regulated learning strategies and behaviours do professionals adopt in MOOCs? Collect and analyse learner data form professionals learning in a MOOC. • Survey of SRL profiles to collect data of overall population. • Learner interviews to explore impact of course on practice. • Data analysis and synthesis of quant. and qual. data.(leading to)• Use Cases of learner behaviours providing a narrative

description of learner behaviour patterns observed. 

page 14

RQ: What self-regulated learning strategies and behaviours do professionals adopt?

• SRL Profile Survey Instrument: http://figshare.com/articles/SRLMQ/866774

• 42 items measuring 11 sub-processes of SRL.• Based on a validated instrument exploring SRL

behaviours in professional learning.• Three open ended questions exploring:

• motivation, • impact,• interaction

• Administered via SurveyMonkey

page 15

Sample questions (measured on 5pt Likert scale)

• My past experiences prepare me well for new learning challenges. (self-efficacy)

• I try to apply my previous experience when learning. (elaboration)

• I ask myself how what I am learning is related to what I already know. (task strategies)

• I ask others for more information when I need it. (help-seeking)

• I try to understand how what I have learned impacts my work/practice. (self-satisfaction)

page 16

Snapshot of Sample: Demographics

Continent Number CountriesSouth America 25 6North America 82 11

Europe 109 23Africa 36 15Asia 87 20Oceania 8 2Not stated 29 -

376 77

page 17

Countries ParticipantsUS – United States 64IN - India 38ES - Spain 20EG - Egypt 13BR - Brazil 12UK – United Kingdom 12

Snapshot of Sample: Demographics

Age profile: 60% under 35Gender F:M 52:48Education: 60% have at least a Masters level

qualificationIntend to Gain Completion Certificate: 96%Participated in a MOOC before: 52%

page 18

SRL – Profile Score

Those who identified as healthcare professionals were selected and split into three groups based on SRL profile score.SRL-L: ave. 125SRL-M: ave. 153SRL-H: ave .180

page 19

42 54 66 78 90 102

114

126

138

150

162

174

186

198

210More

0

5

10

15

20

25

30

35

40

45

50

SRL-Score

Freq

uency

Snapshot of Sample: Roles

50% currently employed as a healthcare professionals

page 20

Clinician (Physician, MD, specific titles) 62Clinical research (specialist job titles) 36Researcher/lecturer 27Pharmacist 18Corporate/ non scientific roles 16Other healthcare (Nurse, Physical Therapist etc.) 12Dentist 5Other 1Total 177

Q1 What motivated you to take this course?

page 21

Low HighInterest (responses indicated a generic expression of interest)

3 6

Topic (referred to the topic of the course but no link to current or future role)

12 4

Career (linked participation to their future career)

5 9

Professional (linked to current practice, or what they hoped to get out of the course)

6 14

Harvard (referred explicitly to certification, or prestige of provider)

11 3

MOOCs  - 1Total 37 37Low and High SRL groups seem to differ in motivation:• Low attracted by prestige of HarvardX Brand or by

topic.• High cite links to career or professional practice

Q2 Please describe how you expect studying on this course to contribute to your work or professional practice (or other studies)?

Low

High

Interest (doing this for interest rather than to gain specific knowledge)

4 0

Knowledge(comments highlight topic focus but do not relate directly to current or future role)

8 12

Practice (focused specifically on current role)

18 16

Career (focused on future role)

6 7

Total 36 35

page 22

Little difference between the two groups:• Only Low-SRL described participation as ‘for interest only’

Q3 Please describe your experience of interacting with other learners in the discussion forum. What went well? What was challenging?

page 23

  Low HighPositive 13 15Negative 6 5 Language /technical 4 2 Frustrated 1 3 Overwhelmed 2 0Not used 13 11 Not interested 6 8 No time 7 3

Broadly similar though:• Low-SRL may be less likely to prioritise time, • Only low-SRL learners were ‘overwhelmed’.

Semi-structured interviews (begin tomorrow).

• Minimum 30 (up to 70) participants (from two groups: low-SRL and high-SRL, all professionals) contacted for interview.

• Research participants asked to reflect on the impact that the MOOC has had/is likely to have on their professional practice.

• SS Interview script: http://figshare.com/articles/PL_MOOC_Semi_structured_interview_script/866773

page 24

Sample questions

• Context: Do you expect to integrate what you have learned in the MOOC into your professional practice? Can you give an example? [PROBE: How?]

• Strategies: Did you share anything with anyone else either formally or informally (e.g. through informal discussions, postings on blogs, contribution to discussion forums etc.) [PROBE: Who?]

• Self-evaluation: Did you talk to anyone in your professional network to discuss the value of the MOOC?

page 25

Semi-structured interviews

• Basis for use case narratives of learning behaviours in a MOOC

• Questions (aligned with SRL-sub-processes) explore:• interaction with others, within and beyond course (e.g. with

professional network)• use of tools to support interaction with course content and

to support learning (using 4c’s: consume, connect, create and contribute)

• explore impact on professional practice.

• First interviews scheduled for 6/12/13• Continue throughout December• Analysis in January 2014page 26

RQ: How are Massive Open Online Courses currently designed to support self-regulated learning?Review current research in SRL and develop a tool to allow mapping of MOOC designs and SRL behaviours.• Desk research of SRL to inform Design Mapping Tool . • Course Design document review. • edX/HarvardX Design team interview. • Data analysis and synthesis mapping design to SRL sub-

processes. (leading to)• Design mapping tool based on current SRL literature.• MOOC design strategy report. 

page 27

RQ: How are Massive Open Online Courses currently designed to support self-regulated learning?• Explore learner behaviour AND MOOC design through

SRL lens• Complement the learner data with data about the

design of the course (at strategic, platform and course level)

• Environments can foster the development of SRL Skills

• Learner’s ability to self-regulate impacts their performance

• Survey research evidence and map to the SRL sub-processes.

• Develop questions for MOOC providers, platform developers, course designers and teaching assistants to build picture of individual MOOCs

page 28

Summary of research evidenceWhat interventions might you make in an environment to promote SRL.

(pre-training/presence of tutor) (Azevedo et al, 2004, 2005) F: Goal-setting tools (Chang et al 2013) (also flexible pathways through

content) F: Planning tools (experimental plans) (White and Frederiksen, 1998) F: Planning tools (cues, hints) (Manlove et al 2008) P: Scaffold interaction (Cho & Kim, 2013) P: Require interaction (Cho & Kim, 2013) P: Feedback mechanisms (Aleven et al, 2006) (Wang, 2011) P: Note-taking and Highlighting tools (Proske, Narciss, and Koerndle, 2007)

(Kauffman, 2004) P: Cues to take structured notes (Kauffman, 2004) P/SR: Diary tools (Arsal, 2010; Neber &Schommer-Atkins, 2002; Schmitz &

Weise, 2006; Winne, 2005) SR: Inline prompts to monitor/reflect on learning (Kauffman, 2004) SR Explicit meta-cognitive tasks (Delfino, Dettori & Persico, 2011 ; Kramarski &

Mizrachi, 2013) P: Tasks structures encouraging participants to take control (Dettori, Giannetti

& Persico, 2005)What SRL-related learner behaviours are expressed in online environments (and correlated with better academic performance):

F: Goal setting (Chang et al, 2013) F: Planning (Azevedo et al, 2004) F: Self-efficacy (Wang and Lin, 2007) (Yang et al, 2006) (Cho and Johanssen,

2009) F: Goal orientation (Chang, 2007) P: Monitoring (Greene and Azevedo, 2009) P: Time management: (Hu and Gramling, 2009; Song et al, 2004) P: Help-seeking (Azevedo et al, 2004; Vighnarajah et al, 2009)

page 29

Design Mapping Tool

page 30

  Domains

Evidence Environment Design

Recommendations

SRL

Sub-processes

Planning      Goal-setting      Self-efficacy      Task-interest/value

     

Help-seeking      Task strategies

     

Elaboration      Self-evaluation

     

Self-satisfaction

     

RQ: How can MOOCs be designed to encourage professionals to self-regulate their learning?Synthesise results as recommendations and scientific outputs.• Data analysis of data collected in first two phases into a

set of recommendations. • Recommendations review with additional professional

learning MOOC. (leading to)• Revised MOOC design recommendations for MOOC

community. • All instruments, datasets and findings from the study

(eg IRRODL article)

page 31

Current and Next steps

page 32

• RQ2 Conduct interviews and analyse learner behaviour

• RQ1 Collect MOOC design data using mapping tool

• RQ3 Develop recommendations for MOOC design

• Refine tools and recommendations in a different MOOC

Current and Next steps

Provide new insight of use to the research & development communities:• Learning behaviours in MOOCs • MOOC environment design recommendations

page 33

Professional Learning through Massive Open Online Courses

Allison Littlejohn, Colin MilliganCaledonian Academy, Glasgow Caledonian University Glasgow, Scotland

Obiageli UkadikeHarvard Medical School – Harvard Catalyst, Boston MA http://gcu.ac.uk/academy/pl-mooc/@allisonl @cdmilligan

Bibliography• Barnard, L., Paton, V., & Lan, W. (2008). Online self-regulatory learning behaviours as a mediator in the

relationship between online course perceptions and achievements. International Review of Open and Distance Learning, 9 (2), 1-11.

• Castells, M. (1996). The information age: economy, society and culture: The rise of the networked society. Oxford, UK: Blackwell.

• Bernacki, M.L., Aguilar, A.C., and Byrnes, J.P. (2011) Self-regulated learning and technology-enhanced learning environments: an opportunity-propensity analysis. In: G. Dettori & D. Persico (Eds.) Fostering Self-regulated learning through ICT. pp. 364-379. IGI Global: Hershey, PA,

• Cho, M-H., & Kim, B.J. (2013). Students’ self-regulation for interaction with others in online learning environments. The Internet and higher Education, 17, 69-75.

• Dettori, G.,& Persico, D. (2009) Supporting self-regulated learning with ICT, in A. Cartelli and M. Palma (eds.) Encyclopaedia of ICT, pp 735-741. IGI Global, Hershey, PA

• Daniel, J. (2012). Making Sense of MOOCs: Musings in a Maze of Myth, Paradox and Possibility. Journal of Interactive Media In Education, 3(0). Retrieved August 8, 2013, from http://www-jime.open.ac.uk/jime/article/view/2012-18

• Downes, S. (2009). Connectivist dynamics in communities. Retrieved 26 February 2013 from http://halfanhour.blogspot.co.uk/2009/02/connectivist-dynamics-in-communities.html

• Engeström, R. (2009). Who Is Acting in an Activity System? In A. Sannino, H. Daniels, & K. D. Gutiérrez (Eds.), Learning and Expanding with Activity Theory, pp. 257-273. New York, NY: Cambridge University Press.

• Eraut, M. (2000). Non-formal learning and tacit knowledge in professional work.British Journal of Educational Psychology, 70, 113-136.

• Fini, A. (2009). The technological dimension of a Massive Open Online Course: the case of the CCK08 course tools. The International Review of Research in Open and Distance Learning, 10(5), 74-99.

• Kop, R. (2011). The challenges to connectivist learning on open online networks: learning experiences during a Massive Open Online Course. The International Review of Research in Open and Distance Learning, 12 (3), 19-38.

• Kop, R., & Fournier, H. (2011). New dimensions to self-directed learning in an open networked learning environment. International J. of Self-Directed Learning,7 (2), 1-18.

• Lefrere, P. (2007). “Business success - the special contribution of self-regulated learning.” In J. Beishuizen, R. Carniero & K. Steffens (Eds.) Self-regulated Learning in Technology Enhanced Learning Environments: Individual Learning and Communities of Learners. pp. 49-53. Shaker Verlag: Aachen.

• Littlejohn, A., Milligan, C., & Margaryan, A. (2011). Collective learning in the workplace: Important knowledge sharing behaviours. International Journal of Advanced Corporate Learning, 4 (4) 26-31.

page 35

Bibliography• Littlejohn, A., Milligan, C., Margaryan, A (2012),"Charting collective knowledge: supporting• self-regulated learning in the workplace", Journal of Workplace Learning, 24(3).226-238.• Littlejohn, A., & Margaryan, A. (2013). Technology Enhanced Professional Learning: mapping out a new domain.

Chapter 1 in Littlejohn, A., & Margaryan, A. (Eds.). Technology-enhanced professional learning: Processes, practices and tools. London, Routledge.

• Liyanagunawardena, T. R., Adams, A. A., & Williams, S.A. (2013). MOOCs: A systematic study of the published literature 2008-2012, International Review of Research in Open and Distance Learning, 14 (3) 202-227.

• Mackness, J., Mak, S., & Williams, R. (2010). The ideals and reality of participating in a MOOC. In L. Dirckinck-Holmfeld, V. Hodgson, C. Jones, M. de Laat, D. McConnell, & T. Ryberg. (Eds.). Proceedings of the 7th International Conference on Networked Learning 2010. pp. 266-275. University of Lancaster, Lancaster.

• Milligan, C. Littlejohn, A., & Margaryan, A. (2013). Patterns of Engagement in Connectivist MOOCs. Journal of Online Learning & Teaching 9 (2) 149-159. http://jolt.merlot.org/vol9no2/milligan_0613.pdf

• Milligan, C., Margaryan, A., & Littlejohn, A. (2013). Goal-setting behaviour in massive open online courses, EARLI 2013 Conference. Munich Germany, 27-31 August, 2013. https://t.co/KsO0uMR5Fv

• Pintrich, P. R., Smith, D., García, T., & McKeachie, W. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor. Michigan.

• Schunk, D. H. (2001). Self-regulation through goal-setting, ERIC Digest. Retrieved August 8, 2013, fromhttp://www.counseling.org/Resources/Library/ERIC%20Digests/2001-08.pdf

• Siadaty, M., Jovanović, J., & Gašević, D. (2013). The social semantic web and workplace learning. Chapter 12 in Littlejohn, A., & Margaryan, A.  (Eds.).Technology-enhanced professional learning: Processes, practices and tools. (forthcoming) London, Routledge.

• Siemens, G. (2005). Connectivism: a learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2(1), 3-10.

• Tynjälä, P. (2008). Perspectives into learning in the workplace. Educational Research Review, 3, 130-154.• Veen, W., van Staalduinen, J-P. & Hennis, T. (2011). Informal self-regulated learning in corporate organizations. In: G.

Dettori & D. Persico (Eds.) Fostering Self-regulated learning through ICT. pp. 364-379. IGI Global: Hershey, PA,• Zimmerman, B. J. (2000). Attaining Self-regulation: A Social Cognitive Perspective. In M. Boekaerts, P.R. Pintrich, & M.

Zeidner (Eds.), Handbook of self-regulation, San Diego, Academic Press.

page 36

Design mapping tool example questions: Help-seekingQuestions Research Evidence- Is interaction with other learners seen as

necessary/essential for course participation? (strategic)

- Are tools/structures provided to encourage interaction? (platform)

- Are tools /structures provided that encourage interaction around specific pieces of content (e.g. collaborative annotation?) (platform)

- Are learners encouraged to interact with each other, and tutors as they learn (course dev.)

- Are learners expected to make contact with other learners to discuss course content (course dev.)

- Are learners expected to make contact with tutors (teaching assistants) to discuss course content? (course dev)

- Have you observed learners making contact with each other in the course discussion fora (teaching assistant).

- Help-seeking correlates with increased academic performance (Azevedo et al, 2004; Vighnarajah et al, 2009)

- Scaffolding interaction promotes SRL (Cho & Kim, 2013)

- Requiring interaction promotes SRL (Cho & Kim, 2013)  

page 37