Embed Size (px)
Citation preview
Digital Badges and Learning Analytics to Explore the Learner Participation
in a MOOCJiyae Bong
Doctoral candidate of Instructional Systems & Learning Technologies, Florida State University
Dr. Vanessa Dennen
Professor of Instructional Systems & Learning Technologies, Florida State University
[email protected], @vdennen1
Research Background
• The positive effects of using digital badges on the completion of learning tasks and learners’ motivation.
• There has been a lack of empirical data on the use of digital badges and its effects on learning engagement.
• Analytic tools embedded in MOOC platforms capture sizable data sets.
• The current study addresses this gap by applying embedded learning analytic tool to gain data related to learners’ viewing and posting behaviors.
2
Research Questions
• How frequently did participants who earned badges access the main learning content across each week of the MOOC?
• How frequently did participants who earned badges engage in main learning activities(discussion, project, quiz) across each week of the MOOC?
• Is there a difference between the badge recipients and non-badge achievers on them?
• This study also looked at relationships between number of badges earned and overall participation patterns.
3
About the MOOC
• Topic: Social media for active learning
• Length: 4 weeks
• Focus: Professional development
• Format: Weekly modules, Video lectures, webinars, discussions, quizzes, & assignments
• Badge criteria: (1) participate in the discussion with at least three posts, (2) earn 80% or higher on the quiz, and (3) complete a project
4
Participants
• A total of 382 participants who participated in at least one or more of the main learning activities during the designated course period.
5
Module 1 Badge Module 2 Badge Module 3 Badge Module 4 Badge # Participants
v 20
v v 8
v v 1
v v v 3
v v v 1
v v v v 24
57 36 29 26 57
Table 1
The Number of Earned Completion Badges
Data Collection and Analysis
• User activity reports at the class level and the individual-learner level including • (a) the total numbers of access to each component of course (e.g. content area,
discussion board)
• (b) the same data rearranged by date, by hour of day, and by day of week
• Independent sample t-test (Welch-Satterthwaite version) was conducted.
• The data were rearranged and visualized considering the digital badge achievement and course design (i.e., modularized lessons).
6
Findings (1) The Degree of Learner Engagement by Badge Achievement
7
Content Area:
Access (#clicks)
Discussion:
#posts
Project:
#posts
Quiz:
Score
Badge Recipients
(N=56)
33.45**
(30.41)
3.88**
(1.40)
2.43**
(1.55)
94.52*
(5.88)
Non-Badge Achievers N=201 6.58**
(6.95)
N=193 .39**
(1.16)
N=193 .41**
(3.70)
N=37 80.84*
(26.93)
Module 1 (Mean, SD)
*p<.05, **p<.001
Content Area:
Access (#clicks)
Discussion:
#posts
Project:
#posts
Quiz:
Score
Badge Recipients
(N=35)
36.74**
(18.85)
3.57**
(1.31)
1.54**
(.98)
94.71
(7.58)
Non-Badge Achievers N=206 5.36**
(13.79)
N=197 .26**
(1.01)
N=197 .05**
(.28)
N=27 88.33
(15.19)
Module 2 (Mean, SD)
**p<.001
Findings (1)(cont.) The Degree of Learner Engagement by Badge Achievement
8
Content Area:
Access (#clicks)
Discussion:
#posts
Project:
#posts
Quiz:
Score
Badge Recipients
(N=29)
35.66**
(24.66)
3.86**
(.13)
2.14**
(1.43)
97.33
(5.51)
Non-Badge Achievers N=229 1.99**
(8.3)
N=229 .13**
(.73)
N=229 .08**
(.84)
N=10 97
(9.47)
Module 3 (Mean, SD)
**p<.001
Content Area:
Access (#clicks)
Discussion:
#posts
Project:
#posts
Quiz:
Score
Badge Recipients
(N=26)
22.04**
(11.73)
4**
(1.94)
1.5**
(.86)
95.77
(6.28)
Non-Badge Achievers N=232 1.06**
(3.31)
N=232 .05**
(.37)
N=232 0**
(0)
N=6 81.67
(29.44)
Module 4 (Mean, SD)
**p<.001
Findings (2)
9
The relationships between
number of badges earned and
overall participation patterns
• The first bar graph shows
• the average numbers of clicks.
• The second and third bar graphs show
• the average numbers of posts.
• The more they earned, the more they
engaged?
• Across the modules, four-badge achievers
showed their consistent engagement level
which is similar to the badge requirements.
Conclusions and Implications
• The participants who earned badges consistently accessed each activity area across the modules. Also, badge owners appeared to have a deeper level of engagement in the required learning activities as noted by their viewing and posting behaviors.
• However, more research is required to explore the relationships between badges and engagement in MOOCs.
• Another challenge relates to learner engagement metrics in MOOCs.
• The numerical data from clickstream analytics do not deeply explore learning. It requires huge demands to extract the meaningful information from the numerical data after exploring all the details of the course context.
10
Findings (3) The Degree of Learner Engagement by Badge Achievement
11
Content Area:
Access (#clicks)
Discussion:
#posts
Project:
#posts
Quiz:
Score
Badge Recipients
(N=56)
33.45**
(30.41)
3.88**
(1.40)
2.43
(1.55)
94.52*
(5.88)
Non-Badge Achievers
“Active” (n=36)
13.97**
(9.57)
1.06**
(1.99)
1.86
(8.46)
80.58*
(27.27)
Module 1 (Mean, SD)
*p<.05, **p<.001
Content Area:
Access (#clicks)
Discussion:
#posts
Project:
#posts
Quiz:
Score
Badge Recipients
(N=35)
36.74
(18.85)
3.57**
(1.31)
1.54**
(.98)
94.71
(7.58)
Non-Badge Achievers
“Active” (N=23)
25.09
(32.58)
1.52**
(2.27)
.39**
(.72)
87.61
(15.43)
Module 2 (Mean, SD)
**p<.001
Findings (3)(Cont.) The Degree of Learner Engagement by Badge Achievement
12
Content Area:
Access (#clicks)
Discussion:
#posts
Project:
#posts
Quiz:
Score
Badge Recipients
(N=29)
35.66
(24.66)
3.86*
(.13)
2.14
(1.43)
97.33
(5.51)
Non-Badge Achievers
“Active” (N=10)
16.2
(31.82)
1.9*
(2.6)
1.3
(3.46)
97
(9.48)
Module 3 (Mean, SD)
Content Area:
Access (#clicks)
Discussion:
#posts
Project:
#posts
Quiz:
Score
Badge Recipients
(N=26)
22.04
(11.73)
4*
(1.94)
1.5**
(.86)
95.77
(6.28)
Non-Badge Achievers
“Active” (N=4)
15
(13.24)
.75*
(1.5)
0**
(0)
81.25
(37.5)
Module 4 (Mean, SD) *p<.05, **p<.001
*p<.05, **p<.001
Thank you! Any Questions?
Contact:
Jiyae Bong – [email protected]
Vanessa Dennen - [email protected] - @vdennen
13
![Certificate Achievement Unlocked: How Does MOOC Learners ... · MOOC learning for a variety of reasons including the scale, the heterogeneity of the learner group [10] with respect](https://img.dokumen.tips/doc/110x75/5ed06e25f00a2361b5505bfd/certificate-achievement-unlocked-how-does-mooc-learners-mooc-learning-for-a.jpg)
![Designing learner centric MOOCssahanamurthy/talks/slides/LCM-IIMB-Jan2018.pdf · Peer Interaction + Reflection quiz] 30 From traditional to Learner-Centric MOOC Jan 15, 2018 Takeaway:](https://img.dokumen.tips/doc/110x75/5f18e7d307ca1f58af2654d7/designing-learner-centric-sahanamurthytalksslideslcm-iimb-jan2018pdf-peer.jpg)
