Embed Size (px)
DESCRIPTION
The past five years have seen a dramatic growth in interest in the emerging field of Learning Analytics (LA), and particularly in the potential the field holds to address major challenges facing education. However, much of the work in the learning analytics landscape today is closed in nature, small in scale, tool- or software-centric, and relatively disconnected from other LA initiatives. This lack of collaboration, openness, and system integration often leads to fragmentation where learning data cannot be aggregated across different sources, institutions only have the option to implement "closed" systems, and cross disciplinary research opportunities are limited. Beyond the immediate concerns this fragmentation creates for educators and learners, a closed approach dramatically limits our ability to build upon successes, learn from failures and move beyond the "pockets of excellence (and failures)? approach that typifies much of the educational technology landscape. The potential benefits of openness as a core value within the learning analytics community are numerous. Learning initiatives could be informed by large scale research projects. Open-source software, such as dashboards and analytics engines, could be available free of licensing costs and easily enhanced by others, and OERs could become more personalized to match learners' needs. Open data sets and reproducible papers could rapidly spread understanding of analytical approaches, enabling secondary analysis and comparison across research projects. To realize this future, leaders within the learning analytics, open technologies (software, standards, etc.), open research (open data, open predictive models, etc.) and open learning (OER, MOOCs, etc.) fields have established a "network of practice" aimed at connecting subject matter experts, projects, organizations and companies working in these domains. As an initial organizing event, these leaders organized an Open Learning Analytics (OLA) Summit directly following the 2014 Learning Analytics and Knowledge (LAK) conference this past March as means to further the goal of establishing "openness' as a core value of the larger learning analytics movement. Additional details on the Summit and those involved can be found at: http://www.prweb.com/releases/2014/04/prweb11754343.htm. This panel session will bring together several thought leaders from the Open Learning Analytics community who participated in the Summit to facilitate an interactive dialog with attendees on the intersection of learning analytics and open learning, open technologies, open data, and open research. The presenters represent a broad range of experience with institutional analytics projects, an open source development consortium, the sharing of open learner data, and academic research on open learning environments.
Citation preview
J O S H B AR O N
S T I AN H Å K L E V
N O R M AN B I E R
H AS H TAG # O P E N L A
Open Learning Analytics Panel
Panel Session Overview
Session Goal: Stimulate discussion around the
importance of open learning analytics to the future of
the larger open education movement
Longer-Term Objective: Form connections between
the OpenEd and Open Learning Analytics networks
Session Format:
Setting the Context – Open Learning Analytics
Examples from the Real World
Discussion/Q&A
What is Learning Analytics?
Academic Analytics Learning AnalyticsA process for providing higher
education institutions with the data
necessary to support operational
and financial decision making*
The use of analytic techniques to help
target instructional, curricular, and
resources to support the achievement
of specific learning goals*
Focused on the business of the
institution
Focused on the student and their
learning behaviors
Management/executives are the
primary audience
Learners and instructors are the
primary audience
* - Source: Analytics in Higher Education: Establishing a Common Language
2014 Open Learning Analytics Summit
Society for Learning Analytics (SoLAR) began
exploring openness in learning analytics in 2011
International OLA Summit held in March 2014
Participants identified OLA “knowledge domains”
as means to organize future work
OLA Knowledge Domains
Open Data and Models
Releasing data sets and models under open licenses
Open Research
Publishing research in open-access journals
Open-Source Software/Platforms
Open software, standards and APIs
Open Strategy and Policy
Open documents on strategy and policy
Open Learning Designs
Combine OER & LA to create new models of learning
N O R M AN B I E R
D I R E C T O R , O P E N L E AR N I N G I N I T I AT I V E
C AR N E G I E M E L L O N U N I V E R S I T Y
Openness = Science
The changing value of content
Changing focus in OER community
Commoditization of content (Wiley: ‘content is
infrastructure’)
Instrumenting content is difficult and expensive
Well-instrumented content and the tools to analyze
student interactions with that content will continue to
increase in importance
Problems with Black-Box Systems
Challenges Opportunities
CC-OLI Research
MOOC Research
UC Davis
Common measures of
outcomes and
achievement
Simon DataLab
Design Analytics
Stanford Outcomes
Analytics Service
Swappable Models
Learner Centered
Examples
S T I AN H Å K L E V
I N S T I T U T I O N AL R E S E AR C H E R , O P E N U T O R O N T O
U N I V E R S I T Y O F TO R O N TO
MOOC RESEARCH AND
REPRODUCIBLE SCIENCE
Supporting three MOOC research projects
(MRI)
Tools to collaborate and document
Connecting database with other data
Clicklog: big data, making it queryable,increasing levels of abstraction
20
J O S H B AR O N
S E N I O R AC AD E M I C T E C H N O L O G Y O F F I C E R
M AR I S T C O L L E G E
Open Data Models & OS
Learning Analytics Platform
OAAI: Overview and Impact
EDUCAUSE Next
Generation Learning
Challenges (NGLC)
Funded by Bill and
Melinda Gates Foundations
$250,000 over a 15 month period
Goal: Leverage Big Data concepts to create an
open-source academic early alert system and
research “scaling factors”
Student Aptitude Data
(SATs, current GPA, etc.)
Student Demographic
Data (Age, gender, etc.)
Sakai Event Log Data
Sakai Gradebook Data
Predictive
Model
Scoring
Identifies
students
“at risk” to
not
complete
course
SIS
Dat
aLM
S D
ata
OAAI Early Alert System Overview
Intervention Deployed
“Awareness” or Online
Academic Support
Environment (OASE)
“Creating an Open Academic Early Alert System”
Model DevelopedUsing Historical Data
Step #1: Developed
model using historical
data
Academic Alert
Report (AAR)
Research Design
Deployed OAAI system to 2200 students across
four institutions
Two Community Colleges
Two Historically Black Colleges and Universities
Design > One instructor teaching 3 sections
One section was control, other 2 were treatment groups
Each instructor received an AAR three times during
the semester:
Intervals were 25%, 50% and 75% into the semester
Intervention Research Findings
Final Course Grades
Analysis showed a
statistically significant
positive impact on final
course grades
No difference between
treatment groups
Saw larger impact in
spring then fall
Similar trend amount low
income students
50
60
70
80
90
100
Awareness OASE Control
Fin
al G
rad
e (%
)
Mean Final Grade for "at Risk" Students
Intervention Research Findings
Content Mastery
Student in intervention
groups were statistically
more likely to “master the
content” then those in
controls.
Content Mastery = Grade
of C or better
Similar for low income
students.
0
200
400
600
800
1000
Yes No Yes No
Content Mastery for "at Risk" Students
Control Intervention
Freq
uen
cy
JAYAPRAKASH , S . M. , MOODY, E . W. , LAURÍA , E . J . ,
REGAN, J . R . , & BARON, J . D . (2014) . EARLY ALERT
OF ACADEMICALLY AT-RISK STUDENTS: AN OPEN
SOURCE ANALYTICS IN IT IATIVE . JOURNAL OF
LEARNING ANALYTICS, 1 (1 ) , 6 -47 .
More Research Findings…
Strategic Vision: Open
Learning Analytics PlatformCollection – Standards-based data capture from any potential source using Experience API and/or IMS Caliper/Senor API
Storage – Single repository for all learning-related data using Learning Record Store (LRS) standard.
Analysis – Flexible Learning Analytics Processor (LAP) that can handle data mining, data processing (ETL), predictive model scoring and reporting.
Communication –Dashboard technology for displaying LAP output.
Action – LAP output can be fed into other systems to trigger alerts, etc.
O AA I P R E D I C T I V E M O D E L D O W N L O AD
H T T P S : / / C O N F L U E N C E . S AK AI P R O J E C T. O R G / X / 8 AW C B
AP E R E O L E AR N I N G AN ALY T I C S P R O C E S S O R D O W N L O AD
H T T P S : / / C O N F L U E N C E . S AK AI P R O J E C T. O R G / X / K W C V B Q
Access to Predictive Model and
related OS Software…
Discussion and Q&A
Discussion Questions
Do you feel LA will be important to OER and Open
Education in the future? How important?
Where do you see connections between the OLA
network and Open Education?
How might we best facilitate making connections
across different “networks”?
[insert more questions]
Additional Resources
European OLA Summit – December 1st (LACE)
http://www.laceproject.eu/
The Asilomar Convention for Learning Research in
Higher Education
http://asilomar-highered.info
Apereo Learning Analytics Initiative
https://confluence.sakaiproject.org/x/rIB_BQ
Society for Learning Analytics and Research
http://solaresearch.org
