Learning Analytics: Gaining good actionable insight

Martin Hawksey@mhawksey

@mhawksey 2

go.alt.ac.uk/join-ALT

@mhawksey 3

@mhawksey 4

ALT Online Winter Conference 2015

 Week commencing 7th December 2015

http://go.alt.ac.uk/ALT-Online-2015-Call (call closes 9am on Mon 2nd Nov. 2015)

@mhawksey 5

#altc

@mhawksey

“

alt.ac.uk6

How much do you know about Learning

Analytics?In small groups share what you know

about Learning Analytics – 4mins

@mhawksey alt.ac.uk7

@mhawksey alt.ac.uk8

Patterns are left behind

Courtesy Marc Smith http://www.slideshare.net/Marc_A_Smith/2013-nodexl-social-media-network-analysis9

@mhawksey

“

alt.ac.uk10

Analytics in Education

Analytics is the process of developing actionable insights through problem

definition and the application of statistical models and analysis against existing

and/or simulated future data

Adam Cooper, What is Analytics? http://publications.cetis.ac.uk/wp-content/uploads/2012/11/What-is-Analytics-Vol1-No-5.pdf

@mhawksey

“

alt.ac.uk11

Learning Analytics Definition

…the measurement, collection, analysis and reporting of data about

learners and their contexts, for purposes of understanding and

optimising learning and the environments in which it occurs

First International Conference on Learning Analytics and Knowledge (LAK11), 2011

@mhawksey alt.ac.uk12

Disciplines

◊ Computer science◊ Statistics◊ Programming◊ Network analysis◊ Psychology of education◊ …

@mhawksey alt.ac.uk13

Methods

◊ Content Analytics  ◊ Context Analytics◊ Discourse Analytics ◊ Disposition Analytics◊ Social Network Analysis◊ …Ferguson and Buckingham Shum (2012) Social Learning Analytics: Five

Approaches 

@mhawksey alt.ac.uk14

Tools/Products

◊ Spreadsheets - MS Excel, Tableau, Open Refine, Google Sheets…

◊ SNA – NodeXL, Gephi, Cytoscape◊ Programming: R, Python,

MATLAB… ◊ VLEs - Blackboard Analytics for

Learn, Desire2Learn Insights …◊ …

@mhawksey alt.ac.uk15

Action

Analytic

Data

Question

@mhawksey alt.ac.uk16

The Absence of Theory

@mhawksey

“

alt.ac.uk17

Amazon cares not a whit *why* people who buy german chocolate also buy cake pans as long as they

get to the checkout buying both

Mike Caulfield - Short Notes on the Absence of Theory http://hapgood.us/2013/12/10/short-notes-on-the-absence-of-theory/

@mhawksey

“

alt.ac.uk18

Counts don’t count much if decontextualized

Wilson, T.D. (1999). Models in information behaviour research. Journal of Documentation, 55(3), 249 – 270.

Dragan Gasevic speaking at DiCE SeminarMoray House School of Education – 9th January 2015

@mhawksey

“

alt.ac.uk19

Dragan Gasevic speaking at DiCE SeminarMoray House School of Education – 9th January 2015

Analytics to account for conditions, operations, products,

evaluation, and standards (COPES)Gašević, D., Dawson, S., Siemens, G. (2015). Let's not forget:

Learning analytics are about learning. TechTrends, 59(1), 64-71, https://bit.ly/techtrends15

@mhawksey alt.ac.uk20

Data Visualization

21

@mhawksey 22

Image credit: http://practicalanalytics.co/2014/06/02/cloud-based-healthcare-platform-apples-health-app-and-healthkit

/

@mhawksey

“

alt.ac.uk23

Graphs can be a powerful way to represent relationships between data, but they are also a very abstract concept, which means that they

run the danger of meaning something only to the creator of the graph … Everything looks like a graph, but almost nothing should ever be

drawn as one. 

Ben Fry in ‘Visualizing Data’

@mhawksey alt.ac.uk24

Exploratory visualizationData visualizations that are used by the designer for self-informative purposes to discover patterns, trends, or sub-problems in a dataset. Exploratory visualizations typically don’t have an already-known story.

Explanatory visualizationData visualizations that are used to transmit information or a point of view from the designer to the reader. Explanatory visualizations typically have a specific “story” or information that they are intended to transmit.

Iliinsky & Steele - Designing Data Visualizations: Representing Informational Relationships

@mhawksey

“

alt.ac.uk25

Visualizations can be harmful

Corrin, L., & de Barba, P. (2014). Exploring students’ interpretation of feedback delivered through learning

analytics dashboards. In Proceedings of the ascilite 2014 conference (pp. 629-633). ascilite.

Dragan Gasevic speaking at DiCE SeminarMoray House School of Education – 9th January 2015

@mhawksey

“

alt.ac.uk26

Visualizations can be useful

…if embedded into and fit learning tasksDragan Gasevic speaking at DiCE Seminar

Moray House School of Education – 9th January 2015

@mhawksey alt.ac.uk27

Ethics, privacy and data sharing

@mhawksey

“

alt.ac.uk28

The worlds of privacy and

analytics intersect

Downes (2014)

…not always happily

© UnknownSource:  http://crooksandliars.com/files/primary_image/14/05/inbloom-drawing.jpg

@mhawksey alt.ac.uk29

Ethics

◊ Clarity ◊ Comfort and care◊ Choice and consent ◊ Consequence and complaint

Legal, Risk and Ethical Aspects of Analytics in Higher Education (Kay, Korn, & Oppenheim, 2012)

@mhawksey alt.ac.uk30

http://www.open.ac.uk/students/charter/essential-documents/ethical-use-student-data-learning-analytics-policy

@mhawksey alt.ac.uk31

https://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics

@mhawksey alt.ac.uk32

Opportunities

@mhawksey

“

alt.ac.uk33

Feedback loops between students and instructors are missing!

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of education research, 77(1), 82-112.

“

alt.ac.uk34

At best analytics can help start a conversation. People have to be willing to take the conversation

on

Roberts, G. Analytics are not relationships http://rworld2.brookesblogs.net/2014/12/12/analytics-are-not-relationships/

@mhawksey

@mhawksey

“

alt.ac.uk35

What might you use Learning Analytics for?

In small groups discuss – 5mins

@mhawksey alt.ac.uk36

Examples

alt.ac.uk

Optimal Video Length

Philip Guo, Optimal Video Length for Student Engagement@mhawksey

@mhawksey alt.ac.uk38

Misconception or mistakes in MCQs

Koller, D The Online Revolution: Education at Scale (2012) http://youtu.be/ixE1YAlHnVU?t=29m16s

Analytics at scale: Course SignalsDeveloped at Purdue University, USA

39Arnold, K. E., & Pistilli, M. (2012). Course Signals at Purdue: Using Learning Analytics

To Increase Student Success. Paper presented at LAK12, Vancouver, Canada.

http://www.slideshare.net/R3beccaF/scaling-up-learning-analytics

Easily accessible OU dataLearning design and analytics at the OU

40

http://www.slideshare.net/R3beccaF/scaling-up-learning-analytics

alt.ac.uk

Scaling up Learning Analytics

41

https://www.youtube.com/watch?v=Ii4EgnOvpxA

@mhawksey

@mhawksey alt.ac.uk42

Wrapping up

@mhawksey

“

alt.ac.uk43

Learning activities don’t happen in a single platform

Dragon Gasevic

@mhawksey

“

alt.ac.uk44

Not everything that can be counted counts.

Not everything that counts can be counted.

William Bruce Cameron

@mhawksey

“

alt.ac.uk45

What kind of learners are we trying to create?

… this should drive our analytics

Simon Buckingham Shum

@mhawksey alt.ac.uk46

Computers are stupid

https://twitter.com/jackyalcine/status/615329515909156865

@mhawksey alt.ac.uk47

Useful links• LACE (learning analytics community exchange) :

http://www.laceproject.eu/• SoLAR : Society for Learning Analytics Research

http://solaresearch.org/• Jisc: http://analytics.jiscinvolve.org/wp• Cetis Analytics Series :

http://publications.cetis.ac.uk/c/analytics• LAK16 Conference #lak16

From http://www.slideshare.net/sheilamac/intro-to-learning-analytics-universities-scotlanddec2014smn

@mhawksey alt.ac.uk48

Thank you!

@mhawksey+MartinHawksey

Association for Learning Technology

Registered charity number: 1063519

www.alt.ac.uk @A_L_T