Learning from meaningful, purposive interaction

  • View
    288

  • Download
    2

Embed Size (px)

Text of Learning from meaningful, purposive interaction

  1. 1. 1 Learning from Meaningful, Purposive Interaction Fridolin Wild Medieninformatik Universitt Regensburg Knowledge Media Institute The Open University Representing and analysing competence development with network analysis and natural language processing
  2. 2. 2 Outline Introduction and overview Theoretical foundation Precursor algorithms (SNA + LSA) Algorithm: Meaningful, Purposive Interaction Analysis Mathematical foundation Visual analytics using vector maps as projection surfaces Implementation Application examples for Learning Analytics Evaluation: verification and validation Summary and Outlook
  3. 3. 3 INTRODUCTION Context, Objectives, Key Contributions
  4. 4. 4 Introduction Fascination with LSA and Matrix Algebra originated in Information Retrieval (UR), then shifted to Technology Enhanced Learning (WU+OU) Research on Technology Enhanced Learning has its place in the canon of Media & Computing (and Knowledge Media) Its a big and growing global Software Market: Adkins (2011, p.6): 9.2% annual growth till 2015 Docebo (2014,p.8): 7.9% annual growth till 2016 Drivers of Innovation: open Grand Challenges to Research and Development in TEL
  5. 5. 5 Bridging informal and formal Create a unified, seamless learning landscape with the help of mobile devices. learning analytics automated feedback using interaction data to predict performance. #6 fostering engage- ment Increasing student motivation to learn and engaging the disengaged using technology. How can we detect (de-) motivation? How can make use intrinsic/extrinsic reward systems? #4 New devices for young childrens expression of scientific ideas Mouse and keyboard are a blocker to natural mapping and new modalities of interaction (touch, gestures) can foster a more tactile learning. #1 Learning to read at home with digital technologies #2 CSCL in teacher training and professional development #3 e-assessment New forms of assessment of learning in social TEL environments #5 Understanding how toddler apps can support learning. early years technology dataTEL Utilising real-time data to improve teaching and learning. #7 #8 networked learning ecologies Interest-driven lifelong learning in personal learning networks #9 #10 Fischer,Wild, Sutherland,Zirn (2014) #1
  6. 6. 6 Objectives for this work (from GC #5,#6,#8) Represent: to automatically represent conceptual development evident from interaction of learners with more knowledgeable others and resourceful content artefacts; Analyse: to provide the instruments required for further analysis; Visualise: to re-represent this back to the users in order to provide guidance and support decision- making about and during learning.
  7. 7. 7 Key Contributions (Wild, 2014, p.21)
  8. 8. 8 THEORETICAL FOUNDATION Concept space, Quality requirements
  9. 9. 9 Information and Learning communicatively successful cooperatively successful [e]= PmO purpose meaning (Janich, 1998/2003/2006; Hesse et al., 2009; Hammwoehner, 2005; Wild, 2014, p.27ff) learns information
  10. 10. 10 Information and Learning (Wild, 2014, p.42)
  11. 11. 11 PRECURSOR ALGORITHMS Foundational examples, Shortcomings
  12. 12. 12 The Foundational Example Particular unit of company with 9 employees All went through trainings recently Offered by universities (UR, OU), MOOCs, informal learning (FaceBook, LinkedIn) Now: Christina is off sick HR manager to identify worthy replacement SNA LSA MPIA (Wild, 2014, p.60)
  13. 13. 13 (Wild, 2014, p.21,61,63) Social Network Analysis (SNA) Foundational Example A =
  14. 14. 14 SNA Paul Joanna Maximilian Peter Christina Simon Ida Thomas Alba Association Matrix
  15. 15. 15 (Wild, 2014, pp.99-102) Latent Semantic Analysis
  16. 16. 16 c1 c2 c3 c4 c5 m1 m2 m3 m4 p1 p2 p3 p4 p5 c1c2c3c4c5m1m2m3m4p1p2p3p4p5 c1c2c3c4c5m1m2m3m4p1p2p3p4p5 original space LSA & Similarity (Wild, 2014, p.104: cosines) (Wild, 2014, pp.229) black = 1, white = 0 c1 c2 c3 c4 c5 m1 m2 m3 m4 p1 p2 p3 p4 p5 c1c2c3c4c5m1m2m3m4p1p2p3p4p5 c1 c2 c3 c4 c5 m1 m2 m3 m4 p1 p2 p3 p4 p5 c1c2c3c4c5m1m2m3m4p1p2p3p4p5 LSA space
  17. 17. 17 Shortcomings Social Network Analysis (SNA) Blindness to content Relationship discovery restricted to incidences captured Popular for analysis, visualization, simulation, intervention (Sie et al., 2012) Latent Semantic Analysis (LSA) Blindness to purposes & structure (relations, groups, ) Lacking instruments for analysis No clear rule for number of factors to retain Popular for essay scoring, information retrieval, dialogue tutoring, recommenders
  18. 18. 18 MEANINGFUL PURPOSIVE INTERACTION ANALYSIS Foundations in Matrix Algebra, Stretch Truncation
  19. 19. 19 Fundamental matrix theorem on orthogonality Calculating the Nullspace Ker A: Ax = 0 Eq.1 (Wild, 2014, p.131; redrawn from Strang, 1988, p.140) (Wild, 2014, p.132) every matrix transforms its row space to its column space (Strang, 1988, p.140)
  20. 20. 20 The Eigenvalue Problem & Singular Value Decomposition (Wild,2014,p.143) For every symmetric, square matrix: (Barth et al., 1998, p.90/E): Bx = x n.b.: B = AAT or ATA Any multiplication of the matrix B with an Eigenvector x yields a constant multiple of the Eigenvector, scaled by the Eigenvalue A = UVT U = Eigenvectors(ATA) V = Eigenvectors(AAT) = UTAV
  21. 21. 21 Base transformation (from Term-Doc space to orthogonal Eigenspace) (Wild, 2014, p.144) Same dims for both Eigenvector types (row and column), same Eigenvalues!
  22. 22. 22 Stretch-Dependent Truncation 0 100 200 300 400 0 20 40 60 index eigenvalues 20%80% 0 100 200 300 400 0 20 40 60 index eigenvalues
  23. 23. 23 Prediction of Threshold Sum of Eigenvalues 2 = Sum of trace of matrix A threshold = 0.8 * sum(A*A) => Calculate only the first k Eigendimensions, for which the sum of Eigenvalues 2 does not yet pass the threshold
  24. 24. 24 Updating using ex post projection v' = aT Uk k -1 a' = Uk k v' T (Wild, 2014, p. 149f; see also Berry et al., 1994, equation 7 and page 16
  25. 25. 25 Point, Centroid, Pathway e1 e2 u2 u3 u1 p 1 1 3 2 Proximity, Identity 1 11 16 25 9 14 24 2 18 3 19 4 13 5 6 10 15 7 20 22 12 17 26 8 21 23 01234 Cluster dendrogram over all meaningvectors Meaningvectors Height cutoff Clustering
  26. 26. 26 Introducing Network Analysis Techniques Still: result is high-volume, sometimes even big data Visualisation techniques from (Social) Network Analysis can help!
  27. 27. 27 In real examples: too high-volume to see structure
  28. 28. 28 Network Visualisation Proximity-driven Link Erosion (Wild, 2014, p.162) Layout with spring-embedder (Wild, 2014, p.163) Wireframe Conversion (Wild, 2014, p.167) Kernel Smoothing (Wild, 2014, p.169) Hyposometric Tints (Wild, 2014, p.171)
  29. 29. 29 Perspective plot (Wild, 2014, p.172)
  30. 30. 30 Topographic map projection and overlays (Wild, 2014, p.173ff)
  31. 31. 31 IMPLEMENTATION Use Cases, Analysis Workflow, Classes, Demo 31
  32. 32. 32 Use Cases
  33. 33. 33 Class Diagram (Wild, 2014, p.209) > 10.000 lines of codeR package MPIA To be: Open Source (GPL-3) Test-driven development
  34. 34. 34 APPLICATION EXAMPLES Concept space, Quality requirements
  35. 35. 35 The SNA/LSA example revisited (Wild, 2014, p.231) C = computer science P = pedagogy M = math + stats
  36. 36. 36 MPIA foundational example (path of Peter) interface socialweb access review system timeusage html management trees clustering intersection agglomerative knowledge learning organisational system html usage c3 learning knowledge p2 system html social c4 clustering trees agglomerative m2
  37. 37. 37 c3, c4 p2 m2 Competences extracted
  38. 38. 38 Example 2: Essay scoring
  39. 39. 39 Essays Collection 1: Programming: define information hiding
  40. 40. 40 EVALUATIONS Concept space, Quality requirements
  41. 41. 41 Evaluations The role of verification and validation (Schlesinger, 1979, as cited in Oberkampf & Roy, 2010, p.23) (Wild, 2014, p.276)
  42. 42. 42 Verification Results + 22 examples in the documentation (tested by the documentation checker) [...] * this is package mpia version 0.60 [...] * checking examples ... OK * checking for unstated dependencies in tests ... OK * checking tests ... Running tests.R OK [...]
  43. 43. 43 Validation Experiments No standardised test collections for conceptual development Effectiveness: Accuracy in application (Essay Scoring) Convergent and divergent validity Annotation accuracy Degree of loss in the visualisation Efficiency: Performance gain
  44. 44. 44 Evaluation of Scoring Accuracy Example of feedback Using holistic scoring (essay = avg. ~ of 3 model solutions)
  45. 45. 45 Performance Gains Savings in calculation time through using the threshold prediction method for SVD calculation truncation (predicted from original doc-term matrix)
  46. 46. 46 CONCLUSION Revisiting Objectives, Summary, Outlook
  47. 47. 47 Innovation in TEL Three Grand Challenges (Fischer et al., 2014) addressed: new forms of assessment for social TEL environments (Whitelock, 2014a) assessment and automated feedback (Whitelock, 2014b) making use and sense of data for improving teaching and learning (Plesch et al., 2012) 47 learning analytics automated feedback using interaction data to predict performance. #6 e-assessment New forms of assessment of learning in social TEL environments #5 dataTEL Utilising real-time data to improve teaching and learning. #8
  48. 48. 48 Summary
  49. 49. 49 The END