Learning from meaningful, purposive interaction

  • Published on
    07-Aug-2015

  • View
    285

  • Download
    2

Embed Size (px)

Transcript

<ol><li> 1. 1 Learning from Meaningful, Purposive Interaction Fridolin Wild Medieninformatik Universitt Regensburg Knowledge Media Institute The Open University Representing and analysing competence development with network analysis and natural language processing </li><li> 2. 2 Outline Introduction and overview Theoretical foundation Precursor algorithms (SNA + LSA) Algorithm: Meaningful, Purposive Interaction Analysis Mathematical foundation Visual analytics using vector maps as projection surfaces Implementation Application examples for Learning Analytics Evaluation: verification and validation Summary and Outlook </li><li> 3. 3 INTRODUCTION Context, Objectives, Key Contributions </li><li> 4. 4 Introduction Fascination with LSA and Matrix Algebra originated in Information Retrieval (UR), then shifted to Technology Enhanced Learning (WU+OU) Research on Technology Enhanced Learning has its place in the canon of Media &amp; Computing (and Knowledge Media) Its a big and growing global Software Market: Adkins (2011, p.6): 9.2% annual growth till 2015 Docebo (2014,p.8): 7.9% annual growth till 2016 Drivers of Innovation: open Grand Challenges to Research and Development in TEL </li><li> 5. 5 Bridging informal and formal Create a unified, seamless learning landscape with the help of mobile devices. learning analytics automated feedback using interaction data to predict performance. #6 fostering engage- ment Increasing student motivation to learn and engaging the disengaged using technology. How can we detect (de-) motivation? How can make use intrinsic/extrinsic reward systems? #4 New devices for young childrens expression of scientific ideas Mouse and keyboard are a blocker to natural mapping and new modalities of interaction (touch, gestures) can foster a more tactile learning. #1 Learning to read at home with digital technologies #2 CSCL in teacher training and professional development #3 e-assessment New forms of assessment of learning in social TEL environments #5 Understanding how toddler apps can support learning. early years technology dataTEL Utilising real-time data to improve teaching and learning. #7 #8 networked learning ecologies Interest-driven lifelong learning in personal learning networks #9 #10 Fischer,Wild, Sutherland,Zirn (2014) #1 </li><li> 6. 6 Objectives for this work (from GC #5,#6,#8) Represent: to automatically represent conceptual development evident from interaction of learners with more knowledgeable others and resourceful content artefacts; Analyse: to provide the instruments required for further analysis; Visualise: to re-represent this back to the users in order to provide guidance and support decision- making about and during learning. </li><li> 7. 7 Key Contributions (Wild, 2014, p.21) </li><li> 8. 8 THEORETICAL FOUNDATION Concept space, Quality requirements </li><li> 9. 9 Information and Learning communicatively successful cooperatively successful [e]= PmO purpose meaning (Janich, 1998/2003/2006; Hesse et al., 2009; Hammwoehner, 2005; Wild, 2014, p.27ff) learns information </li><li> 10. 10 Information and Learning (Wild, 2014, p.42) </li><li> 11. 11 PRECURSOR ALGORITHMS Foundational examples, Shortcomings </li><li> 12. 12 The Foundational Example Particular unit of company with 9 employees All went through trainings recently Offered by universities (UR, OU), MOOCs, informal learning (FaceBook, LinkedIn) Now: Christina is off sick HR manager to identify worthy replacement SNA LSA MPIA (Wild, 2014, p.60) </li><li> 13. 13 (Wild, 2014, p.21,61,63) Social Network Analysis (SNA) Foundational Example A = </li><li> 14. 14 SNA Paul Joanna Maximilian Peter Christina Simon Ida Thomas Alba Association Matrix </li><li> 15. 15 (Wild, 2014, pp.99-102) Latent Semantic Analysis </li><li> 16. 16 c1 c2 c3 c4 c5 m1 m2 m3 m4 p1 p2 p3 p4 p5 c1c2c3c4c5m1m2m3m4p1p2p3p4p5 c1c2c3c4c5m1m2m3m4p1p2p3p4p5 original space LSA &amp; Similarity (Wild, 2014, p.104: cosines) (Wild, 2014, pp.229) black = 1, white = 0 c1 c2 c3 c4 c5 m1 m2 m3 m4 p1 p2 p3 p4 p5 c1c2c3c4c5m1m2m3m4p1p2p3p4p5 c1 c2 c3 c4 c5 m1 m2 m3 m4 p1 p2 p3 p4 p5 c1c2c3c4c5m1m2m3m4p1p2p3p4p5 LSA space </li><li> 17. 17 Shortcomings Social Network Analysis (SNA) Blindness to content Relationship discovery restricted to incidences captured Popular for analysis, visualization, simulation, intervention (Sie et al., 2012) Latent Semantic Analysis (LSA) Blindness to purposes &amp; structure (relations, groups, ) Lacking instruments for analysis No clear rule for number of factors to retain Popular for essay scoring, information retrieval, dialogue tutoring, recommenders </li><li> 18. 18 MEANINGFUL PURPOSIVE INTERACTION ANALYSIS Foundations in Matrix Algebra, Stretch Truncation </li><li> 19. 19 Fundamental matrix theorem on orthogonality Calculating the Nullspace Ker A: Ax = 0 Eq.1 (Wild, 2014, p.131; redrawn from Strang, 1988, p.140) (Wild, 2014, p.132) every matrix transforms its row space to its column space (Strang, 1988, p.140) </li><li> 20. 20 The Eigenvalue Problem &amp; Singular Value Decomposition (Wild,2014,p.143) For every symmetric, square matrix: (Barth et al., 1998, p.90/E): Bx = x n.b.: B = AAT or ATA Any multiplication of the matrix B with an Eigenvector x yields a constant multiple of the Eigenvector, scaled by the Eigenvalue A = UVT U = Eigenvectors(ATA) V = Eigenvectors(AAT) = UTAV </li><li> 21. 21 Base transformation (from Term-Doc space to orthogonal Eigenspace) (Wild, 2014, p.144) Same dims for both Eigenvector types (row and column), same Eigenvalues! </li><li> 22. 22 Stretch-Dependent Truncation 0 100 200 300 400 0 20 40 60 index eigenvalues 20%80% 0 100 200 300 400 0 20 40 60 index eigenvalues </li><li> 23. 23 Prediction of Threshold Sum of Eigenvalues 2 = Sum of trace of matrix A threshold = 0.8 * sum(A*A) =&gt; Calculate only the first k Eigendimensions, for which the sum of Eigenvalues 2 does not yet pass the threshold </li><li> 24. 24 Updating using ex post projection v' = aT Uk k -1 a' = Uk k v' T (Wild, 2014, p. 149f; see also Berry et al., 1994, equation 7 and page 16 </li><li> 25. 25 Point, Centroid, Pathway e1 e2 u2 u3 u1 p 1 1 3 2 Proximity, Identity 1 11 16 25 9 14 24 2 18 3 19 4 13 5 6 10 15 7 20 22 12 17 26 8 21 23 01234 Cluster dendrogram over all meaningvectors Meaningvectors Height cutoff Clustering </li><li> 26. 26 Introducing Network Analysis Techniques Still: result is high-volume, sometimes even big data Visualisation techniques from (Social) Network Analysis can help! </li><li> 27. 27 In real examples: too high-volume to see structure </li><li> 28. 28 Network Visualisation Proximity-driven Link Erosion (Wild, 2014, p.162) Layout with spring-embedder (Wild, 2014, p.163) Wireframe Conversion (Wild, 2014, p.167) Kernel Smoothing (Wild, 2014, p.169) Hyposometric Tints (Wild, 2014, p.171) </li><li> 29. 29 Perspective plot (Wild, 2014, p.172) </li><li> 30. 30 Topographic map projection and overlays (Wild, 2014, p.173ff) </li><li> 31. 31 IMPLEMENTATION Use Cases, Analysis Workflow, Classes, Demo 31 </li><li> 32. 32 Use Cases </li><li> 33. 33 Class Diagram (Wild, 2014, p.209) &gt; 10.000 lines of codeR package MPIA To be: Open Source (GPL-3) Test-driven development </li><li> 34. 34 APPLICATION EXAMPLES Concept space, Quality requirements </li><li> 35. 35 The SNA/LSA example revisited (Wild, 2014, p.231) C = computer science P = pedagogy M = math + stats </li><li> 36. 36 MPIA foundational example (path of Peter) interface socialweb access review system timeusage html management trees clustering intersection agglomerative knowledge learning organisational system html usage c3 learning knowledge p2 system html social c4 clustering trees agglomerative m2 </li><li> 37. 37 c3, c4 p2 m2 Competences extracted </li><li> 38. 38 Example 2: Essay scoring </li><li> 39. 39 Essays Collection 1: Programming: define information hiding </li><li> 40. 40 EVALUATIONS Concept space, Quality requirements </li><li> 41. 41 Evaluations The role of verification and validation (Schlesinger, 1979, as cited in Oberkampf &amp; Roy, 2010, p.23) (Wild, 2014, p.276) </li><li> 42. 42 Verification Results + 22 examples in the documentation (tested by the documentation checker) [...] * this is package mpia version 0.60 [...] * checking examples ... OK * checking for unstated dependencies in tests ... OK * checking tests ... Running tests.R OK [...] </li><li> 43. 43 Validation Experiments No standardised test collections for conceptual development Effectiveness: Accuracy in application (Essay Scoring) Convergent and divergent validity Annotation accuracy Degree of loss in the visualisation Efficiency: Performance gain </li><li> 44. 44 Evaluation of Scoring Accuracy Example of feedback Using holistic scoring (essay = avg. ~ of 3 model solutions) </li><li> 45. 45 Performance Gains Savings in calculation time through using the threshold prediction method for SVD calculation truncation (predicted from original doc-term matrix) </li><li> 46. 46 CONCLUSION Revisiting Objectives, Summary, Outlook </li><li> 47. 47 Innovation in TEL Three Grand Challenges (Fischer et al., 2014) addressed: new forms of assessment for social TEL environments (Whitelock, 2014a) assessment and automated feedback (Whitelock, 2014b) making use and sense of data for improving teaching and learning (Plesch et al., 2012) 47 learning analytics automated feedback using interaction data to predict performance. #6 e-assessment New forms of assessment of learning in social TEL environments #5 dataTEL Utilising real-time data to improve teaching and learning. #8 </li><li> 48. 48 Summary </li><li> 49. 49 The END </li></ol>