Embed Size (px)
Citation preview
Learning Analytics for Practice- and Policy-oriented Educational Research Venia legendi
Kairit Tammets Candidate for the post of senior researcher 17th November 2015
Introduction Rapid advances in technology provide us necessary infrastructure to accept the online learning and support the delivery of education at a large scale; The adoption of educational technologies enable us new opportunities to gain insight into learning
Outline Learning analytics for practice: Competence-based learning in individual and organizational level; Teacher’s learning and knowledge building practices in socio-technical system; MOOCs and dashboards
Learning analytics for policy-oriented educational research
Competence-based approach in education • Evaluation of teacher’s professionalism is
formal and rigid - need for evidence-based evaluation
• Teacher’s evaluation could be promoted by taking on-the-job activities of teachers into account (Tammets et al, 2011)
• Learning activities mapped with competencies (Ley, Tammets, Lindstaedt, 2014; Tammets et al, 2011)
Competence-based approach in education • Profiling individual competencies
• Digimina • eDidaktikum • IntelLEO (Intelligent Learning Extended
Organization) • Profiling organizational competencies
• EpoAbi • eDidaktikum • IntelLEO
Digimina • Web-based environment for
evaluating teachers’ digital competencies
• Three evaluation types: • Self-analysis (with the evidence) • Peer-assessment • Automatic assignment
• Based on the evaluation types, individual competency profile will be created
Põldoja, H., Väljataga, T., Laanpere, M., & Tammets, K. (2014). Web-based Self- and Peer-Assessment of Teachers’ Digital Competencies. World Wide Web, 17(2), 255 - 269
Learning Path Creator Individual learning paths: • Define the required competences • Identify gaps in your competence
development • Plan your learning paths • Monitor your development System: • Accumulates paths as
organizational paths • Recommends appropriate learning
paths
Tammets, K., Pata, K., Laanpere, M., Tomberg V., Gaševic, D., & Siadaty, M. (2011). Designing the Competence-driven Teacher Accreditation. H. Leung, E. Popescu, Y. Cao, R.W.H. Lau & W. Nejdl (Toim.). Advances in Web-based Learning (132 - 141). Springer Verlag
EpoAbi Bottom-up approach to course
design; Course designers plan their
learning activities (nine different activities), assessment methods, technologies;
Course activities and assessment methods are mapped and associated with outcomes;
Systems store the course designs and provides recommendations for similar courses.
Tammets, K., & Pata, K. (2013). The trends and problems of planning outcome-based courses in elearning. Saar, E., Mõttus, R., & Bern, E. (Toim.). Higher Education at a Crossroad: the Case of Estonia (281 - 301). Peter Lang Publishers House
eDidaktikum Continuous and systemic
documentation of knowledge and practices thorough formal studies in teacher education;
Mapping learning activities and digital resources with competencies
Competency profile of pre-service teacher will be created
Why not to create competency profile of the course? Curriculum?
Tammets, K., Tammets, P., & Laanpere, M. (2014). eDidaktikum – online community for scaffolding pre-service teachers into digital culture. In: The proceedings of the ECER conference: 01.- 05. September 2014, Porto
Individual’s documented knowledge and practices enhances the organizational-level knowledge: Competency profiles: strengths of the staff; gaps in
competencies, Learning paths for acquiring certain competencies; Curriculum development
Learning and knowledge building practices • Individual internal learning and collaborative
knowledge building practices in the socio-technical system
• Such practices support professional development, on the job activities needed for evaluation and promote the development of organizational knowledge
• For promoting teacher’s professional development, systemic model is needed: • Knowledge conversion SECI model (Nonaka & Takeuchi,
1995)
Learning and knowledge building practices
Tammets, Pata & Laanpere, 2012
Socio-technical system Socio-technical system promotes enables to: Write reflections and share them within the community; Find and reuse those reflections with the aim to learn from them, and for creating community knowledge; Participate in collaborative and knowledge building activities related with knowledge, competences, or actions with the community members; Plan professional development based on shared social and community norms.
eDidaktikum Initially planned to be learning resource repository for teacher education Based on participatory design approach, it was designed as community-based learning environment of teacher education; Competence-based learning
LKB in socio-technical system • Accumulated knowledge in the socio-
technical system acts as a scaffold for teachers’ professional development
• Data stored in the system can be used for supporting teachers’ learning and knowledge building practices
Tammets, K., Pata, K., Laanpere, M. (2013). Promoting Teachers’ Learning and Knowledge-building in the Socio-technical System. The International Review of Research in Open and Distance Learning, 14(3), 251 - 272.
Learning analytics for policy-oriented research
Learning Analytics – what?
Technologies and systems around us capture learners’ interactions and their online activities Mining and analyzing log data of these systems for identifying patterns, enables insights to into educational practice Is: “Measurement, collection, analysis and reporting of data about learners and their contexts
Learning Analytics – Why?
for purposes of understanding and optimizing learning and the environments in which it occurs”
Siemens, G., & Gaševic, D. (2012). Special Issue on Learning and Knowledge Analytics. Educational Technology & Society, 15(3), 1–163.
Learning Analytics The use of analytics in education has grown in recent years for four primary reasons: • a substantial increase in data quantity; • improved data formats; • advances in computing; • increased sophistication of tools available for
analytics.
Baker, R., Siemens, G. (2014). Educational data mining and learning analytics. In Sawyer, K. (Ed.) Cambridge Handbook of the Learning Sciences: 2nd Edition, pp. 253-274.
Existing research work in LA Predicting learner performance and modeling learners – aim is to estimate the unknown value of a variable that describes the learners, such as performance, knowledge, scores or grades; Suggesting relevant learning resources – recommender systems that analyze learner data to suggest relevant learning resources or - paths Increasing reflection and awareness – analysis and visualization of learning indicators to foster awareness and reflection about learning processes (resource access, time spending, knowledge level indicators)
Existing research work in LA Enhancing social learning environments – make people aware of their social context and enable then to explore this context Detecting undesirable learner behaviors – discover learners who have unusual behavior such as misuse, cheating, dropping out or academic failure Detecting affects of learners – boredom, confusion, flow/engagement for adjusting pedagogical strategies
Verbert, K., Manouselis, N., Drachsler, H., & Duval, E. (2012). Dataset-Driven Research to Support Learning and Knowledge Analytics. Educational Technology & Society, 15 (3), 133–148.
EMMA (European Mul.ple MOOC Pla2orm) aims: § Create a pan-European platform to support ICT-based innovation in higher education § Offer MOOCs provided by accredited institutions from around Europe with different teaching methodologies and learning design approaches § Offer an extensive and multil ingual transcription/translation system § Offer an extensive monitoring system § Give opportunities to small universities and cultural institutions to get in the MOOC market
EMMA LA methodology
• What are the research questions? • What data is needed for answering research
questions? • How this data can be collected? • How to meaningfully present data? • How to design feedback loop? • How to learn from the process?
Dashboards Visualizations (dashboards) are used for visualizing learning analytics results: Progress and overview of the course activities Performance Social structures Recommendations
BUT: Design and use of learning analytics dashboards is far
less understood; What changes after using dashboards in teaching and
learning?
Issues around large scale LA - Numerous small-scale R&D projects have demonstrated successful outcomes, but are dependent on contextual factors -> no strong evidence of the overall effectiveness of LA at scale Few universities have made use of LA at scale; No detailed implementation accounts available; Organizational leaders must require the vision to see how small-scale projects might be scaled to improve teaching and learning across an institution
Ferguson, Rebecca; Macfadyen, Leah P.; Clow, Doug; Tynan, Belinda; Alexander, Shirley and Dawson, Shane (2015). Setting learning analytics in context: overcoming the barriers to large-scale adoption. Journal of Learning Analytics, 1(3) pp. 120–144.
LA in policy-oriented educational research LA as a powerful tool for educational management Merging educational research and practice with LA data
provides novel and real-time approaches to assess issues impacting eudcation: retentions, 21st cent skills, personalised learning; Automating measurements and predictions is promising,
but focus is on outcomes; take into account learning and teaching process
LA in policy-oriented educational research Pedagogical approach: see the pedagogy behind the numbers; Field of learning analytics needs to ground data collection, measurement, analysis, reporting and interpretation processes within the existing research on learning Take the numbers into account in policy-level decisions (curriculum development) Learning analytics is question-driven, not data or technology driven
The Rapid Outcome Mapping Approach (ROMA) Model for guiding an iterative approach to planning the systemic institutional implementation of learning analytics; Seven‐step model is focused on evidence‐based policy change Goal is moving to broader institutional implementation
In agenda What happens after dashboards:
Interpretation skills needed for using dashboards? In which way instructional designs change as a result of using
dashboard? How dashboard may influence learners’ decision-making skills, learning
from failure and becoming self-sufficient?
Integration of LA with educational research: e.g. how students’ learn in online settings and how do they perceive their learning; Integration of LA to higher education; Bringing LA to policy level: LA results of different learning
environments as part of curriculum development process in DTI
Conclusion ROMA model – systemic implementation of LA in institutional level; Participatory design of LA tools and practices – learners, teachers, instructional designers, technologists, researchers and policy-decision makers need to work together for avoiding undesirable practices of LA; Integration with the educational research; When our focus is on improving learning, the critical results we need to monitor and measure are the results that reflect positive educational change .
References Tammets, K., Pata, K., Laanpere, M., Tomberg V., Gaševic, D., & Siadaty, M.
(2011). Designing the Competence-driven Teacher Accreditation. H. Leung, E. Popescu, Y. Cao, R.W.H. Lau & W. Nejdl (Toim.). Advances in Web-based Learning (132 - 141). Springer Verlag Ley, T., Tammets, K., & Lindstaedt, S. (2014). Orchestrating collaboration and
community technologies for individual and organisational learning. LittleJohn, A., & Margaryan A. (Toim.). Technology-Enhanced Professional Learning: Processes, Practices, and Tools (117 - 131). New York: Routledge
