View
215
Download
0
Category
Tags:
Preview:
Citation preview
LEARNING ANALYTICS&
THE LEARNING SCIENCES
Zachary Stein, Ed.M, Ed.D (c)Lectica®, Inc.
Harvard University
OVERVIEW Learning about learning
What is learning? What are the sciences of learning?
Analytics: definitions and approaches Narrowing in on learning analytics and its relatives Lessons from history: testing and technology in US
education
Putting the learning in learning analytics Lectical™ Assessments and the future of educational
assessment
© 2012 Lectica, Inc. all rights reserved
WHAT IS LEARNING?
There are over a dozen definitions currently in use by researchers and educators From learning as “changing connections between
neurons” to learning as “targeted behavior change” to learning as “the active construction of knowledge that is transferable, applicable, and reflectively held”
There is no one “right” definition, but we should set our terms so we know we are talking about the same thing
© 2012 Lectica, Inc. all rights reserved
WHAT IS LEARNING? Individual researchers and innovators should
define what they mean A look at nearly a dozen papers on learning analytics
revealed no attempts at defining learning Yet taking pedagogical or institutional actions
based on learning analytics data assumes a theory of learning, which is often implicit and inarticulate How do you know what to do to promote learning if you
don’t know how learning works? We should look to the learning sciences for help…
© 2012 Lectica, Inc. all rights reserved
WHAT ARE THE SCIENCES OF LEARNING?
They go way back and they are diverse and complex Philosophy, biology, psychology, neuroscience, etc.
They involves a wide variety of models, methods, and motives The emerging interdisciplinary field of Mind, Brain, and
Education (MBE) exemplifies the new science of learning Here I will look at one of the central orienting
generalization in this field, stemming from the cognitive developmental tradition
© 2012 Lectica, Inc. all rights reserved
LEARNING AS A VIRTUOUS CYCLE
An idea that goes back to the first biologically oriented psychologist
A generalization covering learning processes at multiple levels, from neural-networks, to cognitive development, organizational and group learning (dynamic steering), and scientific investigation.
A process that explains the stage-like development of individual learners through learning sequences of increasing complexity
© 2012 Lectica, Inc. all rights reserved
THE CIRCULAR REACTION: LEARNING AT ITS CORE
Baldwin, Piaget, and a baby in a highchair
think you got it... try it... get feedback.... try again (hopefully
at a higher level)....
Virtuous cycles always include tests
© 2012 Lectica, Inc. all rights reserved
VIRTUOUS CYCLES OF LEARNING
Can occur at multiple levels in educational institutions: Students Teachers Administrators Researchers
Consider the example of a classroom A process we refer to as instructional dynamic
steering™ Note the role of embedded assessment…
© 2012 Lectica, Inc. all rights reserved
LEARNING ANALYTICS REDUX
Let’s think through learning analytics with learning defined as a multi-level virtuous cycle…
© 2012 Lectica, Inc. all rights reserved
DATA-DRIVEN DECISION MAKING In a wide variety of contexts modern organizations
have become “centers of calculation and measurement” Scientific management Dynamic steering
In educational contexts this has mostly meant the application of business models and an increase in testing Learning analytics steps in as information technology opens
new opportunities for data generation and analysis (e.g., so-called “Big Data”)
© 2012 Lectica, Inc. all rights reserved
LESSONS FROM THE HISTORY OF TESTING
Preferences for automation and economies of scale
Anti-theoretical orientation leaves us satisfied with weak, under-theorized measures
Measurement structures the system Preferences for objectivity increasingly
marginalize the judgment of those on the ground The informational needs of institutional decision-
makers override the learning needs of the students
© 2012 Lectica, Inc. all rights reserved
THE VARIETIES OF LEARNING ANALYTICS
Learner-centric analytics Learning-centric analytics Classroom/cohort analytics Program evaluation analytics Institutional analytics
These can work at cross-purposes As in the history of testing
These can also work in tandem As is the case with Lectical Assessments
© 2012 Lectica, Inc. all rights reserved
CALL FOR RESEARCH BASED ASSESSMENTS National Research Council Committee on the
Foundations of Assessment (1999; 2001) Assessment design requires research
Tests should be built around a theory of learning, not just psychometrics and expert opinion
Goals and standards should be grounded in evidence about student capabilities
However, high-stakes tests continue to be built around psychometrics and expert opinion, with little input from the learning sciences
© 2012 Lectica, Inc. all rights reserved
LECTICAL ASSESSMENTS R&D
Decide on key subject areas and topics in collaboration with educators and researchers
Conduct basic research in order to build empirically- grounded learning sequences that can be translated into low-inference scoring rubrics
Compile and “developmentally curate” related learning materials
The result: an integrated set of assessments and learning materials that support virtuous cycles of learning
© 2012 Lectica, Inc. all rights reserved
MEASURING LEARNING
The Lectical Assessment System Based on Fischer’s Skill Theory (1980; 2006) A domain general, content independent, developmental
assessment system (Dawson, 2004; 2011) Excellent psychometric properties (Dawson-Tunik, Commons,
Wilson, & Fischer, 2005)
The common core metric behind all Lectcial Assessments And the tool used to build them
© 2012 Lectica, Inc. all rights reserved
A LOOK AT THE LDMA
One of 7 existing Lectical assessments for use in higher-education Currently being used as part of large-scale leadership
education program for a major Canadian city. Like all Lectical assessments, the LDMA asks
students to construct short essays, which are then coded using empirically grounded rubrics based on research into a set of relevant learning sequences.
Feedback fosters learning for students, teachers, decision-makers, and researchers
© 2012 Lectica, Inc. all rights reserved
STUDENT LEARN
Lectical Assessments support student learning by… requiring them to operate on concepts and apply skills encouraging reflection on personal performance (see
growth over time) providing rich, targeted, real-time feedback based on
test results, including suggestions for what comes next delivering targeted (and vetted) learning activities
© 2012 Lectica, Inc. all rights reserved
TEACHERS LEARN
Lectical Assessments support teacher learning by… encouraging deep engagement with student performances
(teachers read and score tests, not computers) providing insight into student learning (through online student
and teacher reports) offering specific information, in real time, about what
individual students have learned and what they are most likely to benefit from learning next
providing learning resources that show how to target instruction to individual students (learning resources)
© 2012 Lectica, Inc. all rights reserved
DECISION-MAKERS LEARN
Lectical assessments support decision-maker learning by providing…
complex multi-focal learning profiles, at the cohort, classroom, and organization (in real time)
multiple, repeated measures that reveal trends and trajectories (growth models)
© 2012 Lectica, Inc. all rights reserved
RESEARCHERS LEARN
DiscoTest support educational research by... contributing to basic research about learning during
their construction generating large relational databases of detailed
individual learning profiles trends and trajectories contribute to our understanding of
learning in general
generating overviews of learning at the level of the cohort, classroom, and organization
having data standardized to a common core metric
SUMMING UP
The emerging field of learning analytics can benefit from taking the learning sciences seriously Otherwise, we run the risk of repeating the mistakes
made during the first wave of data-driven decision making in schools
Recent advances in educational assessments technology can be used to catalyze multi-level learning in education intuitions Leveraging true measures of learning to help students,
teachers, administrators and researchers
Recommended