View
2.602
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Using intelligent tutoring systems, virtual laboratories, simulations, and frequent opportunities for assessment and feedback, The Open Learning Initiative (OLI) builds open learning environments that support continuous improvement in teaching and learning. One of the most powerful features of web-based learning environments is that we can embed assessment into, virtually all, instructional activities. As students interact with OLI environments, we collect real-time data of student work. We use this data to create four positive feedback loops: • feedback to students • feedback to instructors • feedback to course designers • feedback to learning science researchers In this JumpStart Session, we demonstrate how OLI uses the web to deliver online instruction that instantiates course designs based on research and how the learning environments, in turn, support ongoing research. We will discuss the Community College Open Learning Initiative (CC-OLI) and how faculty and colleges across the country can participate in CC-OLI and the connection between CC-OLI and Washington State’s Open Course Library project.
Citation preview
Continuous Improvement in Teaching and Learning:
Candace Thille, Director OLI
WHICH PROBLEM TYPE IS MOST DIFFICULT FOR BEGINNING ALGEBRA
STUDENTS?Story Problem
As a waiter, Ted gets $6 per hour. One night he made $66 in tips and earned a total of $81.90. How many hours did Ted work?
Word Problem
Starting with some number, if I multiply it by 6 and then add 66, I get 81.90. What number did I start with?
Equation
x * 6 + 66 = 81.90
Algebra Student Results:Story Problems are Easier!
The Expert’s Blind Spot
0
10
20
30
40
50
60
70
80
90
100
ElementaryTeachers
Middle SchoolTeachers
High School Teachers
% Making CorrectRanking (which problems hardest)
Nathan, M.J. & Koedinger, K.R. (2000). Teacher’s and researchers beliefs of early algebra development. Journal of Mathematics Education Research, 31(2), 168-190
Expert intuitions about student difficulties are often wrong, systematically biased
OLI Goals• Produce exemplars of scientifically based online
courses and course materials that enact instruction and support instructors
• Provide open access to these courses and materials
• Develop a community of use, research & development that contributes to the evaluation, continuous improvement, and ongoing growth of the courses and materials.
Goal directed practice and
targeted feedback
What is a Cognitive Tutor?
• A computerized learning environment whose design is based on cognitive principles and whose interaction with students is based on that of a (human) tutor—i.e., making comments when the student errs, answering questions about what to do next, and maintaining a low profile when the student is performing well.
Developing component skills and
knowledge, and
synthesizing and applying
them appropriately
Feedback: Changing the Effectiveness of both Learners
and Faculty
“The Killer App” feedback loops for continuous improvement
Learning Curve Analysis on Stoichiometry Data
OLI Review:• Apply learning science research and scientific method
to course development, implementation and evaluation
• Environments are developed by teams of content experts (and novices), learning scientists, HCI, software engineers
• Feedback loops for continuous improvement
What Difference Does This Make?
Accelerated Learning Results
16
OLI students completed course in half a semester, meeting half as often during that time
OLI students showed significantly greater learning gains (on the national standard “CAOS” test for statistics knowledge)
No significant difference between OLI and traditional students in follow-up measures of knowledge retention given a semester later
These results have been replicated with a larger sample
Other Class Results
17
Community College accelerated learning study:OLI: 33% more content coveredOLI: 13% learning gain vs. 2% in traditional
face-to-face class
Large State University: OLI: 99% completion rateTraditional face-to-face class: 41%
completion rate
CC-OLI Goals:• 25% jump in successful course
completion rates in 4 CCOLI gate-keeper courses at participating colleges
• 40 colleges participating in CCOLI course development, adaptation, evaluation
• An established model, platform & tools for testing and scaling innovation
CC-OLI & The Open Course Library Project
• Cable Green is on the CC-OLI Board
• WA CTCs will have faculty (SMEs) on the design teams for the first two courses
• OLI Environments are OER that can be used by the faculty designing courses for the Open Course Library project.
“Improvement in Post Secondary Education will require converting teaching from a ‘solo sport’ to a community based research activity.”
—Herbert Simon
www.cmu.edu/oli [email protected]
Pittsburgh Science Of Learning Center (PSLC)
• Why? Chasm between science & practice Indicators: Ed achievement gaps persist, Low hit rate of randomized control trials (<10%)
• Underlying problem: Many ideas, too little sound scientific foundation
• Need: Basic research studies in the field
• PSLC Purpose: Identify the conditions that cause robust student learning– Field-based rigorous science – Leverage cognitive & computational theory,
educational technologies “rigorous, sustained scientific research in education” (NRC, 2002)
Challenges in Education Research
• Too few experiments that are both rigorous & realistic– Lab studies: short duration, unrealistic content, non-
representative participants– Field studies: lack of controls, change many variables– Lack of fine grain data over long duration
• Inadequate measures of robust learning– Transfer, long-term retention, future learning
• Needed: robust learning studies in real courses with fast feedback loops
• Rigorous studies with real content in real classrooms with real students– On-line courses as lab environment– Authoring tools for learning experiments– Partnerships with schools
• Fine-grain data over months– On-line courses collect interaction-level data – DataShop analyzes data
• Measures of robust learning• Combined theory and practice to develop effective
educational technology and refine learning theory
Pittsburgh Science of Learning Center (PSLC)
Pasteur’s Quadrant• Stokes argues basic/applied goals need not trade off
Low Emphasis on Applied Work
High Emphasis on Applied Work
High Emphasis on Basic Science
How to translate to the real world?
Low Emphasis on Basic Science
What principle can be derived?
X
Data Log Analysis Results
26
Statistics and Biology “Dose Response” data log analysis: positive and significant correlation between student use of OLI learning activities and quiz scores on target topic – no correlation with unrelated topics
A study conducted on the OLI stoichiometry course revealed that the number of engaged actions with the virtual lab explained about 48% of the variation observed in the post test scores. The number of interactions with the virtual lab outweighed ALL other factors including gender and SAT score as the predictor of positive learning outcome.
End of Course Student survey for accelerated online:
• 85% Definitely Recommend
• 15% Probably Recommend
• 0% Probably not Recommend
• 0% Definitely not Recommend
Quotes• Student Quote: "This is so much better than
reading a textbook or listening to a lecture! My mind didn’t wander, and I was not bored while doing the lessons. I actually learned something.“
• Instructor Quote: “The format [of the accelerated learning study] was among the best teaching experiences I’ve had in my 15 years of teaching statistics.”
OLnet: Open Learning network
Network
Research
Fellowships
From producing open resources to use of open resources•Build capacity•Find evidence•Refine the issues
OLnet Research Questions• How can we build a robust evidence base to
support and enhance the design, evaluation and use of Open Educational Resources (OER)?
– How do we improve the process of OER design/reuse, delivery, evaluation and data analysis?
– How do we make the associated design processes and products more easily shared and debated?
– How do we build a socio-technical infrastructure to serve as a collective evolving intelligence for the community?