Embed Size (px)
Citation preview
Project Evaluation: Measuring Learning Impact
Diane Salter, Vice Provost Teaching and Learning
Why measure ‘learning impact’?
To determine whether a pedagogical intervention has influenced student learning as intended
To monitor progress and change to enhance student learning
To provide empirical data to support the pedagogical viability of the learning/teaching method
When might this be important? As part of a scholarly approach to
teaching – ongoing revision and course/ programme review
Projects/grants applications and
reporting to show:Does this project/instructional strategy promote
active learning-centred teaching? Does this project/instructional strategy lead to
significant improvements in quality or outcomes of student learning?
How might the approach to this subject be improved?
To measure impact we need to know:
How do we measure impact on learning?
What are the objectives/expected outcomes
of the intervention?
What are the indicators of success?
How can these be observed/measured?
What is the baseline (what was happening before the intervention) against which changes can be judged?
Measuring impact
Level 1 – Reaction
Level 2 – Knowledge and Skills Gained
Level 3 – Behaviour
Level 4 – Performance Competence *
Measuring impactReaction – How did it feel ? Did they like it?
Do they think it made a difference?
Knowledge and Skills Gained – What can they do that they couldn’t do before, or do better than they could do before?
Behaviour – What do they do now that they didn’t do before?
Performance competence – Has their performance improved as a result of increased learning?
Examples of indicators
Student Feedback
Perceived usefulness and benefitsSelf report on motivation, engagement or study approaches Self-reported learning gains
Instructor Feedback
Observed changes in students’ motivation, engagement and study behaviourTeachers’ reflections, etc.
Classroom Observations
Nature/amount of classroom interactionsStudents’ engagement and participationOther classroom study behaviours
Student performance
Assessment grade/marksQuality of student work Performance in practice settings
How do we know our course design works?
What was the effect of an experience the learners? (learning impact):
Knowledge (e.g. know what, know how and know why)
Skills (e.g. cognitive, meta-cognitive, social and psychomotor)
Attitude (e.g. beliefs, disposition and values)
What’s in a name?
Scholarly Teaching
Scholarship of Teaching
9
Scholarly Teaching
Evidence based approaches to teaching and course design based on best practice.
10
Scholarship of Teaching
Sharing Presentations Publications Reflection Revision
11
What is the difference?
Research for scholarship and publication Ethical approval required
Research/feedback for the purposes of improvement of studies and programs.
See Article 2.5 Tri-council agreement
12
Tri-council policy statement
The Tri-council policy statement on Ethical Conduct for Research Involving Humans guides research done at university and provides clear direction regarding doing research/gathering feedback for the purposes of improvement of studies and programs.
See article 2.5 (next slide)
13
Tri-council policy statement
Article 2.5 states:
Quality assurance and quality improvement studies, program evaluation activities, and performance reviews, or testing within normal educational requirements when used exclusively for assessment, management or improvement purposes, do not constitute research for the purposes of this Policy, and do not fall within the scope of the REB.
14
More information needed?
If you would like a seminar in your department about SoTL and measuring learning impact
please contact:
15
