Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
Gordon SuddabyScholarship of Learning and Teaching
National Centre for Teaching and Learning
Researching your teaching
Rationale Academics typically find themselves questioning their own ideas
and searching out explanations for complex issues. This applies as much to teaching as to research. This module will look at why academic staff might investigate their own teaching practice, determine the possible benefits associated with doing so, consider how they might do so, how they might use the findings to further develop themselves as scholarly and research-led academic teachers, and to advance their careers.
Why should we be interested in researching our teaching?Brainstorm:1.2.3....
Universities are generally interested in teaching... to ensure quality, effectiveness, and accountability to provide recognition and reward to bring about improvement. ... ...The expression of this interest usually involves some form of
evaluation.
Evaluation of teachingThere are two overriding purposes for evaluation;
To measure and accord value to teaching for accountability
To improve the quality of teaching (and learning)
Thinking about evaluating our teaching: Poor evaluation, whether of students or of staff, renders an
unfair judgement and fails to reveal shortcomings in performance. Good evaluation on the other hand provides decision makers with the information necessary for informed choices and teachers with useful feedback for improvement.
Centre, 1993, p.1
Vocabulary Formative: Provides feedback which is used during the
[teaching] process for improvement. It is continuous, diagnostic, remedial, and low stakes.
Summative: ...used after [the teaching] process has been completed. Grading and accountability are major outcomes. It is terminal, finite, descriptive and high stakes.
After Scriven, 1967.
Activity In groups of 3, identify the university processes involving decision-making
about you that requires evaluative information about your teaching
...
What aspects of your teaching provides that information?
What (if any) further information could/should be provided?
How and by whom?
How valid and reliable do you think the information is?
Lets be a little pragmatic... One of the most important and tangible reasons for
evaluating teaching is for promotion so this is a good place to start.
We are effectively evaluating our teaching and to a degree, making that evaluation ‘public’ (at least to a selected audience)
So what information is expected in a promotion application?
5.2.1 Criteria Specific to Teaching Good teaching enhances student development in the directions specified
for particular programmes. Criteria for promotion will allow for different teaching philosophies and contexts.
Indicators of teaching effectiveness include:
(a) the utilisation of current research-based knowledge;
(b) planning and teaching strategies that present knowledge clearly and engage challenge and stimulate students to become independent learners;
(c) evaluation of one’s own teaching activities;
(d) effective communication with students;
(e) appropriate assessment procedures;
(f) evidence of self-evaluation and continuing professional development;
(g) recognition of teaching quality by peers and students;
(h) Teaching Portfolio (Refer Training & Development Unit for assistance).
(i) effective education and teaching of post-graduate students.
5.2.2 Evidence specific to teaching Applicants (except those in research-only positions) should present evidence of teaching
effectiveness, which includes both quantitative and qualitative data from a variety of sources. This should include:
(a) quantitative data on teaching loads, including data on paper numbers, contact hours, on-campus courses, student numbers, supervision loads; number of successful postgraduate student completions.
(b) a brief outline and justification of the applicant's approach to teaching;
(c) examples of paper design (including learning outcomes, curriculum outline/s and teaching and assessment strategies) highlighting aspects that reflect the applicant's approach to teaching;
(d) an overview of student feedback eg. SECAT, etc., including (where appropriate) feedback from the applicant's research students, all with summary comments from the applicant;
(e) a description of steps taken by the applicant to further develop her/his teaching effectiveness;
(f) details of prizes awarded for teaching or other forms of recognition of excellence;
(g) assessment by the Small Group Instructional Diagnosis [SGID] (contact Training & Development Unit).
(h) evidence of incorporation of relevant recent research into paper content.
(i) any other evidence demonstrating teaching effectiveness and commitment;
Something to think about...In what may as well be starkly labelled smug
satisfaction, an amazing 94% [of college instructors] rate themselves above average teachers and 68% rate themselves in the upper quartile of teaching performers.
K. Patricia Cross
Some thoughts about evaluating teaching...When evaluations are used for promotion and tenure, the accepted rule
is that no single evaluation should be considered adequate for decision making
Theall and Franklin, p.94, 1991Use multiple sources of data if you are serious about improving
teaching Cashin, p.93, 1992It is wise to be circumspect about using student ratings to make
judgments on teaching quality and to recognise their complications as well as their virtues.
Ramsden, p.229, 1992
Activity Rule a column down the middle of a piece of paper On the left hand side list things that you believe you have
done well when teaching
On the right hand side for each indicator, map the evidence you have that supports the point
What evidence do you use to know things haven’t gone well?
So the question for you as a teacher is... How do you know you are teaching well?
Or
How well you are teaching?
And
How might you improve?
Think about...A colleague suggests that he/she is a good teacher
What evidence would you need to verify that statement?(NB the evidence needs to be valid, legitimate, verifiable, reliable and ethical)
...
How might such evidence be collected? ...
Researching teaching Apart from promotion, what other reasons might we have for
researching our teaching? ... ... ...
Researching teachingPurposes; For improvement, i.e. Quality enhancement; development and
improvement of learning, teaching etc
For accountability i.e. Quality assurance; competence, assurance for stakeholders etc
For publication i.e. Development of pedagogical research
For portfolios i.e. Promotion, recognition, reward, reflection
For integration i.e. Alignment of curricula in specific disciplines
For reflection i.e. Development of critical reflective practice
Scholarship i.e. Making learning transparent and explicit
Definition: Scholarship of Teaching and Learning (SoTL) ‘The scholarship of teaching and learning is the systematic
reflection on teaching made public”. This involves supporting the development of teaching and learning which fosters ‘long-lasting learning for all students; enhances the practice and profession of teaching, and; brings to faculty members’ work as teachers the recognition and reward afforded to other forms of scholarly work”. In doing so, SoTL “renders teaching public, subject to critical evaluation, and usable by others in both the scholarly and general community”.
http://www.carnegiefoundation.org/scholarship-teaching-learning
Boyer’s Scholarships The Scholarship of Discovery.
The Scholarship of Integration.
The Scholarship of Application.
The Scholarship of Teaching.
Boyer, 1990.
Boyer’s scholarships The Scholarship of Discovery. This comes closest to what is usually meant by ‘basic research’. It is
scholarship that involves “commitment to knowledge for its own sake, to freedom of inquiry and to following, in a disciplined fashion, an investigation wherever it may lead” (Boyer, 1990, p. 17).
The Scholarship of Integration. This is “serious disciplined work that seeks to bring new insight to bear on original work” (p. 19). Giving meaning to isolated facts, putting such facts into perspective, fitting research into larger intellectual patterns, making connections across the disciplines, placing the specialties in larger contexts are all activities that Boyer associates with this mode of scholarship.
The Scholarship of Application. This work, which also requires rigor and accountability, constitutes service to others which calls for the application of special fields of knowledge and associated skills. In the course of this service, new understandings may also arise as “theory and practice vitally interact and one renews the other” (p. 23). Note: Boyer (1996) subsequently renames this type of scholarship, the Scholarship of Engagement.
The Scholarship of Teaching. Boyer viewed teaching as a scholarly enterprise because the “work of the professor becomes consequential only as it is understood by others” (p. 23) and teaching serves to both educate and entice future scholars. “It is inspired teaching that keeps the flame of scholarship alive.” (p. 24). He also observed that “pedagogical procedures must be carefully planned, continuously examined, and relate directly to the subject taught” and “good teaching means that faculty, as scholars, are also learning” (about teaching and learning).
Scholarship of Teaching is about:A. knowing the literature on teaching by collecting and reading
that literatureB. improving teaching by collecting and reading the literature on
teachingC. improving student learning by investigating the learning of
one’s own students and one’s own teachingD. improving one’s own students’ learning by knowing and
relating the literature on teaching and learning to discipline specific literature and knowledge
E. improving student learning within the discipline generally by collecting and communicating results of one’s own work on teaching and learning within the discipline.
(Trigwell et al, 2000, pg 159).
A scholarly approach to teaching involves; Clear Goals
Adequate preparation
Appropriate methods
Significant results
Effective presentation
Reflective Critique
Glassick, Huber and Maeroff (1997)
What does the literature say about good teachers? Good teachers are;Knowledgeable about their subjectAdopt an organised and systematic approach to their
teachingAre enthusiastic and interestingRespect their studentsHave high expectations of their students performance
From: Prebble, Hargraves, Leach, Naidoo, Suddaby and Zepke (2005)
Clarifying our approach:Research
– enhances our theoretical and/or conceptual understanding of teaching and learning
– is firmly situated in its relevant literature and makes a substantial contribution to that literature and or field.
Investigations and Evaluations – enhances our understanding of a local problem or issue, providing
recommendations for policy and / or action – is firmly situated in its relevant literature
Literature Reviews – collection and analysis of literature aimed at describing the various ways in
which the object of the review is thought about and recommendations for practice
Scholarship of Teaching and Learning – evidence based critical reflection on practice aimed at improving practice
Personal circumstances, interests and capabilities
Is there some aspect of teaching and learning that I am really interested in and want to investigate further?
Do I think I can change and improve my own and others’ teaching through SoTL? Am I confident about my research capabilities, including ‘doing’ SoTL? Am I confident about my teaching but ready to investigate it? Given other demands in my academic and personal life, is there space for engaging in
SoTL?
Pull Factors Do other people expect me to be able to offer evidence to support my teaching
decisions or to confirm I am teaching effectively?
Push Factors Would this be another way of fulfilling PBRF expectations? Would this be another avenue for meeting postgraduate research expectations?
(Haigh, Gossman & Jiao, 2010)
Obstacles Personal pre-conditions (as above) are not present
Would what I am interested in investigating be of interest to others?
Would SoTL be given the same status as my other disciplinary research?
Would my colleagues/department be supportive of my engaging in SoTL?
Would I be able to do SoTL when there is limited funding?
Would I be able to get some publications without too much delay?
Would I be able to find relevant, useful, good quality literature on my interests?
Would I be able to develop an acceptable proposal?
Would I be able to handle the ethics considerations and make my way through the ethics approval process?
http://akoaotearoa.ac.nz/sotl
But there are optionsWe can change the focus from; “what is the quality of your teaching” To; “how can we use evidence gathered from student feedback
and other forms of information to improve teaching?” http://akoaotearoa.ac.nz/content/guide-collecting-
evidence-learner-benefitBerk (2005) has listed 12 ways as a starter to which I will add
two more; Small Group Instructional Diagnosis and Classroom Assessment Techniques.
12 sources of evidence(Berk, 2005)1. Small Group Instructional Diagnosis2. Classroom Assessment Techniques3. Student ratings4. Peer ratings5. Self-evaluation6. Videos7. Student interviews8. Exit and Alumni ratings9. Employer ratings10. Administrator ratings11. Teaching scholarship12. Teaching awards13. Learning outcome measures14. Teaching portfolios
Small Group Instructional Diagnosis Developed by Clark in 1979 Small Group Instructional Diagnosis
(SGID) is a form of classroom research which focuses on student learning. It is a formative process enhancing learning and is a safe, non-threatening and transparent mechanism liked by students and staff. It is orally-based, involving concensus and involves little time with a quick turn-around.
http://tdu.massey.ac.nz/pdfs/sgid.pdf
http://tdu.massey.ac.nz/forms/sgidrequest.html
Teaching Awards As a source of evidence of teaching effectiveness, at best, teaching
awards provide worthwhile information only on the nominees, and, at worst, they supply inaccurate and unreliable feedback on questionable nominees who may have appeared on ‘Law and Order’. The merits of teaching awards should be evaluated in the context of an institution’s network of incentives and rewards for teaching.
http://www.massey.ac.nz/massey/about-massey/university-management/avc-academic/awards/teaching-awards-framework-overview.cfm
Student ratings Student ratings is a necessary source of evidence of teaching
effectiveness for both formative and summative decisions, but not a sufficient source for the latter. Considering all of the polemics over its value, it is still an essential component of any faculty evaluation system.
Using the Massey Online Survey Tool (MOST)
http://www.massey.ac.nz/massey/staffroom/teaching/centre-academic-development-elearning/teaching-evaluation--most/en/teaching-evaluation---most.cfm
Fiona Murray will talk to us about it.
Peer ratings Peer ratings of teaching performance and materials is the most
complementary source of evidence to student ratings. It covers those aspects of teaching that students are not in a position to evaluate. Student and peer ratings, viewed together, furnish a very comprehensive picture of teaching effectiveness for teaching improvement. Peer ratings should not be used for personnel decisions.
Using Massey’s Peer Review Processhttp://www.massey.ac.nz/massey/about-massey/university-management/avc-academic/academic-quality/peer-review-of-teaching/peer-review-of-teaching_home.cfm
Teaching scholarship Teaching scholarship is an important source of evidence to
supplement the three major direct observation sources. It can easily discriminate the “teacher scholar” and very creative faculty from all others for summative decisions.
Classroom Assessment Activities (including fast feedback tools)Classroom assessment techniques drawn from Angelo and Cross (1996).
These tools are Learner Centred, Teacher directed, Mutually beneficial, Formative, Context-specific and Ongoing.
http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-1.htm
http://www.ntlf.com/html/lib/bib/assess.htm
Self Evaluation Self-evaluation is an important source of evidence to consider in
formative and summative decisions. Faculty input on their own teaching completes the triangulation of the three direct observation sources of teaching performance: students, peers, and self.
Videos If faculty are really committed to improving their teaching, a video is
one of the best sources of evidence for formative decisions, interpreted either alone or, preferably, with peer input. If the video is used in confidence for this purpose, faculty should decide whether it should be included in their self evaluation or portfolio as a “work sample” for summative decisions.
Student interviews The quality control circle is an excellent technique to provide
constant student feedback for teaching improvement. The group interview as an independent evaluation can be very informative to supplement student ratings. Exit interviews may be impractical to conduct or redundant with exit ratings, described in the next section.
Exit and Alumni interviews Although exit and alumni ratings are similar to original student
ratings on the same scale, different scale items about the quality of teaching, courses, curriculum admissions, and other topics can provide new information. Alumni ratings should be considered as another important source of evidence on teaching effectiveness.
Employer ratings Employer ratings provides an indirect source of evidence for program
evaluation decisions about teaching effectiveness and attainment of program outcomes, especially for professional schools. Job performance data may be linked to individual teaching performance, but on a very limited basis.
Administrator ratings Administrator ratings is typically based on secondary sources, not
direct observation of teaching or any other areas of performance. This source furnishes a perspective different from all other sources on merit pay and promotion decisions.
Learning outcome measures Learning outcome measures should be employed with extreme caution
as a source of evidence for faculty evaluation. It’s safer to use in conjunction with the direct data sources described previously for program improvement.
Teaching portfolio As a collection of many of the previous sources and them some, the
teaching portfolio should be reserved primarily for summative decisions to present a comprehensive picture of teaching effectiveness to complement the list of research publications.
CAT examples The Minute Paper The Muddiest Point Directed paraphrasing Application cards
The proof of the pudding... If we are teaching well, the obvious outcome would be in
student learning. One of the major challenges we face is that of effectively
‘measuring’ student learning. i.e. We are talking about effective assessment!