Embed Size (px)
Citation preview
This article was downloaded by: [University of North Carolina]On: 07 October 2014, At: 08:52Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK
The Law TeacherPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/ralt20
Using student feedback to improve the quality ofteaching law to non‐law studentsAlice Christudason aa Associate Professor, Department of Real Estate , National University of SingaporePublished online: 09 Sep 2010.
To cite this article: Alice Christudason (2006) Using student feedback to improve the quality of teaching law to non‐lawstudents, The Law Teacher, 40:1, 41-58, DOI: 10.1080/03069400.2006.9993196
To link to this article: http://dx.doi.org/10.1080/03069400.2006.9993196
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in thepublications on our platform. However, Taylor & Francis, our agents, and our licensors make no representationsor warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Anyopinions and views expressed in this publication are the opinions and views of the authors, and are not theviews of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should beindependently verified with primary sources of information. Taylor and Francis shall not be liable for any losses,actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoevercaused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
USING STUDENT FEEDBACK TO IMPROVE THEQUALITY OF TEACHING LAW TO
NON-LAW STUDENTS
By ALICE CHRISTUDASON*
Abstract
THE FORMAL semestral student feedback (SF) system at the NationalUniversity of Singapore (NUS) is an online system which students submitanonymously, and upon completion of their Module. In order to reducebias, the exercise is conducted before the semestral examinations. Boththe teacher, as well as the Module are evaluated quantitatively andqualitatively. To reduce bias, staff are provided the evaluation details onlyafter completion of marking of examination scripts and release of results.As in Universities elsewhere, this University has its fair share of scepticsabout the SF system, who view high scores from students as more theresult of popularity, charisma or even "entertainment" value rather than asa true reflection of the teacher's effectiveness. This of course raises therelated question of what is meant by an "effective" teacher is s/he one whocauses higher order learning outcomes? Do students have their own (andoften different) perceptions about how a teacher should be so that theevaluations themselves may be flawed to begin with?
Despite the understandably questionable value of quantitative scores inteaching evaluations, the qualitative formal SF can provide valuable insightson how one's teaching can be improved. This has been found to be ofparticular importance in my context which involves teaching Law to non-Law students. The latter may arrive in my classroom from a singleDepartment such as the one I teach at the Department of Real Estate at theNUS, or from various Departments from across Faculties in NUS includingEngineering, Computing, Business and Science. While the former alreadyposes a unique set of challenges for Law teachers, the latter compound thedifficulties as students' backgrounds are widely heterogeneous and Law isjust one of the Modules they take.
In such circumstances, it is even more critical for a teacher to carefullyexamine students' qualitative feedback. They often provide variousperspectives which a Law teacher may not have even have considered andenable the teacher to improve the quality of learning s/he facilitates. Themost important use of qualitative SF is that they provide useful pointersand act as triggers to activate the teacher into improving his/her course
Dr Christudason is Associate Professor, Department of Real Estate, National University ofSingapore.
41
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
42 ALICE CHRISTUDASON
and teaching. While this may be so, one of the chief disadvantages of theformal semestral online SF system is that improvements can only be madefor the next cohort of students since it is available to a teacher only afterthe Module (and examinations) has been completed. In view of this, it isimperative that the teacher puts in place, more frequent and perhapsinformal SF mechanisms so as to be able to respond more nimbly tostudents' difficulties. This paper examines the formal SF questionnaire andfocuses on some of the ways in which a teacher can obtain SF in a timelymanner so that it can be used for improving instruction. The paper alsooutlines the reasons for some of the changes made (and not made) to aModule as a result of student feedback. Overall, in the final formal SF, thestudents reported enhanced learning outcomes and an enjoyable learningexperience.
Background and introduction
For more than two decades, I had been accustomed to teaching LawModules to non-Law students at the Department of Real Estate, NUS. Suchstudents were not being prepared for a career in Law, and they possessedeither a background in or affiliation to Real Estate. They would graduate topursue careers as valuers, surveyors, architects, and property and projectmanagers. Last year however, I undertook to design and teach a newModule with a colleague. This Module was of a different genre altogether:it was to be taught to students from other disciplines and Faculties whichincluded the Faculties of Arts and Social Sciences, School of Computing,Business School, Design and Environment, Engineering and Sciences.Undertaking to teaching law to non-law students from non-Real Estatebackgrounds posed a different set of challenges which resulted in a newlearning experience for me as a not-so-new teacher.
While this paper reflects on some of the challenges of teaching a widelyheterogeneous student cohort, the focus is on the importance of obtainingtimely SF to improve teaching effectiveness in such a context. Mere relianceon the end-of-semester formal online SF w6uld simply have not yieldedany benefit for the existing cohort: the changes I made (if any) would havecome too late (for them). In this regard, the somewhat limited value of theonline SF system is considered. This deficiency of the semestral online SFsystem revealed the necessity to put in place accessible and practicalmechanisms for students to evaluate more frequently my methods ofinstruction and assessment. This way, I could know how they wereprogressing in their learning. The crucial question was this: I was teachingbut were they learning? Such regular feedback from students enabled me tobe swift and nimble in responding to their concerns about the Moduleduring, not after, the Module. To confirm the effectiveness of the decisionsmade in the course of the module, the paper produces some of thequalitative results of the formal feedback on the Module conducted at the
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
USING STUDENT FEEDBACK 43
end of the Module. This highlights students' final perceptions of theModule and the value of their learning experiences.
The following section provides some background information about theModule's objectives and the heterogeneous mix of students that chose totake the Module. The Module which I co-taught is called "Real EstateDevelopment and Investment Law."
Objectives of the module
This Module introduces students to the legal institutions and frameworkswhich underpin real estate development and investment in Singapore.Through content analysis, students would gain an understanding andappreciation of the circumstances which have obtained in the formulationof the legal principles which underlie the development of and investmentin real estate in Singapore. The Module is targeted at all students acrossFaculties (except Medicine, Law and Dentistry) who have had no exposureto Real Estate Law. By introducing legal concepts and principles throughinteractive lectures and participatory tutorials, the aim was for students togain an appreciation of the various considerations which shape the legalpolicies pertaining to real estate in Singapore. ,
The students
The cohort of students I received was heterogeneous in almost everypossible way: they were from different (i) disciplines (ii) ages (iii) levels intheir Courses (iv) Faculties across the campus (v) nationalities and notunimportantly, (vi) followed different timetables. Such differences,particularly (i)-(v) often translate into different learning abilities,expectations and ultimately, learning outcomes while (vi) posed practicaldifficulties.
What concerned me in particular was that there would be students fromthe "hard" sciences (such as Engineering, Computing and Chemistry).Clearly, such students were accustomed to very different learning modeswhose emphasis is on numbers, formulae, calculations and providingconcrete solutions to problems, rather than the strong application oflanguage skills for persuasive argumentation which this Module required.
In view of the wide differences that my students brought to theclassroom, I had to critically reflect on my usual teaching methods. Whilethese had been suitable and effective for past cohorts of (the relativelyhomogeneous mix of) students within the Department of Real Estate, thiswas a different situation. In order that my students had a more valuable(and enjoyable) learning experience I decided to rely on more informal (andongoing) SE This would serve to provide useful pointers on how I couldcontinually refine my teaching methods; I could then re-assess theireffectiveness for this particular cohort during, not after the Module. In
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
44 ALICE CHRISTUDASON
doing so, I found that I was in effect reacting to the heterogeneitiesoutlined in (i)-(vi) above.
Theories on assessment of teaching performance
Formative evaluation of teaching has been described as "evaluation thattakes place along the way."1 It comprises sources of information (self,students, peers and experts), the method used to collect the information(questionnaires, observations, interviews) and the degree of time, effort andformality involved.2 Such evaluation enables students to know "how theyare doing" and teachers to make mid-course corrections.3 On the otherhand, summative evaluation (also obtained from similar sources namelystudents, colleagues and alumni) yields data that (in addition to other uses)could be used to compare teaching performance and provide a basis fordecisions about staffing contracts, tenure and promotion. This paperconsiders how formative and evaluative information by way of SF usefullyguided me in the conduct of this Module to achieve the desired learningoutcomes outlined in page 3.
Student feedback (SF) and its users
At least five major users of student ratings on campus have been identified4
These are (1) Campus evaluation offices, (2) campus committees, (3)administrators, (4) students and (5) teachers. This paper only discusses thevalue of SF for (4) and (5)
Means and purpose of SF
Reliability
Whether paper-pencil, or online, the overall objective of a SF exercise is "toobtain valid and reliable SF on courses, as well as their teachers."5 In thisregard, various factors may affect the validity arid reliability of SE Theseinclude the situation, content, time, and wording, absence of hypothesis,intent, slippage, and rhetoric in the feedback questionnaire. One otherimportant concern is the anonymity of student responses.6 There is also the
1 Davis, James R. (1993) Better Teaching, More Learning, American Council on Education Series onHigher Education Oryx Press (1993).
2 Smith, R. Formative Evaluation end the Scholarship of Teaching and Learning in Fresh Approachesto the Evaluation of Teaching, Christopher Knapper, Patricia Cranton, (Editors) No. 88, (2001)Jossey Bass Publishers.
3 Davis, op. cit. fn. 1.4 Ory, J. Faculty Thoughts and Concerns about Student Ratings Techniques and Strategies for
Interpreting Student Evaluations, New Directions for Teaching and Learning, Number 87,(2001), Jossey-Bass Publishers p. 3.
5 Johnson, T D. and Sorenson D. L., editors Online Student Ratings of Instruction, New Directionsfor Teaching and Learning, Number 96, (2003) Jossey-Bass Publishers.
6 Tan, W. C. K. Practical Research Methods (2004) Pearson Prentice Hall.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
USING STUDENT FEEDBACK 45
question of whether a standard format can be applicable across allFaculties.
Online SF: benefits
Despite these and other concerns, online SF has a variety of benefits. Thechief benefit is that when compared to paper-pencil feedback, there is lessof a time-lag between the actual providing of the feedback by students andthe teacher's reactionary time. The particular usefulness of end of semesterformal feedback is that at the time of responding, students are in a farbetter position to understand and appreciate the topics covered in the pastsemester. Mid-semester feedback has its limitations, especially where thenature of the course is progressive (as is the case in Real Estate Law) andstudents are unable to see the full picture or relevance of the Module untilthe final topics are completed.
The SF system at NUS
Both the teacher, as well as the Module are evaluated quantitatively andqualitatively. (The questionnaire is considered below). Staff are provided theevaluation results only after completion of marking of examination scriptsand release of results. As is to be expected, this University has its fair shareof sceptics about the SF system who view high scores from students asmore the result of popularity, charisma and even "entertainment" value,rather than a true reflection of the teacher's effectiveness. In the light of thenew learning/instructor-centred paradigm, an "effective" teacher has beendescribed as one who causes effective learning to take place.7 A relatedquestion will be, what are the means a teacher adopts to cause effectivelearning? In this regard, the hypotheses of this paper is that on-goingreflection and action by a teacher in response to SF will lead to enhancedlearning outcomes.
Formal SF
Implementation of the SF exercise: response rates
Many Universities which employ online SF lament students' low responserates.8 At the NUS however, various strategies have been employed toensure optimum participation rates from students in the Feedback exercise:
(a) Initially, students' registration for examinations was dependant onstudents' completion and submission of on-line feedback.
(b) Currently, undergraduate students who participate in the feedbackexercise will be given additional points which may be added to their
7 Ryan, K. E. Evaluating Teaching in Higher Education: A Vision for the Future, New Directions forTeaching and Learning, Number 83, (2000) Jossey-Bass Publishers.
8 Johnson, op. cit. fn. 5.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
46 ALICE CHRISTUDASON
total bid points which they can use to bid for Modules in thefollowing semester.
(c) Graduate students, Dentistry, Law and Medicine undergraduatestudents who do not participate in the feedback exercise will only beable to access their results via the internet five days after the releaseof the examination results.
While the above scheme has indeed been successful in getting almost100% rates, the critical question of course, what is the actual formativevalue of the feedback—can it act as a trigger to activate the teacher intomaking changes to improve the Module? One difficult with qualitativefeedback is that comments are sometimes ambiguous, biased or maygenerate confusion. Svinicki9 cites two reasons for the frequent lack ofstudents' useful feedback: (a) students' beliefs about feedback and (b) theirlack of understanding and practice in giving it. Yet typically, faculty prefersuch comments over numerical scores (Ory and Braskamp, 1981).10
As for (a) above, one of the biggest drawbacks of an end of semesterfeedback scheme is that the feedback can only be used by a teacher toimprove his/her instruction for subsequent cohorts. In other words, thecohort providing the feedback will not be able to experience the benefit ofthe responses (if any) to their feedback. In the light of this, although theremay be a high, or even hundred percent response rate, it is to be expectedthat there is a certain bias in their responses as they may not feel that theirfeedback would benefit them directly. In other words, the perceived"futility" of their feedback (that is, for themselves) could be a disincentiveto being generous with the truth. A variation to this (so-called disincentiveto being truthful) is where students are not inclined to give extensive [oruseful] feedback because they believe it will have no effect on the ultimatetarget of teaching. Thus for example, students may have witnessed enoughexamples of continuing poor teaching in the face of [their] studentevaluation of teaching to make them sceptical about whether anyoneactually uses the feedback.11 General literature on motivation theory statesthat an individual who believes that his efforts will not result in anychange in the situation is less motivated to make any effort. Taken to itsextreme, this phenomenon has been referred to as "learned helplessness."12
As for (b) above, in order for the SF to be "useful" and actionable for theteacher, students must first be primed to provide it. (see discussion below).
9 Svinicki, M. D. Encouraging Your Students to Give Feedback New Directions for Teaching andLearning, Number 87, (2001) Jossey-Bass Publishers at p. 17.
10 Ory and Braskamp, 1981, as cited in Svinicki, M. D. ibid.11 Svinicki, op. cit. fn. 9.12 Peterson et al. (1993) as cited in Svinicki M. D. op. cit. fn. 9.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
USING STUDENT FEEDBACK 47
The questionnaire used at NUS
Quantitative
Table 1. NUS Consolidated teacher performance report: Quantitative
Qn Items Evaluated Fac. MemberAvg Score*
1 The teacher has enhanced my thinking ability.2 The teacher provides timely and useful feedback.3 The teacher is approachable for consultation.4 The teacher has increased my interest in the module.5 Overall the teacher is effective.
* A 5-point scale is used for the scores. The higher the score, the better the rating.
Qualitative feedback about the Teacher
• "What are the strengths of the teacher?"
The information here can reinforce his/her own perceptions of whatmay be working well in the course and the teacher can continue usingthem.
• "What improvements would you suggest?"
This is the section that has the most potential to provide valuableinformation on what a teacher may have been perceived as not doing right.However, this is also the section that shows up most clearly the mismatchbetween a teacher's and students' expectations of (1) what and how theteacher teaches and (2) what and how the students learn.
Thus for example, comments by students that (i) "the teacher should bemore direct in his explanations" or (ii) "the teacher talks about irrelevantmatters" or (iii) "the projects are too demanding and should be accordedmore marks" need to be analysed further. As for (i), the teacher maydeliberately be adopting an oblique approach in his instruction in order toprompt students to read between the lines, or to engage in critical thinkingand inquiry. Such an approach is the antithesis of spoon feeding.Alternatively, the trigger to such a comment may be that students have notdone the required reading before coming to class.
As for (ii), the problem may be that students do not actually see theconnection between what the teacher is elaborating about and the points/he is making. This often arises where the students have long forgotten
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
48 ALICE CHRISTUDASON
what has been studied in earlier foundation Modules and the presentModule/teacher is building upon such knowledge. Another reason for theproblem may be that the connections across Modules are not made explicit.
With regard to (iii), the teacher may wish to re-evaluate the nature of thetasks given to the marks awarded for the tasks. If students feel that only asmall mark is awarded for what they see as a disproportionately difficultproject, motivation (and hence the quality of learning) can be adverselyaffected.
In addition, there may be comments in this section which deal withmatters which are outside the teacher's control. Thus for example, it is notuncommon to see students' opinions that "there should be more tutorials inthis Module" "smaller class sizes" or "more contact time" "this Moduleshould carry more credits." Students often do not realise that such mattersare simply not within the control of the individual or faculty. Rather, suchissues are generally in the hands of a curriculum review committee eitherat Departmental or University level. Nevertheless, the value of suchfeedback may be that they it can be passed on to the relevant committee.However again, the drawback is that the students providing the feedbackwill not actually receive the benefits of their suggestions.
Despite the ambiguities or uncertainties over the reasons why thestudents have made the comments they did, it may nonetheless beworthwhile for the teacher to reflect on the comments. The teacher will beable to see things more clearly from the students' perspective and rectifyany misconceptions future cohorts of students may have about the teacherand the Course.
Qualitative feedback about the Module
"Please comment on the strengths and weaknesses of the module,and suggest possible improvements."
Most of the negative comments pertained to the large volume of readingrequired, asked that less legal jargon be used, and that the specifiedreadings be provided for them.
Informal SF
Providing avenues for ongoing feedback
Other than the formal semestral online system of SI> it is pretty much up tothe teacher whether and how s/he wishes to obtain feedback. There areunlimited opportunities for teachers to obtain and use on-going feedback;this may be oral or paper-pencil feedback and can not only be easilyimplemented, but also allows a teacher a quick response/reaction period.
The initiative to obtain feedback and seeing the importance of such an
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
USING STUDENT FEEDBACK 49
exercise lies with the teacher. This has to be a considered decision as notonly is such an exercise time consuming, but students who give ongoingfeedback would also want to see that their feedback actually makes adifference. This is where the thin line between a considered response andpandering to the students may be crossed.
In addition to putting in place the avenues for ongoing S¥, the followingwere the issues I had to bear in mind when trying to elicit responses fromstudents:
Provide guidelines or structure for feedback.13
• Feedback should be specific: thus for example, where a student emailedto me that I could "improve my pace", I had to seek clarification asto whether I should increase, or decrease my pace. In my mind, Ithought that I was proceeding too fast—I had previous receivedfeedback from students that this was the case—but I was surprisedto learn from this respondent that she actually thought I was goingtoo slowly! This was an eye-opener to me and reminded me thatstudents bring to the classroom a variety of prior learningexperiences and this particular student may have been exposed to aLaw Module before, or be an unusually conscientious student whomay have actually read up before coming for a lesson.
• Feedback should concentrate on observable behaviour rather than inferringwhat the individual is thinking or feeling: Where a student providesfeedback that "the lecturer seems to be too busy for us—it is difficultto get an appointment with the lecturer" it would be helpful if thestudent gives more detailed information regarding the same. Areader will not be able to know whether the student failed onseveral occasions to get to meet the lecturer or perhaps it was justone attempt by the student that left him/her disgruntled.
• Feedback should describe the effect of the teacher's method of instruction onthe student: So for example take this student's comment: "I can'tfollow the explanation-legal concepts are too difficult". While thecomment does point to the problem—the difficulty of grasping legalconcepts, it is too wide and does not assist the teacher to understandwhat in particular the student found difficult.
• Feedback should offer alternatives to the behaviour being criticised: thefollowing comment is more revealing: "I can't follow the explanationbecause of the legal jargon you use." It is helpful in that the studentcould point to use of legal jargon as his problem. In such a case, theteacher can try to use more layman terms to first explain legal
13 Svinicki, ibid.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
50 ALICE CHRISTUDASON
concepts and then only deal with the legal jargon, and then again,• only where necessary. It is certainly not necessary to use Latin forlegal concepts in the way that Law students are exposed to them.
• Provide a "safe" environment—Nothing (well, almost nothing) you saywill be taken against you: Putting any number of feedbackmechanisms in place would be of no use if students did not feel"safe" to voice their concerns about the Module to me. In otherwords, it is important to gain students' trust and assure them thatthe feedback they provide would be taken with a view to improvingthe course as a whole. To this end, I made it clear to students fromthe outset that they should feel free to approach me with genuineconcerns. But these were merely words—I had to take the next stepand actually demonstrate by my actions, demeanour, and even bodylanguage and tone of voice that I was someone that students couldfeel free to speak and make suggestions to, without fear of beingpenalised.
Oral feedback
Semi formal—It is important that feedback be obtained early in the course,even after the first class. For this purpose, I asked for a Classrepresentative. Through him, I could know the first impressions I hadcreated, whether the pacing was right, and whether the level the lecturewas pitched at was understandable to the class as a whole. Thus forexample, I was told that students perceived me as "fierce" and "stern" andto some extent, this even made them feel intimidated and tense. I wassurprised to learn this and decided to tweak my body language to create amore relaxed learning environment.
I also ensured that I would speak to students at each tutorial class whereI met with different and smaller groups. For this purpose, I asked for aGroup Representative. Overall, the Group representatives were to liaisewith the Class representative, who would in turn liaise with me.
I also used a combination of individual and group feedback simply byspeaking to students randomly and to the group representatives on aregular basis;
An important aspect of this simple representative mechanism was thatthe feedback obtained would be considered feedback. In other words, inaddition to spontaneous student responses, feedback which was discussedamong students and then conveyed to the teacher could also beenlightening in that they had actually thought it through.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
USING STUDENT FEEDBACK 51
Written feedback
• Semi-Formal
- Short Questionnaire/survey during tutorial sessions—a few minuteswere set aside during tutorial time for students to address shortquestions on the value of what they were learning, and anydifficulties they were facing.
- Email—this was a channel which students were invited to make useof at any time.
- Integrated Virtual Learning Environment (IVLE)—students were invitedto make use of the Discussion Forum at any time. There was also anavenue for anonymous feedback.
• Informal Written Feedback
For both the above, students can also be asked to provide writtenfeedback. I was not averse to the idea of taking these in the form of notesslipped under my door; I still considered them to see whether there wereany nuggets of truth in the comments;
Using the feedback—Choosing to react
The following section outlines some of the changes that were madespecifically in response to feedback received during the Module.
Type of tutorial questions
Arising from my existing experience of teaching students within the RealEstate discipline, initially (before the Module had commenced) I hadprepared fairly complex case-studies for analysis during tutorials. However,after the first tutorial, feedback indicated that they were problems. Thegeneral sentiments are reproduced in italics below:
I'm from the [Faculty of] Arts and Social Sciences. I find this module veryinteresting and useful as it has given me more insight into everything aboutand related to land. So far the only difficulty is pertaining to the tutorialquestions. I don't really understand some of the questions and I find that it'squite hard to sieve through large amounts of materials when trying topinpoint the relevant information.
This alerted me to students' immediate difficulty of handling complexlegal issues. As such, tutorial questions were re-formulated to provide somehand-holding in the form of structured questions which led up to the casestudies.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
52 ALICE CHRISTUDASON
Model answers/answering techniques
• . . . as students coming from Arts, Engineering, Computing etc., thetutorials need to train us to answer questions legally. The tutorialspreviously only achieved in reinforcing the concepts we learnt in lectures,but did not effectively equip us with the techniques to answer legalquestions.
Again, a difficulty similar to (1) above was brought to our attention. Wfepromptly decided to provide students with a bare-boned structure on theapproach to legal analysis and our focus on the rationale behind the legalpositions.
Collaborative/peer learning opportunities
My original plan had been for students to operate in Affinity Groups. Thismeant that they would have engaged in collaborative or peer learningprior to tutorial groups on a case-study or questions which I had set priorto the tutorial slot. However it transpired that this was highly impracticaldue to the students' widely differing time tables across Faculties.Accordingly I adopted another collaborative/peer learning strategy: thebuzz group. Through this means, after self-learning prior to tutorials,students would engage in collaborative/peer learning during tutorialdiscussions.
Participation and Contribution during tutorial time
My original intention had been to assess Affinity Groups' performanceduring tutorials; however with the switch to the buzz group strategy thefollowing alternative assessment procedures helped:
- Informal appointment of sub-group leaders and scribes for each BuzzGroup;
- Providing discussion time and playing "roving" tutor among the"buzzing" groups;
- Pitting one Buzz Group against another by allocating roles of plaintiffand defendant—this increased the "fun" element and the competitioninjected a sense of responsibility and loyalty to the group;
- Setting pop quizzes—this could be an assessment method forindividual, and group learning.
Format for mid-term test
Arising from feedback that students were having difficulties in graspingfundamental concepts, I decided to temporarily move away from thestructured type of questions I had originally intended to set and beganexperimenting with the use of Multiple Choice Questions. This forced me to
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
USING STUDENT FEEDBACK 53
rethink my questioning techniques and was in itself a learning exercise forme. A test comprising 20 MCQs was set in the mid-semester rather than atest comprising an essay question, which had been my original intention.The results of this test were really an eye-opener. It revealed that even aslate as half-way through the Module, there were students who could noteven understand the implications of say, joint tenancy as a method of coownership, or leasehold (as opposed to freehold) as a method oflandholding. I was therefore able to address these matters.
Final Assessment method
The type of format typically adopted for assessment in law modules isthe case-study/essay question. However, for this Module, arising fromthe nature of difficulties that students voiced they were facing, wedecided on an examination format of allowing a choice of shortstructured hypothetical questions followed by compulsory case-studyquestions.
Providing more frequent breaks during lectures
The Module comprises 2 contact hours per week. Effectively, the actualcontact time for the lecture slot is approximately one hour and forty-fiveminutes as students must be given time to move on to their subsequentclasses at various other venues across campus.
During this one hour and forty-five minutes, the teacher has to makedecisions on some tough questions:
• How much elaboration should there be of bulleted points in aPowerPoint presentation? Should some concepts be just introducedand left for further discussion in small groups during tutorials?V\fould this be adequate priming for further development of theconcept in the course of the lecture?
• What if the students have not done their background reading (orforgotten what was done in a previous foundation Module) andcannot make much sense of what is presently being lectured? Shouldthe teacher press on and complete what was originally intended andspelt out in the lecture sequence which was given to the students atthe beginning of the Course?
• What if unexpected questions crop up during the lecture period—this is very likely where the teacher adopts an interactive teachingstyle and it is important for the teacher to be mindful of not losingthe attention of the rest of the class.
• What if the students are especially inattentive or the concepts beingdealt with are more complex than usual so that a straight walkthrough the lecture is not possible? (This often happens where the
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
54 ALICE CHRISTUDASON
students are going to be having a test or doing a presentation at• another class after your lecture.)
Research shows that students' attention span lasts all of fifteenminutes.14 Despite all the above which serve to cut down even further theactual contact time you have with your students, it is imperative that abreak (or even some breaks) be given during the lecture time so as tominimise this problem of attention deficit.
Closer liaison between tutors conducting tutorials
The Module I co-taught comprised 140 students. The students were dividedup into six groups for tutorial purposes. The tutorials groups were taughtby the other lecturer as well as me. While my colleague and I did meetfrom time to time to discuss tutorial answers and the approach we shouldtake, we learnt that students from the different groups perceived ourapproaches "irreconcilably" different. This made us aware that there shouldbe a greater degree of consistency (though not uniformity) in ourapproaches so that students would know the basic concepts and theirapplication which we intended for them as their learning outcomes for agiven topic. We therefore made the effort to discuss the ground coveredprior to each Tutorial session.
Using the feedback—Making the choice not to react
One can make the mistake of being too responsive and reactive to SE Thisleads to unpredictability, may foster the perception that students can"bargain" with you and worst of all, make you seem indecisive andunprofessional. This is where discernment is necessary before adjustmentsare made to initially stated objectives. If you eventually make the choice tonot react to the feedback, you must still respond to students, and explainyour reasons. These were some of the SF that I did not react to:
• Providing extracts of relevant pages Students asked that I make copiesof readings available to them. However I did not accede to thisrequest as it came within my definition of "spoon feeding" and Iwas averse to that. I reiterated to students the importance of self-directed and independent learning skills. I also emphasised tostudents that one of the abilities that needed to be cultivated in thisera of information overload is the ability to conduct research and
14 Wankat, P. and Oreovicz, F. (2003). "Teaching: Breaking the 15-minute Barrier". ASEE Prism Vol. 12No. 8. http://www.prism-magazine.org/aprilO3/teaching.cfm. See also, Christudason, A(2005), "Your 15 minutes of 'Fame'-Maximising the Effectiveness of Lectures", Centre forDevelopment of Teaching and Learning, Ideas on Teaching, 3: 26-27.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
USING STUDENT FEEDBACK 55
actually locate relevant material. I stuck to providing just the titles of• recommended references and sometimes, Chapter Numbers.
• Marks for IVLE participation I received feedback from several studentssuggesting that I removed the mark allocation of 10% for students'participation on the IVLE Discussion Forum. They cited reasons suchas "we find it difficult to find something original to say"; "most ofthe postings on the IVLE are cut and paste from the Internet". Irealised that students were unhappy about participating on the IVLEbecause it involved a little more effort to come up with someoriginal contribution and students were just not prepared to put thateffort in. I turned down the request and reiterated the importance ofthe exercise. This was to encourage in students the ability to look forinformation on their own, provide an opinion, and be able to defendtheir opinions.
• Can you provide us "model" answers? There were numerous requestsfor "model" answers to tutorial questions. However, despitestudents' unfamiliarity with Law as a module, I firmly refused toprovide "model" answers as I did not want to foster the idea ofuniformity and rigidity in analysis and writing of creative andpersuasive argument. However, I did provide a framework or orderof sequencing that students could use in their approach to writinganswers.
• Webcasting all lectures I received many requests from students to webcast all my lectures. Despite the oft-advocated advantages of webcasting (such as availability of lectures-on-demand, convenience,distance-learning etc.) I must admit to not being a believer—I stillprefer "real" time contact. Nevertheless, upon receiving requestsfrom students in this and particularly bearing in mind that thesewere students from other Faculties across campus, I relented andweb cast several lectures.
Feedback from alumni
This is an avenue the importance of which must not be discounted. Therewere various opportunities to obtain feedback from former students. One ofthese was the liaison that was possible as a result of my involvement asChairperson of the Practical Training or Internship Scheme wherebystudents from my department are attached at various positions in industryas a form of "apprenticeship" Arising from this, there was contact with theformer students, who were now captains of industry and who couldprovide valuable feedback on what the specific and current concerns inindustry were.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
56 ALICE CHRISTUDASON
Is there a down-side to all this?
Yes.First, setting up the mechanism for informal feedback, getting the
feedback, studying the feedback and deciding on how to respond or reactto it, take time. In these days of instantaneous communication, whenstudents email feedback, they may expect almost instant responses to then-suggestions. So this creates a certain pressure on the teacher.
The second issue relates to the teacher's attitude. It is important to keepan open attitude towards the feedback as there will be negative perceptionsabout some of your teaching approaches. These must be taken in yourstride and in a positive light; there is no room for defensiveness for ateacher who genuinely wants to be receptive to feedback. Nevertheless, abalance must still be maintained between structures which the teacher hasestablished and flexibility to make adjustments to the structures. This iswhere discernment is necessary.
Reflections
Reflecting on the experience of having taught this module and how I had tobe nimble and decisive in my responses to the frequent and informal SF, Icame to the realisation that although the routes I had taken (in teaching themodule) did change along the way (from those I had originally set beforethe start of the Module), the destination (my learning objectives) anddesired learning outcomes for my students had not. My online semestral SFwas overwhelmingly positive and here are a few extracts which verified myrealisation:
I got to learn so so much [sic] in the area of real estate andinvestments law!! Very excited to learn more and more each week.
[This is] one of the most pragmatic modules I've ever taken. Itconcerns issues in reality and especially knowing how to manageyour future real estate assets. It summarises the legal issuespertaining to tax and real estate superbly well in one module.
Concepts are clearly explained, everything is explained just right fornon law student to follow at the right pace. Its an interesting modulethat I never dread going for lectures [sic], in fact I look forward tolearning more about it. Tutorials are full with discussions and casescenarios that set us thinking interesting concepts . . .
If you don't ask you won't get
"You" above includes the teacher, and the student.If I did not ask for feedback and provide a conducive setting for it, it is
unlikely that I would have received it so promptly or insightfully. I would
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
USING STUDENT FEEDBACK 57
then have not been able to respond in a timely manner so as to enhancemy students' learning experience.
Conversely, if the students had not asked for modifications orclarifications about the Module, the chances are that I would not haverealised that they were facing a problem in the first place.
Summary and conclusion
To be a master teacher, one needs to be engaged in continual on-goingreflection and critical thinking about various aspects of the Module. Theseinclude course content, modes of delivery and assessment. This paper hashighlighted some of the problems with the formal end-of-semester onlineSF system at NUS. To begin with, there is the questionable value of thequantitative scores in teaching evaluations. There is justifiable scepticism(and cynicism) about the reliability and validity of such numbers: it hasbeen suggested that quantitative data collection methods may not be anaccurate or certain measurement of what the [collector of the data] actuallywants. Thus for example, where students are indirectly coerced to providequantitative feedback (as described on page 6), the results may not be anaccurate reflection of what the students really think as the students may bemore interested in quickly getting the evaluation out of the way andassigning numbers indiscriminately. This undermines the internal validityand reliability of the system.15
As for qualitative feedback, their validity and reliability can also bequestioned from various angles. First, such feedback often lacks detail orexplanation and can lead to difficulties in interpretation; this is especially sowhere there are contradictory comments. Second, where students know thatit is not possible for them to benefit from their feedback there may be adisincentive to be truthful.16 Third, students may not provide usefulfeedback because they are accustomed to it not being acted upon. The latterhas been termed "learned helplessness."17 In addition, there are variousother factors that can affect the validity and reliability of qualitativefeedback. These include the situation (or circumstances) under which thefeedback was given and the wording and intent of the questionnaire.Nevertheless, the discussion has shown that qualitative SF can providevaluable insights on how one's teaching can be improved. However, themain problem with the formal SF system is that such feedback comes in alittle too late for adjustments to be made during the course. Onlysubsequent cohorts will benefit from any adjustments made as a result ofchanges made in response to the feedback.
This underscores the importance of putting in place, informal and
15 Bryman, A. Quality and Quantity in Social Research, (1993) Routledge, London.16 Tan, op. c i t . fn. 6.17 Svinicki, op. cit. fn. 9.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
58 ALICE CHRISTUDASON
accessible mechanisms for students to provide timely feedback during thecourse. In such a context there are various opportunities for the teacher toclarify and thus obviate some of the problems discussed above. This can inturn, translate into actionable feedback and produce results for students tothemselves experience.
By providing pointers to students about how to provide the feedback,students learn to think through the suggestions they make, make practicalsuggestions for improvement, and take responsibility for their suggestions.When you obtain the feedback, it is critical that you adopt an open andresponsive attitude to it. Where their suggestions cannot be acceded to,students must be told why this is the case, and be persuaded as to yourreasons. This will assist students to see the bigger picture in the light ofyour intended learning outcomes for them.
The difficulty of course, lies in deciding when and how you shouldrespond to the SE To respond is not to pander. Admittedly, the line betweenthe two may be thin, and one would have to tread very carefully. In thisregard, the importance of discernment cannot be overemphasised. If unsure,it may be a good idea to check out senior colleagues' opinion.
A responsive approach is beneficial in that it not only signals to studentsthat you are involving them in the teaching/learning process, but it givesstudents a sense of connectedness with the Module. Based on a study ofstudent/teacher interactions and motivations underlying student behaviour,it has been found that when courses are designed so that teachers andstudents share responsibility and work together to achieve common goals,there are very positive effects on educational outcomes; not only is learningsuperior, but harmonization of teaching/learning motives also improves theemotional quality of the educational experience for both professors andstudents.18
Besides, there is always a better way of doing things. There is a valuablelesson here and I cannot articulate better than Marilla Svinicki:19
None of us is so good that we cannot be better.
18 Teeples, R. and Wichman, H. The Critical Match between Motivation to Learn and Motivation toTeach CDTLink Jan 1998 Vol. 2 No. 1.
19 Svinicki, op. cit. fn. 9.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
orth
Car
olin
a] a
t 08:
52 0
7 O
ctob
er 2
014
