1

The Scholarship of Teaching and Learning: implications for its

assessmentMichael Prosser

Centre for the Advancement of University Teaching

The University of Hong Kong

2

1. Introduction

2. The Scholarship of Teaching, and the UK Research Assessment Exercise

3. Example of Pedagogic Research

What is pedagogic research?

How does it relate to the scholarship of teaching?

How does pedagogic research relate to other forms of scholarly inquiry in teaching and learning in higher education?

How is pedagogic research assessed in the UK RAE

3

Growing Interest in various forms of Scholarly Inquiry in Teaching and Learning in Higher Education

• Carnegie Academy for the Scholarship of Teaching and Learning (CASTL) Program in Higher Education - USA

• Carrick Institute for Learning and Teaching in Higher Education – Australia

• Higher Education Academy – UK – The Academy

• Research Assessment Exercise – RAE – UK

Conferences, Research and Development Journals, Graduate Certificates in Higher Education, PhD programs in T&L in Higher education

The UK Professional Standards Framework for teaching and supporting learning in higher education:

‘the ability to incorporate research, scholarship and/or professional practice into those activities’

4

The Higher Education Academy in the UK supports various forms of scholarly inquiry into teaching and learning in higher education, and has developed analytical distinctions to support its aims of supporting and encouraging the various forms.

A concern is that the RAE in the UK will distort the forms of inquiry, by institutions encouraging all forms of inquiry to be consistent with and to meet the definition of research for the RAE.

5

Boyer’s four scholarships

(Scholarship Reconsidered, 1990)

Discovery – more traditional UK RAE type research

Integration – making connections across the disciplines and placing things in larger context

Application (or now often termed engagement) – goes beyond application and develops an interactions that each informs the other (research and application)

Teaching – research informed, critical, evidence based, collegial reflection on practice to improve practice

Separate, but overlapping

6

Forms of educational research and Inquiry

Outcomes include

Scholarship of Teaching and Learning

Critical, evidence based and research informed reflection on practice to improve practice

Investigative studies Recommendations for action on HE issue/problem, locally and nationally

Optionally, enhanced understanding of theoretical or conceptual underpinnings

Evaluative studies Judgement about value or worth, or actions for local enhancement

Recommendations for wider practice Optionally, enhanced understanding of theoretical or

conceptual underpinnings

Theory-driven studies Enhanced understanding of theoretical or conceptual issues for longer term thinking about policy and practice

Optionally, recommendations for action on HE issues

Analytical distinction between forms of educational research and inquiry

7

Theoretical study

Investigative or evaluative

study

Scholarship of teaching

8

Investigative practice-based and policy-based projects

Theory-driven projects

Primary Purpose developing recommendations for action on issues, policies or practices within HE.

enhancing the conceptual or theoretical understanding which underpins the topic under investigation

May also increase the conceptual or theoretical understanding which underpins the recommendations.

lead to recommendations for action on issues, policies or practices within HE.

Starting point may be

an issue, problem or set of concerns relating to HE policy or practice, perhaps articulated within a local context

conceptual or theoretical, perhaps informed by a practical concern within HE, or the project may adopt an approach leading to theory generation.

9

Definition of research for the UK RAE

‘Research’ for the purpose of the RAE is to be understood as original investigation undertaken in order to gain knowledge and understanding.

It includes work of direct relevance to the needs of commerce, industry, and to the public and voluntary sectors; scholarship*; the invention and generation of ideas, images, performances, artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction.

It excludes routine testing and routine analysis of materials, components and processes such as for the maintenance of national standards, as distinct from the development of new analytical techniques.

It also excludes the development of teaching materials that do not embody original research.

10

Definition of pedagogic research for the RAE (generic statement)

‘60. Pedagogic research in HE will be assessed where it meets the definition of research for the RAE. It is research which enhances theoretical and/or conceptual understanding of:

• teaching and learning processes in HE• teacher and learner experiences in HE• the environment or contexts in which teaching and learning in HE take place• teaching and learning outcomes in HE• the relationships between these processes, outcomes and contexts.

Reports of studies providing descriptive and anecdotal accounts of teaching developments and evaluations do not constitute pedagogic research.Pedagogic research is firmly situated in its relevant literature, and high quality pedagogic research makes a substantial contribution to that literature.’

11

UOA 35, Accounting and Finance

16. Teaching materials and non-text output may be included in submissions where they contain original research. Such materials will be assessed in the same way as other work. Where the research aspects of submitted work may not be clear to the sub-panel, departments should describe this briefly (in up to 50 words) in the ‘Other relevant details’ field in RA2.

17. Where this is relevant to the UOA the sub panel welcomes the submission of research into the teaching and learning of the field or sub-field, at HE or other levels. This will be assessed in the same way as other work.

12

This does not imply that that theoretical and conceptual understanding cannot be developed through investigative studies or the scholarship of teaching – but that the outcomes of the study enhances theoretical or conceptual understanding – it is then the outcomes that bare judged.

13

Training of UK RAE Reviewers of Pedagogic Research in the Disciplines

The RAE and the Higher Education Academy organised a training day for members of the discipline panels responsible for pedagogic research in disciplines

The Academy identified what it considered to be high quality published papers, and asked the attendees to rate and discuss the papers in terms of the definition of high quality pedagogic research for the RAE (4*).

4 the Academy judged to have met the criterion for high quality pedagogic research

1 was more a description of the results of a survey

1 was more a description of practice

1 was an unconceptualised case study

1 was the report of an investigation

14

The attendees independently judged the 4 judged by the Academy to be of high quality, to be of high quality

The other 4 were judged to be research, but not of high quality.

The issue here is that the research needs to be well contextualised in its appropriate literature and / or field and be seen to be making a significant contribution to that literature and / or field.

15

  Overview of the student learning perspective

Figure 1: Model of Student Learning

CHARACTERISTICS OF THE STUDENT (e.g. previous experiences, current understanding) STUDENTS'

PERCEPTIONS OF CONTEXT (e.g. good teaching, clear goals)

COURSE AND DEPARTMENTAL LEARNING CONTEXT (e.g. course design, teaching methods, assessment)

STUDENTS' APPROACHES TO LEARNING (how they learn e.g. surface/deep)

STUDENTS' LEARNING OUTCOMES (what they learn quantity/quality)

16

STUDENT APPROACHES TO LEARNING

 

Surface Approach

Intention to reproduce

- rote memorise information needed for assessment

- failure to distinguish principles from examples

- treat tasks as external impositions

- focus on discrete elements without integration

 

Deep Approach

Intention to understand

- meaningfully memorise information for later use

- relate new ideas to previous knowledge

- relate concepts to everyday experiences

- relate evidence to conclusions 

17

Approach to learning represents the relationship between the student and the task or subject

Approaches vary between subjects

Approaches vary between tasks within subjects

Approaches not stable but variable

Surface and deep approaches describe a fundamental variation in the way students approach learning – they do not describe the rich detail in the way students approach their learning

18

STUDENTS’ PERCEPTIONS OF THE LEARNING CONTEXT

Research by Entwistle and Ramsden (1983) using interviews and questionnaires identified a number of student perceptions relating to the way they approached their studies

Student perceptions of:

Quality of teaching – including quality of feedback

Clearness of goals of course and standards of assessment

Workload so high that it was not possible to understand everything

Assessment measuring reproduction and not understanding

were found to relate to how they approach their studies and to learning outcomes (exam results and other indicators)

19

RELATIONSHIP BETWEEN PERCEPTIONS AND APPROACHES

Study of 8837 students in 51 first year subjects around Australia

(3 first year accounting courses)

Amongst the data collected were students responses to:

contextualised Ramsden's Course Experience Questionnaire and a

contextualised Biggs Study Process Questionnaire

1994-1996: Australian Research Council ; Academic Departments and the Quality of Teaching and Learning; Paul Ramsden, Griffith University, Elaine Martin, RMIT, Michael Prosser, La Trobe University, Keith Trigwell, UTS

20

Approaches to Study

Surface Approach

32. Although I generally remember facts and details, I find it difficult to fit them together into an overall picture

35. The best way for me to understand what technical terms mean is to remember the textbook definitions

 

Deep Approach

28. I try to relate ideas in this subject to those in other subjects, wherever possible

34. In trying to understand new ideas, I often try to relate them to real life situations to which they might apply.

 

Biggs Study Process Questionnaire

21

Student Perceptions Of Learning Context

Good Teaching

15. The staff made a real effort to understand difficulties students might be having with their work.

Clear Goals and Standards

1. It was always easy to know the standard of work expected

6. I usually had a clear idea of where I was going and what was expected of me in this subject.

Appropriate Workload

25. The sheer volume of work in this subject meant that it couldn't all be thoroughly comprehended (-).

Appropriate Assessment

8. To do well in this subject, all you really need is a good memory (-).

Student Independence

20. Students had a great deal of choice over how they learned inthis subject.Ramsden’s Course Experience Questionnaire

22

Factor Analysis of Perceptions of T & L Context and Approach to Study – accounting students

_________________________________________________________________

Scale Factors

____________________________

1 2

_________________________________________________________________

Perceptions of Context

Good teaching .83

Clear Goals and Standards .58 (.38)

Appropriate Workload .73

Appropriate Assessment (.36)

Student Independence .80

Approach to Study

Surface Approach -.83

Deep Approach .63 -.37

_________________________________________________________________Principal Components, Varimax Rotation, n=463

23

  

1.   A deep approach is associated with perceptions that the teaching is good and the goals and standards are clear

 

2.   A surface approach is associated with perceptions that the workload is too high and assessment tests reproduction

That is, variation in individual students’ perceptions of the learning context across all subjects is associated with the approaches to study (and learning outcome)

1994-1996: Australian Research Council ; Academic Departments and the Quality of Teaching and Learning; Paul Ramsden, Griffith University, Elaine Martin, RMIT, Michael Prosser, La Trobe University, Keith Trigwell, UTS

24

  Overview of the student learning perspective

Figure 1: Model of Student Learning

CHARACTERISTICS OF THE STUDENT (e.g. previous experiences, current understanding) STUDENTS'

PERCEPTIONS OF CONTEXT (e.g. good teaching, clear goals)

COURSE AND DEPARTMENTAL LEARNING CONTEXT (e.g. course design, teaching methods, assessment)

STUDENTS' APPROACHES TO LEARNING (how they learn e.g. surface/deep)

STUDENTS' LEARNING OUTCOMES (what they learn quantity/quality)

25

Having looked at the way students approach their learning, how do university academics approach their teaching?

How does the approach to teaching relate to approach to learning and how does?

Prosser, Martin, Trigwell, Ramsden, Lueckenhausen (’05)

In a series of phenomenographic interview studies aimed at identifying key aspects of the differences in the way university teachers approach their teaching, the following key differences were identified:

26

A: Teacher focused, teacher activity with an intention of transferring information

B. Teacher focused, student activity with the intention of transferring information to students

C. Teacher focused, student activity with the intention of student acquiring the concepts of the discipline

D. Student focused, student activity with the intention of students developing their own conceptions

E. Student focused, student activity with the intention of students changing their conceptions

27

Intention Strategy

___________________________________________

Teacher focused

____________________________

Student focused

Teacher activity Student activity Student activity

Information Transfer

A B

Concept acquisition

C

Conceptual development

D

Conceptual change

E

28

ATI scales measuring:

Information Transmission / Teacher Focused (8 items)

It is important to present a lot of facts to students so that they know what they have to learn for the subject

Conceptual Change / Student Focused (8 items)

A lot of time in this subject should be used to question students’ ideas

29

Trigwell, Ramsden, Prosser and Martin (‘98) – individual class unit of analysis (n=55 first year courses)

Correlations between Approach to teaching and Approach to learning

Approach to Learning

Biggs (ASI)

(Mean)

Approach to Teaching (ATI

_________________________________

Information transfer Teacher focused

Conceptual change Student focused

Surface .37** -.46 **

Deep -.14 .34 *

30

Information Transfer / teacher focused approach to teaching in first year associated with student adopting more reproductive approaches to learning

Conceptual change / student focused approaches to teaching associated with students focusing more on meaning and understanding

31

Eleven of those courses were first year business studies courses.

The following scatter plots show the relationship between the two approaches for those courses

32

33

34

There is a clear relationship between the way student approach their learning in first year business courses and the way academics approach their teaching.

We know that there is a relationship between the way students approach their learning with the quality and quantity of what they learning

Is there a relationship between the way academics see their teaching and how they understand their subject matter

Substantial amount of ongoing work, including that of Lynne Leveson’s, who has addressed this issue in accounting:

More than the classroom: A study of approaches to teaching in accounting and its relationship to lecturers’ disciplinary conceptions

35

In her study Lynne interviewed 30 accounting academics about the way they see accounting as a discipline and the way they approach their teaching in first year curses.

The study was again a phenomenographic study in which the aim was to identify key differences in the way these accounting academics saw they discipline and approached their teaching

36

A Technical process for recording, measuring and presenting financial information.

B Technical processes providing financial information for decision-making.

C Processes to facilitate enhanced decision-making.

D System support for the management and control of an organisation’s resources

E Accounting as a social phenomenon shaping and maintaining social structures.

F A culturally-determined system of principles and rules which operationalise the moral principles of the right-to know in society.

Accounting Academics Conceptions of Accounting

37

The variation in the way they saw their discipline ranged from a focus in the discipline as substantially technical in orientation to seeing the discipline going beyond the technical and seeing it as being socially and culturally constituted and open to critical inquiry and analysis.

But the interesting outcome of this study was in the relationship between these ways of seeing and experiencing the discipline and that way they approached their teaching.

38

Approaches to Teaching

Conceptions of Accounting

A B C D E F

A 6

B 3 4 2 1 1

C 7 1

D 1 1

E 3

39

A relatively strong association between the way accounting academics see their subject matter and the way they approach their teaching.

Seeing accounting in terms of technical processes only is associated with more teacher focused information transfer approaches to teaching

Seeing the accounting in its broader social and cultural context is associated with more student focused conceptual change approaches to teaching

40

Is it too long a bow to draw to say that if we wish:

• to engage our student in more meaningful approaches to study,

• to enhance their perceptions of the quality of teaching and learning in accounting and

• to improve their learning outcomes,

accounting academics need

to adopt more conceptual change and student focused approaches to teaching, which may entail them seeing the discipline they teach not purely or substantially in technical terms but in its broader social and cultural context.

41

Conclusions

The UK RAE judges pedagogic research in terms of its contribution to enhancing theoretical and conceptual understandings through pure, applied, policy and practice-based research.

Accounts of such research need to go beyond being descriptive and anecdotal, and to show how it is making an important contribution to the literature of the field. A high rating would mean that it is judged to be comparable to the best work in the field worldwide.

Research in teaching and learning in higher education is highlighting the importance of the way academics see and understand their subject matter on the way they teach and, in turn, on the way their students learn.