The external examiner system at Cardiff University

Andy Lloyd, Clive BrownRegistry and Academic Services21st April 2014

Welcome from Prof. Patricia PricePro Vice-Chancellor (Student Experience and Academic Standards)

The external examiner system at Cardiff University

• Presentation and discussion (11.00 – 12.30)– Introduction to the external examiner system;– The information you will receive to support

your role;– The University’s Regulations;– The evidence you should use to make

judgements;– The reports you are requested to complete;– Fees and payments.

• Lunch (12.30 – 2.00)– With an opportunity to meet colleagues from

academic schools;

• Afternoon (2.00 onwards)– As arranged with academic schools.

Points you wish this session to cover

• Understanding regs and process (Exam Boards)

• Expectations of XX• Guidance on referencing• Timescales• Moderation• Nature and balance of the role• Role of the Exam Board

External examining in UK Higher Education

• “The fundamental importance of external examining to maintaining academic standards and assuring and enhancing quality”

• The indicator – “Higher education providers ensure independent and external participation in the management of threshold academic standards.”

• QAA (2011) UK Quality Code for Higher Education, Part B: Assuring and enhancing academic quality, B7: External examining

[www.qaa.ac.uk/Publications/InformationAndGuidance/Documents/B7.pdf]

External examining at Cardiff University

• External examiners shall ensure that the Assessment process is valid, reliable and explicit and provide informative comment and recommendations upon whether or not:

– the University is maintaining the threshold academic standards set for its awards in accordance with the frameworks for higher education qualifications and applicable subject benchmark statements;

– the assessment process measures student achievement rigorously and fairly against the intended outcomes of the Programmes(s) and is conducted in line with Senate Regulations and related policies and guidance;

– the academic standards and the achievements of students are comparable with those in other UK higher education institutions of which the external examiners have experience.

The evidence base on which judgements can be made

External Examiners

Assessment tasks / Exam

papers

Meetings with

students

Samples of student work

Assessment outcomes /

reports

External reference

points (e.g. QAA / PSRBs)

Cardiff Assessment Regulations

Programme Information

The typical calendar of assessment

• September to March – Draft assessments / exam papers sent to external examiners for approval;

• April to June – Samples of marked scripts / assessments given to external examiners for moderation;

• June – Exam Board meetings;• June to July – External examiners to submit annual report

form (within 1 month of exam board meeting);• July to November – Institutional response issued where

required (and if needed, to be informed by the academic school).

Information you will receive

• General information from the University (through the External Examiners handbook)

– Background information about the University;

– Guidance on the role of external examiners in the assessment process;

– Assessment Regulations that cover marking, extenuating circumstances and exam boards;

– Information on taught master's dissertations that will be made available to external examiners;

– The reporting procedures (including guidance on the annual report form);

– Equality and Diversity matters (including the University’s Equality Plan, information about specific provision regulations, and rules on the assessment of students through the medium of Welsh, religious observance days etc.);

– Information on rates of fees and expenses payable to external examiners.

Information you will receive

• Programme specific information from academic schools, including:

– A copy of the school brochure for the Programme(s); – Information about the aims and learning outcomes of the programme, the methods of assessment,

and syllabus details of each module or unit of study; – A copy of the marking scheme and details of school procedures that ensure the reliability and

consistency of marking;– The final report of any retiring external examiner for the programme

(where available at the time of appointment), and; – Details of opportunities to meet with students in the absence of staff

(either in person or by electronic means).

• and in addition to the above, as they become available:

– The date of the examining board(s); – Draft examination papers contributing to the final award for approval; – Arrangements for the approval of supplementary assessments, and;– Any proposed changes to the forms of assessment of any module or unit of

study contributing to the final award for comment.

Cardiff assessment Regulations

• “Where institutional audit reports have noted that institutions are operating multiple sets of assessment and classification arrangements, they have generally tended to recommend measures to reduce the number of such arrangements, with a clear preference that institutions adopt frameworks, or a single framework, within which classification decisions can be taken more consistently and fairly.” (QAA, 2006)

• Comments from external examiners that raised concerns over the different rules being used within different schools;

• A desire to improve the student experience of assessment.

Consistency Transparency Equity

Cardiff assessment Regulations

• Changes are being made through a University-wide project (2010-14) that was designed to:

– Enhance feedback to

students on assessment; and

– Improve consistency in assessment processes and practices.

“the rationale for differing approaches to regulations has become obscure”Burgess Report

Assessment Changes at Cardiff

• A limited number of rule sets to manage taught modular degrees across the University;

• The removal of ‘discretion’ from exam boards;

• Consistent rules to manage extenuating circumstances;

• Consistent rules and practice for:– late submission of coursework

– Penalties for exceeding word limits

– anonymous marking of all assessments (where possible)

– second marking and moderation

• Reduction of the volume of assessment, and better management and co-ordination of assessment within programmes of study, and;

• Development of a common, shared understanding of feedback on assessments between staff and students.

External Examiners are not responsible for …

• Resolving disputes between internal examiners or markers;

• Making recommendations on, or changing the marks of, individual students;

• The assessment of individual students (except in the case of clinical and/or practical-based assessments);

• Judgements related to extenuating circumstances or unfair practice;

• Communication with individual students

External Examiners and programme information

An information source that can be used to help shape judgements on:

• the coherency and currency of the programme and its component parts;

• the extent to which the programme reflects any additional Professional, Statutory and Regulatory Body requirements.

• whether the programme is maintaining the threshold academic standards set for its award in accordance with the frameworks for higher education qualifications and applicable benchmark statements;

• whether the academic standards and the achievements of students are comparable with those in other UK higher education institutions of which the External Examiner has experience.

External Examiners and setting assessments

• External examiners are required to consider, comment on, and approve all assessments that contribute 50% or more to the mark for any module or unit of study that contributes to the final Award. It is good practice for schools to provide a response to external examiners on any suggested amendments to assessments.

• When approving assessments external examiners should seek to ensure that these: – are appropriate to the level of study; – will enable learners to demonstrate achievement of the learning outcomes; – cover the subject content appropriately; and – are accessible and fair, so that all learners have equal opportunity to demonstrate

achievement of the learning outcomes.

• Schools shall consult with external examiners on major changes to the methods of assessment for any module or unit of study contributing to the final award.

External examiners and marking and moderation

• External examiners should be provided with a sample of examination scripts, class test scripts, outcomes of practical assessments and coursework to determine that internal marking and classifications are of an appropriate standard.

• At least 10% of dissertations for a taught Master's programme, or a minimum of 10 (whichever is the higher figure)

• You will be invited to consider and comment upon marking and moderation procedures, to ensure the reliability and consistency of marking and feedback.

• External examiners may ask the board to review the marks of whole cohorts or of individual assessments

External examiners and the confirmation of results

As well as reports on student achievements there will be a report to help you look at data over the past 5 years at programme level.

External examiners and extenuating circumstances

• A sub-committee will consider student requests to consider extenuating circumstances in advance of the board and will determine if a student’s performance has been impacted by their circumstances or otherwise. The sub-committee will present its outcomes to the board;

• Exam boards allow students shall be given the opportunity to re-take failed assessments where they have had extenuating circumstances (as a 1st attempt);

• Where a student has passed modules impacted by extenuating circumstances, the exam board shall remove the marks of these modules from a student’s final mark (up to 1/6 of the contributing credit) where it benefits the student’s final award – ‘Discounting’;

• Exam boards shall not:– receive or discuss the details of individual extenuating circumstances cases,– consider possible extenuating circumstances where these have not been reported through

appropriate channels,– seek to review or overturn decisions made by extenuating circumstances groups.

• To discuss personal circumstances at an exam board would risk compromising confidentiality and lead to judgements being made on the basis of partial and/or incomplete information.

External examiners and meetings with students

• External examiners and students are encouraged to meet (either in person or by electronic means) to discuss the programme of study and its assessment. Such meetings may lead to the identification of possible programme enhancements and provide the external examiner with a rounded overview of the quality and standard of the student experience and enhance the external examiner’s contribution to the examining board. The annual reports of external examiners often usefully include headline issues raised in such meetings.

• Schools are expected to confirm to external examiners, at the start of the academic year, the opportunities available to them to meet with students in the absence of members of staff.

• Such meetings should normally include the opportunity to meet with student representatives in the later stages of the programme. Such students should be well placed to provide informed views of the overall student experience.

The evidence base on which judgements can be made

External Examiners

Cardiff Assessment Regulations

Programme Information

Assessment tasks / Exam

papers

Meetings with

students

Samples of student work

Assessment outcomes /

reports

External reference

points (e.g. QAA / PSRBs)

External Examiners and any serious concerns

• External examiners have a right to raise any matter of serious concern with the Vice-Chancellor, if necessary by means of a separate confidential written report. The University will provide a considered and timely response to any confidential report received, including details of any action that will be taken as a result.

• Where an external examiner has a serious concern relating to systemic failings with the academic standards and has exhausted all published internal procedures, including the submission of a confidential report to the Vice-Chancellor, he/she may invoke QAA’s concerns scheme or inform any relevant professional, statutory or regulatory body.

External examiners and quality and standards

External examiner reports

Student feedback(including NSS outcomes and

comments)

Programme review conducted by the

School

Annual Review and Enhancement

Process

Academic Quality and Standards Committee

Action plans for enhancement

Annual reports on taught programmes

• To be submitted within one month of the Examining Board;

• Consistent and transparent approach regardless of the size or level of the programme.

The ways in which reports are considered and used

• The University reviews the reports to identify:

– Enhancement issues that require a response from the University (often informed by dialogue with the Schools);

– Particularly positive aspects of the reports.

• University Annual Report on Assessment

• Reports and institutional responses are published online

• Please avoid mentioning staff or students by name

2012/13

Matters for further

attention

Positive comments

Validity 1.1 Style of Assessment 33 12

1.2 Word Limits 8 0

1.3 Feedback Opportunities and Enhancement 1 1

1.4 Over-assessment 4 0

1.5 Variety and appropriateness of assessments 17 20

Reliability 2.1 Double Marking/Internal Moderation 32 8

2.2 Use of the Full Mark Range 21 6

2.3 Marks/Information Presented to Examining Boards 16 3

2.4 Analysis of Mark Trends/Comparative Statistics 9 0

2.5 Plagiarism / Citations 7 0

2.6 Handling of Extenuating Circumstances 6 1

2.7 Resolution of Marking Disputes 4 0

2.8 Time available to External Examiners 5 1

2.9 Opportunities for External to meet students 12 3

2.10 Consistency of marking/Mark Schemes 6 4

2.11 Rounding of Marks 1 0

2.12 Anonymity 2 0

2.13 Joint Hons. / Examining Board Discretion to Raise Classification 4 5

2.14 Alignment of Marking Scheme with Learning Outcomes 9 1

2.15 Group-work Assessment 1 2

2.16 Error on Papers 3 0

Explicitness 3.1 Sufficiency / Timing of Explicit Feedback 29 60

3.2 Information / Induction / Audit Trail Needs of External Examiners 28 29

3.3 Sampling Issues 13 0

3.4 Role of External Examiner (including moderation of exam papers) 6 5

Example (1)

Example (2)

Masters’ dissertations

• There is a separate, and briefer, Master’s degree report form that is issued to external examiners along with dissertations for moderation.

• These are managed in the same way by the University.

Fees and expenses

• Fees will be cleared for payment upon receipt of the Annual Report Form

• Claims for expenses are processed upon receipt of the Claim Form

• Payments are made by BACS processing (monthly) – the arrangements are set up at the time of appointment

• Payment runs occur on the last working day of each month

Further information

• Online resourceshttp://learning.cf.ac.uk/quality/review/external-examiners/

• Clive Brown, Registry OfficerBrownC@Cardiff.ac.ukTel: +44 (0)29 2087 5249

• General enquiriesExternalExaminers@Cardiff.ac.ukTel: +44 (0)29 2087 9222