Regional Quality Review meeting

5th July 2012

2

Agenda topics

1. The Regulatory Framework

2. Centre Monitoring

3. The OCNYHR Portal

4. Qualification Update

5. New and Revised Policies

6. CPD

3

1. Regulatory Framework• QAA

– Regulates Access to HE Diplomas

– By licensing Access Validating Agencies (AVAs)

– Require AVAs to regulate Centres

– Regulations stable for last few years

• Ofqual– Regulates other qualifications

– By licensing Awarding Organisations (AOs)

– Require AOs to regulate Centres

– Regulations changed in May 2011

4

OFQUAL’s role

• ensure assessments and qualifications are reliable and consistent over time

• promote public confidence in regulated qualifications and assessments

• secure efficiency and value for money

5

OFQUAL Regulations

• Introduced May 2011

• More stringent than previous regulations

• Criteria for Recognition – apply to Awarding Organisations seeking recognition

• General Conditions of Recognition– apply to recognised Awarding Organisations

6

General Conditions of Recognition

• A1 - A8 How AO is set up and run

• B1 - B8 General Compliance

• C1 - C2 Arrangements with partners eg Centres

• D1 - D7 Qualifications

• E1 - E5 Information about Qualifications

• F1 - F3 Fees

• G1 - G8 Setting up Assessment

• H1 - H6 Conducting Assessment

• I1 - I4 Appeals & Certificates

7

Impact of Conditions on Centres

• Directly – none

• However Conditions affect:– the way AOs regulate Centres

– policies and procedures required

– how qualification and assessment standards are set and maintained

– quality assurance arrangements

8

2. Centre Monitoring

2.1 Risk based approach

2.2 Sanctions

2.3 Monitoring Visits

2.4 Direct Claims Status

2.5 Quality Assurance of Centre Monitoring

9

2.1 Risk-Based Approach

• Each qualification at each Centre will be risk rated, based on:– risk indicted by Ofqual

– risk indicated from EV reports

• Each Centre will be risk rated, based on:– risk rating for the qualifications at the Centre

– outcomes of annual monitoring visits

– Centre’s record of responding to action

10

Banding ScoreColour code General descriptor Priority of actions

required

Low Risk 1 Green

Little or no risk to integrity of assessment, qualifications, Centre

approval criteria, regulatory Conditions and/or reputation of

OCNYHR or NOCN. Centre performance is good.

No action required or action is merely

suggestions towards improvement or best

practice

Marginal Risk

2 YellowAny risk to integrity is marginal.

Centre performance is satisfactory.

Suggested action, provided usually as Recommendation(s)

Moderate Risk

3 Amber

Concerns about a specific risk indicator or indicators. Customized or specific action required to ensure

integrity. Sanctions may be imposed.

Action is required and provided as

Condition(s)

High Risk 4 Red

Major concerns about one or more risk indicators which threaten

integrity. Urgent action required. Sanctions may be imposed.

Immediate and urgent action is required,

provided as Conditions

11

Table of Risk Indicators

1Centre uses valid, reliable, fair and safe assessment methods and tasks. Clear, comprehensive assessment guidance for assessors.

2

Assessment decisions made against unit learning outcomes and assessment criteria at appropriate level, so evidence is valid, authentic, sufficient and current. Clear and constructive feedback to learners explains assessment decisions and helps them improve.

3Robust internal verification, including pre-issue verification, confirming validity and consistency of assessment tasks and decisions, constructive feedback to assessors, appropriate standardisation, accurate records.

4 Centre provides appropriate and timely response to action points.

5Centre compliant with qualification and Centre approval criteria and requirements, quality assurance criteria and regulatory Conditions.

Risk Indicators

12

Allocating Risk Bands

• Initial Risk banding:– Based on qual mix at each Centre;

– Plus previous EV and QR reports;

– Centres informed September 2012.

• From September 2012:– Risk Banding a key purpose of each visit;

– qual Risk updated following external verification;

– Centre Risk updated following compliance visit.

13

2.2 Sanctions

• Five Sanction Levels

• Applicable where Risk Band is 3 or 4

• Not always used, even where Risk is 3 or 4

• Intention is to apply Sanctions where they will help to ensure the required action is implemented

14

Table of Sanctions that may be applied

Level Sanction Rationale

0 None Little or no risk

1Escalation to Centre management, eg, QIP monitored by Centre senior manager.

Limited confidence in specific risk indicators

2

Additional development or quality intervention visit, eg to check action plan completion or EV following learner re-assessment, etc

Non-compliance Not responding to action points

3(a) Suspension of registration(b) Suspension of certification

Threat to learners Loss of integrity of assessment Danger of invalid certification claims

4Withdrawal of Centre approval for specific qualifications

Irretrievable breakdown in QA and management of specific quals

5Withdrawal of Centre approval for all qualifications

Irretrievable breakdown in QA and management of all quals

15

Risk Sanction Action

1 0 Course may continue unchanged, no action required

2 0 Course may continue, there are Recommendations.

3 0 Course may continue, there are Recommendations.

3 1Course may continue subject to completion of Conditions. Monitored by Centre Quality Manager, copy OCNYHR.

3 2Course may continue subject to completion of Conditions. Monitored via a quality intervention visit.

3 3Registration and/or Certification for the quals listed to be suspended until Conditions outlined in QIP have been met.

4 2Course may continue subject to completion of Conditions. Monitored via a quality intervention visit.

4 3Registration and/or Certification for the quals listed to be suspended until Conditions outlined in QIP have been met.

4 4 Withdraw Centre Recognition for the qualification.

4 5 Withdraw Centre Recognition for all qualifications.

16

2.3 Centre Monitoring Visits

• One visit always an overall compliance review.

• If only one, visit will combine this with EV.

• Number of visits will depend on number of qualifications offered, number of learners and Centre Risk band.

• Extra visit if Risk 3 or 4; fee charged.

• Fee charged for extra EV visits.

17

Before the visit

• QRs & EVs to contact Centres in first two weeks of September to arrange visits

• Centres will need to respond quickly - within two weeks please

• QR/EV will ask Centres for pre visit information and will send a Visit Plan

• Please respond promptly – as most of you do

• Failure to respond can affect your Risk rating

18

During the visit

• Where visit involves EV:– random sample scrutinised, as now

– also any records of malpractice

• Where visit is compliance monitoring:– quality systems and management scrutinised, as now

– Centre Malpractice Policy checked

– Check that malpractice procedures are implemented and recorded

– Check of requirements for Direct Claims Status

19

Upholding standards

Standards high in most Centres, however:

• QR/EV must check additional work where initial sample identifies an issue.

• Where assessment decisions incorrect:– QR/EV can reverse assessment decisions;

– QR/EV must ask for work to be re-assessed;

– QR/EV should not sign off achievements.

• If previous decisions could be affected, QR/EV may ask for a review of assessment tasks and/or for all relevant work to be re-assessed.

20

After the visit

• Visit report will show reviewed and/or revised Risk Banding

• Centre to report completion of Conditions – which many of you do, thank you

• Quality team will follow up in any case – but

failure to report may affect your Risk rating.

21

2.4 Direct Claims Status (DCS)

• DCS awarded to Centre for particular qual, if: – Centre has an Accredited Internal Verifier (AIV);

– Centre has delivered the qual for at least a year;

– assessors & IVs satisfactory, Centre Risk Band = 1 or 2.

• Criteria reviewed during compliance visit

• 5% sample of assessed & verified work scrutinised

• Direct Claims Status removed if:– AIV no longer verifies, or Centre stops delivering qual;

– Centre Risk = 3 or 4, or monitoring reveals concerns.

22

2.5 Quality Assurance of Centre Monitoring

• OCNYHR check all reports (as now).

• Formal moderation of visit reports.

• QRs and EVs to be qualified (most already are).

• Centres will be asked to complete an evaluation following each visit.

23

Exercise: Centre Monitoring Visits

In groups:

• Discuss the changes to Centre Monitoring

• Agree three changes that you welcome

• Identify three changes that may be less easy to implement

• Suggest actions that might help

24

5. New and revised policies

New:• Withdrawal

• Notification to Ofqual

Revised:• Malpractice and Maladministration including Plagiarism

• Complaints

• Appeals

• Centre Agreement

25

New: Withdrawal Policy

• If Centres withdraw, learners must be protected.

• OCNYHR and Centres must check qualification end dates to prevent withdrawal.

• Centres to inform OCNYHR as soon as they believe learners may not be able to complete.

• Action plan must be developed, eg, transfer learners to another Centre.

• Ofqual may need to be notified.

26

New: Notification to Ofqual

• Ofqual to be notified if anything happens that might have an Adverse Effect.

• Has affected a number of processes, eg, need to tell us if you will use formal exams or tests.

• Examples of Adverse Effects:– Learners unable to get certificates

– Learners get certificates when they haven’t completed or achieved the standard

– Integrity of qual threatened, eg, by security breach, assessment conditions not met, etc

27

Revised; Malpractice, maladministration including plagiarism (MMP)

• More stringent – required by Ofqual

• Sensible approach to maladmin through error.

• More guidance, including about plagiarism

• Instances to be logged, action plan drawn up.

• Still able to deal with most plagiarism informally

• Training will be available

28

Revised: Complaints and Appeals

• Appeal and Complaints processes clearer

• Appeals based on processes not being followed, eg, Centre Monitoring, EV, etc

• Centre processes to be accessible to learners

• Need formal logs and meticulous records

• Makes clear what to do if not satisfied

• Appeals and complaints must be dealt with by Centres in first instance

29

Revised: Centre Agreement

• Longer and more detailed – required by Ofqual

• New elements – Centres must:– not advertise courses made of Units as though they

were regulated qualifications

– ensure learners can complete (withdrawal policy)

– expect sanctions if Agreement not met

– tell OCNYHR if assessment includes formal tests or exams

30

Further information

• Centre Handbook

• www.certa.org.uk

• www.ofqual.gov.uk

• www.qaa.ac.uk

• www.accesstohe.ac.uk