Assessment without levels

Tim Oates

Group Director

Assessment Research & Development

Criteria relating to assessment – ‘Cambridge Approach’

Reliableconsistent measurement

Validmeasures precisely what it claims to measure

Sound construct basemeasures something consistent with curriculum aims

Consequential validitythe uses to which the assessment is put are technically and ethically sound

Beneficial impactthe full range of effects are beneficial

Utilitycost, resource  

Problems in UK assessment

Underdeveloped formative assessmentAssessment dominating curriculum thinking Relentless transformation into high stakes Creep in functionEscalation of purposes  

In assessment, the concept of ‘construct’ is vital

Multiply three digit numbers Understands and is inventive with metaphorReads a wide range of books for pleasureUnderstand the concept of percentage and calculate pc Part played by evaporation and condensation in the water cycle

Use the concept of inequality to analyse social relationsUnderstands conservation of massMeasures accurately to quantify oxidationVerbal reasoningExternalising behaviour

Diagnoses malignant melanomaLands with one engine on fire and undercart locked in ‘up’ position

 

Assessments – what do we have?

1National curriculum tests and tasks - which most pupils in State schools are expected to take

2Public examinations - which most pupils are expected to take

3Optional tests and progress tests - which schools can elect to enter pupils for

4Tests other than national curriculum tests (available from commercial companies) - which many schools choose to use

5In-course or school-based assessments - which are set throughout a pupil's provision 

Detail

KS11 maths test1 reading test1 spelling test1 writing task

KS23 maths (non-calculator; calculator; mental arithmetic); 3 English (reading, writing, spelling & handwriting); 2 science

KS31 reading & writing1 Shakespeare2 Science3 maths

GCSEAverage number taken = 9.5Mode of 2 papers per subject (some have more, others fewer)

Advanced level AS in four subjects, for many one of the three units is coursework. Mode approx 10 papers.

A2 as for AS, for three subject mode approx 8

Number of tests/ exams

KS1 4

KS2 8

KS3 7

GCSE 19

AS 10

A2 8

Total 56

2002Daily Mail 105

Trends to 2002

•Modularisation in A level

•Increasing elaboration of National Assessments (eg mental maths)

•Refinement in the form of the national tests

•Enforcement of League Table measures (performance tables and targets)

Trends from 2002

•Increase in the battery of national assessments (eg ICT KS3)

•Development of new test forms (Single Level National Tests)

•State-initiated formative assessment (Assessing Pupil Progress)

•Shift from external testing in KS1 to teacher assessment  

 

DetailEarly Years

Reported as a profile of 13 scales, using a score of 0-9, covering cognitive, social, physical and emotional development

Yr 1 (age5) Phonics screening check Statutory test, based on reading 40 words and non-words, taken in one week window in June.

KS1 Yr 2 (age7) statutory teacher assessment – levels for reading, writing, speaking and listening; overall level for mathematics; and a level for each attainment target in Science. P-scales for pupils with special educational needs.

KS2 Yr 6 (age 11) English reading test Level 3-5 (plus Eng level 6 reading test); English grammar, punctuation and spelling test Level 3-5 (plus Eng level 6 test); Mental maths test; Maths test A; Maths test B; (plus level 6 paper 1, paper 2)

KS3 Yr 7, 8, 9, Optional tests Eng, Maths; yr 9 Science; Yr 9 (Age 14) – statutory teacher assessment in core and non-core subjects (13 subjects) – scale 1-8 in 10 subjects.

Modular GCSE Average number of GCSE or equivalent taken = 10.9Mode of 2.5 papers per subject (some have more, others fewer) plus controlled assessment

Advanced level AS in four subjects, for many one of the three units is coursework. Mode approx 10 papers.

A2 as for AS, for three subject mode approx 8

Number of tests/examinations

EYFS 0

KS1 1

KS2 5 (7 for higher ability)

KS3 0

GCSE 27

AS 10

A2 8

Total 51 (56 – 2002)

2012

Trends to 2012

Rise in shift to GCSE ‘equivalent’ qualifications, attributed to schools striving to meet national targets

Crisis in the practical administration of National Tests

Cessation of expansion of National Testing – abandonment of development of KS3 ICT tests

Increase in attention to formative assessment including introduction of Assessment of Pupil Performance (oriented to determining ‘levels’ through formative assessment processes)

Exploration of innovations in marking (eg rank ordering /paired comparison for national tests in English)

Reduction in National Tests (removal of KS3 tests, reduction of KS2 tests from Sci Eng Math to Eng and Math only)

Elaboration of targets and measures including introduction of EBac (English Baccalaureate) as a school performance measure

Trends from 2012

Introduction of phonics screening test

Linear qualifications promoted and modular examinations discouraged or abandoned

Contracting model for GCSE-level qualifications (move to single board per subject)

Abandonment of APP as a national initiative

Greater HE involvement in design and operation of A Levels encouraged

 

2014 Benchmarking – a means of measuring progression (Bew recommendation)Phonics check KS2 tests

Nick Gibb 2010 – ‘..In Primary, apart from KS2 and the phonics check, I have no interest in the assessment which is done…’

Be very careful to read this in the right way – it confirms the professionalism of teachers and deliberate contraction of the role of the State

High autonomy in formative assessment

Constructs are all-important

Progress is:

uneven in pace

not always ‘upwards’ – spiral curriculum and revisiting for consolidation

‘Levels’ could leave serious gaps and misconceptions because of ‘best fit’ – focus on secure learning in key constructs

Expansion may be as important as progression

A child who understands A may not ‘get’ B and vice versa

High quality assessment and learning – rich questionning, high density assessment (externalisation for child and teacher)

Stigler and Stevenson on ability

2010

Levels

3 contrasting, co-existing models

1 the score on a compensation-based test2 best fit 3 threshold

Poor construct integrityContradictions between school and State Poor communication with parents – parental understandingUndue pace – expectations of Ofsted Labelling – contrary to TGAT

The lessons from transnational comparisons

High attainment, high equity and high enjoyment is possible

Don’t assess everything which moves

High density formative assessment is valuable to pupil (externalisation of inner processes) teacher (information on progress, misconception and concept development) and parent (how can I best help my child)

‘Has Alex developed a sufficient understanding of conservation of mass to move to next segment of the learning progression?’ – hence year by year in Primary

Spiral curriculum – application of concepts – Singapore Different models of ability and progression – Stigler and Stevenson

Living in a levels-free world

Wroxham does it Finland, Singapore do itSoft landing as use decays Focus on deep, secure learning of key constructsImplement learning progressions (Schmidt and Prawat)

What we may have from 2014consultation 11 Oct close

Year by year statement of contentEach school publishing its school curriculum and assessment schemeLevels no longer used

Assessment modelBenchmarking statutory tests at KS1 M&E – progress measurePhonics screening check end of Yr1 – with cut score Statutory tests at KS2 M&E (scale score and decile reporting) KS2 reported against prior attainment measure – baseline assessment

Non-modular GCSE conditioned by accountability measures Non-modular A level VQs

Let’s not fool ourselves

We have one of the most diverse systems in the world(institutional forms, local structures, size of schools, school transfer, ideas about education….)

This was true in 2000 and remains trueMany of the structural shifts began years ago (GM, Academies)

My ideal Educationally focussed measurement

Potent and valid formative assessmentDiagnostic assessment – CAT, PIPS, VESPARCH

High density, low weightHigh autonomy in selection and use (assessment schemes)A pull down bank of items

Independent measurement for monitoring national standards

A switch to high equity and high attainment through attainment measures not progress measures

No regression to thresholds (the grade D phenomenon)

Professionalisation of assessment expertise – use of local collaborative mechanisms for development and promotion of good practice – getting the right unit of collaboration