www.cemcentre.org

CEM Conference

www.cemcentre.org

Welcome

Peter Tymms

www.cemcentre.org

Outline• The day• Assessment for excellence

– Why Monitor?– Choices– Layers– Traps

• CEM• E-Assessment• Metrics

www.cemcentre.org

Why Monitor

• From “Assessment for Curriculum for Excellence”:– to support learning; – to give assurance to parents and others; – to provide a summary of what learners have

achieved; – to inform future improvements

www.cemcentre.org

Some key points

• From time to time teachers also take stock of their learners’ progress. – This is vital in ensuring that learners’ progress is on track. – This stocktaking is particularly important at transitions.

• CfE emphasises literacy and numeracy • Greater breadth and depth of learning and a greater focus on skills

development • The practices for arriving at a shared understanding of standards and

expectations involve teachers • Self-evaluation in schools

– It is important that staff in schools reflect on a wide range of sources of information. They need to have easily available valid and reliable performance data to help to identify areas of strength, aspects which prompt further questions, and scope for improvement

www.cemcentre.org

www.cemcentre.org

Choices: Assessment Domains

Physical

Cognitive

Affective

Behavioural

www.cemcentre.org

Layers of Information

Brain modules

SchoolsAuthorities

Classes

Pupils

www.cemcentre.org

Traps• Relying entirely on teacher assessment

– Reliability? Comparability? Bias? Discrimination

• Believing that top down accountability raises standards– Failed south of the border

• Using only aggregate data– Ecological fallacy

www.cemcentre.org

CEM• 1,000,000+ students each year• 1000s schools• Ages 3 to 18+• Scotland, England, Australia, New Zealand, Netherlands, Hong

Kong, China, S. Africa • 100s of international schools• Confidential and accurate• Translations into a dozen languages• Norm and criterion• Paper based and e-assessment• Cognitive, affective, physical and behavioural• With support

www.cemcentre.org

E-Assessment• Two main approaches

– Flat assessment – paper move directly on to computers

– Adaptive assessments – capitalising on advances in psychometrics and IT capacity

www.cemcentre.org

Traditional approach

Low Average High

www.cemcentre.org

Individual approach

Low Average High

www.cemcentre.org

Disadvantages

• Closed questions• Computer access • Atmosphere in computer room• Not suited to high stakes tests.• Gremlins• Takes years of development

www.cemcentre.org

Advantages• Efficient• Economical• Rapid• Good student experience• Detailed feedback• LAN or web• Green • Standards referencing and/or norm referencing

www.cemcentre.org

Figure 1 Cognitive Profile of Children Starting School in Scotland Logits Distribution Map of items Difficult of children 6 + 42-17=? | | Answer ‘What is 21 more than 32?’ | 5 . + Answer ‘What is 8 more than 13?’ ‘ What is a quarter of 8?’ . | . |T 15+21=? . | Answer ‘What is half of six?’ 4 . + 4+11=? . | 9-6=? . | 7+3=? . | Answer ‘What is 3 less than 7?’ 3 . + Answer ‘What is 3 more than 8?’ . | Point to some cosmetics. . | Point to a full stop. .# T|S Read simple sentences, e.g. ‘The cat went for a walk’. 2 .## + Identify several two-digit numbers. .### | Recognise high-frequency words e.g. dog, tree .#### | Point to a capital letter. Point to a yacht. .###### S| 1 .######## + .########## | Point to a microscope. .############ | Identify all letters. Recognise some three-digit numbers. .############ | Point to some cash. Point to a hexagon. 0 .############ M+M Identify approx half of letters and do informally presented addition problems. .########### | Do informally presented subtraction problems. .######### | Repeat words such as ‘Frigglejang’ and recognise one or two letters. .####### | Identify all single digits. Point to a windmill. -1 .###### S+ Understand meaning of math concepts such as ‘most’ and ‘least’. Point to first letter of his/her first name. .#### | Detect some rhyming words. .### | Count to 7 and recall counting 7 objects. .## | -2 .# T+ Identify half of single digits. .# |S . | Point to a circle and a triangle. Count to 4. . | Understand meaning of math concepts such as ‘tallest’ and ‘shortest’. -3 . + Point to some cherries. . | Point to a kite. A knife. Can repeat the word ‘stop’. . | . | Point to a wasp? -4 . + Understands meaning of math concept of ‘smallest’ . | Can point to someone writing and someone reading. |T Can point to a castle. . | Can point to a fork. -5 . + Understands meaning of math concept of ‘smallest’. Point to some carrots?

Easy

What children know and can do

www.cemcentre.org

Which metric?

www.cemcentre.org

Choices: Criterion and Norm Referencing

www.cemcentre.org

Possibilities

• Norm referenced– IQ style– T scores– Z scores– Age equivalents

• Criterion Referenced– A, B C etc– Alpha, beta etc– 1 st, 2:1, 2:2 etc

www.cemcentre.org

Issues

• Criterion referencing generally preferred

• Because – Progress is clear– We can work together– Goals are clear– Can tracking progress

www.cemcentre.org

ButFigure 1.2: Growth Curve for the InCAS Reading Module

7

8

9

10

11

12

7 8 9 10 11 12

Age at Test (years)

Ag

e E

qu

iva

len

t A

ss

es

sm

en

t S

co

re (

ye

ars

)

www.cemcentre.org

Reporting in CfE• Experiences and Outcomes @ levels

– Developing– Consolidating – Secure

• Are these categories discriminating enough?• Will they satisfy parents?• How will modest progress be shown?• How will agreement be ensured across Scotland?

www.cemcentre.org

Possibility

www.cemcentre.org