ASSESSMENT IN EDUCATION ASSESSMENT IN EDUCATION. Reliability Test-re-test, equivalent forms, internal consistency. Test-re-test, equivalent forms,

ASSESSMENT IN EDUCATIONASSESSMENT IN EDUCATION

ReliabilityReliability

Test-re-test, equivalentforms, internal consistency.

Test-re-test, equivalentforms, internal consistency.

Consistency across markers, occasions, test items, test types, marking conventions, grading procedures, contexts.

Consistency across markers, occasions, test items, test types, marking conventions, grading procedures, contexts.

ConsistencyConsistencyInter-raterInter-raterreliabilityreliability

DependabilityDependabilityDegree of confidenceDegree of confidence

TransparencyTransparency

Threats to reliabilityfrom markers

Threats to reliabilityfrom markers Errors in Errors in

markingmarking

Inter-raterInter-raterreliabilityreliability

Inconsistency inInconsistency inmarkersmarkers

Variation atVariation atgrade boundariesgrade boundaries

Halo effectHalo effect

Copyright Keith Morrison, 2004

THREATS TO RELIABILITYFROM STUDENTS AND TEACHERS• Motivation and interest in the task• Relationship between assessor and assessee• Test conditions/environment• Hawthorne effect• Distractions• Teacher expectations• Time of day/week• Students are not always clear on what they

think is being asked


THREATS TO RELIABILITYFROM STUDENTS AND TEACHERS

• Students may see two similar questions as dissimilar

• Teachers teach to the test• Teachers and students practise test-like materials• Students can perform the skill in the test but not in

real life, and vice versa• Cultural and ethnic background affect

meaningfulness of the test• Unreliability of marking practices


THREATS TO RELIABILITYFROM ASSESSMENT ITEMS

• Task may be multi-dimensional• Validity of items• Language of the assessment• Readability levels• Size and complexity of numbers• Number and type of operations• Form and presentation of questions• Early error in a sequence may block later

stages of the sequence• Questions using mechanical toys may favour

boys over girls


THREATS TO RELIABILITYFROM ASSESSMENT ITEMS

• Questions requiring use of dolls or kitchen work may favour girls over boys

• Essays favour boys if they concern impersonal topics and girls if they favour personal topics

• Boys perform better than girls on multiple choice items

• Girls perform better than boys in written work• Continuous assessment favours girls• Questions are culture-bound• The length of the test affects performance


CONSTRUCT Predictive

validityContentvalidity

Face validity

Consequentialvalidity

VALIDITYVALIDITY


KEY FEATURES OF ASSESSMENT (1)• Assessment to empower pupils as learners.• Classroom assessment impacts significantly on

the pupils’ sense of self, expectations, motivation and confidence.

• Assessment should provide guidance to both teachers and pupils about what needs to be learnt next.

• Assessment should embody an approach to teaching and learning in which the development of long-term dispositions is more important than short-term performance.


KEY FEATURES OF ASSESSMENT (2)• The purposes are to be diagnostic and formative,

providing feedback and being educative.• Teaching should be adjusted in light of assessment

evidence.• Assessment should promote, not damage, student

motivation and self-esteem.• Assessment should be constructively critical and

provide rich, positive feedback and feedforward.• The assessments should be criterion-referenced

and the criteria should be public.• The assessments should lead to diagnostic

teaching.• Assessment should promote student self-evaluation.


KEY FEATURES OF ASSESSMENT (3)• The assessments should be built on evidence

rather than on intuition.• Assessment data should be derived from

everyday classroom activities.• Assessment opportunities should be sought in

everyday classroom activities.• Semi-structured approaches to gathering data

are recommended, generating words rather than numbers (measures).

• Assessments should be linked to the student teacher’s and the student’s action planning and target setting.


KEY FEATURES OF ASSESSMENT (4)• Involve the students in the assessment process.• Communicate the assessment criteria to

students.• Demonstrate validity and reliability.• Demonstrate fitness for purpose in deciding the

method(s) of gathering assessment data and setting assessment tasks.

• Select assessment methods that accord strongly with everyday teaching and learning processes.

Documents

ASSESSMENT IN EDUCATION ASSESSMENT IN EDUCATION. Reliability Test-re-test, equivalent forms, internal consistency. Test-re-test, equivalent forms,