Computer-Based AssessmentsLiteracy / Numeracy

Autumn 2012

Principals’ Information Seminars3-21 May 2012

Overview of the Seminar

9.30 1.30 Setting the context for the CBAs

9.45 1.45 NILA / NINA Assessment Overview

10.45 2.45 Tea/Coffee Break

11.15 3.15 Assessment Outcomes / Reports

11.30 3.30 Preparation for the Autumn Term

12.00 4.00 Feedback from Trials & Developments

12.15 4.15 Guidance & Support

12.30 4.30 Evaluation & Close

2

Learning Intentions

At the end of this session principals will be aware of the:

•NINA, the new computer-based numeracy assessment for introduction in 2012/13;•NILA, the new computer-based literacy assessment for introduction in 2012/13;•Feedback and developments from the Trials;•Quality Assurance in May;•Timescales involved; and•Guidance and support available from CCEA. 

3

Part 1

Setting the Context for the CBAs

4

Legislative Context

“Pupils shall be assessed in the autumn term, using such computer-based assessment method [for literacy and

numeracy] as the Department may specify …”

The Education (Assessment Arrangements) (Foundation to Key Stage 3) Order (Northern Ireland) 2007 SR 2007 No. 45

5

Focus on Raising Standards

6

“The use of data from computer-based assessments remains a key aspect of the Department’s school improvement policy”

Paragraph 3 – DE Letter to Principals Nov 2011

New Providers

The Department of Education (DE) initiated a public procurement process in 2011 (as the InCAS contract with CEM ended in January 2012) and this procurement process identified two new suppliers for the computer-based assessments in Literacy and Numeracy:

•Tribal Education Ltd. for the literacy assessment (NILA); and•Rising Stars UK Ltd. in partnership with Ardluce for the numeracy assessment (NINA).

Information can be found at : www.nicurriculum.org.uk

7

8

Computer-Based Assessments

Northern Ireland Literacy Assessment – NILA

Northern Ireland Numeracy Assessment – NINA

•Assessments will be available in English & Irish Mediums.

•To be completed by all Year 4 to Year 7 pupils in the Autumn Term 2012/13.

•The purpose of the computer-based assessments is to help teachers plan the learning and teaching during the school year to meet the needs of pupils in their classrooms.

9

Computer-Based Assessments

Statutory Assessment Exemptions

•Pupils with a statement of special educational needs who have been assessed as having severe learning difficulties.

•Pupils exempted from any of the requirements of the Education (Northern Ireland) Order 2006.

Paragraph 21 - DE Circular 2011/15

•A new DE Circular is expected in August 2012.

10

The questions items in the Numeracy / Literacy Assessments have been developed using the following criteria:

•Definition of Numeracy / Literacy as set out by CCEA;•The expectations of the NI Mathematics & Numeracy / Language & Literacy Curricula;•CCEA’s revised Lines of Development;•Advice from ELBs regarding expectations at the end of each academic year; and•Levels of Progression.

11

Computer-Based Assessments

Key Dates

12

January 2012 Small observation trial

March 2012 Trial with 200 schools

May 2012 Principals’ Information Seminars (3-21 May)

May 2012 Quality assurance process (8-18 May)

May 2012 Irish Medium Trial (21-25 May)

May – Oct 2012 Online Teacher Training (modular course)

Autumn 2012 Use of NILA / NINA in schools

2013/14 onwards Ongoing training support (principals, teachers,

assessment coordinators, classroom

assistants etc.)

Part 2

Literacy / Numeracy

Assessment Overview

13

Introduction to the Literacy Assessment

14

The Literacy Assessment

• There will be two age related bands in the assessment, one for Primary 4 and 5 and one for Primary 6 and 7.

• The questions will have a range of difficulty within each band, designed to help pupils show what they can do whatever their ability.

• As they go through the assessment the pupils will be given questions adapted to their responses.

• The assessment should take the majority of pupils less than 50 minutes.

15

The Literacy Assessment

• It uses the potential of the on-screen presentation to provide interactive and engaging experiences in different ways.

• It draws varied topics from across the Northern Ireland Curriculum.

• It is varied in form, ranging from everyday texts, such as advertisements and signs, to high quality literary passages.

• It assesses crucial aspects of Literacy from the Reading, Writing and Listening elements of the NI Curriculum and provides teachers with diagnostic information about the skills and understanding of their pupils.

16

From the Reading Curriculum …

• Read, explore, understand and make use of a wide range of traditional and digital texts.

• Use a range of reading and comprehension strategies and skills for different purposes.

• Locate, select and use texts and sources for a particular task.

• Understand how texts are structured and identify features of written language.

• Use evidence from the text, and in key stage 2, inference and deduction.

17

NILA assessment of Reading

Focuses on understanding explicit and implicit meanings and how texts

are organised.

The questions relate to these four strands:

• Understanding information and ideas;• Finding and using information;• Understanding different types of texts; and• Responding to different layers of meaning.

18

Question focus – Reading: understanding information and ideas

19

Question focus – Reading: finding and using information

20

Question focus – Reading: understanding different types of texts

21

Question focus – Reading: responding to different layers of meaning

22

From the Writing Curriculum …

In the context of learning to write for a range of purposes in different

forms …

• Use a variety of skills to spell words correctly.

• Develop increasing competence in the use of grammar and punctuation to create clarity of meaning.

23

NILA assessment of Writing

Focuses on technical accuracy in punctuation, spelling and grammar.

The questions are related to these three strands:

• Punctuation;• Spelling; and • Grammar.

24

Question focus – Writing: punctuation

25

Question focus - Writing: spelling

26

Question focus - Writing: grammar

27

From the Listening Curriculum …(within Talking and Listening)

• Listen and respond to a range of fiction, poetry, drama and media texts.

• Respond to guidance, directions and instructions.

• Respond to ideas and points of view, using evidence.

28

NILA assessment of Listening

Involves listening to and understanding information and ideas that

pupils hear.

The questions for the Listening strand focus on a range of skills

such as:

• retrieving; • summarising; and • inferring meanings.

29

Question focus - Listening

30

Question Types & Answer Techniques:

• Click one• Click multiple• Cloze• Drag into order• Drag to box• Drag punctuation• Match• Select word

This means looking at the instruction and also at how the mouse is controlled to make the right selection of answer.

NILA Question Types

31

32

NILA Practice Assessment

Admin / Teacher Section

33

Pupil Details & Assessment History

34

Introduction to the Numeracy Assessment

The Numeracy Assessment

• Samples the key concepts of Numeracy through two assessments / strands:

Strand 1 Understanding & Using Number Strand 2: Shape & Space, Measures; Handling Data

• Each strand will be assessed using an adaptive approach to assessment.

• Assessment for each strand will take approximately 30 minutes.

• Assessment will include a variety of interactive questions and a range of response techniques.

36

Strands and Threads

Strand 1 includes:

• Counting & Understanding Number;• Facts and Mental Calculations;• Operations with Whole Numbers, Integers and Decimals;• Fractions, Decimals, Percentages, Ratio & Proportion; and• Problems – including Money, Function Machines & Algebra.

Strand 2 includes:

• Shape & Space;• Measures; and• Handling Data.

37

Threads and Objectives

• Within each thread, there are objectives covering Years 1 - 7. (Some threads do not appear in the early years e.g. there are no questions on Fractions in the early years as Fractions is not taught until later.)

 • Each Objective is targeted at a particular curriculum year and has

questions of varying difficulty. As these assessments are taken early in the first term, the target curriculum year is the one below the pupil’s current curriculum year.

38

Counting & Understanding Number

From Year 3 curriculum – Order 2-digit numbers. Know where a number should lie on a number track, number line, 50 array or 100 square.

39

Number Facts, including Mental Calculations

From Year 5 curriculum - Use knowledge of addition and subtraction facts and place value to derive sums and differences of pairs of multiples of 10, 100 or 1000. 40

Operations with Whole Numbers, Integers & Decimals

From Year 6 curriculum - Exact division of whole numbers and decimals to 2DP by a single digit.

41

Fractions, Decimals & Percentages

From Year 4 curriculum - Understand and identify fractions of shapes; use diagrams to compare fractions and establish equivalents.

42

Problems

From Year 7 curriculum - Understand that a letter can represent an unknown number.

43

Shape & Space

From Year 5 curriculum - Identify and draw lines of symmetry in shapes. Given a mirror line, show a shape's reflection.

44

Measures

From Year 4 curriculum - Read the time on analogue clocks - including hour, half and quarter hour intervals.

45

Handling Data

From Year 2 curriculum - Sort objects into groups using one criterion. 46

NINA Practice Area

47

Admin / Teacher Section

48

2 Assessments (Numeracy)

49

Assessment Status

50

Part 3

Assessment Outcomes / Reports

51

Assessment Outcomes / Reports

As pupils answer questions their responses are saved automatically

online.

Once pupils complete the assessments, the assessment is processed.

Schools will be able to access the following reports within 24-48 hours:

• Pupil Objectives Reports (showing each question outcome); and• Pupil Diagnostic Reports (showing strengths and areas for

development).

52

53

The Objectives Reports provide information on the 8 threads of Numeracy and 8 strands Literacy Assessments.

The information gained will complement teachers’ own assessments in the classroom.

•Age Related Outcomes will be from 5 to 12 years (available as quartiles from January 2013).

•Standardised Scores will also be available from January 2013.

Assessment Outcomes / Reports

Features of the new CBAs

• NINA Practice area available to pupils (no passwords required)• NILA Practice assessment available to pupils (no passwords

required)• Pupil usernames/passwords are pre-assigned • Assessment ‘Pause’ facility available• Questions are not timed• Headphones do not need to be worn all the time• Assessment status information (started, paused and completed)• Automatic online saving of question responses• Automatic marking• Diagnostic Feedback• Unique Teacher log-ins – can work through their specific class

information56

Part 4Preparation for the Autumn Term

57

Autumn Term 2012

• Encourage your Year 4-7 teachers to complete the modular online training course;

• Encourage your staff to access www.nicurriculum.org.uk – CBA section – for information, guidance and support materials;

• Ensure your SIMS information contains full and up-to-date records for Year 4-7 teachers and pupils prior to the SIMS data transfer;

• Assign CBA roles and responsibilities; and

• Schedule pupil practice times and assessment times.

58

Roles and Responsibilities

Administrator (or Senior-Administrator for NINA) can access all

areas in the administration section of the software, including reports.

Teachers will have access to own class (view passwords and reports).

Pupils can access the practice assessment without log-in and will have

a username & unique password for the assessments.

59

Shortcuts on your Desktop

60

61

Initial Preparation

Check that:

• each PC and laptop is working before use with pupils;

• the links to the NILA (www.niliteracyassessment.org.uk) and NINA (www.ninumeracyassessment.org.uk) websites work on all PCs and laptops; and

• Shockwave is loaded. This can done by using the Practice Area of the Numeracy Assessment - if questions do not appear, you will need to contact the C2k Helpdesk to load Shockwave.

62

Initial Teacher Preparation

Teachers should check that:

• they have the usernames/password list for their class;

• wireless laptops (if used) are within ‘range’ of the wireless network connection;

• laptop batteries are sufficiently charged;

• the sound is working on all the computers/laptops; and

• there are a sufficient number of working headphones and that these are plugged in firmly—also ensuring the sound is not set too loud.

NB - Pupils should be provided with pencils and paper for the NINA assessment.

63

Initial Teacher Preparation

Ensure that pupils:

• are familiar with the computer/laptop that they will be using for the assessments, especially if they will be using a touchpad;

• know how to log-on to NINA/NILA;

• know how to pause or exit an assessment by clicking on the red x at the top right hand side of the screen and closing the window;

• know how to resume an assessment by clicking “Start Assessment button”. Pupil should log in again with the same username and password and see “Continue first assessment”; and

• understand the assessments they will be completing and have had sufficient opportunity to use the Practice Area / Practice Assessment.

The Practice Area / Practice Assessment

Be sure to:

• take your pupils through the various question/response types;

• draw attention to the instructions for each question type;

• demonstrate how to answer using a data projector or smart board;

• give your pupils plenty of opportunity to use the practice area (NINA) / practice assessment (NILA);

• show pupils how to undo/change an answer;

• evaluate how pupils cope with question types and advise; and

• show that an answer can be deleted or changed any number of times before the ‘Submit’ (NINA) / ‘Next’ (NILA) button is clicked.

NB - Pupils can access the Practice Area / Practice Assessment without logging on to do an assessment.

64

65

Reporting CBA Outcomes to Parents 2012

• Schools are required to report in writing, the outcomes from the statutory computer-based assessments.

• Only formative/diagnostic feedback will be provided in autumn 2012.

• Schools are required to offer parents a meeting to discuss outcomes by the end of the autumn term.

Part 5Feedback from Trials & Developments

67

Feedback from March Trial

68

Literacy Numeracy

Number of Schools Registered for Trial 185 193

Number of Pupils Registered for Trial 20,560 pupils 21,497 pupils

Number of Pupils Started Assessments 12,626 13,224

Number of Assessments completed(each pupil completes 2 numeracy assessments and 1 literacy assessment)

11,065 26,077

The aim for the March Trial was to have 200 schools, 400 teachers and 10,000 pupils involved.

The figures above indicate that the pupil number target had been exceeded by more than 30%. THANK YOU to all the Trial Schools.

Teacher Evaluation of NILA (March)

• 91.1% of teachers (n=224) rated the pupil practice assessment highly;

• 69.1% of teachers (n=170) reported that pupils engaged well with the assessment;

• 77.2% of teachers (n=190) indicated they provided support to pupils during the assessment; and

• Three quarters of teachers (78.5%, n=193) felt confident about using the assessments with pupils in future.

* based on 246 questionnaire responses

69

Pupil Evaluation of NILA (March)

70

Pupils were asked if they found NILA ‘easy’, ‘ok’ or ‘difficult’. The

following table illustrates pupils’ responses:

Therefore, results showed that the majority of pupils felt that the

assessment was either easy or ok (85.4%, n=6806).

% of Pupils Number of Pupils

Easy 33.1 2,641

Ok 52.3 4,165

Difficult 14.6 1,162

Total 100 7,968

Teacher Evaluation of NINA (March)

• 90.6% of teachers (n=221) rated the pupil practice area highly;

• 92.2% of teachers (n=225) reported that pupils engaged well with the assessment;

• 69.7% of teachers (n=170) indicated they provided support to pupils during the assessment;

• 80.3% of teachers (n=196) felt the assessment was user-friendly for their pupils; and

• 86.9% of teachers (n=212) felt confident to administer the assessments with pupils in future.

* based on 244 questionnaire responses

71

Pupil Evaluation of NINA (March)

72

Pupils were asked if they found NINA ‘easy’, ‘ok’ or ‘difficult’. The

following table illustrates pupils’ responses:

Therefore, results showed that the majority of pupils felt that the

assessment was either easy or ok (90.7%, n=8543).

% of Pupils Number of Pupils

Easy 38.7 3,646

Ok 52.0 4,897

Difficult 9.3 881

Total 100 9,424

Trial Feedback & Developments

Based on feedback received from teachers & pupils during the trials,

the suppliers have carried out the following developments:

Aligning the look & functionality of the two CBAs

• Upload of SIMS data;• Automatic pupil password creation;• Administration section;• Reporting section; and• Feedback reports.

73

Trial Feedback & Developments

Developments from a pupil perspective:

• Practice assessment (NILA) with easier, more colourful questions;• Practice area (NINA) now with instructions and ‘correct/incorrect’

response;• Greater number of lower ability questions in the assessments;• Functionality of some questions (drag & drop / punctuation / mouse

control / drag & rotate);• Voice-over; and• Pause facility.

74

Part 6Guidance & Support

75

CCEA Support and Resources

• Modular Online Training Course for all Year 4-7 Teacher

• Preparation & Support Booklet re: SIMS data report and accessing passwords (via General Dispatch)

• Parent Leaflet 2012 (via General Dispatch)

• Website, online support & guidance materials at www.nicurriculum.org.uk

• CCEA Assessment Operations Team Helpdesk: 028 9026 1274

• ‘Help’ sections within NILA / NINA

76

77

Evaluation

Please complete the evaluation.

This is important as your comments are valued

and will inform future improvements.