CBA Autumn 2014 Principal Information Seminar September 2014 Diane Owens and Anne Cromie 1

CBA Autumn 2014

Principal Information Seminar

September 2014

Diane Owens and Anne Cromie

1

Learning Intentions

At the end of the session you will be aware of the:

•context for CBA in Northern Ireland for autumn 2014;•findings of the recent independent reviews of computer-based assessments (CBA) commissioned by the Minister of Education;•developments made to CBA in response to feedback from schools; •reports and outcomes produced and how they can be used to inform planning; and •guidance and support available from CCEA and C2k.

2

Overview

9.00 or 13.00 Registration

9.30 or 13.30 Welcome

CBA Autumn 2014

Recent independent reviews

10.30 or 2.30 Tea/Coffee

10.45 or 14.45 Developments to CBA

11.45 or 15.45 Questions, evaluation & close

3

Focus on Raising Standards

“The introduction of the CBAs is consistent with the national strategy to improve pupil performance within literacy and numeracy.” p61

“…fit well with an official assessment strategy that focuses on assessment for learning and a wider national strategy to reduce inequity in student outcomes.” p61

4

OECD Review of Evaluation and Assessment in Northern Ireland Education 2013

Recent Evaluation and Review

5

Evaluation and Review of CBA Since 2012

Standard Evaluation

Commissioned Review Additional

Annual and ongoing evaluation by CCEA

1.CCEA review the operation of the new assessments.2.ETI review to determine how effectively schools make use of assessment information.

3.Independent review of the technical issues - why they occurred and what lessons can be learnt for the future.

4.Independent review of policy and practice.

5.Department review the policy of statutory CBA.

OECD Review

6

Evaluation and Review of CBA Since 2012

• Covered all aspects from procurement to operation – Phased approach in autumn 2013– 2014 mini trial– Phased approach in autumn 2014

• Overall findings– Consistent– Reinforce key messages

• Since 2012:– 184 schools chose to use CBA in 2013 – 194 in 2014– 105 2012-2014

7

Moving Forward

Procurement Cycle

•Five year procurement cycle

Technical Support/ Transformed Primary Network

•Continued partnership – CCEA/C2k/Capita/suppliers•Enhance CCEA/C2k helpdesk•Monitoring network stability•Ensure ICT equipment set up for CBA•Pre-flight check

– Support CBA on transformed network

8

OECD Review

9

OECD:Particular Strengths of CBA

OECD has identified the following as particular strengths:

•Standardised to NI school population•Aligned to revised curriculum•Adaptive (difficulty of the questions adapts to the level of the pupil)

– reduces the risk of demotivation– provides nuanced, fine-grained profile of the pupil’s performance.

•Consistent with the national strategy to:– improve pupil performance in literacy and numeracy– focus on assessment for learning

10

OECD:Main Benefits of CBA

OECD has highlighted CBA as a reliable basis for…

•assessing progress and learning needs •planning teaching to meet learner needs•supporting self-assessment and individualised target setting•reducing the workload of teachers as no marking is involved•providing teachers with clear and reliable information for parents on their children’s strengths and weaknesses

11

OECD:Learning for CBA

OECD identified the following regarding CBA:

•Technical challenges– procurement process– screen freezes/sound– physical dexterity (keyboard and mouse)

•Feedback of results– adaptivity– delay in standardised scores– parent reports

•Lack of continuity– procurement process– InCAs– longitudinal data– two providers

•Timeframe12

Ministerial Statement

• OECD Review positive

‘The practice of having this sort of universal, formative assessment in primary school, mapped to our own curriculum and delivered at the start of the academic year, is noted with approval by the OECD’

John O’Dowd, Oral Statement (11 March, 2014)

13

Legislative Context

In an email to schools on April 10, 2014, the Assessment and Qualifications Team (DENI) noted that:

•NILA and NINA will not be specified for mandatory use in 2014/15;•the Department expects schools to continue to carry out diagnostic assessment using an assessment, or assessments, of their choice; and •the results of this assessment should be used as the basis for engagement with the pupil’s parent/guardian.

14

NILA and NINA

15

Roles and Responsibilites

Role Responsibility

PrincipalOverview of school CBA

• accessing the school username and password • arranging the SIMS upload to WebXchange• allocating the CBA administrator role

AdministratorManages at school level

• teacher passwords• SEN/HI Entry Points

TeacherManages at classroom level

• pupil usernames and passwords for class• preparing classroom• practising with pupils• administering assessments

Classroom assistantSupports pupils

• continuation of general classroom good practice

16

SIMS Upload to WebXchange

Role to be assigned by Principal

•Quick to complete•Complete well in advance of using assessments•Need to have:

– Instructions sent to principals (approx. 26 August) – School username and password – Principal can access this from ‘Principal Documents

with MY-SCHOOL area of C2k exchange (school username DENI number)

•Does not affect use of practice assessment/area by pupils•Last date to complete is 26 September 2014

If you experience any difficulty with SIMS/Webxchange or accessing Principal Documents, please contact the C2k Service Desk on 0870 601666.  

17

Moving Forward with the CBA 2014

18

Admin passwords available to schools 11 August

Preparation instructions provided to schools 26 August

Manuals and posters delivered to schools 5 September

Phase 1 assessments completed 8-12 September

Phase 2 assessments completed 15-18 September

Final date for SIMS upload 26 September

Phase 3 assessments completed 29 September-12 December

Evaluations live 8 September – 12 December

What do they have in common?

•May be completed only by Year 4 to Year 7 pupils•Web-based and pupil responses to questions are saved automatically online•Adaptive and not time-bound•Longer a pupil takes to answer the longer the assessment will take•Can be paused and re-started from that point•Available in English & Irish•Special Entry Points available•Pupils cannot repeat assessments•Similar tables and reports•Reports and outcomes available within 48 hours of completion

NILA and NINA

19

20

Focus on Skills

Aspect of Literacy Skills Focus Pupils’ ability to …

Listening Information Retrieval

Listen to and understand information and ideas.

Reading Sequence Understand information and ideas

Information Find and use information

Features of Texts

Understand different types of texts

Opinions Respond to different layers of meaning

Writing Punctuation Punctuate text

Spelling Recognise accurate spellings

Grammar Use grammar accurately21

The Literacy Assessment

•Single assessment •No time limit•Most pupils complete 46 questions•Average time of 40 minutes•Four difficulties of questions

– age appropriate for the pupil (=), for a younger pupil (<), an older pupil (>) or an even older pupils (>+)

•SEN Entry Point starts with (<)•Question number is displayed at the bottom of the screen•8 different answer techniques

– cloze, drag and match, drag into order, drag punctuation, drag to hot spot, pick multiple, pick one and select word

•Practice assessment

22

Adaptive Routing in NILA

23

24

Focus on Skills

Skills Focus Pupils ability to …

Strand 1 Number Counting & Understanding Number;

Facts Facts and Mental Arithmetic

Operations Operations with Whole Numbers, Integers and Decimals

Fractions Fractions, Decimals, Percentages, Ratio & Proportion

Problems Problems – including Money, Function Machines & Algebra

Strand 2 Shape and Space

Shape and Space

Measures Measures

Handling Data Handling Data

Skills Focus Pupils’ ability to …

Strand 1 Number Counting & Understanding Number;

Facts Facts and Mental Arithmetic

Operations Operations with Whole Numbers, Integers and Decimals

Fractions Fractions, Decimals, Percentages, Ratio & Proportion

Problems Problems – including Money, Function Machines & Algebra

Strand 2 Shape and Space

Shape and Space

Measures Measures

Handling Data Handling Data25

The Numeracy Assessment

• 2 strands:

– Strand 1 Understanding & Using Number – Strand 2: Shape & Space, Measures and Handling Data

• Neither has a fixed time for completion• Each strand assessed using an adaptive approach• Pupils take an average of 20 minutes per strand• Only mental arithmetic questions are timed• Irish Medium version only available to IM schools or those with IM

units• Pencil and paper should be available to pupils• Practice area• Voiceover available for every question

26

Adaptive Assessment

• Start in their target year - one year below their current school designated year e.g. Year 3 questions for a Year 4 pupil

• SEN Entry Point is 2 years below target year• 3 difficulties of questions

– Easy, medium and difficult

• First question is a ‘medium’ one• First set of 6-8 questions assesses general numeracy• Move up, down or stay in the year• Move up or down can be reversed• Process repeated until assessment determines they are answering

questions appropriate to their ability

27

Adaptive Assessment

28

29

Enjoy!

See you at 10.30/14.30

CBA Developments

30

Developments

31

For Autumn 2013

SEN Entry Point Both

HI Entry Point NILA

Skills Focus Report NINA

Year Group Report NINA

Improved Parent Report Both

SS and AROs available within 48 hours Both

Improved IM selection Both

Modified pupil checklist Both

Improved assessment monitoring NINA

Parent Report

32

Special Entry Points

SEN (Special Educational Needs) Entry Point:

•for pupils experiencing difficulties with literacy/numeracy– quicker access to easier questions – more positive and potentially shorter assessment experience– offers same access to the full range of questions as other pupils

HI (Hearing Impaired) Entry Point:

•for pupils who are receiving support for a hearing impairment – removes the Listening aspect of the assessment– allows pupils to be assessed in Reading and Writing

Use of the Special Entry Points will not be obvious to pupils.

33

Qualifying for Special Entry Points

The teacher and the Senior Management Team must:

For the SEN Entry Point•reasonably expect that the pupil is performing two or more years below their chronological age; and•be in agreement that an SEN Entry Point is appropriate for the pupil.

For the HI Entry Point•have evidence that the pupil is receiving support for a Hearing Impairment; and•be in agreement that an HI Entry Point is appropriate for the pupil.

Only the Administrator can set a Special Entry Point 34

Developments

For Autumn 2014

Software adapted to operate on the C2k transformed network Both

Improved SEN Entry Point selection NILA

Improved HI Entry Point selection NILA

Skills Focus Report NILA

Question number added to the bottom of the screen NILA

Improved filtering of reports NILA

Reduction in question techniques NINA

Improved practice area NINA

35

Transformation

Transformation of C2k network

•Impacts all primary and special schools•Close liaison between CCEA, C2k, Capita and suppliers•Thorough piloting:

– December 2013– June 2014– Phased introduction Autumn 2014

36

Computers

• Only use pupil workstations• Teacher laptops cannot be used • Connect to the C2k network at least 30 minutes

before CBA use • Computers not been connected to the C2k network

recently may take longer to update• Pupils must first log into the C2k network with their

individual C2k username and password. • Multiple pupils logged on with the same username

and password will not be able to access NILA and NINA.

37

Check the Sound

NINA•Go to the practice area •Select any practice question and •Click the sound icon•You should be able to hear the sound play

NILA•Go to the practice assessment•Click the sound icon•You should be able to hear the sound play

38

Accessing CBA

39

Teacher ‘MY-SCHOOL’ homepage

• ‘View All’• ‘Curriculum Links’• ‘NILA’ or ‘NINA’

Accessing CBA

40

Pupil ‘MY-SCHOOL’ homepage

• ‘My Links’• ‘NILA’ or ‘NINA’

Pre Flight Check (PFC)

Key Points

•Simple and user-friendly resource for schools•Automatically runs when pupils click on NILA or NINA link via their ‘MY-SCHOOL’ homepage•Completed prior to commencing assessments•Checking correct versions of software are installed•In case of incorrect set-up - provide guidance and advice to schools•When successful takes you straight to the assessment homepage

NINA NILA

http://preflightcheck.ccea.org.uk/nina.html http://preflightcheck.ccea.org.uk/nila.aspx

41

PFC Help Identify Issues

• Every time the PFC runs CCEA is notified• Gives us frequency of information • Completing the pop up gives details of:

– school, – workstation – teachers comment

• Help identify patterns • Need to include both DENI number and workstation number – little

sticker on computer

42

Improved Entry Point Selection

43

Individual Entry Point Selectionfor NILA

44

NILA Skills Focus Report

45

Monitor Assessments

NILA EM completed NINA completed

NILA EM started NINA started

NILA IM completed NINA not started

NILA IM started

NILA – to help monitor progress during an assessment the question number is displayed at the bottom of the screen (most pupils answer 46 questions).

NINA – each strand takes approximately 20 minutes.46

Reduction of Question Techniques

47

Pupils are allowed to use pencil and paper

48

During an Assessment

Question not fully loaded (images/buttons are misplaced/missing)

Step 1: If this issue continues for more than 10 seconds click on ‘F5’ once.

Step 2: If the problem persists for this user, close Internet explorer (IE) and get the pupil to log back into their assessment.

Step 3: If the problem is still not resolved, please remove this device from the assessments and use another workstation if available.

Step 4: If the problem remains, contact the C2k Service Desk urgently on 0870 6011666.

49

Phoning C2k

Please make a note of the following information before calling the C2k Helpdesk:

•date and time of issue •assessment being attempted (NINA or NILA)•description of issue•workstation ID •pupil ‘C2k / NINA / NILA’ account user name•if possible please take a screenshot.

50

Outcomes and reports

51

What NILA / NINA Tables and

Reports are available?

• NILA / NINA Results Tables provide schools with Age-Related Outcome (ARO) Ranges and Standardised Scores (SS). These are available for individual pupils, classes, year groups and at whole school level. These build over time to show additional data as it becomes available for new academic years.

• NILA / NINA Parent Reports contain the results table for an individual pupil, showing only the Age-Related Outcome (ARO) Range.

• NILA / NINA Objectives Reports provide specific information about the questions asked and whether the pupil answered correctly or incorrectly.  

52

Results Table (School, Year Group and Class)

53

Results Table (Individual Pupil)

54

Skills Focus Reports

55

Using Outcomes

CBA “… are intended for diagnostic purposes and the assessment data is primarily used for:

•Assessing pupil progress and identifying their learning needs

•Supporting self-evaluation and target-setting

•Helping teachers shape their teaching to address the learning needs of their pupils

•Providing information for parents on their children’s strengths and weaknesses within literacy and numeracy.”

OECD Review of Evaluation and Assessment in Education (2013) p.54  

56

CCEA Support and Resources for Autumn 2014

• Face-to-face training (10 September - 1 October)

• Online Training Course (From 25 August)

• Preparation instructions – SIMS upload, school username/password

• NILA / NINA Guidance Documents, including guidance for pupils with SEN

• ‘How to change an answer before you submit it in NINA’ poster

• Pupils demos and scripts

• NINA practice area

• NILA practice assessment

57

Websites and Helpdesks

58

Resources and Information for Schools•Latest information about CBA and online resources for schools on www.ccea.org.uk

Computer-Based Assessment Helpdesk

Hardware/ IT Difficulties or help with computers/laptops•C2k helpdesk 0870 6011666•C2K Exchange (desktop icon)

Monday – Friday (except for public holidays)

Email cba@ccea.org.uk

Questions, Evaluation and Close

59