FFT Data Analysis Project © Fischer Family Trust, 2009 Using FFT Live 3.0 How FFT data can help your school improve Primary Schools (KS2) Fischer Family

FFT Data Analysis Project © Fischer Family Trust, 2009

Using FFT Live 3.0How FFT data can help your school improve

Primary Schools (KS2)

Fischer Family Trust

www.fischertrust.orgJune 2009


KEY AIMS OF THIS TRAINING SESSION

To gain an overview and use some of the key features AND reports available on FFT Live

To interpret and use some of the main reports available on FFT Live

To gain an understanding of how FFT data, alongside other data, can be used in key situations (SEF, self evaluation, inspection, target setting) by a variety of staff to help your school and LA improve

Allow time for discussions on FFT Live, FFT reports, ‘best practice’ and potential ‘misuse’ of data


FFT Aims and Ethos

Background to the Fischer Family Trust (FFT)

Independent charity, founded by Mike Fischer, co-founder of Research Machines (RM). FFT supports projects in health and education

Education Projects include the data analysis project managed by Mike Treadaway, using national data sets provided by DCSF and Welsh Assembly Government

Aim is to help schools make effective use of value-added test and TA data to raise individual pupil attainment and improve progress

Analyses are provided to support effective school self-evaluation:

Data provides questions not answers

Data analysis should be used by schools to promote discussion, evaluation and planning

Analyses for different groups of pupils, and a range of indicators, to help identify strengths or areas for development/intervention

Use the past to inform the future


Background to using dataBackground to using data


A five-stage cycle for school improvement

Where can FFT data

support school improvement?

Source: DCSF


Before we start – the concept of triangulation

FFT analysisRAISEonline/

other data models

Professional Judgement

Basis for action Investigate Further

Check Accuracy Challenge Assumptions


Introduction to FFT LiveIntroduction to FFT Live


How can I access FFT data?

Direct via FFT Live 3.0Direct via FFT Live 3.0From your LAFrom your LA

Paper reportsElectronic reports (pdf)

Excel filesSchool FFT database

www.fftlive.org

Logins are availableFrom your LA FFT contact

or School FFT contact

http://www.fftlive.org/


FFT Live 3.0

What is FFT Live 3.0?

FFT Live 3.0 is FFT’s updated online system giving FFT Live 3.0 is FFT’s updated online system giving schools and LAs online access to a range of FFT data schools and LAs online access to a range of FFT data and reports to support target setting and self evaluationand reports to support target setting and self evaluation

The reports are based on similar methodologies and The reports are based on similar methodologies and models used in the standard database reportsmodels used in the standard database reports

FFT Live 3.0 gives users access to a greater range of FFT Live 3.0 gives users access to a greater range of reports than the previous version of FFT Live and also reports than the previous version of FFT Live and also includes a number of reports which are only available includes a number of reports which are only available onlineonline


FFT Live 3.0

Who is FFT Live 3.0 aimed at?

FFT Live includes a broad range of reports to support target FFT Live includes a broad range of reports to support target setting and self evaluation and can be used by a wide variety of setting and self evaluation and can be used by a wide variety of staff in schoolsstaff in schools

Teachers, Department Heads and Heads of Year will find the new Teachers, Department Heads and Heads of Year will find the new pupil/subject estimates and value added information particularly pupil/subject estimates and value added information particularly useful in helping to review progress and set challenging targets.useful in helping to review progress and set challenging targets.

SLTs can now access a greater range of strategic data to support SLTs can now access a greater range of strategic data to support whole school and group evaluation and target settingwhole school and group evaluation and target setting

School governors can also access school level data to support School governors can also access school level data to support SLTs in evaluation and planning.SLTs in evaluation and planning.

LA staff including advisers, SIPs, information teams and senior LA staff including advisers, SIPs, information teams and senior managersmanagers


FFT Live 3.0

Why use FFT Live?

Allows quicker access to a wide range of FFT reports and data for school Allows quicker access to a wide range of FFT reports and data for school and LA staffand LA staff

Allows access to pupil, group and school data to support target setting Allows access to pupil, group and school data to support target setting and self evaluationand self evaluation

Some reports (e.g. pupil estimates showing a range of probabilities) are Some reports (e.g. pupil estimates showing a range of probabilities) are only available on FFT Liveonly available on FFT Live

Greater level of flexibility allowing users to choose their own options (e.g. Greater level of flexibility allowing users to choose their own options (e.g. model types and ranks)model types and ranks)

Allows users to export data directly from FFT LiveAllows users to export data directly from FFT Live

Gives access to Self Evaluation bookletsGives access to Self Evaluation booklets

Improved look and feelImproved look and feel

Online help and guidance available for individual reportsOnline help and guidance available for individual reports


Logging in to FFT Live 3.0

How do I log-in to FFT Live 3.0 (www.fftlive.org)?

Schools can now access FFT Live in a 4 ways:Schools can now access FFT Live in a 4 ways:

1.1. SCHOOL ADMINSCHOOL ADMIN account supplied by the LA (e.g. account supplied by the LA (e.g. 9992000Admin9992000Admin) or ) or created by a school itself (e.g. created by a school itself (e.g. ASmit9999ASmit9999) – max 2 per school) – max 2 per school

2.2. SCHOOL USERSCHOOL USER account (e.g. account (e.g. ASmit9999ASmit9999). An FFT Live school admin ). An FFT Live school admin account holder can create as many user accounts as needed for staff in account holder can create as many user accounts as needed for staff in schools (e.g. teachers).schools (e.g. teachers).

3.3. GENERIC SCHOOLGENERIC SCHOOL account (your school number – e.g. account (your school number – e.g. 99920009992000). Using ). Using this account you can access school and group level data. The account is this account you can access school and group level data. The account is supplied by your LA.supplied by your LA.

4.4. GENERIC PUPILGENERIC PUPIL account (your school number plus ‘P’ – e.g. account (your school number plus ‘P’ – e.g. 9992000P9992000P). ). Using this account you can access school, group AND PUPIL level data. Using this account you can access school, group AND PUPIL level data. The account is supplied by your LEA.The account is supplied by your LEA.





If you don’t have an account of your own, you can also access FFT If you don’t have an account of your own, you can also access FFT Live using an ANONYMOUS account:Live using an ANONYMOUS account:

1.1. ANONYMOUS SCHOOL ACCOUNTANONYMOUS SCHOOL ACCOUNTAccess school reportsAccess school reports

Username: 9992004Username: 9992004Password: ANON (Password: ANON (case sensitivecase sensitive))

2.2. ANONYMOUS PUPIL ACCOUUNTANONYMOUS PUPIL ACCOUUNTAccess school AND PUPIL reportsAccess school AND PUPIL reports

Username: 9992004XUsername: 9992004XPassword: ANON (Password: ANON (case sensitivecase sensitive))



FFT Live 3.0 Main Menu


FFT Live 3.0 Help Facilities

You can access a range of help You can access a range of help topics from any screen using the topics from any screen using the HelpHelp menu menu

You can also access specific help You can also access specific help on each report by choosing the on each report by choosing the ‘‘Help with this reportHelp with this report’ option.’ option.


FFT Live 3.0 Report Screen Layout

(1) Menu Bar (2) Drop-down options

(3) View Report buttonPress this button whenever

you make any changes

(6) Export and Print optionsExport options include

PDF, Excel, CSV, and XML

(5) Resize screen options(4) Page options


FFT Live 3.0 Report Screen Layout

(1) Report title and details (2) Description of report

(3) Main report (4) Key

(2) School and LA name


What’s on FFT Live 3.0?

Value Added Significant Areas 3 Year Summary Report (KS1-2)

Value Added Significant Areas 3 Year Detail Report (KS 1-2)



Pupil VA Summary Report (KS1-2)



Pupil Estimates Summary Report (KS2)

Pupil Estimates Detail Report (KS2)*

*Only available on FFT Live3.0*Only available on FFT Live3.0



School Estimates Report (All KS)



Analyses to Support Self Evaluation Booklet**

**Available through school logins only


A framework for self evaluation A framework for self evaluation and target setting using FFT and target setting using FFT

LiveLive


A framework for Self-Evaluation and target setting using FFT Live

(1) VA Sig Areas Grid (2) VA Sig Areas Detail

(3) Pupil VA Summary ReportHow well are our learners doing (pupil/group/school)?How well are our learners doing (pupil/group/school)?What about in-school variation?What about in-school variation?

What interventions have/haven’t worked?What interventions have/haven’t worked?What are the big issues for our school?What are the big issues for our school?

What other information can we use to ‘triangulate’What other information can we use to ‘triangulate’our evaluation?our evaluation?


What more should we aim to achieve?What more should we aim to achieve?Which areas require intervention?Which areas require intervention?

What must we do to make it happen?What must we do to make it happen?How are we going to track pupils?How are we going to track pupils?

How do we measure success?How do we measure success?

(4) Pupil Estimates Detail Report (5) School Estimates Report



PART 1Using FFT Live for school self evaluation

Improving self evaluation in schools: Using FFT estimates


A description of value added

What is Value Added?

Basic Basic Value Added (VA)Value Added (VA) measures the PROGRESS made by measures the PROGRESS made by an individual pupil, or group of pupils, between Key Stage 1 an individual pupil, or group of pupils, between Key Stage 1 (the starting point) and Key Stage 2 (the end point).(the starting point) and Key Stage 2 (the end point).

Basic VA compares the progress made by each pupil with Basic VA compares the progress made by each pupil with the average progress made by ‘similar pupils’, defined by the average progress made by ‘similar pupils’, defined by FFT as pupils with similar prior attainment, gender and FFT as pupils with similar prior attainment, gender and month of birth.month of birth.

Contextual Value AddedContextual Value Added, or , or CVACVA, is a version of VA which , is a version of VA which also takes into account school context factors and individual also takes into account school context factors and individual pupil context factorspupil context factors


What is Value Added?

VA or CVA can be calculated for individual pupils, VA or CVA can be calculated for individual pupils, groups of pupils or for a whole school.groups of pupils or for a whole school.

Group or school level VA scores are simply based on Group or school level VA scores are simply based on the average of individual pupil VA scores.the average of individual pupil VA scores.

A description of value added


Value Added Models - Overview

Model PA (Prior Attainment) - Value Added

Prior Attainment

Gender Month of Birth

2 basic models are used in FFT Live:

Model SX (School EXtended) – Contextual Value Added

GenderPrior

AttainmentMonth of Birth

School Context

Pupil Context


Factors included in FFT VA Models

Pupil Factors PA SX

Mean Test Level (where available) Mean TA Level Subject Variations Gender Month of Birth EAL

FSM

SEN Stage, Statemented

Ethnicity

Mobility (joined late / time in school)

School Factors PA SX

Mean Intake Test Level

Spread of Intake Test Level

FSM Entitlement (Percentile Rank)

Geodemographic Data (Percentile Rank)







Step 3: ‘Drilling down’ - KS5 Subject EstimatesGuidance: KS1 to KS2 Value Added (Significant Areas Grid)

What does the report show?This report highlights potential strengths, weaknesses and trends across a range of indicators, subjects and pupil groups in the school. The reports covers a 3 year period.

In this example Science appears to be a potential issue with a large number of areas highlighted in blue (showing that actual attainment was significantly below the estimated attainment over the last 3 years.

However, the upward arrows also show that there was a significant improvement in value added in one year (e.g. the % of girls achieving Level 5+ Science).

In contrast however, a number of other areas are highlighted in green (e.g. % of girls attaining L5+ in Maths) showing that performance in this these areas was significantly better than estimated.

The blank areas show where actual attainment was broadly in line with the estimated performance (i.e. no statistically significant difference between estimated and actual performance).

VA Basis: Select from 2 different value added models – PA (Prior attainment) and SX (School Extended)

PA (Prior attainment) is a simple value added model where estimates are based on the progress of similar pupils nationally taking into account pupil prior attainment, gender and month of birth.

SX (School Extended) is a full contextual value added model where estimates are based on the progress of similar pupils nationally (PA model factors plus SEN status, FSM status, ethnicity, mobility and EAL) in similar schools (based on free school meals, deprivation, cohort ability).

Value added scores are calculated by comparing the attainment of each individual student with that of ‘similar’ pupils nationally. Where attainment is better than expected then value added is positive and vice versa.

Matched Pupils: Those pupils who’s prior attainment results can be matched with the most recent results. When using this report be careful if there are large differences between the total number of pupils in a group and the number of matched pupils used in the report.

Key Stage: Select KS1-2, KS2-3, KS3-4 and KS2-4 VA reports where available.

Year: Select report for a 3 year period (e.g. 2004/05 – 2006/07).

Pupil Group: Select All pupils (default) or ‘multi-select’ any combination of pupil groups (girls, boys, FSM etc.)

Group Size: Only show Pupil Groups with a minimum of 5,10, 15 or 20 pupils. Default setting is 5.

What do the colours and arrows indicate?

The colours show where actual attainment is statistically significantly above or below estimated performance over a 3 year period (based on the progress of similar pupils nationally).

A green highlight shows where actual attainment is significantly higher than estimated attainment over a 3 year period. A blue highlight shows where actual attainment is significantly lower than estimated attainment. If there is no highlighting this shows that there is no statistically significant difference between the actual and estimated performance.

Arrows show where there is a statistically significant change in VA performance over a 3 year period compared to similar pupils nationally. Arrows pointing upwards show there has been a significant improvement in one year (↑) or both years (↑ ↑). Arrows pointing downwards show where there has been a significant decline in performance in one year (↓) or both years (↓↓). A significant improvement in one year followed by a significant decline the following year (or vice versa) is represented by a ↑ ↓ symbol.

Pupil Group: Groups are split by gender, prior attainment, FSM, SEN, ethnicity and LAC. Girls/Boys Lower, Middle and Upper Groups are based on an average of Test and TA prior attainment results. Pupils in the Lower group, for example, are in the lowest third nationally based on prior attainment. Pupils in the Middle group are in the middle third and those in the Upper group are in the top third nationally based on average test/TA prior attainment results.

Step 3: ‘Drilling down’ - KS5 Subject EstimatesExample: KS1 to KS2 Value Added (3 Year Significant Areas Grid)


Where the Actual result is:

Significantly above the estimate (in statistical terms)

above the estimate / below the estimate / same as estimate

Significantly below the estimate (in statistical terms)

How is statistical significance shown in reports?

Differences between estimated and actual results are shownusing colours:


Significantly better in one year compared to another

Significantly better two years running

Significantly worse in one year compared to another

Significantly worse two years running

Volatility – significant improvement and significant decline over 3 year period

How is statistical significance over time shown in reports?

Significant changes over time (trends) are shown using arrows:

Where the Actual result is:


Significant Changes

Value-added significantly

below

Value-added broadly in-line with other schools


above


below



above

Yr1

Yr2

Yr3

Yr3

Yr1

Yr2

Significant Improvement

The arrows show significant changes NOT a change in significance

Significant Improvement Significant Improvement

Example 1

Example 2


Significant Changes


below



above


below



above

The arrows show significant changes NOT a change in significance

Yr1

Yr3

Yr2

Yr2

Yr3

Yr1

Significant Improvement

Significant decline

Significant decline

Yr1 – Yr 2: Small change in value added rank but significant change in actual value added as it is at the extreme. This occurs when VA is very high or very low

Example 3

Example 4


Statistical significance

In statistics, a result is called statistically significant if it is unlikely to have occurred by chance.

"A statistically significant difference" simply means there is statistical evidence that there is a difference; it does not mean the difference is necessarily large, important or significant in the common meaning of the word.


When is something likely to be “statistically significant”?

It is difficult to say without actually calculating statistical significance as each individual situation will vary. However, in general, the likelihood of something being statistically significant will increase:

as the difference between the estimated and actual result for a group of pupils become larger

as the number of pupils in a cohort or group increases

As the ‘spread’ of results for a group of pupilsreduces


Activity 1: Using the Significant Areas Grid Report







Step 3: ‘Drilling down’ - KS5 Subject EstimatesGuidance: KS1 to KS2 Value Added (Significant Areas Detail)

What does the report show?This report should be used alongside the significant areas grid report to highlight potential strengths, weaknesses and trends in more detail. Data is only shown for those indicators which are statistically significant (i.e. where actual attainment is significantly above or below estimated attainment or where there has been a significant change in performance over a 3 year period).

In this example, Maths Level 5+ is highlighted for Girls. This is because for the 3 year period 2004/05 – 2006/07, the combined girls achievement was significantly above estimated achievement (based on the progress of similar pupils nationally.

It is important to understand that for an area to be significantly better than expected, this does not mean that every single year has to be green. In this case, the estimate/actual difference is only significant in one year (05/06). The other years show actual results above estimates (positive numbers) but they are not significant. However, the combined effect over 3 years is significant and thus Maths Level 5+ is shown as ‘Significantly above’ for Girls in the Detail report and on the Significant Areas Grid.

VA Basis: Select from 2 different value added models – PA (Prior attainment) and SX (School Extended)


SX (School Extended) is a full contextual value added model where estimates are based on the progress of similar pupils nationally (PA model factors plus SEN status, FSM status, ethnicity, mobility and EAL) in similar schools (based on free school meals, deprivation, cohort ability).


Matched Pupils (3yrs): Those pupils who’s prior attainment results can be matched with the most recent results. When using this report be careful if there are large differences between the total number of pupils in a group and the number of matched pupils used in the report.

Key Stage: Select KS1-2, KS2-3, KS3-4 and KS2-4 VA reports where available.

Group Size: Only show Pupil Groups with a minimum of 5,10, 15 or 20 pupils. Default setting is 5.

What do the colours indicate?

The colours show where actual attainment is statistically significantly above or below estimated performance over a 3 year period (based on the progress of similar pupils nationally).

A green highlight shows where actual attainment is significantly higher than estimated attainment over a 3 year period. A blue highlight shows where actual attainment is significantly lower than estimated attainment. If there is no highlighting this shows that there is no statistically significant difference between the actual and estimated performance.


Actual: Actual performance of MATCHED pupils

Step 3: ‘Drilling down’ - KS5 Subject EstimatesExample: KS1 to KS2 Value Added (Significant Areas Detail)


Activity 2: Using the Significant Areas Detail Report







Step 3: ‘Drilling down’ - KS5 Subject Estimates

KS1 Results: Includes Reading, Writing and Maths Test Levels and English, Maths and Science TA levels where available.

VA Basis: Select from 3 different value added models – PA (Prior attainment), SE (Socio Economic) and SX (School Extended)


SE (Socio Economic) is a contextual value added model where estimates are based on the progress of similar pupils nationally (as in the PA model) in similar schools (based on free school meals, deprivation, cohort ability).

SX (School Extended) is a full contextual value added model where estimates are based on the progress of similar pupils nationally (PA model factors plus SEN status, FSM status, ethnicity, mobility and EAL) in similar schools (based on free school meals, deprivation, cohort ability)


Guidance: KS1-2 Pupil Value Added Summary

Overall VA ScoreAn overall estimate and actual points score is shown for each pupil. The points score is an average across all KS2 subjects and uses the QCA points scoring system (decimalised level x 6). For example a mid level 4.5 is 27 points (4.5 x 6). Level 5.0 = 30 points and Level 5.5 = 33 points etc.

What does the report show?

KS2 Value Added: This section shows the estimated and actual level achieved for each subject along with the probability of achieving Level 4+/5+. The Levels are shown in decimal form (based on marks). As an example, a level 4.0 to 4.33 is equivalent to a 4C whilst a level 4.67 to a 4.99 is equivalent to a 4A.

A pupil’s level is highlighted where a pupil’s actual performance is half a level or more above (green) or below (blue) the estimated level (based on the progress of similar pupils nationally). This is not the same as statistical significance. It is simply a highlight to draw the readers attention to potentially high/low value added.

In the example report , Keira Neon looks to have made excellent progress in Maths compared to similar pupils nationally. Her actual performance (5.4) was more than half a level above her estimated performance (4.3) with her actual level highlighted in green.

By scanning horizontally, you can also see if this pattern is consistent across subjects for individual pupils. In Keira’s case, she looks to have done well in both English and Science. However, although it in both cases her actual level is above her estimated level (based on the progress of similar pupils nationally), the difference is less than half a level and so neither is highlighted in green.

It is important to remember that the half a level or more difference is not a significance test and is simply an arbitrary level.

KS1 NCSS – National Curriculum Standardised Score

Based on an average of KS1 test results and month of birth (age standardised). Other things being equal, the higher the KS1 average the higher the standardised score. Where 2 pupils have the same KS1 average, the younger pupil (according to the month of birth) will have a higher age standardised score.

The standardisation of results was carried out in 1998 using a a mean of 100. Since then the national mean has increased above 100 as test/TA outcomes have improved. The national mean for the Year Group is shown at the bottom of the report.

Prior attainment Group: L (Lower), M (Middle) and U (Upper) Groups are based on an average of Test and TA prior attainment results. Pupils in the Lower group, for example, are in the lowest third nationally based on prior attainment. Pupils in the Middle group are in the middle third and those in the Upper group are in the top third nationally based on average test/TA prior attainment results.

Step 3: ‘Drilling down’ - KS5 Subject EstimatesExample: KS1-2 Pupil Value Added Summary


Activity 3: Using the KS1-2 Pupil VA Summary Report


Keeping a Balance

A school might wish to emphasise CVA: As the closest approach to eliminating factors outside of

the school’s control The fairest way to evaluate school effectiveness

A pupil will be more concerned with ‘What did I attainment?’ than with ‘Did I make good progress?’

Which will matter most to a potential employer?

Raw Scores Progress (VA)

Effectiveness (CVA)

AND not OR


PART 2Estimates, Predictions and Target Setting

Improving target setting in schools: Using FFT estimates


Estimates, targets, predictions

Predictions

Estimates

Targets

Deciphering the vocabulary:


Estimates, targets, predictions

ESTIMATE + PROFESSIONAL KNOWLEDGE

PREDICTION

PREDICTION + CHALLENGE

TARGET!TARGET!

FFT data will only ever give you an estimate.It is not a replacement for targets or target setting!


Why do FFT include these factors when producing pupil estimates?

For pupils, we know that in general: KS2 attainment is highly dependent on attainment at KS1 Girls make different progress than boys Autumn born pupils have higher attainment than Summer born

pupils Pupils’ prior-attainment in English often has a greater impact on

subsequent progress

but we also know that School Context has an impact on pupil progress:

Pupils from disadvantaged backgrounds tend to make less progress (geodemographic data)

Prior attainment for the cohort can have an impact on future achievement


What does FFT include in their estimates for pupils and schools?

Pupil Factors

Test Levels (Fine levels)

Teacher Assessment Levels (where available)

Variation between subjects

Gender

Month of Birth

School Factors

Average Test level for whole cohort

Spread of Test Levels for whole cohort

FSM Entitlement (Percentile Rank)

Geodemographic Data (Percentile Rank)


Estimate Models - Overview

PA - Type A (estimate based on similar pupils nationally)

Prior Attainment

Gender Month of Birth

There are 2 basic types of estimates:

School Context

SE - Types B (50th percentile) and D (25th percentile)

(estimates based on similar pupils AND school context)

GenderPrior

AttainmentMonth of Birth








If only life were this simple!

Estimate

Year 2 Year 6

Att

ain

men

t


It’s more like this:

Year 2 Year 6

Att

ain

men

t

Estimate

Guidance: KS2 Pupil Estimates (Subject) Report

Est Basis: There are 2 Basic estimates – PA (based on Prior Attainment, gender and month of birth) and SE (School Extended) based on PA plus school context factors such as FSM, deprivation and cohort ability. Select All to view both estimates at the same time.

KS2 Estimates: The report shows estimates for each individual pupil and subject level from 2 to 5+. In this example, Tina Argon has a 69% chance of achieving a Level 4 and a 25% chance of achieving a Level 5 in English. The probabilities are based on the progress of similar pupils nationally in the previous year (i.e. 25% of pupils similar to Tina achieved a Level 5 last year). Level 5 is highlighted in Green as it is within the top 25% of levels achieved achieved by similar pupils last year Level 4 is highlighted in orange indicating that it is the ‘most likely’ or highest probability level (69%).

Showing estimates as probabilities rather than a single estimated level allows users to see the whole range of levels that similar pupils have achieved in the past. The reports can then be used to set both challenging and realistic targets. Probabilities can also be more empowering for students and teachers. A 20% probability of achieving Level 4+ reminds us that last year, 2 in 10 similar pupils did actually achieve this level!

Highlight Top: Select options for colour coding grades within the top 5-25%

Coloured boxes (Linked to ‘Highlight Top option’ above):

Orange highlight – shows the Level with the highest probability (orange).

Green highlight - shows the Levels achieved by pupils within the TOP 5%,10%,15%,20% or 25% (user option)

Estimate Rank: This option is used in conjunction with the Estimate Basis (PA/SE) and allows users to select a rank from 50 to 5.

Selecting PA (Prior attainment) and 50, for example, will produce estimates based on the progress made by similar pupils (prior attainment, gender and month of birth) in schools at the 50th percentile for value added. This is how Type A estimates are produced.

Selecting an est basis of SE (School Extended) with a rank of 50 will produce Type B estimates and changing the rank to 25 will produce Type D estimates (progress of pupils in schools at the 25th percentile for value added). The higher the rank (e.g. 5), the more challenging the estimates will be.

Subject: Select individual subjects (English, Reading, Writing, Maths, Science) or ALL to see all subjects on one report.

Level 4+ Summary: Calculated by adding the Level 4 and 5+ probabilities together. If the figures do not match exactly this will be due to:

a) rounding, or

b) minimum and maximum probabilities are set at 1% and 99%. This is to avoid the situation whereby pupils appear to have no chance of achieving a particular grade or are 100% certainties to achieve a particular Level.

KS2 Estimates: The report shows estimates for each individual pupil and subject level from 2 to 5+. In this example, Toby Chromium has a 72% chance of achieving a Level 4 and a 20% chance of achieving a Level 5 in English. The probabilities are based on the progress of similar pupils nationally in the previous year (i.e. 72% of pupils similar to Toby achieved a Level 4 last year). Levels 5+ is highlighted in Green as it is within the top 25% of levels achieved achieved by similar pupils last year (this can be changed using the Highlight Top menu above). Level 4 is highlighted in orange indicating that it is the ‘most likely’ or highest probability level (72%).

Showing estimates as probabilities rather than a single estimated level allows users to see the whole range of levels that similar pupils have achieved in the past. The reports can then be used to set both challenging and realistic targets. Probabilities can also be more empowering for students and teachers. A 20% probability of achieving Level 4+ reminds us that last year, 2 in 10 similar pupils did actually achieve this level! What support will pupils like this need to attain Level 4 or higher?

Example: KS2 Pupil Estimates (Subject) Report


Activity 4: Using the KS2 Pupil Estimates Report

Using Estimate reports with pupils (KS2 example)

Using reports appropriately and effectively

Probabilities are included for all levels from 2 to 5+.

The colour coding shows the most likely grades within the top 5,10,15, 20 or 25% (as selected by the user) in green and the grade with the highest probability in orange.

When looking at potential progress for individual pupils (or a group of pupils) the estimates and colour coding can be used to support the process of target setting but should not take the place of target setting.

Take the example of Nina Caesium to the left. Think for a moment about what the information is telling you. Whilst her highest probability level in English is a 4 (73% and highlighted in orange), the estimates are telling you that last year, 19% of similar pupils (nearly 1 in 5) attained level 5 (shaded green as these grades are within the top 25%). With support, could Nina be one of those 5 or could she actually attain a level 5 quite easily? It’s rarely as simple as just using the ‘highest probability’ or ‘most likely’ grade!

The data shows that there’s a chance that Nina may be able to attain far more but what other questions would you need to consider when setting a target for Nina?

• What do you think Nina could achieve?

• What have similar pupils in your school achieved in the past?

• What are your aspirations? What are Nina’s aspirations?

• What other data is available to help you set an ambitious but appropriate target for Nina?

• What additional work would Nina need to do to achieve a level 5?

• Which areas of the curriculum is Nina strong/weak in?

• What additional resources would be required to ensure that Nina (and similar pupils) have a chance of achieving a level 5 in English?

Remember, use the reports ALONGSIDE other data, your own professional judgement and aspirations AND THE ASPIRATIONS AND MOTIVATIONS OF CHILDREN THEMSELVES!







Example: KS2 School Estimates

Estimates OverviewEstimates are now shown as a range from lowest (on the left hand side of the report) to highest (on the far right hand side of the report). However, the order of the estimate types themselves may vary for individual indicators and pupil groups.

Estimates based on national progress (Type A, B and D estimates)Estimates based on the historical progress of similar pupils (Type A estimates) in similar schools (Type B and D estimates) nationally last year.

Estimates based on your LA’s/school’s own Value added (Green box)

This estimate is a good starting point for target setting discussions at school or LA level. The estimate itself is based on how similar pupils (based on prior attainment) have performed in YOUR school on average over the past 3 years. The estimate is shown in a green box and its place on the report is dependent upon where the estimate sits in comparison to other estimates.

High LA/school value added in the past would be reflected by a higher estimate which would appear at or towards the right hand side of the report.

Low value added in the past would be represented by a lower estimate which would appear at or towards the left hand side of the report.

In this example, the data suggests that the progress of lower prior attaining boys and girls has been lower than the progress of similar pupils nationally (with green box estimates of 32% and 30% appearing at the left hand side of the report.

Pupil Group: Options to show pupil groups based on FFT criteria (default) or DSCF Pupil Groups used for LA SALTS Target setting.

Key Stage: Select KS2,3 or 4 estimate reports where available

Group Size: Only show Pupil Groups with a minimum of 1, 2 5 or 10 pupils. Default setting is 5.

Group Size is based on ‘matched’ pupils only (i.e. pupil’s where prior attainment results can be matched with the most recent results). When using this report be careful if there are large differences between the total number of pupils in a group and the number of matched pupils used in the report


Example: KS2 School Estimates


Activity 5: Using the KS2 Summary Estimate Report

FFT Data Analysis Project

FFT Live Administration ToolsFFT Live Administration Tools




Schools can now access FFT Live in a 4 ways:Schools can now access FFT Live in a 4 ways:

1.1. SCHOOL ADMINSCHOOL ADMIN account supplied by the LA (e.g. account supplied by the LA (e.g. 9992000Admin9992000Admin) or ) or created by a school itself (e.g. created by a school itself (e.g. ASmit9999ASmit9999) – max 2 per school) – max 2 per school

2.2. SCHOOL USERSCHOOL USER account (e.g. account (e.g. ASmit9999ASmit9999). An FFT Live school admin ). An FFT Live school admin account holder can create as many user accounts as needed for staff in account holder can create as many user accounts as needed for staff in schools (e.g. teachers).schools (e.g. teachers).

3.3. GENERIC SCHOOLGENERIC SCHOOL account (your school number – e.g. account (your school number – e.g. 99920009992000). Using ). Using this account you can access school and group level data. The account is this account you can access school and group level data. The account is supplied by your LA.supplied by your LA.

4.4. GENERIC PUPILGENERIC PUPIL account (your school number plus ‘P’ – e.g. account (your school number plus ‘P’ – e.g. 9992000P9992000P). ). Using this account you can access school, group AND PUPIL level data. Using this account you can access school, group AND PUPIL level data. The account is supplied by your LEA.The account is supplied by your LEA.


FFT Live 3.0 Administration Utilities

What modules are available to School FFT Live Administrators?

All Administration tools are available from the Admin Tools menu under Manage Users.

Manage Users

- Add User- View/Modify All Accounts- View Account Statistics

Schools can now create and administer their own FFT Live accounts using the school administration facility



The Add User module allows School FFT Live Administrators to create new school user accounts or an additional Administration account (maximum 1 additional administration account per school). The process is carried out using a simple 2 step wizard. Once you’ve created a new account, FFT Live also gives you the ability to modify it using the View/Modify All Accounts module.

Adding a User



The View/Modify All Accounts module allows School FFT Live Administrators to view details for all accounts on a single sheet. Administrators can also change account settings, modify accounts and export account details – useful when securely providing usernames and passwords for new users.

Viewing and Modifying All Accounts



The View Account Statistics module allows School FFT Live Administrators to view login statistics for all available accounts. This can be useful for analysing use by school staff, to find users having problems logging in (due to incorrect passwords) or to check security (where a user is concerned that someone else may be using their account for example).

Viewing Account Statistics


FFT Live 3.0

Using FFT Live accounts in school – First 6 Steps

1.1. Your Headteacher should receive details of your FFT Live Your Headteacher should receive details of your FFT Live School Admin Account (username and password) from your School Admin Account (username and password) from your LA.LA.

2.2. The Headteacher should nominate 1 member of staff to be an The Headteacher should nominate 1 member of staff to be an FFT Live Admin account holder and then provide them with FFT Live Admin account holder and then provide them with the username and password. No other members of staff the username and password. No other members of staff should have access to this account.should have access to this account.

3.3. When the nominated Admin user first logs in to the account When the nominated Admin user first logs in to the account they should add their personal details (name, job type, email they should add their personal details (name, job type, email address) using the address) using the MODIFY USERMODIFY USER feature in the feature in the VIEW/MODIFY ALL ACCOUNTSVIEW/MODIFY ALL ACCOUNTS module module


FFT Live 3.0

Using FFT Live accounts in school – First 6 Steps

4.4. A school can create one extra FFT Live Admin accounts A school can create one extra FFT Live Admin accounts itself. We recommend that the Headteacher should also itself. We recommend that the Headteacher should also nominate a 2nominate a 2ndnd member of staff to be an additional FFT Live member of staff to be an additional FFT Live Admin Account holder (the existing FFT Live Admin account Admin Account holder (the existing FFT Live Admin account holder can create one other Admin account in a school)holder can create one other Admin account in a school)

5.5. The Headteacher should then consider which members of The Headteacher should then consider which members of staff require access to FFT Live and ask the FFT Live Admin staff require access to FFT Live and ask the FFT Live Admin account holder to create these accountsaccount holder to create these accounts

6.6. All account details should be passed SECURELY to All account details should be passed SECURELY to individuals (e.g. face to face, through SECURE email system, individuals (e.g. face to face, through SECURE email system, via 2 letters – one with username, the other with password)via 2 letters – one with username, the other with password)

Documents

FFT Data Analysis Project © Fischer Family Trust, 2009 Using FFT Live 3.0 How FFT data can help your school improve Primary Schools (KS2) Fischer Family