Using Data to Improve Student Learning

8/6/2019 Using Data to Improve Student Learning

1/12

Victor ian Institute of Teaching

RESEARCH DIGEST2008/3

What does research

tell us about how

schools and teachers

can use data to

inform teaching

in ways that lead

to improvements

in student

achievement?

ThisResearchDigest is focused on studies that have

investigated how data can be used in schools to examine

teaching practices in order to improve student learning.

A selection of relevant websites is listed, and a full reference

list is provided. Links to those references for which full-text

online access is freely available are also included.

School systems, principals and teachers have access to anextensive range of data that can be used for a variety of

purposes. Accountability processes and data have come to

play a significant place in policy development and reform

effor ts. There is a large body of research about the use of

data for improvement at the system level. The key question

addressed by this digest is What does research tell us about

how schools and teachers can use data to inform teaching in

ways that lead to improvements in student achievement?

The digest draws on searches of a number of databases and

bibliographic resources, including the Australian Education

Index, Education Resources Information Centre (ERIC,)Education Research Complete, British Education Index

and Scopus.

The Research DigestsThis Research Digest is one of a series of periodic digests produced

by the Australian Council for Educational Research (ACER) for the

Victorian Institute of Teaching. The digests will be published on the

Institute website under Publications.

Each digest focuses on a single topical issue, and provides a review

of major messages from research on the issue. A key feature of thedigests is an emphasis on what the research means for teachers and

teaching. Over the course of several editions, a wide range of issues

will be covered, so that teachers from different areas of schooling will

find topics of particular relevance to their needs.

Previous Issues

2008/1 Writing to learn

2008/2 Managing student behaviour in the classroom

Student LearningUsing data to improve

IN THIS EDITION

Data in schools 2

Understanding andinterpreting data 3

Purposes for using data 4

Does performance feedbacklead to improvement? 8

On a large scale 9

Comment: Using datain classrooms 10

Useful websites 11

References 12

The Victorian Institute ofTeaching has commissioned theAustralian Council for EducationalResearch to prepare this series ofelectronic research digests.

This digest has been preparedbyMarion Meiers, SeniorResearch Fellow and JenniferTrevitt, Librarian, Cunningham

Library, ACER.

This Research Digest is available in a PDF

version on the Victorian Institute of Teaching

website at: htt p://www.vit.vic.edu.au/


2/122

Using data to support learning has

recently become a matter of significant

interest in schools.

Earl (2005) suggested some key reasons why this is so.

In the past several decades, a great deal has changed. The 21st

century has been dubbed the information age. There has been an

exponential increase in data and information, and technology has

made it available in raw and unedited forms in a range of media.

Like many others in the society, educators are trying to come to

grips with this vast deluge of new and unfiltered information, and

to find ways to transform this information into knowledge and

ultimately into constructive action.

Accountability and data are at the heart of contemporary reform

efforts worldwide. Accountability has become the watchword ofeducation, with data holding a central place in the current wave of

large-scale reform.

Technological advances have led to data from external

assessments being made accessible to schools in a range of

ways so that data can be used meaningfully. Schools also have

access to a wide range of formal and informal data generated

in the course of everyday teaching.

Data can be used to generate change. Earl (2005) points out

that educational change depends on collaborative professional

learning . She reminds us that

Becoming inquiry-minded and data literate are major changes in

practice that are consistent with the notion of professional learning

communities and that warrant concerted attention to new shared

learning.

Matters (2006) in a review of a series of papers focused on

research about the use of data to improve learning, describes

how data shape the landscape of our professional lives. She

highlights recent changes in data use, suggesting that increased

accessibility opens up wider possibilities for using data in many

contexts, including classrooms.

In the not-too-distant past, educational data were slow to

turn around, unwieldy to manage, and too disparate to enable

meaningful comparisons to be made between groups or over

time. Today, owing to advances in computing and communications

technology, the widespread use of data for decision making is

practical and possible at every level of the education system

from students and teachers to parents and school administrators

to stakeholders and policy makers.

Matters (2006) drew from a categorisation of sources of data

provided by Earl (2005) to identify potential sources of data

about student achievement:

standardised, norm-referenced, criterion-referenced tests

questioning in classperformance and standards-based assessments

teacher-made tests, projects, quizzes

teachers observations

student work.

schools

Data in


3/12

interpreting data

Understanding and

The availability of this extensive range of data poses challenges

for schools and teachers, not least in understanding and

interpreting data. Work done in the Western Australian project

that has come to be known as the Data Club has focused on

representing data from state-wide assessments in ways that

help schools. Schools own data has been the focus of this

project. The developers of the Data Club indicate that

There is evidence that the best support for interpreting data is a

combination of information about data analysis and representation

strategies together with application of this information to

participants own data. (Wildy, 2003)

In New South Wales, another example of suppor t provided

to schools to use data effectively is found in the School

Measurement, Assessment and Repor ting Toolkit (SMART) that

facilitates the gathering, monitoring, analysis and reporting of

data in NSW public schools.

Smith (2005) describes the vast store of information availablewithin the system and its schools on academic, social and

affective student outcomes; and the vital impor tance of

providing suppor t to schools to use this data.

Accessing, managing, analysing and interpreting this store

of information are tasks fundamental to the success of the

Department, its schools, and for high quality provision for the

students in their care. Significant challenges have been overcome

through the development of state-of-the- art information and

communication systems (ICT) that bring complex data to the

finger tips of staff in schools and regions in highly usable forms.

The Victorian Curriculum and Assessment Authority (VCAA)

offers a Data Service to schools. This service provides data

about students results in the Victorian Certificate of Education

to schools in ways that help school staff to use the data

effectively and easily for planning improvement.

Charts and tables provided to schools through this service can

address a range of key questions:How were our schools overall results this year?Were

they better or worse than in previous years?How do they

compare with schools like ours?

How did our students perform, study by study in terms

of completions, Study Scores, examinations and school

assessment grades?

How did our schools results compare to reasonable

expectations?Did our students perform as well as students

of comparable ability in other schools?

Is our student cohort changing over time, in ability and/or

achievement?How can we develop better understandings of the patterns

of group performance by identifying how individuals

contribute to those patterns?(Rowley, 2005)

These services are interesting examples of ways in which

performance feedback from large scale assessment has been

provided to schools, so that it can inform the future planning of

classroom programs.

3


4/124

System leaders, principals and teachers all use data, for

overlapping yet different purposes. For teachers, the central

purpose of analysing data is described by Allen (2005) as

being to

improve the learning of one or more particular students. That

is, the individual teacher and the school take the students who

come to them and seek to improve the learning of those students.

This purpose is different from that of the sociologist seeking to

understand patterns of participation, or that of the policy analystseeking to understand the impact, if any, of policy settings.

Interest in the uses of data has accelerated, as, increasingly,

schools receive performance feedback from a variety of

sources. Hattie (2005) notes that schools are awash with data.

He argues that the discussion about using data needs be

located in the classroom, and to move

away from data towards interpretations, from student

outcomes to teaching successes and improvements, and from

accountability models located in schools to located first in the

classroom to support such evidence-based teaching and learning.

By locating evidence in the classroom we can influence the

major agent that influences student and learning the teacher,

can highlight the debate about what is worth teaching, and most

importantly, can begin to establish a teacher-shared language

about the achievement progression (Hatt ie, 2005).

Research about what makes a difference to teaching and

learning indicates that what students bring to learning accounts

for 50% of the variance in students achievement. Teachers

account for account for 30% of the variance, with home, peers,

schools and principals making up the remaining 20%. (Hattie2003) In further research, Hattie (2005) emphasises the

impor tance of teachers using data:

The reason for locating the power of data to enhance student

outcomes at the teacher level comes from the many recent

studies on the epicentre of causal effects on learning: the teachers.

(Hattie, 2005)

Studies at the school level provide insights into how teachers,

suppor ted by school leaders, use data for the purpose of

improving learning. Much can be learnt from these studies.Three examples illustrate this.

In the first example, a recently published case study (Waddell,

2008) from an elementary school in the US, provides

insights into how an ongoing review of summative student

achievement data collected over three years transformed

teaching practices and closed the achievement gap for

students.

By locating evidence in the classroom we can influence the major agent that

influences student and learning the teacher

using data

Purposes for


5/125

A second, school-level study is described by Timperley (2005)

who reported on a study of a New Zealand school where the

assistant principal:

wanted to use student achievement information during team

meetings as a catalyst for professional learning about improving

the delivery of literacy programs. Over the time of the study,

the teachers shifted from initial beliefs that achievement wasdetermined primarily by outside influences, and therefore, the data

were irrelevant to their practices, to using data to improve their

practice. Indicative measures of student achievement showed that

it improved as a result of their efforts.

In this school, located in a poor suburban area, ninety-five per

cent of children enrolled were Maori or from one the Pacific

Islands, groups that traditionally underachieve in New Zealand

schools. The assistant principal and the teachers collected a

range of literacy achievement data in the early years, including,

for example, running records and reading tracking sheets, and

the Observation Survey (Clay 2002). The assistant principal

had considerable expertise in using data to plan programs to

meet student needs, but she observed that the teachers were

not using the available data for teaching purposes. I actually

dont see it in their planning. I dont see that it informs their next

steps. (Timperley, 2005)

She began a process of professional learning in a team meeting

by giving teachers the Observation Survey data taken from herrecords and showed how she had started to graph the results,

which indicated that the students were not progressing at a rate

expected for their age. (Timperley, 2005) This process continued,

with the assistant principal helping teachers to make links

Data as an impetus for examiningpractice: one schools journey

At Viewmont Elementary School in North Carolina the

incoming principal noticed that the data from state

assessments showed that some groups of students were

failing to meet state benchmarks, and the achievement gapbetween the highest and lowest performing students was

around 40%. The decision was made to change this situation.

Over three years the school worked towards becoming a

professional learning community. They began by reviewing data

from current and previous years, which showed that what

was happening in classrooms did not benefit all students.

The school set the goal of establishing a focus on data-driven,

collaborative professional work. Dialogue and data were key

words, and data became the impetus to examine practice.

Staff became engaged in conversations about data and

teaching practices in small- and large- group meetings.

By the end of the first year, teams had made presentations

to the whole staff, and it was recognised that there was a

need to change instruction to be more responsive to the

needs of all students. In the second year, staff had developed

a deeper understanding of literacy instruction, and a model

of peer coaching had been established. Significant changes

in instructional practices had been made by the end of the

second year by many, but not all, staff members. The analysis

of data at the end of this year showed that the achievement

of students overall had increased, and that there were largespikes in the achievement of lower achieving students.

The evidence provided by the data convinced teachers who

had resisted the changes, and school wide implementation

commenced. Support was continued in the form of half-

day meetings, model lessons, peer coaching and whole-

group discussions.

By the end of the third year, the data validated the work

of the principal and the whole staff. All students continued

to perform well, and the achievement gap had closed

from 40% to less than 10% over three years. The staff scommitment to reflection, research and professional growth

had become embedded in the schools daily work, and much

had changed. (Waddell, 2008)

A characteristic of good data is its potential

to help teachers make good decisions aboutchildrens learning. Data tell a story.

The two questions uppermost in

teachers minds should be: What does it all

mean and; how can we use it to improve

childrens achievements?


6/126

between the skills and understandings of the students, and

teaching practice.

As the project continued, gains in student achievement were

evident. Teachers described gains associated with changes in

teaching practices, and commented on the achievements of

some students:

... one student who had written nothing the first time had 27words the second time. (Timperley, 2005)

A year after the focus on using data to plan teaching, teachers

described how they had been working together:

You can identify where you need to put more effort in, We all

support each other we ask, Hey what are your doing to get

yours [text levels] up, and What do we need to do? (Timperley,

2005)

Timperley (2005) teases out the issues that, in this context,

needed to be addressed simultaneously to support teachers touse the data to change practice:

identifying the knowledge and skills required to use

achievement data;

setting up a situation whereby the data on initial

improvement in students achievement challenged the

teachers initial assumptions, and teachers came to realise

that the achievement data related to teaching practices;

providing the opportunity for the teachers to make explicit

how they might teach differently to improve achievement.

The third example relates to work in numeracy teaching.

It is clear that data tell stories for all; the child, the teacher,

the school, the parents and the nation. Without informed

analysis or in fact interrogation of data and a precise

understanding of the story it tells, little or moderate impact

will be made on childrens achievement. The data provides

the base for constructive management of the implicit

complexities of teaching in a dynamic classroom. (Tozer &

Holmes, 2005)

using data

Purposes for


7/127

A characteristic of good data is its potential to help teachers

make good decisions about childrens learning. Data tell a

story. The two questions uppermost in teachers minds

should be: What does it all mean and; how can we use it to

improve childrens achievements?(Tozer & Holmes, 2005).

These comments come from an account of New Zealand

work in using numeracy interviews to provide the teacher with

immediate and detailed information about childrens number

knowledge and mental strategies and allowing behavioural

observations to be made in response to oral questions. This

work demonstrates a process of data gathering in classrooms,

and leads to conclusions such as:

The diverse range of childrens numeracy needs as evidenced by

the data cannot be ignored. The challenge is for the classroom

teacher to plan a classroom programme in response to the

highlighted strengths and weaknesses in student knowledge and

strategies. (Tozer & Holmes, 2005)


8/128

improvement?

Does performance feedback lead to

A paper from the National Foundation for Educational

Research (NFER) repor ts on a study which investigated

whether this kind of feedback is useful to schools, and whether

it leads to improvement in schools. The results of the study,

in general, suggested that feedback did have an impact on

performance. The study was focused on primary schools that

had received feedback from NFER projects and found that

school receiving feedback had better attainment. It appears

that supplying feedback to schools is a positive approach. This

method should be encouraged, but support systems for schools,

teachers, policy maker and practitioners should be widely available.

This would enable them to make the most of the rich data that is

available to them in todays society. (Hammond, 2007)

Visscher and Coe (2002) reviewed the available literature on

the impact of performance feedback in schools, and, despite

finding that its effects are extremely complex and not well

understood, identified a number of tentative suggestions for

optimising the beneficial effects of performance feedback.

The main requirement is that feedback should direct attention

to an achievable gap between desired and actual performance.

(Visscher 2002)

Recognition of the role of assessment for learning has been

significant in recent years. Research on formative assessment

has indicated that there is clear evidence that formative

assessment raises standards and that current practices are weak.

(Black, Harrison, Lee, Marshall & William, 2003)

The OECD Centre for Educational Research andDevelopment (CERI) published a study analysing the formative

approach to assessment in eight countries, including Australia

(OECD, 2005). Case studies of schools indicated how teachers

used formative assessment as a framework for teaching and

learning. Key elements were found to be the use of varied

teaching methods and varied approaches to assessing students

understanding, in recognition of the diversity of students needs.

The classroom culture in these schools encouraged interaction,

and students were actively involved in learning. Learning goals

were clearly established, and were used to track students

progress. Feedback to students was critical, together with the

ongoing adjustment of teaching and learning activities to keep

pace with students learning.

assessment may also serve a formative function. In classrooms,

formative assessment refers to frequent, interactive assessments of

student progress and understanding to identify learning needs and

adjust teaching appropriately. Teachers using formative assessment

approaches and techniques are better prepared to meet diverse

students needs through differentiation and adaptation of

teaching to raise levels of student achievement and to achieve

greater equity of student outcomes. (OECD, 2005)

The main requirement is that feedback should

direct attention to an achievable gap between

desired and actual performance.


9/129

On a

large scaleA recent large scale study in England (Kirkup, Sizmur, Sturman& Lewis, 2005) was designed to investigate how data was usedto promote learning in schools, and to identify good practice in

the effective use of data to improve learning. A key finding of

this study was that

Good practice emerged from the use to which the data was put

rather than specific systems or tools. A recurrent theme was thatdata only becomes effective if it stimulates questions about the

actual learning that is taking place and how it can be developed

further. (Kirkup, al., 2005)

This study found that, at pupil level, effective use of data

informs accurate curricular targets for individual pupils

highlights weaknesses in specific topics for classes or groups

highlights specific weaknesses for individual pupils

provides evidence to suppor t decisions as to where to

focus resources and teaching

informs setting and grouping of pupils. (Kirkup et al., 2005)

Using a questionnaire survey and case studies, another English

study by Kirkup (2006) looked specifically at the extent to

which schools and teachers are able to integrate analyses of

summative tests within an assessment for learning approach

to teaching. Indications from the survey were that information

from formal tests was considered very useful by teachers at

the class or year group level, as it could feed into teaching

strategies, planning, grouping and target setting strategies.


10/12

Using data to improve learning poses many challenges.

A significant challenge is that of transforming data into

information which leads teachers to improve the learning of

their students. Axwor thy (2005) identifies some of the lessons

that have been learnt in Western Australia when this challenge

has been addressed. Teachers are willing to look at test data if it gives them

some insight into their students learning of relevant

curriculum outcomes.

Creating a dichotomy between test results and teacher

judgements that privileges one over the other is

counterproductive. Assisting teachers to see how test results

can refine and sharpen their judgements is very powerful.

Presenting data in ways that encourage teachers to take on

a questioning, problem-solving role (scientist practitioner) with

respect to their students learning causes changes in their teachingpractice and results in improvements in student learning.

(Axworthy, 2005)

The research indicates the potential value of using data for

classroom planning. Understanding and interpreting the wide

range of data available to teachers, ranging from external

assessments as well as formal and informal data gathered

from everyday teaching, provides a basis for teaching directed

towards suppor ting all students to make progress from current

levels of achievement.

Collaborative work with teams of colleagues, sometimes

the whole staff, sometimes with smaller groups, suppor ted

by school leadership, builds a context in which teachers caninterpret and use data more effectively. Key questions to be

addressed in planning are: What story does the data tell us?

What does it mean?How can we use it to improve childrens

achievements?Asking and answering these questions helps

teachers to make informed decisions about childrens learning.

Data from their own observations and from many other

sources provides feedback to teachers about the performance

of individuals and groups of students, and teachers can draw

on this data to provide feedback to students. Feedback

improves learning by identifying achievable gaps betweendesired and actual performance. Teaching successes and

improvements in learning thus develop from evidence-based

teaching and learning.

comment

Using data inclassrooms

A significant challenge is that of transforming

data into information which leads teachers to

improve the learning of their students.

Collaborative work with teams of colleagues,

sometimes the whole staff, sometimes with

smaller groups, supported by school leadership,

builds a context in which teachers caninterpret and use data more effectively.

10


11/12

USEFUL WEBSITESTeachers TV Australiahttp://www.teacherstv.com.au/

This site provides professional development videos for

teachers at no cost. For example, there are two videos

on formative assessment featuring Paul Black andChristine Harrision.

American Association of School Administrators. (nd).

Using data to improve schools: Whats working. Retrieved

August 11, 2008 from http://www.aasa.org/files/PDFs/

Publications/UsingDataToImproveSchools.pdf

asTTle

asTTle stands for Assessment Tools for Teaching and

Learning. These are tools for assessing literacy and

numeracy, and can be further tailored for any language.They have been developed for the Ministry of Education

by The University of Auckland and are now being

developed in several other countries through Auckland

UniServices Limited. These tools enable teachers to track

the progress and achievement of both individual students

and groups of students against national standards.

http://www.asttle.com

The website of the Assessment Reform Group in the UK.

http://www.assessment-reform-group.org/

The Assessment for Learning website has been developed

by Curriculum Corporation on behalf of the education

departments of the States, Territories and Commonwealth

of Australia.

http://www.curriculum.edu.au/assessment

How to cite this Digest:

Wilkinson, J. & Meiers, M. (2007). Using data to improve

student learning,Research Digest, 2008(3). Retrieved

Month DD, YEAR, from http://www.vit.vic.edu.au/

11


12/12

Allen, R. (2005). Using the evidence

of student achievement for

improvements at individual, class and

school level. In Using data to supportlearning:Conference proceedings(pp.87-92). Melbourne: ACER. Retrieved

August 11, 2008 from http://www.

acer.edu.au/documents/RC2005_

RegAllen.pdf

Axwor thy, D. (2005). Turning data into

information that improves learning:

The WA experience. In Using datato support learning: Conferenceproceedings(pp. 127-130). Melbourne:ACER. Retrieved August 11, 2008

from http://www.acer.edu.au/

documents/RC2005_DavidAxwor thy.

pdf

Black, P., Harrison, C., Lee, C., Marshall, B.

& William, D. (2003). Assessment forlearning:Putting it into practice. NewYork, Open University Press.

Clay, M. M. (2002).An observation survey ofEarly Literacy Achievement. Auckland,Heinemann.

Earl, L. (2005). From accounting to

accountability: Harnessing data

for school improvement. In Usingdata to support learning:Conferenceproceedings(pp. 6-10). Melbourne:Australian Council for Educational

Research.

Retrieved August 11, 2008 from

http://www.acer.edu.au/documents/

RC2005_Earl.pdf

Hammond, P., Beddow, K., & Yeshanew, T.

(2007). Feedback - does it have any

bite?Practical Research for Education,37, 38-45.

Hattie, J. (2003). Teachers make a

difference: What is the research

evidence?Paper presented atthe ACER Research Conference

2003, Building Teacher Quality:

What Does the Research Tell Us?

Melbourne: ACER. Retrieved August

11, 2008 from http://www.acer.edu.

au/documents/RC2003_Hattie_

TeachersMakeADifference.pdf

Hatt ie, J. (2005). What is the nature ofevidence that makes a difference

to learning?In Using data to supportlearning:Conference proceedings(pp.11-21). Melbourne, ACER. Retrieved

August 11, 2008 from http://www.acer.edu.au/documents/RC2005_

Hattie.pdf

Kirkup, C. (2006). Using assessmentinformation to inform teaching

and learning.Education 3-13,34(2),153-162.

Kirkup, C., Sizmur, J., Sturman, L. & Lewis, K.

(2005).Schools use of data in teachingand learning. (DfESResearch Repor tNo. 671). London: DfES. Retrieved

August 11, 2008 from www.dcsf.gov.

uk/research/data/uploadfiles/RR671.pdf

Matters, G. N. (2006).Using data tosupport learning in schools:Students,teachers,systems. Camberwell: ACER.(Australian Education Review No.

49). Retrieved August 11, 2008 from

http://www.acer.edu.au/documents/

AER_49-UsingDatatoSuppor tLearninginSchools.pdf

OECD (2005).Formative assessment:improving learning in secondaryclassrooms,. Paris, OECD.

Rowley, G. Congdon, P. (2005). Data-dr iven

school improvement through the VCE

data service. In Using data to supportlearning:Conference proceedings(pp.46-57). Melbourne: ACER. Retrieved


GRowleyandPCongdon.pdf

Smith, M. (2005). Getting SMART with

data in schools: Lessons from NSW.In Using data to support learning:

Conference proceedings(pp. 38-457).Melbourne: ACER. Retrieved August11, 2008 from http://www.acer.edu.au/

documents/RC2005_MaxSmith.pdf

Timperley, H. S. (2005). Instructionalleadership challenges: The case

of using student achievement for

instructional improvement.Leadershipand Policy in Schools,4(1), 3-22.

Tozer, L. & Holmes, M. (2005). Moving on

from count me in too: Evidence-based

teaching and learning in numeracy

in the early and middle years ofschooling. In Using data to supportlearning:Conference proceedings(pp.32-37). Melbourne: ACER. Retrieved


LTozerandMHolmes.pdf

Visscher, A. J. & Coe, R. (Eds.). (2002).

School improvement throughperformance feedback. (Contexts ofLearning). Lisse: Swets & Zeitlinger.

Waddell, G. & Lee, G. (2008). Crunching

numbers, changing practices: A close

look at student data turns the tide inefforts to close the achievement gap.

JSD:The Journal of t he Nat ional Staf fDevelopment Council,29(3), 18-21.Retrieved August 11, 2008 from

http://www.nsdc.org/publications/

getDocument.cfm?articleID=1680

Wildy, H. (2003). Data Club: Suppor tingschools to use data.ProfessionalEducator,2(3), 6-7.

REFERENCES

Prepared by the AustralianCouncil for Educational

Research for the

Victor ian Instit ute

of Teaching

Documents

Using Data to Improve Student Learning