• 1 •

SHARING LESSONS from the fieldQUARTERLY NEWSLETTER DECEMBER 2016

Assessment for learning and the Girls’ Education Challenge

© C

fBT

Over the last few years, culminating in the adoption of Sustainable Development Goal 4 (“Ensure inclusive and quality education for all and promote lifelong learning”), the international community has been grappling with the pressing need to be able to better measure progress in learning. At the same time, the Girls’ Education Challenge (GEC) has been working to rigorously assess the changes in girls’ learning across the programme.

Assessment of learning – where progress is assessed at key points during the teaching and learning process – across such diverse and difficult environments, from primary to upper secondary and using a range of different assessment types, has been a rich source of learning for all concerned.

But the current debate addresses a wider problem and is exploring how assessment can in some cases also be used for learning. This quarter’s newsletter acknowledges the growing trend across many of the countries we are working in to consider assessment for learning. It draws together

general feedback from GEC projects and a recent survey of them on the types of learning assessments that are being carried out in the schools and learning centres with which they are working. It also looks in greater depth at three projects which have implemented different forms of learning assessment.

• 2 •

within an effective assessment system: one that can collect, analyse and use the data generated to improve their education system. But a recent GPE report found that only 19 out of 60 GPE developing country partners had a learning assessment system that was sufficiently established to monitor learning over time – so these too tend to be summative and are not generally used to improve the teaching and learning process.

Formative assessmentInside the classroom, there is also opportunity for formative assessment for learning, and many teachers use this as an integral part of the day-to-day teaching and learning process. The goal of formative assessment is generally to provide ongoing feedback that can be used by instructors to improve their teaching, and by students to improve their learning. It involves a range of activities such as: • Planned questioning during the

teaching process around important points to gauge understanding and to encourage a culture of thinking and readiness to answer (as well as one of listening to and not punishing a wrong answer). This questioning could be verbal or written, and could use for instance techniques such as quizzes, essays, tests, or assignments

• Effective feedback from the teacher to students on what they have achieved and where improvement is necessary, and specific suggestions for that improvement.

• Use of intuitive software resources which assesses a child’s strengths and weaknesses as they complete tasks and focuses future tasks on areas of weaknesses

Summative and formative assessment

There are two main types of assessment: summative (roughly equated to assessment of learning which demonstrates a learner’s success at meeting assessment criteria) and formative (roughly equated to assessment for learning which involves giving and applying feedback to students and teachers and is intended to improve the teaching and learning process). According to experts, education systems require both to varying extents.

Summative assessment

The predominant form of assessment in the countries within which the GEC is working is summative. The goal of this type of assessment is to evaluate student learning at the end of an instructional unit by comparing it against some standard or benchmark. The results of summative assessments are often recorded as scores or grades that form a part of a student’s permanent academic record. The girls that the GEC works with are familiar with summative assessments such as end-of-year tests, or end-of-primary leaving examinations that will determine their eligibility for secondary places. In most cases, these test the knowledge of pupils against the curriculum that they should have covered. If a girl is unable to pass the end-of-year examination then, in many contexts, she will be retained in her present year, delaying her progress and costing her family an extra year (or more) of school fees and associated costs. If she fails her primary examination, she cannot gain entry to secondary level schooling. And in many countries, since places in secondary are scarce, then she will not only have to pass the end-of-primary examination but also be within the top 30% of pupils, in order to have any chance of securing a secondary school place. Education systems within which we are working are also involved in other regional or international assessments. These assessments are designed to test the system and highlight achievement and concerns so that policy can be adjusted accordingly. Based on a sample of pupils, they return a snapshot of learning across a range of school levels in a number of subjects, although predominantly reading and mathematics. To be effective these type of assessments have to be used

Interestingly, it is important to acknowledge that the distinction between formative and summative often proves fuzzy in practice for a variety of reasons. Two elements in particular make differentiating between the two challenging at times. Firstly, summative assessment can also be used in a formative way. At its simplest, this could be a teacher discussing the outcomes of a test that determines a student’s final grade with the student, outlining where they went wrong and how they could have improved. But it could also be the systematic analysis of results to identify particular areas of difficulty and to develop strategies to overcome these. Secondly, it is possible for summative and formative assessments to appear similar. Quizzes, monthly tests, essays, or assignments can, for instance, all be used in both a summative and a formative manner. However, what makes an assessment formative as opposed to summative is the way it is used—a summative assessment would be used to evaluate performance at the end of the teaching process, while the formative assessment would be used primarily to inform in-process teaching and instructional practice. Most classroom assessment in GEC contexts is summative – measuring a learner’s progress at the end of a specific period. In fact, testing done to assess success of projects more broadly at the GEC through, for instance, assessment systems like EGRA/ EGMA or ASER/UWEZO, is also summative.

© M

aths

-Whiz

z

• 3 •

The use of assessment to improve and accelerate learning can be limited if teachers are not trained or encouraged to use this approach and/or due to difficult classroom conditions such as overcrowding, exam pressure and a packed curriculum which limit the time to integrate it into teaching methods.

“Our survey says…” A recent survey was administered to the staff of GEC projects to understand the various assessments learners were exposed to during the course of their project. Across the GEC, survey responses show limited evidence of the use of formative assessment within classrooms. 94% of the survey respondents reported that their beneficiaries undergo some kind of in-classroom assessment. The frequency of these classroom assessments ranged from daily to annual assessment. However, only four out of the 21 respondent projects reported the use of frequent in-class formative assessment. Camfed Zambia reported that this regular formative assessment was a key part of their intervention (see Case Study below). Link Community Development and BRAC Afghanistan reported that teachers participating in their projects conduct daily in-class assessment as part of instruction to identify problem areas for students and help teachers understand focus areas in teaching. In the project led by the Varkey Foundation, teachers organise weekly mini-plenaries in class for the same purpose. Two projects reported the use of slightly different forms of assessment,

which also appear to have a formative function. CfBT’s Teacher Coaches carry out monthly assessments of a sample of students to assist them in giving feedback to teachers and the VIVA project in Uganda carries out monthly assessments against students’ individual learning plans to assess progress and assist teachers to understand where they should focus their future teaching.Three projects reported carrying out pre- and post-tests to assess student progress in specific components such as remedial or accelerated learning classes; academic and life skills teaching; and consolidation classes (Relief International, ENGINE/Coca-Cola and STAGES). In all three cases, the initial tests given as part of the pre-post regime appear to be used in a formative fashion in order to help teachers identify the concepts students are struggling with (see ENGINE Case Study below). Summative assessments, on the other hand, were much more commonly reported in the survey. While most respondents reported the use of termly assessments (mid-term and end of year exams), some of the projects also indicated the use of monthly assessment tests (I Choose Life and STAGES) or quarterly assessment (STAGES and BRAC Afghanistan). These tended to be formal and structured but were sometimes also used to identify problem areas for students and help teachers understand what areas to focus on in future. Almost all projects reported that their learners are subject to end-of-year exams. These were generally conducted as part of the formal schooling system

and appear to be exclusively for the purpose of determining end-of-year grades and therefore which students will progress to the next grade. In sum, much of the assessment projects reported in the survey appears to be summative and geared either to establish ranking or to identify which students will progress to the next grade. There are a number of possible reasons for this: • Teachers are unfamiliar with the type

of pedagogy that would encourage more frequent formative assessment in the classroom, and have not generally had access to the relevant training.

• Teachers may find that the prevailing culture and pressures within their school system mitigate against using the new skills that they have been taught.

• Pressures of examinations and packed curricula leave insufficient time to go into any one topic area in depth, to check understanding of all pupils and to offer remedial assistance to those that need it.

• There can be low levels of subject matter and pedagogy mastery amongst teachers which would allow them to offer remedial or enrichment activities.

• Overcrowded and poorly resourced classrooms, often with a wide range of age and ability within the same class often mean the teacher has little scope for a more individualised approach.

• Formative assessment is taking place in the classroom but is being done in such an ad hoc and informal way that it is not being recognised or reported by either teachers or projects as an established approach.

© D

FID

• 4 •

The following case studies illustrate the ways in which some GEC projects are using various forms of assessment. They highlight how the results of the assessments have been used, in different ways, to customise learning and teaching in the classroom to student needs, and the impact this is starting to make. Contact details are included with each case study so that you can get in touch with the relevant organisation for more information.

Parents want their children to learn and understand the opportunities that education offers, but are frustrated by poor resourcing, teacher absenteeism, and other contextual challenges. There are significant problems with grade repetition and dropout, which are high across the project schools and higher for girls in upper primary grades. Camfed’s interventions in their project schools include: • Improvement of the school

environment, to create a safe and supportive environment for girls.

• Provision of safety net funds designed to provide small, targeted assistance to pupils at risk of dropping out, managed by the school.

• Training in child-centred pedagogy for upper primary school teachers from 90 schools, as well as establishment of micro-centres for teachers to meet and exchange ideas that are designed to improve teacher motivation and attendance

• Provision of study guides for all upper primary children in project schools.

Assessment

Students have been assessed in three ways in project schools:• Assessment 1 is the annual school

exams, which are set by each school in the core subjects of English, Maths and Sciences.

There is a particular focus in Zambia on developing approaches to assessment to be ‘for’ rather than ‘of’ learning (diagnostic and formative rather than to determine who progresses).

• Assessment 2 is national assessments for English and Maths at the key transition points of Primary 7 (transition to secondary), Grade 9 (transition to upper secondary) and Grade 12 (school leaving).

• Assessment 3 is informal teacher-led formative assessments for all subjects in upper primary, using assessment principles drawn from the child-centred methodology. This includes:

- informal assessment through targeted questioning before, during and after lessons

- observation of group work and individual students reading to the class

- formal (structured) assessments in week 5, mid-term, and end-of-term.

Impact on learning

Early results and qualitative feedback suggest these approaches are increasing student confidence in learning and engaging in class, and teachers are more engaged with

the progress of their classes. The informal, teacher-led formative assessments during the teaching and learning process aim to influence and inform the overall instructional process, by gauging the aspects students have understood well and which they have struggled with. By informally measuring learning on a regular basis, teachers can then cater their class plans to individual class needs. Whether or not the use of formative assessment techniques will translate directly into higher summative scores for the beneficiaries of Camfed Zambia’s project at endline is hard to say at the moment. Catering to the learning needs of students can improve the quality of teaching and is therefore likely to enhance learning results. However, this has to be viewed in the context of the project as a whole and how the intervention has tackled other critical barriers.

For further information, please contact: ksmith@camfed.org

Camfed, Zambia

The Camfed GEC project is being delivered in three districts in northern Zambia: Mpika, Chinsali, and Shiwangandu. These are remote, deeply rural districts. Pupils and teachers face very significant resource and infrastructure constraints and levels of poverty are very high.

© C

amfe

d

• 5 •

Assessment

Every student learns at a different pace. The Maths-Whizz computer based programme simulates the behaviour of a human tutor to identify and support each child’s individual needs. Students complete an initial assessment, which pinpoints their needs across several maths topics (measured by Maths Age; Whizz’s international standard for maths achievement). The Virtual Tutor then develops a personalised and continuously adapting learning journey, filling the student’s specific knowledge gaps and helping them work towards a rounded learning profile. In the following example, one student, Nancy, showed an obvious weakness in Place Value in her placement assessment (the red markers). After two years of dedicated support from the virtual tutor, Nancy now enjoys a more rounded profile; she has improved in her weakest areas (blue bars). The Tutor delivers lessons in sequence based on each child’s needs. Each lesson has a test component, which the Tutor uses to keep track of students’ specific knowledge gaps. It acts on these insights to constantly update their learning journey.

As students interact with the Tutor, it feeds back key learning insight to educational stakeholders at all levels. Students can visualise their progress from lesson to lesson, reinforcing a growth mindset. Teachers can keep track of the spread of learning needs in their class. School leaders can similarly keep track of the progress of individual classes, in terms of their effort (time spent learning on Maths-Whizz) and progress (improvement in Maths Age).

Impact on learning

This model of continuous assessment and real-time reporting is positively reshaping the role of testing in education.• It displaces the snapshot judgements

of summative assessments with a more refined scale that focuses on learning processes.

• It lowers the stakes – and therefore the anxiety – around any individual assessment and removes the temptation of teaching to a single, narrow test.

• It enables a shift from assessment of learning to assessment for learning, empowering teachers with the diagnostic insights they need to understand and meet each child where they are. In practical terms

teachers are only just starting to deploy this capability to inform their curriculum delivery and classroom learning. There is a huge potential yet to be tapped.

Research carried out by Maths-Whizz around the world has consistently demonstrated accelerated learning gains for students on Maths-Whizz. In Project iMlango the added challenges of rural and often marginalised environments are brought to bear. They have seen that the underlying quality of education is such that, prior to implementation, little over half a year of progress in maths is made for each year of primary schooling in iMlango communities. However the continuous assessment data, gained as a bi-product of focusing on learning, shows that:• On average, students require

52 minutes of virtual tutoring each week to progress at a rate comparable to their international peers, representing a 72% increase in learning compared to students in rural Kenya not on Maths-Whizz.

• Continuous improvements to the Maths-Whizz service, informed by real-time data insights that are contextualised by local field teams, continue to drive down these usage thresholds (this year alone they have seen over 15% improvements in progress rates per unit of time on task).

In large-scale implementations, Maths-Whizz relieves stakeholders of the expense and unreliability that comes with sending data collection teams into the field, though there are technology infrastructure requirements which need to be considered ahead of implementation.

For further information, please contact: Junaid.mubeen@whizzeducation.com

Whizz Education, KenyaIndividualised Learning, Continuous Data Collection and Assessment Overview

Whizz Education’s virtual tutoring system, Maths-Whizz employs an innovative model of continuous assessment and real-time reporting to deliver personalised learning at scale. This is being delivered by the iMlango Project in Kenyan primary schools.

© W

hizz E

duca

tion

• 6 •

Through the in-school intervention, 5,700 marginalised girls are enrolled in a 9 month education cycle in one of 228 Safe Spaces. These girls receive academic tutoring for at least two hours per week and financial education and leadership and life skills training. In the out-of-school intervention, ENGINE has enrolled over 13,000 girls in a 9 month course where they receive training in business and entrepreneurship skills and a greater focus on leadership skills and financial education. They also have the opportunity to enter the Coca Cola value chain and/or other business related support.

Assessment

Students participating in ENGINE’s courses are assessed in three ways:1. Formative assessment by the project

at the beginning of the course All students are given formative

assessment tests when they enrol in the programme to gauge their level of academic knowledge. Junior secondary level students are tested in Mathematics, English, Hausa and Basic Science, while senior secondary students are also tested in Mathematics, English, Biology, Physics, Hausa and Chemistry. The out-of-school girls are also tested in business and life skills.

2. Summative assessment by the school In-school students are also tested

on a regular basis – often three times a year – in all subjects mentioned above by their schools. Schools use the results to determine a student’s final class rank. But over the life of the project, ENGINE has worked with school managements to create a system for allowing schools to share the results of their students enrolled on the project.

3. National assessment at completion of junior and senior levels by government

In Nigeria, students sit the Junior Secondary School Examination at the end of JSS 3. Those completing upper secondary school (SS 3) sit the Senior Secondary Certificate Examination (SSCE). Testing is in multiple subjects, including but not limited to Mathematics, English, Physics, Biology, Chemistry, Hausa, Science, Civic Education, History and Geography.

Impact on learning

According to the project, their use of formative assessment has had a positive impact on learning and on teacher reflection and self-assessment. Although they acknowledge that learning for not all girls has improved in some subjects, their internal pre and post-test analysis indicates a strong upward trend. To understand why this is the case, it is important to consider the way the programme uses its formative assessments. • The results of the enrolment tests are

shared with teachers with the core purpose of helping them understand where students need assistance, and to encourage them to focus their teaching on areas in which students are struggling. In practical terms, the results of these tests are discussed not only in monthly review meetings between teachers and the Safe Space lead, but are also considered with project staff during quarterly refresher training sessions. So, for instance, a teacher might highlight through either of these forums that her cohort is weak in Physics or that she herself is unfamiliar with some concepts in Chemistry. Solutions are then discussed in the meetings and adjustments to teaching are made Where teachers themselves are unfamiliar with content, this is addressed through refresher sessions

or by referring the teacher to other teachers in the Safe Space better versed in the subject.

As an example, one student was failing physics when she joined ENGINE’s programme. Over the course of the programme, her physics grade improved dramatically and she is now passing the subject. When asked what had allowed her to improve her grade, she noted two things. Firstly, she highlighted how the ENGINE teacher had not only simplified the subject matter, but had also taught her at her own learning speed. Secondly, she mentioned that the smaller class sizes – up to a maximum of 25 students as compared to over 40 in some government schools – allowed the teacher to respond to her needs in a much better way.

• ENGINE also conducts an ‘exit’ assessment with students. Comparison of the two assessments is conducted by subject and results are shared with key stakeholders. This analysis shapes future programme decisions, particularly around teacher training.

• ENGINE uses the results of student performance in termly school assessments in exactly the same way they use their own formative assessment – teachers cater their teaching to those areas in which students are lagging behind.

Initial endline results for the project appear to support the link to improved learning. At endline, in-school girls performed slightly better than the control group on literacy. On numeracy, in-school girls tested lower than the control group at baseline, but at par at endline, thus indicating accelerated learning during the life of the project. For out-of-school girls, results further indicate improvements in financial literacy.

For further information, please contact: rasani@mercycorps.org

ENGINE, Nigeria

The Educating Nigerian Girls in New Enterprises (ENGINE) project is being implemented in Northern Nigerian and the city of Lagos by Mercy Corps Nigeria and other partners, in conjunction with Coca Cola. ENGINE aims to improve learning outcomes and the economic status of over 18,000 marginalised girls, aged 16-19, in two ways – one targeted at in-school girls and the other at out-of-school girls. ©

EN

GIN

E

The Girls’ Education Challenge is managed on behalf of the UK Department for International Development by PricewaterhouseCoopers LLP in alliance with FHI 360, Nathan Associates Ltd. and Social Development Direct Ltd.

Contact us: Email: girlseducationchallenge@uk.pwc.com | Tel: +44 (0)20 7213 5969

Conclusions

Although the international discourse is emphasising a shift from simple assessment of learning towards more assessment for learning (see Brookings Institute, 2015 and Black and William, 2001), the GEC experience to date shows that this is no easy feat given the context in which the GEC is working.However, there are examples of this type of assessment for learning being used across the programme and, in some cases, there is evidence of its efficacy. Internationally, there is much work going on to try to improve national assessment systems to include both assessment of learning and assessment for learning. Ideally, within the next phase of the GEC, we hope to see assessments which not only allow

us to track the progress of learners but which also allow projects, schools and teaching staff to use these assessments for learning. The efficacy of such approaches will depend not only on how well they are suited to individual contexts and constraints, but also on how projects are more broadly addressing the key barriers to learning they have identified for their beneficiaries. Nonetheless, there does appear to be scope and appetite to adopt assessment for learning practices within wider approaches to overcoming the learning barriers that confront girls in the classroom. Some project staff, for instance, have already explored the data that they and their external evaluators have gathered through the EGRA type assessments

© C

amfe

d

and analysed it with a view to adapting and improving their interventions around literacy and numeracy. We encourage projects to continue to share their experiences of how they are using assessment and developing their practice in this area.

The Girls’ Education Challenge has a zero tolerance policy on misconduct, including mistreatment of individuals and misappropriation of funds. If you would like more information on the whistle-blowing mechanism, or to report misconduct please email gecpmo@uk.pwc.com. The e-mail account is accessible only by a small number of individuals who have been trained on the requirement to keep the information confidential. We will follow up matters on an anonymous basis and are committed to investigate claims thoroughly and fairly.