Himachal Pradesh State Assessment - Main Report

7/24/2019 Himachal Pradesh State Assessment - Main Report

1/56


2/56

Educational Initiatives Pvt. Ltd.

1

We would like to thank the officials of Education Department and Sarva Shiksha Abhiyan, Himachal Pradesh. The

successful completion of this assessment is attributed to help from many individuals in the department of Himachal

Government.

This study would not have been possible without support from Michael and Susan Dell Foundation (MSDF) and

Learning Links Foundation (LLF). We are highly indebted to them for their guidance and constant support for

successful roll out of State wide assessments.

Special gratitude is due to all the DIET Principals and officers, Research team-SSA and LLF team at Himachal for

their kind co-operation and the great support. We were able to carry out the assessments smoothly because of

the great support provided by the all the Government school teachers who carried out the test administration.

Last but not the least; we are also thankful to the students, and principals of the schools of Himachal thatparticipated in the Assessments.

We sincerely hope that this illustrative study will be useful to the policy makers and schools.

MR. SRIDHAR RAJAGOPALAN MR. VISHNUTEERTH AGNIHOTRI

(Managing Director) (Vice President - Assessments)

Educational Initiatives Pvt. Ltd. Educational Initiatives Pvt. Ltd.

ACKNOWLEDGEMENT


3/56


2

Himachal State Education Department

Sarva Shiksha Abhiyan, Himachal Pradesh

Michael and Susan Dell Foundation

Learning Links Foundation


Mr. Sridhar Rajagopalan, Managing Director

Mr. Vishnuteerth Agnihotri, Vice-President, Assessments

Ms. Vyjayanthi Sankar, Vice-President, Strategic Relationships

Mr. Venkatesh Reddy, Manager Communications

Ms. Neha Miglani, Research Associate

Ms. Archana Dwivedi, Research Fellow

Ms. Bharti Porwal, Educational Specialist

Ms. Swati Jha, Education Specialist

Mr. Nishchal Shukla, Educational Fellow

Ms. Sidhya Balakrishnan, Jr. Research Associate

Ms. Kaveri Malik, Programme Implementation Manager

Ms. N.V.N.S.S. Durga, Graphic Designer

Mr. Rahul Mathur, Software Development ManagerMr. Jaikishan Keswani, Jr. Software Engineer

Ms. Jayashree T.S., Educational Specialist

Mr. Devpal Shah, Manager- Design and Printing

Mr. Ketan Beherawala, Asst Manager - Accounts

Mr. Drupad Shah, Manager - Logistics

TEAM


4/56


3

EXECUTIVE SUMMARY 4

1.

DESIGN AND IMPLEMENTATION OF THE STUDY 6

1.1 Need For The Study 6

1.2 About the Study 7

1.3 Salient Features Of The Study 8

1.4 Question Paper Design 9

2. MAIN FINDINGS 152.1 Performance Across Subjects and Classes 15

2.2 Comparative Performance Against Indian Private Schools 15

2.3

Learning with Understanding - Performance on Higher Order Questions 162.4 Performance On Practical Competencies 18

2.5 Performance Across Districts 19

2.6 Performance Of Boys And Girls 21

3. INSIGHTS ON SPECIFIC LEARNING AREAS 22

3.1. Misconceptions And Common Errors 22

3.2. Performance In Different Skills 38

4. FIELD AUDITS 42

5. A PEEK INTO THE DISTRICT REPORTS 465.1 A Birds Eye View Of The Districts Performance Across Classes And Subjects 46

5.2 Score Distribution Of Students Across Subjects, Class 47

5.3 Comparison With Other Districts (Composite Index) 48

5.4 Performance On Different Skills 49

5.5 Question-wise Performance Across Classes And Subjects 50

5.6 Subject Level Recommendations 51

6.

APPENDIX 52

Appendix A: Comparison With Asset 52

Appendix B: Difference In Performance In Straight Forward (SF) And Higher Order (HO) Questions 52

Appendix C: District-wise Performance 53

Appendix D: District-wise Composite Performance 54

CONTENT


5/56


4

Why this Study?

Understanding the importance of a comprehensive educational policy, the state of Himachal Pradesh conducted anassessment of learning levels with Educational Initiatives, in partnership with the Michael and Susan Dell

Foundation and support from the Learning Links Foundation. The Learning Assessment was conducted in June

2013 in an attempt to undertake a scientific and systemic study to detect state learning levels. . It seeks to provide

independent third-party assessment of student learning to be used by policymakers and researchers and establish

student learning in terms of knowledge and skills (or competencies) and provide feedback on the learning gaps,

common errors and misconceptions. This report will argue that assessment must promote good education rather

than undermining it.

Design of the study

Coverage:The study was aimed to cover all the 12 districts in Himachal Pradesh. Around 1000 students per class

from each district were sampled and tested for classes 3, 5 and 7. Diagnostic assessment was conducted with

Hindi as the medium of instruction. The study included a detailed scientific process of test development supported

by curriculum checking; development of test items based on the previous studies conducted by EI, fine-tuning and

assembling of the final papers; detailed test adaptation, translation, validation of papers; master trainer workshops,

field audit, data entry, and diagnostic data analysis prior to the release of the report.

Test Format: The learning assessment was conducted for two subjects-Language (Hindi) and Mathematics for

classes 3, 5 and 7. These test papers were designed for a wider audience of MSDF grantees. For each class and

subject, one set of papers was made and adapted into Hindi language. The question papers includedstraightforward questions as well as higher order questions detecting deeper understanding and application of

learnt concepts. The test papers were completely written but the question paper was read aloud once in Class 3

Students ofclass 3 filled in their answers on the question booklets itself. At class 5 and 7 level, the students directly

marked their answers in the given OMR sheets. However, they answered the free response questions in the question

booklets. After the test, the students responses (OMRs as well as question booklets) were sent back to EI where the

class 3 complete question responses and class 5 and 7 free response questions were coded on OMR sheets by the

evaluators at EI.

Test Conduction and Analysis: All schools got the same paper to detect learning levels and to undertake a

comparative analysis of student performance. Further, in order to benchmark achievement of Himachal Pradesh

students with those of Indian private schools (not low-cost schools), sufficient number of test items were used from

ASSET1

1ASSET is a diagnostic test from Educational Initiatives, which is administered every year to more than four lakh students studying in Indian private

schools (not low-cost). Details can be obtained through www.ei-india.com.

.

A team of EI assessment experts conducted training for Principalsof District Institute of Education & Training

(DIET) as well as some District Primary Education Officers (DPEOs), District Education Officers (DEOs) to ensure

standardized test administration. Test conduction was managed by the State of Himachal Pradesh with the help of

LLF. EI teams also carried out field audits during the day of the test in over 3 schools spread across 5 districts.

EXECUTIVE SUMMARY


6/56


5

Based on the student performance on these tests, different types of analyses were carried out using advanced

statistical methods to extract patterns in performances and to understand differences in learning levels across

different groups and patterns in learning. The findings from this analysis have been presented in this report.

MAIN FINDINGS

Based on the analysis of student performance at the school, district as well as the state level, several key findings

have been presented below and detailed in the sections that follow.

1. The state average performance ranges from 48.5% to 59.5% in Hindi and 38.1% to 59.7% in Maths

across the three tested classes. It was observed that the performance was more homogenous within

higher classes, (indicated by lower standard deviation).

2. Students in the State perform lower than the Indian Private School Average (assessed through ASSET),

typically by 5-13 percentage points. Further, this gap between the state and ASSET performance increases

in the higher classes.

3. Lower performance of students was observed in higher order questions involving deeper understanding of

concepts. This indicated that learning with understanding is weak, as the students find it difficult to applylearnt concepts in unfamiliar question formats.

4. The students were observed to be the weakest in questions testing practical competencies such as visual

interpretation, map reading, observation and estimation. Practical competencies are essential for the

students all round development and this is a matter of concern.

5. A comparative analysis of the performance of all the districts revealed that Hamirpur outperformed all

other districts, performing significantly better than the state average. At the same time, the districts of

Chamba, Bilaspur and Sirmaur observed lower performances, significantly below the state average.

6. Students seem to have acquired several misconceptions during the process of learning. These were

observed amongst the students across competencies and subjects. These misconceptions can arise at any

time, and should be tackled at the earliest through strategic pedagogical improvements.

7. The student performance was weak in particular skills of Hindi and Maths. These include intermediate

skills of Fractions, Surface Area and Volume and Integers in Maths and reading comprehension, particularly

in inference based questions and passages based on authentic material in Language.

These findings have been comprehensively discussed in chapters 2 and 3 of this report. The report will present

these findings with relevant statistical methods, representation and sample questions.


7/56


6

1.1 NEED FOR THE STUDY

Education is recognized across the world as perhaps the most vital public service of all. However, the

measurement of its impact is still in question. Assessments are essential to ensure that students get the educational

support that they need to succeed, to see the effectiveness of different educational methods, and to ensure that

education budgets are being utilized effectively. Therefore, inevitably, assessments also risk marking teachers,

learners and institutions as successes or failures.

Various Assessments have been in existence as an informal tool for even longer than the actual annual school tests.

The emphasis has been on ways in which an assessment can help to advance learning rather than merely

measuring it. Current developments in education; such as the reinvention of assessment processes, suggest that

the views of different experts on the correct and incorrect ways to assess educational progress are becoming

more influential, and the research projects have helped to build on the evidence needed to improve assessments,

and the educational outcomes which it is meant to focus on.

In India, independent assessments conducted by organisations like NCERT, ASER and Educational Initiatives show

that students in Indias Government schools are not learning well, and not acquiring the expected competencies in

basic literacy and numeracy.

EI has been carrying out research in learning outcomes for several years. It is a pleasure to acknowledge

the Government of Himachal Pradesh and especially all officials of the Education Department for their contribution

in advancing our ideas about appropriate ways of assessing educational outcomes.

Diagnostic scores are of increasing interest in educational testing due to their potential benefits, both remedial and

instructional. Naturally, the number of educational tests that report diagnostic scores is on the rise across the

globe, as are the number of researches based on such scores.

This report provides a critical evaluation of the diagnostic testing attempted by 28,898 students from classes 3, 5

and 7 across the state of Himachal Pradesh. It is demonstrated, using results from operational and simulated data,

that diagnostic scores have to be based on a sufficient number of items and have to be sufficiently distinct from

each other to be worth reporting and that several operationally reported sub-scores are actually not worth

reporting. The format of the state-wide diagnostic testing has taken care of these basic elements.

A diagnostic test helps to identify a student's learning problems, so that teachers can provide specific

instructions to remedy them. Moreover, teachers must often undertake complicated and time-consuming

analyses of students' responses to individual items. Legitimate diagnostic tests permit teachers to use a test's

results without having to devote hours to intricate interpretations.

This report on the Diagnostic Assessment in the state of Himachal Pradesh is a joint initiative undertaken by the

Education Department Govt. of Himachal Pradesh along with Sarva Shiksha Abhiyan (SSA) HP, Learning Links

Foundation (LLF), Michael & Susan Dell Foundation (MSDF) and Educational Initiatives (EI).

1. DESIGN AND IMPLEMENTATION OF THE STUDY


8/56


7

1.2 ABOUT THE STUDY

In its first year of conducting this study, Educational Initiatives conducted diagnostic assessments for 1310 schools

for class 3, 1246 schools for class 5 and 401 schools for class 7 (chosen as sample) across the 12 districts which

fall under the jurisdiction of Education Department, Himachal Pradesh.

The tests were done for classes 3, 5 and 7 in Language (Hindi) and Maths.

Around 1000 students per class from each district were sampled and tested for classes 3, 5 and 7. Only classes

with Hindi as the medium of instruction were covered in the diagnostic assessment. The sampling was performed

by the State based on several criteria. It was ensured that in smaller districts almost all blocks are covered. In

larger districts all blocks that are different in demography were covered to ensure a complete representation of

the district. The clusters inside a block were chosen to represent different demographics of the block. First the

class 7 schools were selected (these are secondary schools). Primary schools attached to secondary schools were

selected post that. It was ensured that 1000 students from class 7 were selected. More primary schools were

selected in same clusters to ensure 1000 students in each grade (3, 5). In some districts the total population is very

less hence the sample size is low.

This report will argue that assessment must, in all cases, promote, rather than undermine, good education.Policy-

makers need to keep the needs of pupils at the fore, and ensure that any evaluation of new developments in an

assessment is carried out with careful consideration of the consequences, both intended and unintended.

It is helpful to make a distinction here between the intended uses of assessment data, and their actual uses.

Assessments are often designed quite differently to ensure their adaptability for different purposes. We must

therefore deliberate on the different ways in which such assessments are used. Paul Newton has identified 22 such

uses2

1. Be part of effective planning

. These are, however, only broad categories. If one considers each purpose in detail, the number of uses for

the data can multiply.

Similarly, the term assessment for learning is often interchangeably used as formative assessment. In 1999, the

Assessment Reform Group defined assessment for learning as the process of seeking and interpreting evidence

for use by learners and their teachers to decide where the learners are in their learning, where they need to go

and how best to get there. ARG also identified ten principles for formative assessment practice, arguing that it

should:

2. Focus on how pupils learn

3.

Be central to classroom practice

4. Be a key professional skill

5. Be sensitive and constructive

6. Foster motivation

7. Promote understanding of goals and criteria

8. Help learners know how to improve

9. Develop the capacity for self-assessment

10.

Recognize all educational achievement

2This list was submitted as part of the evidence from QCA to the inquiry on assessment and testing conducted by the House of Commons Children,

Schools and Families Committee in 2007/08. See: http://www.publications.parliament.uk/pa/cm200708/cmselect/cmchilsch/169/16906.htm#n35


9/56


8

The persistent question thus is what is required to fulfill the aim of formative feedback; i.e., help learners to grow

in capacity to manage their own learning, in order to progress effectively in developing capability in specific subject

domains. As highlighted by the previous definition, learning can hence, be seen as a journey in which learners need

to know where they are at present, where they are going, and how to get from one place to another.

The goals of the diagnostic assessment were to:

Provide independent third-party assessment of student learning to be used by policymakers andresearchers.

Establish student learning in terms of knowledge and skills (or competencies) and provide feedback on the

learning gaps, common errors and misconceptions.

Provide analysis of student performance on specific competencies for classes tested.

Analyze the learning patterns.

Identify (if any) significant differences in performances across districts.

Identify (if any) significant differences between the performance of boys and girls

Diagnose the learning issues at the systemic level and provide pointers for further research.

In short, this diagnostic assessment report is a comprehensive study that aims to provide detailed question-wise

and student-wise data for understanding and monitoring learning across classes.

1.3 SALIENT FEATURES OF THE STUDY

Coverage: This was a sample study covering around 1000

students per class per district from the government schools

across 12 districts of Himachal Pradesh. All the students had

Hindi as the medium of instructions. Details of the coverage ofthe assessment are given in the table alongside.

Scientifically Designed Test Development Cycle:The study

included a detailed scientific process of test development

supported by curriculum checking; development of test items

based on the previous studies conducted by EI, fine-tuning and

assembling of the final papers; detailed test adaptation,

translation, validation of papers; master trainer workshops, field

audit, data entry, and diagnostic data analysis prior to the release

of the report.

Specially Constructed Papers: The test papers for the

students were drawn from EIs tests particularly designed for a

wider pool of audience of the Michael and Susan Dell Foundation.

These tests assess basic competencies as well as deeper

conceptual understanding of the students through straightforward and higher order questions testing

understanding and application of concepts, reasoning ability and critical thinking.

Benchmarking:In order to benchmark achievement of Himachal Pradesh students with those of Indian private schools

(not low-cost schools) sufficient test items were used from ASSET

3

3ASSET is a diagnostic test from Educational Initiatives, which is administered every year to more than four lakh students studying in Indian private

schools (not low-cost). Details can be obtained through www.ei-india.com.

.

District

Planned

Sample

(as shared

by the state)

Actual

Sample*

BILASPUR 2,861 2,225

CHAMBA 3,816 2,904

HAMIRPUR 3,439 2,704

KANGRA 3,270 2,536

KINNAUR 791 319

KULLU 3,272 2,544

LAHUL & SPITI 785 501

MANDI 3,491 2,745

SHIMLA 3,474 2,606

SIRMAUR 3,773 2,699

SOLAN 3,403 2,266

UNA 3,704 2,712


10/56


9

Papers:For each class and subject, one set of papers was made and adapted into Hindi language. All schools got

the same paper set for reasons of testing level and comparison across districts.

Specially Trained Test Administrators and Evaluators: A team of EI assessment experts conducted the

training of Principalsof District Institute of Education & Training (DIET) to roll out the test evaluators in their

respective districts and to ensure standardized test administration. Test dates were decided by the State during

the master trainers workshop after consulting the present DIET principals. It was decided that the master trainers

would impart training to the teachers in their respective blocks and clusters on similar lines.

Conducting the tests: Test conduction was managed by the State of Himachal Pradesh with help from LLF.

Printed papers were dispatched from Shimla to all the districts through DIET offices. The papers were completely

written, however, the papers were read out aloud once for class 3. The teachers of the respective school

invigilated the test after receiving appropriate trainings. The test timings were in line with the school timings of

that area, however almost all the schools conducted the test on the same day.

Field Audits: EI teams carried out field audits during the testing day to check the quality and fairness of test

administration in different schools. On the testing day, 31 schools spread across 5 districts were spot-visited by a

team of 10 members. These 5 districts - Kangra, Kullu, Shimla, Sirmaur, Mandi - were chosen to cover parts of

lower, central and upper Himachal. The team provided feedback to the test implementation and management

team to ensure that the test administration was carried out smoothly. A report on the field audit was prepared by

collating the feedback from all the team members and has been submitted as a part of this report in Chapter 4.

Data Entry:Students ofclass 3 filled in their answers on the question booklets itself. At class 5 and 7 level, the students

directly scored their answers in the given OMR sheets. However, they answered the free response questions in the

question booklets. After the test, the students responses (OMRs as well as question booklets) were sent back to EI

where the class 3 complete question responses and class 5 and 7 free response questions were coded on OMR sheets

by the evaluators at EI.

Analyses:Different types of analyses were carried out on the collected data to extract patterns in performances

and to understand differences in learning levels across different groups. Advanced statistical methods were used to

confirm patterns of learning. Distracter analysis enabled identification of misconceptions and common errors. The

data was studied across different units of analysis: schools, districts, competencies, gender, etc. Tests of statistical

significance were done to compare the performance of districts and girls and boys.

1.4 QUESTION PAPER DESIGN

Most education tests measure the status of students with respect to a cognitive skill or body of knowledge, which

we can refer to as assessed attributes. The test needs to include a sufficient number of items to measure each

assessed attribute so that the teacher can arrive at a reasonably accurate inference about how an individual studentstands with regard to each of these attributes. For instance, if the teacher wants to know whether a student can

multiply pairs of double-digit numbers, one or two items in a test won't provide an accurate estimate. The number

of items required, of course, will depend on the nature of the skill or body of knowledge being measured, but one

item per measured attribute definitely doesn't serve the purpose.

To arrive at a sound inference about student competency, the test should (1) measure a modest number of

significant, high-priority cognitive skills or bodies of knowledge; (2) include enough items for each assessed

attribute to enable teachers to make a reasonably accurate judgment on a test taker's mastery of that attribute; (3)

describe with clarity what the test is assessing; and (4) not be too complicated or time-consuming.

The purpose of this assessment is to provide a detailed diagnostic snapshot on the strengths (students achievements)

and weaknesses (learning gaps). The assessments were developed for Language and Maths for Classes 3, 5 and 7.


11/56


10

Paper Format:A simple, easy-to-administer test as well as its results may be better accepted by teachers and

other stakeholders. This will also ensure that the findings of the assessments would be used to improve teaching

and learning. The format of the test papers has been summarized below:

The written test had items that were read and answered by students themselves. However, all the

questions that appear in the written test for Class 3 were read out by the evaluators at the beginning of

the test to familiarise the students with the questions.

The Written test included Multiple Choice Questions, where the students chose an answer from four given

options; and Free Response questions, where the student wrote down his/her answer in a fill-in-blanks format.

The students in class 3 marked their answers on the question papers itself, and these responses were then

recorded on the OMR sheet by a team of evaluation coders at EI. The students of class 5 and 7 recorded

their own responses on the OMR sheet, but Free Response questions were answered on the question

paper itself and similarly these questions were recorded on the OMR later.

The invigilator manual that was provided along with the test papers. This had general and specific

instructions on what needs to be read out to the students for each question in each test paper etc.

Competency Framework:The tests were comprehensive and included a number of competencies to ensure a

more reliable measurement of student achievement. National Curriculum Framework documents, the Minimum

Learning Levels (MLLs), the state textbooks, standard international framework of tests such as TIMSS and PIRLS,

and EIs large-scale benchmarking studies for Government schools, were referred to while finalizing the

competencies for each paper. The tests were thus competency-based such that every question was linked to a

particular competency in the skill framework or the test blueprint.

Types of Questions: The tests included questions to test not just knowledge (recall and procedure) but

understanding and higher order skills. The focus of the questions was on testing for learning with understanding,

i.e., the real understanding of students in concepts they have learnt in their specific classes. A percentage of

questions were also drawn from EIs national level benchmarking test, ASSET.

The following table provides details about the classes, subjects tested, paper length, duration of each paper, the

proportion of free response (FR) and multiple choice questions (MCQs) and the proportion of straight forward

(SF) and higher order (HO) questions:

Subject Class # Questions

# SF Questions # HO Questions

Duration# FRQuestions

# MCQs # FRQuestions

# MCQs

Maths

3 26 -- 13 -- 13 90 mins

5 32 -- 16 -- 16 105 mins

7 36 -- 18 -- 18 105 mins

Language

(Hindi)

3 32 8 8 1 15 105 mins

5 38 2 16 5 15 105 mins

7 44 2 20 3 19 105 mins


12/56


11

Sample Questions

A.

Questions that check for learning that is straightforward or text-bookish

Generally, it is believed that though the students may not be learning with conceptual understanding, they perform well

on questions which test concepts in a format that is familiar to them. This can be tested by placing questions which are

similar to examples or questions of text books. They are described as straight forward or text-bookish questions.

B.

Higher Order Questions - that check for a deeper understanding of concepts:

While students perform well on easy and familiar questions, these questions check for a conceptual understanding

of the subject in question and whether students can apply what they have learnt in different formats.

C.Higher Order Questions - that check for application of concepts

These types of questions checks the students ability to answer the questions based reasoning, visual interpretation, and

estimation. These are the kind of questions which students are faced with in their day-to-day interaction with the world.

MathsClass 3HindiClass 5

MathsClass 7MathsClass 3

MathsClass 7


13/56


12

D. Questions that check for Reading Comprehension

Several questions in the Language papers are based on unseen passages and authentic material seen in daily life such as

tables, labels, notices, etc. These passages test comprehension which is both fact based and implied in the text.

Scoring Process: At EI, test papers for every class are supplemented with a scoring card. The scoring card

provided a question-wise rubric (with scores / codes / answers) for how each response is to be scored. The scoringcard captured common mistakes that children tend to make while answering questions. The data that was collected

by this process was critical for the diagnostic feedback and analyses. This form of scoring is common international

practice in tests such as TIMSS.

MathsClass 5

MathsClass 7


14/56


13

The evaluators from EI for class 3 captured the answer responses of the students in the form of codes as specified

in the scoring card and entered them on to ICR and OMR sheet. The codes thus entered were scanned using

intelligent character recognition (ICR) and optical mark recognition techniques and was converted

into a database of student responses for further analysis. This eliminated human error while transcribing, etc., and

also saved time. While coding responses for multiple choice items, only the option letter ticked was to be

recorded. For free response items, the responses were recorded based on the detailed scoring rubrics. All

scoring cards also had general codes assigned for specific types of responses common to all papers. The details are

given below:

General Codes Assigned Across Papers:

1. For questions with options A, B, C, D: Option ticked (enter option A, B, C, and D in English).

2. Items that have not been attempted at all should be coded as 88.

3. Invalid answers (e.g. question copied, more than one option ticked in multiple choice questions, crossed

out answers, illegible answers) should be coded as 86.

4.

Spellings are not to be checked unless indicated specifically.

5. Instead of ticking, if a child crosses or circles or marks an option in any other way, or rewrites one of the

options, that option should be treated as their choice.

6. Free Response Questions were coded as follows:

a. Correct Answer - 01

b. Partly Correct Answer- codes vary with every question - 11, 21, 31 (depending on the type of FR

question)

c.

Any other answer - 85

Sample Test Items and Their Scoring Rubrics:

Class 7 Maths - MCQ Question

Option ticked (enter option A,B,C,D in English) A

B

C

D

Invalid Answer Code/More Than One Option Ticked 86

Not Attempted 88

MathsClass 7


15/56


14

Class 5 Language- Free Response Question

Meaning comes out clearly; no grammatical errors; sentence complete and related to the given picture 1

Meaning comes out clearly; 1-2 grammatical errors; sentence complete and related to the given picture 11

Meaning not completely clear; some grammatical errors; sentence incomplete but related to the given picture 21

Meaning not clear at all but 1-2 appropriate words related to the given picture are used 31

Sentence not related to the given picture 81

Any other answer 85

Invalid Answer Code/More Than One Option Ticked 86

Not Attempted 88

LanguageClass 5


16/56


15

2.1 PERFORMANCE ACROSS SUBJECTS AND CLASSES

The overall performance for each class and subject is presented in the table below. The state average performance

ranges from 48.5% to 59.5% in Hindi and 38.1% to 59.7% in Maths across the three tested classes. It was

observed that the performance was more homogenous within higher classes, indicated by lower standard

deviation.

Subject ClassN

(Total students)

% Score

(Average)Std. Dev.

Language (Hindi)

3 8,231 59.5 % 21.4

5 9,151 59.0 % 18.4

7 9,516 48.5 % 15.6

Maths

3 8,231 59.7 % 24.3

5 9,151 48.2 % 20.6

7 9,516 38.1 % 18.4

While the overall performance in Hindi has been largely consistent in Classes 3 and 5, the dip in performance in

Hindi class 7 and the decline in overall performance across classes in Maths cannot be explained through an

analysis of the average scores. A comparative analysis may not suffice for this purpose, and a system of

benchmarks against which the scores can be analysed is essential. A question-wise analysis is also essential to

understand what particular skills and competencies the students are not performing well in, and whether the

decline in the overall performance can be traced to weaker conceptual understanding in particular skills.

2.2 COMPARATIVE PERFORMANCE AGAINST INDIAN PRIVATE SCHOOLS

A comparative analysis was undertaken between the performance of Himachal Pradesh Students and the

performance of Indian private schools as represented by ASSET (A diagnostic benchmarking national level test) in

English, Maths, Science, Hindi and Social Science. Himachal Pradesh students typically score lower than Indian

Private Schools.

2. MAIN FINDINGS


17/56


16

Sufficient common questions4

(7 -11 questions in each paper) were used from ASSET to compare the

performance of Himachal Pradesh students with English medium private schools. Gap between Himachal Pradesh

students and students of Indian Private Schools was found to be comparatively high in Class 7 and lower for Class

3 (Hindi and Maths). A closer look at Class 3 questions reveals that Himachal Pradesh students have performed

better than Indian Private School students on certain questions. In Language, for all the classes those questions

wherein students had to infer from the given passages/ stories/ notices, were found to be low performing

compared to those questions which had answers directly stated in the passages/ stories/ notices. The examples

below illustrate the difference seen between Himachal Pradesh schools and Indian private schools on commonASSET questions.

2.3 LEARNING WITH UNDERSTANDING

A high level analysis of questions which tested higher order (HO) skills as opposed to questions which were straight

forward (SF) reveals that students have performed better on SF questions. Broad trend on these lines also indicates that

performance on HO questions goes down as we move higher classes.

425% ASSET items used for the analysis

MathsClass 7MathsClass 3

Sample Question: This question tests students understanding of

operations and the relation between them. 50% of HP students

answered this question correctly compared to 46% of ASSET students.

Sample Question: This question checks if students are able to observe

a pattern and logically think of a general rule being followed in that

pattern. Only 29% of HP students answered this question correctlycompared to 49% of ASSET students.


18/56


17

Deeper analysis of question-wise performance of students highlights that learning with understanding is weaker in

higher classes for questions testing conceptual learning. This indicates that the conceptual understanding in

specific topics is not deep enough for the student to understand and attempt questions that are presented in a

non-familiar format or which go beyond the textbook. Rote-based learning among the students indicates that

conceptual understanding is not entrenched enough for the student to progress to higher order questions and

skills which require a strong basic understanding of the concept. This has been explained though the examples

given below.

This question tests the students understanding of basic shapes. Only half of the students of class 5 have been able

to correctly identify the rectangle among the given shapes. About 28% of students have chosen option C, a

rounded rectangle. This indicates that these students are not able to recognise a rectangle in a non-standard

orientation. They may have a mental picture of a rectangle in a standard orientation, and may have chosen the

image that is closest to this picture, irrespective of whether it satisfies all the properties of a rectangle or not.

MathsClass 5


19/56


18

This question tests the understanding of place value system in decimals. Students are expected to realise that 7

tenths and 8 thousandths make 0.708. Only about 24% of students have been able to answer this correctly.

Students may be used to converting conventional expanded forms like 7/10 + 0/100 + 8/ 1000 into decimals. But

here when the conventional order is changed and hundredths are missing, many students may have been confused.

About 34% have chosen option C, 0.8007 and 15% have answered D, 0.87, retaining the order in which the

numbers appear in the question.

MathsClass 7


20/56


19

2.4 PERFORMANCE ON PRACTICAL COMPETENCIES

Practical competencies are defined as visual interpretation, map reading, observation and estimation- all of which

are essential skills for daily life and for the overall development of the student, irrespective of the level of

education that student pursues in the future. Based on the student performance on questions testing these skills, it

was observed that the overall performance is low in these questions. An example of a question testing practical

competencies is given below:

This question checks if students are able to estimate the length of an object relative to another given object. Only

38% students have answered this question correctly. Around 30% students have chosen the wrong answer option

A. They have probably just looked at the height of the scale given in the image and chosen 100 cm as their answer.

For more detailed analysis by skill and question, please refer to Chapter 4 of this report. The skills, questions have

also been analysed at a district level to understand the strengths and weaknesses in competencies for a particular

district. A district wise analysis report for each district is also shared with this report.

MathsClass 5


21/56


20

2.5 PERFORMANCE ACROSS DISTRICTS

2.5.1 District Performance on Composite Performance Index

To have a comprehensive understanding of the states

performance at all levels, a district wise analysis was

performed and relative performance of the districts

were analysed. A Composite Performance Index was

calculated for all districts, which is a simple average of

performance across classes and subjects. This

composite index was calculated for all districts and

their scores on this index were analysed.

At the same time, an analysis only based on an overall

performance may be insufficient as well as misleading;

a particular district may be the top performer, but

may have scores very close to the average. Toincorporate differences of the district scores from the

state average, effect sizes were calculated for each

district through Cohens D5

Based on the district wise performance on this index

and their effect sizes, a few observations have been

presented.

, which would highlight if

the districts performance is significantly higher or

lower than the state average.

Hamirpur outperformed all districts, with a score significantly higher than the state average.

Chamba, Bilaspur and Sirmaur were among the bottom three districts when compared to state average

Kinnaur and Lahul & Spiti removed from the district-wise analysis due to low sample size for both the districts

The remaining districts of Mandi, Kangra, Kullu, Solan, Una and Shimla performed close to the state average

score.

5Cohen's d is defined as the difference between two means divided by a standard deviation for the data


22/56


21

2.5.2 District Performance across classes and subjects

The composite performance index was also calculated separately for each subject to analyse if the performances of

the districts vary between the two subjects. The difference of this score from the state average was also calculated

through the Cohens D.

Based on the district wise performance on the index in Hindi and Maths, observations have been presented below

Hamirpur was the top performer in both subjects, performing significantly better than the state average

Mandi performed significantly better than other districts and the state average in Maths, but performed

close to average in Hindi, leading to a decline in its overall performance.

For a detailed view on each district, please refer to the district wise report (provided for every district) along

with this report.

MathHindi


23/56


22

2.6. PERFORMANCE OF BOYS AND GIRLS

Over 13 thousand girls and a similar number of boys participated in the study. This section of report attempts to

analyse any noticeable difference between the average performance of the boys and girls in the study. The table

below indicates the mean difference in the performance according to gender.

Class SubjectGirl Boys T

Criticvalue

Cohen's

d

Effect Size-

interpretationN Avg SD N Avg SD

3 Hindi 4189 60.5 21.2 3956 58.4 21.4 4.4 0.1 -

5 Hindi 4409 59.6 18.4 4716 58.6 18.3 2.6 0.05 -

7 Hindi 4682 49.4 15.8 4830 47.6 15.3 5.6 0.12 -

3 Maths 4189 60.6 24.3 3956 58.6 24.3 3.7 0.08 -

5 Maths 4409 48.6 20.6 4716 48 20.6 1.4 0.03 -

7 Maths 4682 38 18.4 4830 38.1 18.4 -0.3 -0.01 -

On comparing performances according to gender, it was observed that girls perform better than boys in Hindi

class 3, 5 and 7 and Maths class 3 but the magnitude of difference was marginal for all classes. In Maths class 5 and

7, both boys and girls performed similarly.

Legend

Boys and girls are performing at the same level.

Girls are performing significantly better than boys with


24/56


23

3.1 MISCONCEPTIONS AND COMMON ERRORS

Students develop a cognitive understanding of the world around them through interactions based on their daily

experiences. Teachers and schools help build this understanding. Misconceptions are concepts that students

acquire which are not in line with or do not match the conventional expert view in that topic. These result in

cognitive gaps in their understanding. Identifying the exact nature of student misconceptions is difficult through

regular classroom interactions. Any question in an assessment test that attempts to identify the misconceptions of

the students must force the students to actively use their conceptual understanding. A detailed understanding of

these misconceptions through large scale diagnostic assessments provides the teacher with a starting point to

explore these in the classroom and eventually help to build correct notions or conceptual understanding. Detailed

analysis of the answers given by students in each question in all the subjects tested reveal that students

understanding of the concepts is often faulty as they seem to harbor many mistaken notions. They also seem tocommonly make errors in answering some questions. Identifying the exact nature of student misconceptions is

difficult through regular classroom interactions. Any question in an assessment test that attempts to identify the

misconceptions of the students must force the students. Some misconceptions and common errors found in the

analysis of HP state study include:

The bigger the area of a shape, the larger will be its perimeter

Confusing Part-Whole relationship with Part-Part relationship in Fractions and Ratios

Integers can be added without checking their positive/ negative signs

Using prior knowledge to answer reading comprehension questions

An item wise analysis of all the questions reveals these patterns through a graphical representation known as Item

Response Curve (IRC). A short description on how to read an IRC is given below.

How to read the graph:

These are the item response curves for a

question. The X axis is the total score in the

paper and the Y axis is the percentage ofstudents. The graphs show what percentage of

students scoring different total scores in each

class, chose which answer options for the

question. The legend box shows the total

percentage of students who chose each option

in the paper. In this graph, the percentage of

students choosing option A increases as one

moves from a low total score to a high total

score, while percentage of students choosing

option D increases up to a certain total score

and then starts reducing.

y

x

3. INSIGHTS ON SPECIFIC LEARNING AREAS


25/56


24

3.1.1 Misconceptions and Common Errors in Mathematics

Question Graphs

Class 3 / Question 15

Explanation: This question checks if

students can understand how a paper

would look like when it is folded, cut and

then unfolded. Students are expected to

visualize the shape that would be formedafter the paper is unfolded. Only 30%

students chose the correct answer (D).

However, a lot of students chose option C

(19%) and were probably just looking at

the image in the question and choosing this

option, not realizing that when the paper is

folded into half and cut, double the part of

the paper will be cut. Around 35%

students have not attempted this question.


Explanation: This question tests the

understanding of place value. About 50% of

students have answered it correctly as

option A, and 22% have answered it as C.

These students may have mechanically put

down the digits in the order in which they

appear in the question. They may not have

noticed that the question asks for 3 ones

and 7 tens. This could be because they are

used to solving problems in the

conventional order 7 tens and 3 ones and

are not sufficiently exposed to equivalent

alternate forms of writing the expandedforms of a number.

x

y


26/56


25

Question Graphs



students are able to represent a part of a

group as a fraction. Students are expected

to understand that 10 students out of total

40 (30 girls + 10 boys) are boys and hence

the fraction of boys would be . However

a lot of students have chosen option A

(25%) and option C (39%). Students

choosing option A probably are notconsidering the fact that the total number

of students is 40 and are calculating the

ratio of boys to girls rather than the fraction

of boys in the class. Students choosing

option C are probably making the same

error but are giving the ratio of girls to

boys. These students have not been able to

differentiate a part-whole relationship from

a part-part relationship.



students can apply fractions to the units of

capacity. Students are expected tounderstand that a litre has 1000 millilitres

and litre would be equal 250 millilitres.

Only 6% students have chosen the correct

answer option A. However, most of the

students (41%) have chosen option B.

These students seem to have the

understanding of fractions but are probably

choosing this option as part of the jug is

filled here. Students choosing option C

(31%) may be matching the 4 of with

400 ml. They do not seem to understandthe concept of fractions.


27/56


26

Question Graphs



understanding of perimeter as the length of

the boundary of a shape. Students are

expected to identify the shape that has the

longest boundary. But almost 31% of

students have answered D. Possibly they

are not able to differentiate between the

terms area and perimeter, and have chosen

the piece that has the largest area. It is alsopossible that they have a notion that the

shape with the largest area has the largest

perimeter too.



students understanding of basic shapes.

Only half of the students have been able to

correctly identify the rectangle among the

given shapes. About 28% of students have

chosen option C, a rounded rectangle. This

indicates that these students are not able to

recognize a rectangle in a non-standard

orientation. They may have a mentalpicture of a rectangle in a standard

orientation, and may have chosen the image

that is closest to this picture, irrespective of

whether it satisfies all the properties of a

rectangle or not.


28/56


27

Question Graphs


Explanation: This question checks if the

students can read a pictograph and answer

questions based on it. The students are

expected to understand from the key that

since the picture of one book stands for 2

books, Reena read total 32 books in her

vacation. Only 27% of the students chose

the correct answer. Students choosing

option C (49%) have probably notunderstood the key or not used it at all.

They have just counted the number of

books and chosen this option.



students are able to estimate the length of

an object relative to another given object.

Only 38% students have answered this

question correctly. Around 30% students

have chosen the wrong answer option A.

They have probably just looked at the

height of the scale given in the image and

chosen 100 cm as their answer.


29/56


28

Question Graphs



students are able to logically solve a

problem based on a daily life context.

Students are expected to understand that

they need 3 more marbles for equal

distribution and look for a number which

would become divisible by 10 on adding 3.

Only 32% students have answered this

question correctly. An equal percentage ofstudents (32%) have chosen the wrong

answer option A. They probably have not

understood the question and have just

added the numbers given in the question.



students are able to find the sum of given

integers. Only 36% students have

answered this question correctly. A lot of

students (31%) have chosen the wrong

answer option A. They probably haveadded all the 3 numbers and used the

negative sign with the answer. This

indicates that the concept of

integers/addition or subtraction of integers

is not clear to them.


30/56


29

Question Graphs



students are able to find the perimeter of a

given figure. They are expected to

understand that the side of each small

square on the grid is 1 cm and then

calculate the perimeter based on that. Only

23% students have chosen the correct

answer option C. A lot of students (41%)

have chosen option A. They have probablyeither just counted the number of small

squares that are shaded or have found out

the area instead of the perimeter. One

reason for this could be that traditionally

such grids in the textbooks are used to find

out the areas and not perimeters and

students are also confused between the

two concepts area and perimeter.



students are able to add two fractions withdifferent denominators when given in a real

life context. Only 32% students have

answered this question correctly. A lot of

students (34%) have chosen option D.

They have probably just added the

numerators and denominators separately.


31/56


30

Question Graphs



students are able to simplify a simple

algebraic expression. Students are expected

to know that when the brackets are

opened the sign (- or +) and the digit

outside the brackets get multiplied with

each term inside the brackets. Only 20%

students have answered this question

correctly. Students choosing option A(36%) have probably not multiplied 5 with

the y inside the brackets. Students

choosing option C (20%) have probably

multiplied 5 with the y inside the brackets

but ignored the - sign.



understanding of place value system in

decimals. Students are expected to realize

that 7 tenths and 8 thousandths make

0.708. Only about 24% of students havebeen able to answer this correctly. Students

may be used to converting conventional

expanded forms like 7/10 + 0/100 + 8/

1000 into decimals. But here when the

conventional order is changed and

hundredths are missing, many students may

have been confused. About 34% have

chosen option C, 0.8007 and 15% have

answered D, 0.87, retaining the order in

which the numbers appear in the question.


32/56


31

Question Graphs



students are able to observe a pattern and

logically think of a general rule being

followed in that pattern. Students are

expected to understand that 2 corners in

each face will not have antennae and hence

a face having n corners will have n-2

antennae. Only 29% students have

answered this question correctly. A lot ofstudents (31%) have chosen the wrong

option C. They probably have thought that

the 50th figure will have 50 antennae but

not considered the fact that the face having

50 corners will not be the 50thfigure.



students are able to express a part of a

group in terms of ratio. Students are

expected to calculate the number of boys in

the class by subtracting the number of girlsfrom the total number of students and then

give the ratio of number of boys to number

of girls. Only 25% students have answered

this question correctly. A lot of students

(33%) have chosen the wrong answer

option D. They are either just looking at

the numbers given in the question, i.e., 30

and 20, and are choosing their answer as

3:2 or are not careful about what is being

asked in the question.


33/56


32

3.1.2 Misconceptions and Common Errors in Language

Question Graphs


Explanation: The question tests students

logical analysis of information in a poster.

The poster is about a contest where

students have to send hilarious responses

to the questions asked. The judges will have

to read the responses before announcing

the winner. Students who have understood

this have chosen the correct answer, option

B. Most students (over 50%) have chosen

to skip answering the question probably

because they were looking for direct facts

mentioned in the poster. It is also possible

that due to long length of answer options

students had difficulty reading and

therefore skipped this one. It is likely that

students who chose options A, C, or D

relied on guesswork.

Class 3/ Question 22

Explanation: This question tests students

understanding of words and their meanings

based on the context. In this question, theword tested is snatched () and the

answer is A: took quickly. Some students

have understood this in the context and

have chosen option A. 32% students have

processed the information a little further

and have chosen option C, ate fast, which

is what the goats may do after snatching,

but that is not the act of snatching. These

students may either have not understood

the question or the context, or may have

been short of time.


34/56


33

Question Graphs



understanding of explicit and implicit

information presented in a given text. In

this question, there is a mention of the first

three prizes and then there is a mention of

additional prizes being given away. Students

who have got the correct count after

reading the authentic text have chosen the

correct answer, option D. Students whochose options B or C, (56%), as their

answer have missed out reading the details

in the passage. It is also likely that those

who chose option B have been influenced

by the prior knowledge that there are

generally only three prizes given away.



ability to identify the main theme in a given

text. In this context, the main activity as

required by the poster is to complete a

dialogue. Students who have been able to

abstract this information have chosen the

correct answer, option D. Many students

(35%) have misunderstood the question

and have chosen option C, use the pen or

buy it. They probably have thought that

the main activity the boy in the comic is

doing, is being asked about. Those choosing

other answers are merely referring to

details in the passage.


35/56


34

Question Graphs


Explanation: This question aims to test if

student understand implied facts. In the

question, the line is written in a certain

manner to imply that it is not the norm and

is in fact the opposite of what generally

happens. 37% students have incorrectly

chosen option D which is very similar to

the correct answer (A) but is not exactlywhat it means. This shows that they have

either resorted to prior knowledge or have

not understood what the sentence means.

Those choosing other options have tried to

locate facts from the passage to supply as

the answer.



student understand implied facts. In the

question, the meaning of a phrase is being

tested in context of the story. Majority of

students (38%) have wrongly chosen

option B. This could be because of they fact

that they didnt understand the meaning

and chose the answer which seems like a

genuinely good quality generally seen in the

protagonist of a text. Those choosing other

options are basing their responses on either

what they themselved would do, or what

they assumed the protagonist may do in

such a situation.


36/56


35

Question Graphs


Explanation: In the question, the traits of a

character based on the context in the text

are being tested. Most students (43%) have

failed to understand this and have based

their answer on the text that follows this

context, choosing option A which is an

explicit detail as the answer over an implicit

one. Those choosing options B and C have

either not understood the detail beingtested and have picked a response based on

the overview of the context, or they have

used prior knowledge based on such stories

where the antagonist has negative qualities

and tries to get the protagonist in trouble.



student can recall stated facts. The tested

sentence is very clearly mentioned in the

passage. Only 20% students have been able

to recall this fact correctly and have chosen

the correct answer, option B. Students who

have chosen the wrong answers have done

so either because they were trying to

locate implicit details or because they were

looking at other elements of the situation in

the passage.


37/56


36

Question Graphs


Explanation: In this question, the

underlying meaning of a given statement is

being tested. Only 21% students have

understood the overall text and have been

able to derive the meaning by choosing the

correct answer, option A. Students

choosing other options are looking for clues

in the preceding or following content.


Explanation: This question aims to teststudents understanding of the text and the

references made in certain places. Here,

the focus of the word is on the incident,

option D. Most students (44%) have

chosen option A, referring to the person

with whom the incident happened rather

than the incident itself. They have failed to

understand that the reference is being

made to the case and not the person

involved. Some students have chosen

option B, wrongly assuming that the

reference is being made to the knife lodged

in the patient.


38/56


37

Question Graphs



understanding of the text by being able to

differentiate between facts and opinions.

The given passage is in the form of a story.

The phrase is the writers opinion about

what happened. Students who have chosen

option A (20%) have probably not

understood the phrase and have assumed it

to be a cause and effect question. They

have chosen an answer which tells us thatnothing really happens to the protagonist

despite his errors. 30% students chosen

option C, probably because they have not

understood the phrase tested and have

gone for something which relates to both

Vasco and Malindi.



ability to draw inferences from a story. In

the given passage, it is Meena who thinks

she is a brave girl as she chooses the brave

thing to do to overcome her fears. 24%

students have chosen option A; this could

either be due to prior knowledge thatparents always think that their children are

brave and are the best. It could also be

because they assumed that she may have

later told her mother that she faced the

situation bravely and her mother must have

called her a brave girl. Students who chose

option B must have done so thinking along

the similar lines as those who chose option

A, only replacing mother with the music

teacher. Students who chose C probably

thought that the teacher thinks is brave as

she overcomes her fears and writes downthe bravest thing she will do. Also, in the

end it is mentioned that the teacher praises

her, though not for the same reason.


39/56


38

3.2 PERFORMANCE IN DIFFERENT SKILLS

Analysis was carried out on the performance of the students in the various skills across different classes and

subjects. This analysis enables one to identify patterns in performance across the different skills and understand

which skills are the strongest and which are the weakest. This understanding will pave the way for appropriate

action in the classroom.

3.2.1 Skill-wise Performance in Hindi

Class 3

S.

No.Medium Class Subject Skill

No. of

Questions

Sample

Size

%

Score

1 Hindi 3 LanguageRecognises and writes letters and knows

starting sound of words3 8231 85.3

2 Hindi 3 Language

Reads and writes simple words that are

moderately difficult and have 3-4 letters

and knows names of objects, birds,

animals, etc. seen in daily life

4 8231 81.7

3 Hindi 3 Language

Uses words appropriate to the context

based on their meanings, number and

gender

5 8231 75.6

4 Hindi 3 Language

Reads and writes 3 simple and short

sentences that have less than 5 words in a

sentence

5 8231 52.6

5 Hindi 3 Language

Reads descriptive text, of 3-5 sentences

independently and comprehends beyond

stated facts

12 8231 48.2

6 Hindi 3 Language

Understands written information

presented in various forms as Tables,Notices, Tickets, Posters, Labels, etc seen

in real life

3 8231 34.0

High Performing Skills Low Performing Skills

Class 5

S.


No. of

Questions

Sample

Size

%

Score

1 Hindi 5 Language

Reads and writes simple words (that are class

level appropriate) and knows names of

objects, birds, animals, etc seen in daily life

5 9151 81.3

2 Hindi 5 LanguageReads and writes sentences using grammar

concepts8 9151 68.2

3 Hindi 5 LanguageKnows synonyms, antonyms and deduces

word meanings from clues in context4 9151 70.6

4 Hindi 5 LanguageReads descriptive text and comprehends

explicit and implicit details for class level16 9151 53.9

5 Hindi 5 Language

Understands written information presented in

various forms as Tables, Notices, Tickets,

Posters, Labels, etc seen in real life for class

level

5 9151 29.3



40/56


39

Class 7

S.


No. of

QuestionsSample

Size

%

Score

1 Hindi 7 Language

Reads and writes simple words (that are class

level appropriate) and knows names of

objects, birds, animals, etc seen in daily life

4 9516 71.9

2 Hindi 7 LanguageReads and writes sentences using grammar

concepts8 9516 65

3 Hindi 7 LanguageKnows synonyms, antonyms and deducesword meanings from clues in context

3 9516 56.9

4 Hindi 7 LanguageReads descriptive text and comprehends

explicit and implicit details for class level20 9516 38.6

5 Hindi 7 Language

Understands written information presented in

various forms as Tables, Notices, Tickets,

Posters, Labels, etc seen in real life for class

level

9 9516 42.4


In Hindi language, the skill of Recognizes and writes letters and knows the starting sound of wordsobserved the

highest performance in class 3 while Reads and writes sentences using grammar concepts was the highest

performing in classes 5 and 7

The weakest skills in all classes in Hindi were skills of reading comprehension: Understands written

information presented in various forms as Tables, Notices, Tickets, Posters, Labels, etc. seen in real life and Reads

descriptive text and comprehends explicit and implicit details for class level. Thus understanding both the stories

as well as authentic text material seems to be a challenge.

Students seem to be weak in reading comprehension beyond stated facts. Performance is poorer on

inference based questions as compared to straight forward fact based questions. Examples given below

elucidate the same.

Students should be encouraged to write their own stories, as well as discuss stories that they have read.

Students can be asked to list the different qualities for each character in the story and describe how they feel

the characters will react in different situations and contexts.

HindiClass 7HindiClass 7

Sample Question: Q31 is a straight forward question testing a fact

cited in the passage. Student performance on this question is 65.7%.

Sample Question: Q36 requires students to infer after reading the

entire story, keeping in mind the facts stated. Student performance on

this question is 17.1%


41/56


40

3.2.2 Skill-wise Performance in Maths

Class 3

S.


No. of

QuestionsSample

Size

%

Score

1 Hindi 3 Maths Pre-maths skills 3 8231 78.9

2 Hindi 3 Maths Number sense and basic number competency 5 8231 57.5

3 Hindi 3 MathsArithmetic operations: addition and

subtraction4 8231 55.7

4 Hindi 3 Maths Arithmetic operations: multiplication 3 8231 67.3

5 Hindi 3 Maths Basic shapes and geometry 4 8231 59.6

6 Hindi 3 MathsApplication in daily life: money, time,

calendar, length etc.4 8231 52.2

7 Hindi 3 Maths Problem Solving 3 8231 51.8


Class 5

S.


No. of

Questions

Sample

Size

%

Score


2 Hindi 5 Maths Four arithmetic operations 5 9151 56.3

3 Hindi 5 Maths Fractions: concepts and applications 4 9151 40.5

4 Hindi 5 MathsApplication in daily life: money, time,

calendar, length etc.5 9151 41.1

5 Hindi 5 Maths Basic shapes and geometry 6 9151 51.5

6 Hindi 5 Maths Data interpretation and analysis 4 9151 54.2

7 Hindi 5 Maths Problem Solving 4 9151 38.3


Class 7

S.


No. of

Questions

Sample

Size

%

Score


2 Hindi 7 Maths Factors and Multiples 3 9516 37.8

3 Hindi 7 MathsApplication in daily life and word/visual

problems3 9516 33.7

4 Hindi 7 Maths Area and Perimeter 4 9516 37.3

5 Hindi 7 Maths Geometry: concepts and application 4 9516 45.1

6 Hindi 7 Maths Algebra: concepts and application 3 9516 35.8

7 Hindi 7 MathsFractions, Decimals and Ratios: concepts and

application4 9516 42.2

8 Hindi 7 Problem Solving 4 9516 27.8

9 Hindi 7 Data interpretation and analysis 4 9516 46.5

10 Hindi 7 Integers: concepts and application 4 9516 33.7



42/56


41

In Maths, the strongest skills were Pre-Maths skillsand Multiplication in class 3,four basic arithmetic operationsin

class 5 andData Interpretation and Analysis and Geometry in class 7.

The weakest skills wereApplications in daily lifeand Problem Solvingacross the three classes.

Transition from basic skills in primary classes to intermediate skills like Integers, Algebra in higher classes seems

to be a challenge

o

Students of class 7 seem to be struggling with Area and Perimeter problems a concept that is

introduced in Class 4

o Integers, a concept introduced in Class 5, also seems particularly weak as many students are

unable to answer straight forward questions

o The transition from concrete to abstract concepts in maths is a challenge and can be systematically

addressed through targeted curriculum efforts

Within a skill, different kinds of questions are covered - some could be straightforward or procedural, some

could be conceptual and some could require higher order thinking. In skills found weak, students are not ableto answer the conceptual and higher order thinking questions. This could be because of the stress laid on the

procedural ways of answering or solving questions. If instead, stress is laid on conceptual understanding,

students are more likely to answer different kinds of questions correctly.


43/56


42

EI conducted field audits as a part of this assessment. Over 30 schools were visited during test conduction by 10

Field Audit Resource Persons. The purpose of the audit was to ensure that the execution of the test met certain

standards of quality, maintained a certain level of standardization and to assess whether any issues of bias or

subjectivity can be traced at any of the schools. To ensure that the observations are comprehensive, the auditors

not only observed the Test Administration, but also interacted with the Teachers, Head Teachers and Students to

take feedback and opinions about the test and the testing process.

Each auditor spent an average of 40 minutes in each school and recorded his/her observations on a detailed

checklist. The key observations which the auditors were asked to make in their visits included the adherence to

the processes given in the evaluator manual, whether the teacher had participated in the evaluator training,

infrastructural observations such as issues of disturbance and cluttered classrooms, the responsiveness of the

teachers to the tests and testing processes, issues of cheating and bias through observations on seatingarrangements and teacher involvement, and perception of the teachers and students regarding the test and the

testing process. The responses were then analysed to deduce some key test conduction practices and

observations. Some key observations from the field audits are discussed below:

1. All the schools audited (except one) were ready with the test material and started the test at the scheduled

time. Test duration and timings were followed in all the audited schools. Seating arrangement was proper and

students were sitting comfortably. In most of the audited schools, students from an individual class sat

separately from other classes. No instance of mass assistance observed in the audited schools.

2. Over 80% of the evaluators present during test conduction did not get any formal training from the master

trainers who were trained by EI for this purpose. Though most of them were aware of the broad procedures;

in some schools this caused a little confusion and needed intervention by the Resource Persons in the

classroom from time to time.

3. The processes from the evaluator manual were not consistently followed in all the schools. This can also be

associated with the unfamiliarity of the evaluators with the processes indicated in the Manual.

4. Apart from the evaluator, there were other people present in the classrooms in around 40% of the observed

schools. The presence of external persons during test administration can have certain repercussions of bias

and unnatural environments in the classroom.

5. Over 80% of the school evaluators were aware of the school codes for their schools and helped the students

write the code on their sheets.

6. Over 80% of the classes observed conducted the tests with no serious disturbances.

7. Almost all the schools observed practiced seating arrangements that ensured that the room was not

overcrowded and that the students were seated in rows or a circle; in a way that ensured that the

arrangement minimised the issues of cheating and were deemed appropriate for a testing environment.

8. Most of the evaluators had enough test material for all students and for themselves to conduct the test

according to the procedures. However, in four of the observed schools, testing material was not sufficient for

all the students to test independently. Issues of shortage included the unavailability of the paper for the teacher

4. FIELD AUDITS


44/56


43

to read out the questions for class 3, exclusion of some students due to unavailability of papers as the

enrolment data shared was for previous year.

9. In a few schools, (about 26%) there were a few instances noticed where the teachers were assisting the

students in answering the questions, usually by giving clues.

10. In one school, the test was not conducted during the Resource Persons visit. Since the conduction of the test

was supposed to be at the same time across the schools, this school was probably not informed about the

testing.

11.With respect to the time durations of the tests, most schools followed the time durations prescribed for the

test. Issues of time included delayed commencement of the test and confusion and unfamiliarity about the test

practices such as School Codes and OMRs.

12.Some of the schools reported multi-grade classrooms. While some of the classrooms were effectively

partitioned to ensure minimum disturbance, others were more chaotic and not suitable for a testing

environment. However, no significant patterns were observed with respect to the classroom structure.

13.

Most students shared that they found the test interesting and mentioned a few questions / passages form thetest which were fun to do / read.

14.Some of the students and teachers reported that the testing format was unfamiliar and difficult, while others

commented on the difficulty levels of the Hindi papers and Reading Comprehension sections. This observation

however, varied across the interactions with the students and teachers.

The observations from the field audit visits have been summarized in the table given below.


45/56


44

School Code

Did the teacher

attend the

evaluator

training?

Were

processes

from the

Evaluator

Manual

followed?

Were

there

other

people

present

during

testing

?

Did the

teacher

know

the

school

code?

Did she

help

student

s write

it?

Disturbance

in class?

Was the

Class

overcrowded?

Was the

seating

arrangement

APPROPRIAT

E for testing?

Did the

teacher

have

sufficient

test

material?

Was

time

schedule

and

duration

followed

?

Comments

2110907702 N Y Y Y Y Y N Y NBig hall where classes were being conducted simultaneously,disturbance, friendly teachers, teachers said that the paper was

tough

2110907701 N N Y Y N N Y Y Y Students and teachers both found the tests very difficult

21109079011 N N Y Y N N Y N N Teacher filled OMR sheet himself

2020602602 N N Y Y Y N Y Y Y Teachers on rotation, thought language paper tested facts, givenno formal training, shortage of papers

2020602601 N N N Y N N Y Y Y No formal training for teachers, shortage of papers

2020601402 N N N Y N N Y Y YNo formal training for teachers, shortage of papers, did not find

papers tough

2110904801 N Y N N N N Y Y Y

2110904802 Y Y Y NA N N Y Y Y

2110904501 Y Y N Y N N Y Y Y Teachers and students liked tests

2110904501 N Y N Y N N Y Y Y Teachers and students liked tests

2040304307 N Y Y Y N N Y Y Y

2040301802 N N Y Y N N Y Y YUninformed teachers, paper found difficult, no break b/w tests for

class 3

2040304301 N N Y Y N N Y Y Y Difficult format of test, shortage/excess of papers

2050304201 N N N Y N N Y Y N Papers reached late, no teacher had read the manual

2050307601 N N N Y N N N Y NIssues of time of test-easy but lengthy, papers reached late, noteacher had read the manual

2050404701 Not tested during field audit

2040301102 N N N Y N N Y Y YHindi paper was observed to be tough, there was no extra copyfor teacher to read question for class 3

2040301701 N N N Y N N Y Y YHindi paper was observed to be tough, there was no extra copyfor teacher to read question for class 3

2040301905 N Y N Y N N Y Y Y Hindi paper was tough, test conducted in school ground

2100501201 N Y N Y N N Y Y NNo evaluator guide made available, Reading Comprehension wasfound to be tough

2100501202 Y Y N N N N Y Y N No evaluator guide, positive feedback

2100501401 N Y N Y Y N Y Y Y No evaluator guide, positive feedback

2100501302 N Y N Y N N Y N Y No evaluator guide, positive feedback

2100500902 N Y Y N N N N Y Y Trained teacher was not present, papers up to their level

2100501101 N N Y N N N Y N N Trained teacher was not present, issues in dealing with OMR

2100501001 N N N N Y N Y Y NInstructions not clearly followed, teacher was helping students toomuch

2050103601 N Y N Y N N Y Y Y

2050103801 N N N Y N N Y Y Y Teacher kept leaving class, students scared of test

2050203603 N Y N Y Y N Y Y Y


46/56


45

School Code

Did the teacher

attend the

evaluator

training?

Were

processes

from the

Evaluator

Manual

followed?

Were

there

other

people

present

during

testing

?

Did the

teacher

kno

Documents

Himachal Pradesh State Assessment - Main Report