An Introduction to TIMSS& How We May Use the Data?

Berinderjeet KaurNational Institute of Education

Singapore

29 May 2013

TIMSS (Trends in International Mathematics

and Science Study)

TIMSS • Is a series of international mathematics and science

assessments conducted every four years.

• It is designed to provide trends in fourth- and eighth-grade mathematics and science achievement in an international context.

• TIMSS 2011

- was the fifth and most recent cycle of assessment.

- 45 countries participated at the 8th grade level &

52 countries participated at the 4th grade level.

- data were collected from representative samples of

students, in participating countries, at the respective

grade levels.

- teacher participants were not representative samples as

they were the teachers of the students.

• The TIMSS 2011 International Results in Mathematics and Science are reported in two respective reports.

• These reports are comprehensive.• They contain analysis of data that spans from

achievement of participants to home environment that supports mathematics and science achievement, school resources for teaching mathematics and science, school climate, teacher preparation and classroom instruction.

• Singapore has participated in all the five cycles of TIMSS so far.

What can we learn from International Assessments such as TIMSS and PISA

Introduction• Large scale international assessments of schooling effects

attempt to provide comparative data for participating countries.

• Two examples of such assessments are

- TIMSS conducted by the IEA (International Association

for the Evaluation of Educational Achievement

- PISA conducted by the OECD (Organisation for

Economic Cooperation and Development)

• These studies use student outcomes as measures of school effectiveness and educational achievement.

• They focus on student achievement mainly in three school subjects: mathematics, science and language.

• A large corpus of data (student, teacher, school, country) is collected during every cycle of the studies.

• But often achievement of students in mathematics, science and language are the highlight – league tables reported in media.

• Needless to say, other than achievement scores the data from such studies have much to offer for both within and between country insights.

• This presentation will use Singapore as a case for deliberation for the school subject mathematics and clarify the impact such studies may have on educational developments of participating countries.

Case Study: Singapore

Why does Singapore participate in TIMSS & PISA ?

Singapore participates in international studies such as TIMSS and PISA for four main purposes.

i)to benchmark the outcomes of schooling, viz-a-viz the education system against international standards;

ii)to learn from educational systems that are excelling;

iii) to update school curriculum and keep abreast of global advances; and

iv) to contribute towards the development of excellence in education internationally.

Participation in TIMSS and some learning points

• Singapore has participated in TIMSS since 1995 for both grades 4 and 8.

• Generally, the results of every cycle of TIMSS for Singapore, has affirmed that students at both grades 4 and 8 have mastery of content knowledge according to international standards.

• In addition they are also highly proficient in the application of their knowledge and reasoning with their knowledge of mathematics.

TIMSS: Singapore students achievement in Mathematics

Rank Rank

TIMSS Grade 4 Grade 8

1995 1 1

1999 - 1

2003 1 1

2007 2 3

2011 1 2

International benchmarks• The international benchmarks presented as part of the

TIMSS data help to provide participating countries with a distribution of the performance of their 4th and 8th graders in an international setting.

• For a country the proportions of students reaching these benchmarks perhaps describe certain strengths and weaknesses of mathematics education programs of the country.

• The benchmarks delineate performance at four points of the performance scale, advanced, high, intermediate and low.

• Due to time constraint, only grade 8 data will be used for discussion.

Percentages of grade 8 students reaching TIMSS 2011 international benchmarks of mathematics achievement

Country Advancedbenchmark(625)

Highbenchmark(550)

Intermediatebenchmark(475)

Lowbenchmark(400)

Chinese Taipei 49 (1.5) 73 (1.0) 88 (0.7) 96 (0.4)

Singapore 48 (2.0) 78 (1.8) 92 (1.1) 99 (0.3)

Korea, Rep of 47 (1.6) 77 (0.9) 93 (0.6) 99 (0.2)

Hong Kong SAR 34 (2.0) 71 (1.7) 89 (0.7) 97 (0.3)

Japan 27 (1.3) 61 (1.3) 87 (0.7) 97 (0.3)

Australia 9 (1.7) 29 (2.6) 63 (2.4) 89 (1.1)

New Zealand 5 (0.8) 24 (2.6) 57 (2.8) 84 (1.6)

Malaysia 2 (0.4) 12 (1.5) 36 (2.4) 65 (2.5)

Thailand 2 (0.4) 8 (1.3) 28 (1.9) 62 (2.1)

Indonesia 0 (0.1) 2 (0.5) 15 (1.2) 43 (2.1)

International Median

2 17 46 75

• As shown in the table almost half of the students from Chinese Taipei, Singapore and Korea were at the Advanced benchmark.

• So what does this signal to mathematics educators in Singapore?

- It affirms that achievement of the top group of students in Singapore is at par with those from Chinese Taipei and the Republic of Korea.

- These students can• reason with information, draw conclusions, make generalizations, & solve linear

equations;• solve a variety of fraction, proportion, and % problems and justify their conclusions;• express generalisations algebraically and model situations;• solve a variety of problems involving equations, formulas, and functions;• reason with geometric figures to solve problems; and• reason with data from several sources or unfamiliar representations to solve multi-step

problems.

Figure 1: An advanced international benchmark item.

Content Domain: NumberCognitive Domain: ReasoningDescription: Given two points on a number line representing unspecified fractions, identifies the point that represents their product

Country Percent correct

Chinese Taipei 53 (2.0)

Hong Kong SAR

47 (2.5)

Singapore 45 (2.0)

Korea, Rep of 44 (2.0)

Japan 43 (2.1)

Australia 23 (2.1)

New Zealand 19 (2.3)

Malaysia 18 (1.4)

Thailand 12 (1.5)

Indonesia 10 (1.7)

International Avg

23 (0.3)

• Figure 1 shows an advanced benchmark item in TIMSS 2011. The performance of grade eight students from Singapore on this item is significant to mathematics educators in Singapore. It affirms that the “Thinking Schools, Learning Nation” (Goh, 1997) vision of the Ministry of Education for all schools in 1997 is bearing fruit.

• However, at the other end of the achievement spectrum 1 % of the grade 8 students from Singapore did not even make it to the low benchmark level. At the low benchmark level students have some knowledge of whole numbers and decimals, operations, and basic graphs. Figure 2 shows a low benchmark item in TIMSS 2011.

Figure 2. A low international benchmark item

Content Domain: AlgebraCognitive Domain: KnowingDescription: Evaluates a simple algebraic expression

Country Percent correct

Korea, Rep of 92 (1.0)Chinese Taipei 91 (1.0)Singapore 91 (1.1)Japan 86 (1.5)Hong Kong SAR 83 (1.8)Australia 71 (2.6)Indonesia 65 (2.4)New Zealand 61 (2.6)Thailand 56 (2.2)Malaysia 47 (2.1)International Avg 71 (0.3)

Item analysis

• At the national level, item analysis of all released items after every cycle of TIMSS is of interest to mathematics educators in Singapore.

• Such an analysis helps to check the performance of students on items with respect to the content and cognitive domains.

• Prior to 2006, the topic probability was only taught to grades 9 and 10 students in Singapore but our participation in TIMSS 95, 99, 2003 convinced us that perhaps we need to introduce our students to the basic concepts of probability in grades 7 and 8.

• So following a review of the school mathematics curriculum by TIMSS 2007 grade eight students in Singapore were introduced to the basic concepts of probability in grade 8.

Figure 3. TIMSS 2003 & TIMSS 2007 data and chance item 1

Content Domain: Data and ChanceCognitive Domain: Knowing

Country

% Full CreditTIMSS 2003

TIMSS 2007

The smaller box contains 20 tickets numbered from 1 to 20. The larger box contains 100 tickets numbered from 1 to 100.

Without looking at them, you can pick a ticket from either box. Which box would give you the greater chance of picking out a ticket with the number 17 on it?a)The box with 20 ticketsb)The box with 100 ticketsc)Both boxes would give the same chanced)It is impossible to tell

Rep of KoreaChinese TaipeiHong Kong SARJapanSingaporeInternational Average

94.1

87.5

84.3

84.375.3

62.8

94.6

87.8

79.3

87.878.5

59.4

       

Figure 4. TIMSS 2003 and TIMSS 2007 data and chance item 2

Content Domain Data and ChanceCognitive Domain: Applying

Roland’s spinner has three sectors of different colours, orange, purple, and green. Roland spins the pointer 1000 times. The chart below shows how many times the pointer stops on each section.

Country% Full Credit

TIMSS 2003

TIMSS 2007

Rep of Korea

Japan

Singapore

Chinese Taipei

Hong Kong SAR

International Average

68.2

65

48.9

47.4

47.3

32.1

50.5

70.4

53.7

39.6

46.6

27.3

Colour Times stoppedOrange 510Purple 243Green 247

Draw lines on the spinner above to make the three sectors the approximate size you would expect them to be. Label them orange, purple, and green.

     

• Our national analysis of the TIMSS data also shows that the achievement of grades 4 and 8 students on some specific mathematics items warrants attention, particularly when the content domain of the items is an integral part of the school mathematics curriculum.

• Such analysis is useful for our classroom teachers.

• Following the release of TIMSS 2007 and TIMSS 2011 data the Centre for International Comparative Studies (CICS) at the National Institute of Education (NIE) published reports that present to educators in Singapore achievement data for students in Singapore and countries that ranked within the top 5 as well as those of interest for comparative purposes.

• It also presents student attitudes data for the same.

• Most importantly the reports highlight items that students found easy and difficult.

Recognition and Contribution to Education Internationally

Textbooks• Singapore’s mathematics education gained international

recognition following repeated good performance in TIMSS.

• The data of the studies showed that not only did the average Singapore students perform very well against international benchmarks they also had a positive attitude towards the learning of mathematics.

• In a study of mathematics textbooks from Singapore the American Institutes for Research noted that in the textbooks the topics were treated in depth, with appropriate illustrations and mathematical representations.

• It also noted that of all the elements of Singapore’s successful mathematics system, its textbooks were the easiest to transfer to US schools, certainly with adaptations.

• So, presently in the US, Singapore Math is a teaching method based on the primary textbooks and syllabus from the national curriculum of Singapore.

• Like the United States many other countries, Indonesia, Philippines, Israel and others, have also adopted and adapted Singapore mathematics textbooks for use in their schools.

• It must be noted that textbooks are only tools of the teacher as without comprehensive understanding of the underlying philosophy of the books the implementation may be problematic.

• Therefore in many of these countries, educators from Singapore are invited to provide professional development for key instructional leaders in mathematics.

An International Systemic Benchmark• To improve educational practices and move up the

educational value chain Singapore always benchmarks itself with the best systems in the world.

• For example Singapore’s mathematics curricula were developed after reviewing mathematics research and practice from around the world.

• Following participation in TIMSS (1995, 1999, 2003, 2007 and 2011) and PISA 2009 and subsequent achievement of students from Singapore in the studies Singapore has become notably an international benchmark for others in the world.

• In a report entitled: How the world’s best performing school systems come out on top by Mckinsey & Company lessons that the world can learn from Singapore as one of the world’s best performing school system are detailed.

• In another report produced by OECD for the US entitled: Strong performers and successful reformers in education: Lessons from PISA for the United States, a case study of Singapore’s education system is presented as an example of a nation that has had rapid improvement followed by strong performance.

• Both these reports and the many other research papers that have drawn on Singapore’s data have presented very succinctly Singapore as an international systemic benchmark worthy of emulation by nations desiring change and growth.

THANK YOU