IB Programme Impact Research: Findings and implications of recent research projects

IB Programme Impact Research: Findings and implications of recent research projects

2012 IB Conference of the Americas, MexicoJuly 2012

Olivia HalicResearch analyst

IB Global Research Department

© International Baccalaureate Organization 2007

Presentation objectives

Brief introduction to IB Global Research departmentProgramme impact research agendaStudent performance on ISATitle I IB schoolsBrief overview of in-progress research projects


IB Global Research

Programmeimpact

Quality Assurance

Professional development

School Services

Global coordination, support, and

services

Assessmentresearch

Programmedevelopment


Programme impact research agenda and priorities

Learner profileTo what extent do learners demonstrate characteristics of the IB learner profile?

What distinguishes IB learners in levels of motivation, values, and attitudes?

Student performance

How do IB learners perform on external measures of academic achievement?

How do they compare with non-IB peers?

Standards

How do IB standards compare to those at national/ state levels?

To what extent are IB graduates prepared for postsecondary success?

Programme implementationWhat is the impact/value-add of

implementing IB programmes in schools?What changes, if any, result from the implementation of IB programmes?What are the enablers/ inhibitors of

successful implementation?

Impact/value-add of IB programmes on learners and

schools


Recently completed studiesProject Title Programme Date

IB Students’ High School and Postsecondary Experiences in Chicago Public Schools

DP March 2012

PYP and MYP Student Performance on the International Schools’ Assessment (ISA), Phase 2

PYP & MYP March 2012

Student Performance and Student Engagement in the IB MYP

MYP July 2011

International Baccalaureate students studying at UK Higher Education Institutions: How do they fare?

DP April 2011

Postsecondary Enrollment Patterns of IB Certificate and Diploma Candidates from U.S. High Schools

DP March 2011

Postsecondary Enrollment Patterns of IB Certificate and Diploma Candidates from International High Schools

DP March 2011

First college courses taken by Florida IB students DP March 2011


Performance comparison between IB school students (PYP and MYP) and non-IB school students on the International Schools’ Assessment (ISA) – Phase II

Australian Council for Educational ResearchMarch 2012


International Schools’ Assessment (ISA)Areas of assessment

Mathematical Literacy

Uncertainty

Quantity

Space and Shape

Change and Relationships

Reading

Retrieving Information

Interpreting

Reflecting

Narrative Writing

Content

Language

Spelling

Expository Writing

Content

ESOL Language

Structure and Organisation


International Schools’ Assessment (ISA)Grade 3 to 10Two sessions per year (October/ February)Multiple-choice & open-ended items2 essaysAligned with OECD's Programme for International Student Assessment (PISA) scales for reading and mathematics


Study design-Sample

2009-10 & 2010-11 ISA sessions270 participating international schools

117 PYP and 86 MYP schools

50,714 students 34,690 IB students16,024 non-IB students


Study designReplication of phase I: 2009-10 & 2010-11 ISA data− Comparison of student performance by:

assessment areas & sub-strandsregion

− PISA benchmark analysis− Multilevel analysis of school variance (new)

Closer examination of particular findings from phase I− Continuum schools compared with single or dual programme

schools− Impact of the length of programme authorization

Student Learning and Wellbeing questionnaires (new)


Analysis

Sign Meaning Direction Cohen’s d+ Small effect size Higher IB scores 0.1 ≤ d < 0.2++ Medium effect size Higher IB scores 0.2 ≤ d < 0.5+++ Large effect size Higher IB scores d ≥ 0.5▬ Small effect size Lower IB scores 0.1 ≤ d < 0.2▬▬ Medium effect size Lower IB scores 0.2 ≤ d < 0.5▬▬▬ Large effect size Lower IB scores d ≥ 0.5

Significance legendStatistically significant difference: t-testPractical significance: effect size


Findings – Comparison of student performance on ISA



Grade IB Non-IB Significance Effect SizeMean S.D. N Mean S.D. N

3 310 84 6,455 322 90 2,903 – -0.134 379 84 3,788 376 89 1,995 0.035 425 83 6,872 425 89 2,577 -0.01

6 467 84 3,167 453 97 2,011 + 0.157 499 88 4,767 495 95 2,010 0.04

8 517 83 3,653 526 91 1,589 – -0.10

9 551 85 3,227 535 87 1,699 + 0.18

10 570 91 1,948 529 84 1,085 ++ 0.46


Reading

GradeIB Non-IB

Significance Effect SizeMean S.D. N Mean S.D. N

3 253 90 6,523 242 100 2,888 + 0.114 323 91 3,771 303 101 1,981 ++ 0.205 371 87 6,844 363 96 2,574 + 0.096 421 97 3,148 388 108 2,015 ++ 0.317 464 97 4,868 446 106 2,004 + 0.188 489 86 3,617 489 99 1,570 -0.019 533 90 3,352 504 94 1,694 ++ 0.3110 568 94 1,924 529 100 1,076 ++ 0.40


Narrative Writing

GradeIB Non-IB

Significance Effect SizeMean S.D. N Mean S.D. N

3 364 59 6,540 365 62 2,895 0.004 411 62 3,759 406 67 1,987 + 0.085 452 64 6,826 456 68 2,565 – -0.076 483 66 3,160 472 74 2,010 + 0.167 512 69 4,867 510 72 2,008 0.038 535 69 3,629 539 73 1,586 -0.069 557 74 3,337 545 77 1,697 + 0.1610 578 74 1,934 551 80 1,085 ++ 0.35


Expository Writing

Grade IB Non-IB Significance Effect SizeMean S.D. N Mean S.D. N

3 395 50 6,506 394 54 2,879 0.024 433 54 3,771 426 60 1,979 + 0.115 469 57 6,831 471 62 2,565 -0.046 493 59 3,141 486 65 2,019 + 0.117 524 61 4,858 517 66 2,002 + 0.108 548 62 3,616 551 68 1,570 -0.049 571 69 3,341 553 71 1,682 ++ 0.2610 594 66 1,921 563 81 1,073 ++ 0.43


FindingsIn general, IB students had significantly higher mean scores than non-IB students

In reading, IB students had higher scores in 7 out of 8 grade levelsIn expository writing, IB students had higher scores in 5 out of 8 grade levels

Grades 6, 9 &10: IB students outperformed non-IB students in each area; strongest results at grade 10


Europe & the AmericasDomain Grade IB Non-IB Significance Effect SizeMean N Mean N


3 319 2,364 322 407 -0.044 389 1,101 378 276 + 0.145 434 2,630 421 367 + 0.176 478 972 432 441 +++ 0.557 501 1,891 483 329 ++ 0.228 529 1,221 519 305 + 0.139 561 1,183 530 357 ++ 0.4210 559 565 531 424 ++ 0.34

Reading

3 259 2,457 221 406 ++ 0.404 336 1,100 297 275 ++ 0.445 383 2,611 350 363 ++ 0.376 433 968 359 444 +++ 0.757 466 1,997 436 328 ++ 0.318 503 1,188 501 298 0.029 545 1,316 515 354 ++ 0.3710 581 550 544 425 ++ 0.37

Narrative Writing

3 363 2,462 352 406 + 0.184 409 1,089 394 274 ++ 0.245 450 2,596 445 363 0.076 483 967 444 442 +++ 0.517 510 1,998 512 327 -0.028 539 1,198 547 304 – -0.139 560 1,299 549 357 + 0.1610 581 551 560 424 ++ 0.28

Expository Writing

3 396 2,442 390 402 + 0.124 434 1,098 420 273 ++ 0.245 469 2,597 466 359 0.066 494 963 463 447 +++ 0.517 522 1,991 516 328 0.098 550 1,186 547 298 0.059 572 1,310 550 354 ++ 0.3410 593 552 559 423 ++ 0.43


Comparison with PISA – Mathematical Literacy, Grade 9/10

9 10

551

570

OECD countries Partner countries

496


Comparison with PISA – Reading, Grade 9/10

9 10

533

568

OECD countries Partner countries

493


Student Questionnaire-Primary Years

Primary Years(Grades 5 & 6) Example items

Student and Teacher Interaction Most of my teachers really listen to what I have to say.

Social Connectedness I feel like I belong.

Personal Development Outcome I like helping someone with a problem.

Study Engagement I try very hard to complete all my work.


Student Questionnaire-Secondary Years

Secondary Years(Grades 8 & 9) Example items

Student and Teacher Interaction

If I need extra help, I will receive it from my teachers.

Social Connectedness I feel like I belong.

Personal Development Outcome I am good at solving conflicts without fighting.

Learning Goals I enjoy trying different approaches to see which one will work.

Deep Learning I am keen to know how the things we do in class are meaningful to me.

Surface Learning I don’t spend time learning things that I know won’t be in the exam.

Academic Outcome Orientation

I like to work on tasks that I have done well on in the past.


Student Questionnaire

Findings - High agreement rates • Supportive schools: 88% -PYP students; 80% MYP• Feeling of connectedness: 83% - PYP & MYP• Engaged in their study: 83% - PYP & MYP• Deep learning: 80% MYP• Focus on learning for assessment: 56% MYP

Sample size

PrimaryN = 11,632

SecondaryN = 10,058

Response rate

Primary> 80%

Secondary> 87%


ImplicationsStrongest results at grade 10 – growing trend

Sustained results relative to ISA 2007/2009

Performance of PYP & MYP students measured against external benchmarks, with favorable results

Comparison with PISA – additional opportunity for evaluation against international benchmarks

Questionnaire results inform about student perceptions and non-scholastic outcomes


Questions?


Title I IB Schools

IB Global ResearchMay 2012


BackgroundIB commitment to expanding access among underserved student populations

Title I of the Elementary and Secondary Education Act of 1965

Two types of Title I assistance:Schoolwide programmes: schools with 40% or more of students from low-income familiesTitle I Eligible programmes: schools with less than 40% of students from low-income families


Data2009–2010 National Center for Education Statistics (NCES) Common Core of Data (CCD)

103,959 public schools in the United States

IB school data: 1,389 public schools in the U.S.


Findings - IB Schools

56% of IB public schools (IB World Schools or candidate schools) were designated Title I


Findings - IB Programmes

AllIB World Schools Candidate schoolsAll DP MYP PYP All DP MYP PYP

Total IB Programmes 1,623 1,196 622 360 214 427 65 192 170

Title I 885 645 277 209 159 240 34 112 94

% Title I 55% 54% 45% 58% 74% 56% 52% 58% 55%

Schoolwide Title I 633 475 181 174 120 158 15 75 68

% Schoolwide Title I 39% 40% 29% 48% 56% 37% 23% 39% 40%


Findings - IB Programmes1,623 IB programmes in US public schoolsOverall, 55% of programmes offered in Title I schools74% of PYP offered in Title I IB world schools

74%

45%58%


ImplicationsIB’s goal to develop a more inclusive IB community by enabling access to education regardless of personal circumstances

Understand who IB students and schools are, to provide better services to a increasingly diverse community


Questions?


In-progress studiesProject Title ProgrammeThe Relationship between MYP Student Moderation Performance and DP Student Performance MYP & DP

DP Extended Essay Series DP

Enrolment and Achievement of IB DP Graduates in the Australian Tertiary Education Sector DP

Examining Gender Differences in the IB DP STEM Subjects DP

Performance and Engagement in the MYP - Continuation study MYP

MYP UK MYP

India PYP PYP

IB Teacher Continuum


BRIEF OVERVIEW

The Relationship between MYP Student Moderation Performance and DP Student Performance


Purpose

To investigate whether DP students benefit from previously completing the MYP

DataMYP moderation scores from 2007-2009DP exam scores from 2009-20116,352 MYP to DP students from 48 countries


Preliminary results

Higher MYP moderation scores ↔ higher DP exam scores

One unit increase in the MYP score leads to a 0.4 increase in the DP scores

MYP students develop knowledge and skills that prepare them for the DP


BRIEF OVERVIEW

Performance and Engagement in the MYP - Continuation study


PurposeTo examine the influence of MYP on later high school performance and course enrollmentTo further explore student and teacher experiences with the MYP

Data5 middle schools with MYP & 5 IB high schoolsStudent performance and course enrollment (grades 9 & 10)Student surveys (including global mindedness scale)Teacher surveys & interviews


Preliminary findings554 MYP students & 175 non-MYP students

Moderate or large amount of time spent learning

MYP Non-MYP

Issues affecting people around the world 51% 43%

Issues related to the environment 61% 52%

Other cultures 52% 45%


Questions?


Thank you!Website

Research overview • http://ibo.org/research/

Programme Impact studies• http://ibo.org/research/policy/programmevalidation/

Email• [email protected]• [email protected]

http://ibo.org/research/

http://ibo.org/research/policy/programmevalidation/

mailto:[email protected]

mailto:[email protected]

Documents

IB Programme Impact Research: Findings and implications of recent research projects