Information About the Accountability Provisions of No Child Left Behind

California Department of Education

Policy and Evaluation Division

July 2003

2

Timeline of Events

Accountability Workbook submitted by January 31, 2003 deadline

Peer Review occurred February 26

Submitted State Plan and additional material for the Accountability Workbook on May 1

Discussions and negotiations continued with US Department of Education through June 6, 2003

3

Final Workbook Approval

Final approval of workbook by US Department of Education (USED) on June 10

State Board of Education (SBE) approval of revisions required by USED on June 11

For information on California’s state plans: http://www.cde.ca.gov/pr/nclb/stateapp.html

4

Approved Without Additional Revision API as additional indicator CAHSEE as 10th grade academic

measure Subgroup size (100 or 50 if 15%) Intermediate goals for meeting annual

measurable objectives Definitions of “mobility”

5

Significant Revisions Participation Rate – for grades 2 – 8, parent

exemptions must be counted in “number of students enrolled”

Graduation Rate – CAHSEE proxy replaced with NCES formula

EL Subgroup – EL for 3 years of proficiency in ELA CST

ASAM indicators replaced with AYP Small school AYP determination done by

state

6

Timeline for Release of Reports July 2003:

– 2002 Base APIs for districts and ASAM schools

– 2002 Baseline AYP report (2002 testing data)• Advisory to LEAs• Videotape• Information Guide• Staggered Release

7

Timeline for Release of Reports August 2003:

– Phase 1 2003 AYP report (AMO’s and participation rate)

October 2003: – Phase 2 2003 AYP report (API and graduation rate) – 2003 Growth API release

December 2003: – Phase 3 2003 AYP report (updated data and

application of “safe harbor”)– Certified 2003 Growth API report

8

2002 AYP Baseline Report The CDE had hoped to release the 2002 AYP

baseline information in late May 2003 With the approval of our Accountability

Workbook, CDE will release this information in July

This report provides a starting point for each school using the new AYP definition and metric (e.g., percent proficient)

Districts can see which schools may be at risk for not meeting AYP when the August 2003 is posted

Adequate Yearly Progress

10

Adequate Yearly Progress (AYP):The Basics Based on English language arts and

mathematics separately

All students held to same high academic standards

Goal is 100% proficiency by 2013-14

Inclusion of all students– 95% participation on assessments– Accountability for all students

11

Components of AYP1) Achievement of the statewide Annual

Measurable Objectives (AMO’s) in both English language arts (ELA) and math

• “Percent proficient”

2) Achievement of a 95% participation rate on all applicable assessments

3) Achievement on the “additional” indicators

• API for all schools, and

• Graduation rate for high schools

12

AMOsin ELA and

Math

95% Participation

Rate

API

Graduation rate

AA YY PP

13

Annual Measurable Objectives (AMO’s)

For Elementary and Middle Schools are based on:

– The California Standards Tests (CSTs) in English language arts and math

– The California Alternate Performance Assessment (CAPA) for students with severe cognitive disabilities

For High Schools are based on:

– Results from the Grade 10 California High School Exit Exam (CAHSEE) administration

– The California Alternate Performance Assessment (CAPA) for students with severe cognitive disabilities

14

High School Data ETS completed the technical procedure to

set three performance levels on the CAHSEE for NCLB purposes (see slides15 and 16)

Starting points for high schools were set using the new CAHSEE performance levels and the methodology set by NCLB

Annual measurable objectives and intermediate goals were set for high schools and approved by the State Board of Education (see slides 22 and 23)

15

Using CAHSEE to Generate “Percent Proficient” – High schools only

NCLB requires that three performance levels (i.e. advanced, proficient, basic) be set on all assessments used for AYP

Linked CAHSEE ELA to grade 10 CST-ELA

Linked CAHSEE Math to grade 7 CST-Math

Technical process done by ETS

16

Cut Scores on the CAHSEE for NCLBHigh schools only

English Language Arts

Advanced = 413 or above

Proficient = 387-412 Not Proficient = Below

387

Math

Advanced = 417 or above

Proficient = 373-416 Not Proficient =

Below 373

These Cut Scores are Independent of the CAHSEE Pass Score.The CAHSEE Pass Score will Remain Unchanged.

17

More On The AMO’s…

Statewide goals are applicable to ALL– Schools, including alternative and charters– Subgroups– Districts– State

NCLB requires – Annual goals– Intermediate goals (no more than 3 years apart)

18

2003 AMOs for Schools2003 AMOs for Schools

ELA Math

Elementary or Middle School

13.6% 16.0%

High School 11.2% 9.6%

19

School and District AMOs

Elementary/Middle Elementary District School AMOs AMOs

Unified District, High District (7-12) AMOs

High School High SchoolAMOs District (9-12)

AMOs

20

2003 AMOs for Districts2003 AMOs for Districts

ELA Math

Elementary School District

13.6% 16.0%

High School District(Grades 9-12)

11.2% 9.6%

Unified or High School District (Grades 7-12)

12.0% 12.8%

21

Defining the Starting Point for the AMO’s

USE THE HIGHER VALUE

Option 2:

•Statewide % of students proficient in lowest achieving group:

-Economically disadvantaged

-Major racial/ethnic groups

-Disabled students

-ELL Students

Option 1:

•Rank all schools by % proficient

•Count from bottom up to to reach 20% of total enrollment

•Percent of students at proficient at that school is the starting point

22

AMO’s: English language artsElementary and Middle Schools and Elementary Districts

89.2%

24.4%13.6%

67.6%56.8%

46.0%35.2%

78.4%

100.0%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2001-2002

2002-2003

2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

2008-2009

2009-2010

2010-2011

2011-2012

2012-2013

2013-2014

23

AMO’s: Math Elementary and Middle Schools and Elementary Districts

37.0%47.5%

58.0%68.5%

79.0%89.5%

26.5%16.0%

100.0%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2001-2002

2002-2003

2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

2008-2009

2009-2010

2010-2011

2011-2012

2012-2013

2013-2014

24

AMO’s: English language artsHigh Schools and High School Districts

11.2%

88.9%

22.3%

100.0%

77.8%

33.4%44.5%

55.6%66.7%

0%

20%

40%

60%

80%

100%

2001

-200

2

2002

-200

3

2003

-200

4

2004

-200

5

2005

-200

6

2006

-200

7

2007

-200

8

2008

-200

9

2009

-201

0

2010

-201

1

2011

-201

2

2012

-201

3

2013

-201

4

25

AMO’s: Math High Schools and High School Districts

9.6%

88.7%100.0%

77.4%66.1%

54.8%43.5%

32.2%20.9%

0%10%20%30%40%50%60%70%80%90%

100%

26

AMO’s: English language artsUnified Districts and High School Districts with Grades 7/8

12.0%23.0%

89.0%

100.0%

78.0%

34.0%45.0%

56.0%67.0%

0%10%20%30%40%50%60%70%80%90%

100%

2001-2002

2002-2003

2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

2008-2009

2009-2010

2010-2011

2011-2012

2012-2013

2013-2014

27

AMO’s: Math Unified Districts and High School Districts with Grades 7/8

12.8%23.7%

89.1%

67.3%56.4%

45.5%34.6%

78.2%

100.0%

0%10%20%30%40%50%60%70%80%90%

100%

2001-2002

2002-2003

2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

2008-2009

2009-2010

2010-2011

2011-2012

2012-2013

2013-2014

28

Participation Rates

95% required on any assessment used for AYP under NCLB

The remaining 5% is the maximum allowable percentage of non-participants, including students who are exempted from testing at parental request.

29

Additional Indicators

The API will serve as the “other” indicator for all grades– How would a school meet the “other” indicator?

• API above the “status bar”, OR• Show growth of at least one point

Graduation rate will be an additional indicator for high schools– Demonstrate a one-tenth of a percent increase up

to 100%

30

The API “Status Bar’

Defining Progress on the API as the "Other" Indicator of AYP

770740

710

680

650620

800

560 590

500

550

600

650

700

750

800

2001-02

2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-2011

2011-2012

2012-2013

2013-2014

AP

I Sco

re

31

Graduation Rate National Center for Educational

Statistics (NCES) four year completion rate

Progress = increase of one tenth of one per cent per year until the school reaches 100%

32

Graduation Rate

High School Graduates, year 4

[ High School Graduates, year 4 + (Grade 9 Dropouts, year 1 +

Grade 10 Dropouts, year 2 + Grade 11 Dropouts, year 3 +

Grade 12 Dropouts, year 4) ]

Four year graduation rate as required by NCLB:

33

Graduation Rate Example2002 2003

100 / (100+2+1+3+4) =90.9% Grad Rate

120 / (120+5+2+1+3) = 91.6% Grad Rate

Change in rate: 91.6% - 90.9%

= .7%

Met requirement

Must increase Grad Rate by at

least .1% to meet requirement

34

Safe Harbor Definition Alternate method of meeting AYP if a subgroup is

showing progress in moving students from “basic” to “proficient”

“All students” is considered a subgroup

If a subgroup or the school fails to make the AMO, they may make AYP if:

– The percentage of students below proficient decreases by 10% over the prior year

– The group has at least 95% participation– The group meets the “other” indicator

35

Safe Harbor ExampleYear 1 Year 2

60% of the students are performing below proficient

54% of students are performing

below proficient

Met participation rate and

other indicator

Met AYP

10% of 60% is 6 percentage

points

+

Where Are Results Counted?

37

NCLB Student Mobility Rules

Student was enrolled since CBEDS date

Count in school accountability

report

Count in district accountability

report

Student was enrolled in more

than one school in the same district

since CBEDS date

Yes

Yes

No

Count in state accountability reportNo

38

Mobility Definitions Full academic year = Enrollment from CBEDS date to

first day of testing

“Continuously enrolled”– The student did not withdraw or was not dropped from the

school’s (or LEA’s) enrollment any time between the CBEDS census date and the first day of testing”

2002 baseline AYP report will use the current API rule (enrolled in district since CBEDS date)

New mobility rules begin go into effect with the August 2003 AYP report (2003 testing cycle)

Other Issues

40

Subgroup Size Reporting will occur for groups with at least 11

students

Schools will be held accountable for groups that have:– 100 students, OR– 50 students that comprise 15% of the student population

This rule will apply to schools and districts

CDE is pursuing legislation to align API with AYP rules for sub group size

41

Subgroup Definitions

All racial/ethnic definitions will remain the same as with the API (collected via STAR)

Socio-economically disadvantaged will be used per API definition

Students with disabilities included if they have a disability coded on the STAR answer document

42

Subgroups African American (not of Hispanic origin) American Indian or Alaska Native Asian Filipino Hispanic or Latino Pacific Islander White (not of Hispanic origin) Socioeconomically disadvantagedNEW: Students with disabilities English Learners

43

English Learners All students designated on the student answer

document as EL (English Learners) or as RFEP (Redesignated Fluent English Proficient).

RFEP students will continue to be included until

they have attained the proficient level on the CST in ELA for three years consistent with the federal definition of limited English proficient students in paragraph (25) of Section 9101 of Title IX of NCLB.

44

District Accountability Held to same AYP criteria as schools; will be held

accountable for all students enrolled in the district for a full year (not just those who aren’t counted at the school level)

Districts will receive a 2002 Base report; first AYP report in August 2003

Will be identified for Program Improvement (PI) in the same manner as schools The first year a district could be identified is proposed to be 2004-05.

The CDE will provide additional guidance

45

Schools With Fewer than 100 Valid Scores CDE will assume responsibility for

establishing AYP for schools with fewer than 100 valid test scores:– Step 1: Apply pairing and sharing for schools

with grade spans outside the testing program

– Step 2: Aggregate test results across years

– Step 3: Apply statistical test to achieve a 95% confidence interval

46

Schools With Fewer than 100 Valid Scores School results with a small number of scores

tend to fluctuate For these schools, California’s NCLB

accountability plan requires that determination of AYP be based on statistical procedures to adjust for fluctuations

These procedures are posted on CDE’s AYP web site at http://www.cde.ca.gov/ayp

Identification of Program Improvement Schools and Districts

48

AYP for Title I Schoolsand Districts Applies to all schools and districts that

receive Title I funds Title I schools and districts must meet all four

components of AYP Percent of students proficient or above on

statewide assessments Student participation rate in the statewide

assessments API Graduation rate (high schools)

49

2003-04

Title I Schools Identified for PI

Did not meet the annual measurable objectives (AMOs) in the same content area (English-language arts or math) in both 2001-02 and 2002-03

or Did not meet any one of the other

components of the AYP.

50

Identification of Schools for Program Improvement 2002 data was used ONLY to exit schools

from Program Improvement (PI)– See letter from the CDE dated February 6, 2003

2003 data will be used to determine AYP for all schools and districts– New schools may enter PI– Schools may advance to later years under NCLB– Schools may exit– (Districts will not enter PI until after 2003-04)

51

Identification of Schools for Program Improvement NCLB Requirement:

– Schools enter PI when they fail to make AYP for two consecutive years

In California– A school will enter PI only if the school fails

in the SAME content area for two consecutive years (participation rate or AMOs)

52

Title I Districts Identified for PI

Did not meet the annual measurable objectives (AMOs) in the same content area for two consecutive years for any significant subgroup or district-wide

or The district did not meet any one of the

other components of AYP.

53

AYP for School-wide Programs vs. Targeted Assistance For 2002, AYP was evaluated differently for

TAS than for SWP schools– For SWP: AYP = meeting school-wide API target

and targets for all numerically significant subgroups

– For TAS: AYP = meeting API target for socio-economically disadvantaged subgroup only

NCLB allows for differential treatment of TAS

54

Appeal Process for PI Schools and Districts

A district may appeal on its own behalf or for a school.

Appeal must be based on substantive or statistical error (to be defined).

30-days to file appeal and to receive final determination

55

NCLB PI School RequirementsYear 1 Program Improvement

Revise school plan.

Use 10% funds for staff development.

Provide school choice with paid transportation.

District provides technical assistance (TA).

56

NCLB PI School RequirementsYear 2 Program Improvement

Continue

– Staff development

– Choice

– District TA

Add

– Supplemental services/tutoring

57

NCLB PI School RequirementsYear 3 Corrective Action

Continue

– District TA

– Choice

– Supplemental services Add

– District corrective action

58

NCLB PI School RequirementsYear 4 Corrective Action

Continue

– District TA

– Choice

– Supplemental services Add

– Development of plan for alternative governance

59

NCLB PI School RequirementsYear 5 Restructuring

Implement alternative governance plan

– Reopen as charter.

– Replace staff.

– Contract with external entity.– Takeover by state.

60

PI District Requirements Year 1 Program Improvement

Revise LEA Plan. Use 10% funds for staff development. Target students not making AYP. Provide extended learning

opportunities. Involve parents. Receive TA from state.

61

State takes one corrective action:– Reduce funds;– Institute new curriculum and staff

development;– Replace personnel;– Appoint trustee;– Restructure LEA;

Authorizes choice and one of the above actions.

PI District Requirements Year 2 Implement PlanYear 3 State Corrective Action

62

PI Schools Identified Prior to NCLB Placed in Year 1, Year 2, or Year 3 for

the entire 2002-2003 school year, in accordance with NCLB.

63

Options for Existing PI Schools(Years 1 and 2) in 2003-04

Schools will exit PI status:

– Made AYP in 2002 and 2003. Schools will remain in place:

– Made AYP in 2003. Schools will advance to the next

level under NCLB:– Did not make AYP in 2003.

64

Options for Year 3 Corrective Action Schools 2003-04

Identified for 2001-02 school year.– Made AYP in 2002 and 2003, will exit PI.– Did not make AYP in 2003, will advance to

Year 4.– Made AYP in 2003, will remain in Year 3.

Identified for 2002-03 school year.– All will remain in Year 3 for 2003-04.

65

After August 2003 AYP Report

New PI schools must move immediately to implement school choice.

Advancing PI schools must move immediately to implement new requirements.

PI schools remaining at the same level must continue required activities.

66

After October 2003 AYP Report

Schools that made AYP for August Report, but did not make AYP for October Report, must immediately move to implement the requirements of NCLB.

67

After December 2003 AYP Report Schools that made AYP for August and

October reports, but did not make AYP for Final December Report, must immediately move to implement the requirements of NCLB.

Timeline for AYP Information

69

2002 Baseline AYP Reports

Starting points for each school and district.

CDE will post on the Internet on July 22, 2003.

Districts and schools may determine whether they are at risk for not meeting AYP criteria when the 2003 report is released in August.

70

2003 AYP Reports

CDE will post 2003 AYP reports on the Internet on August 15, 2003.

Reports will include the “percent proficient” and participation rates based on 2003 testing data.

Will be used to identify districts and schools that are not making AYP for 2003.

71

2003 Reporting Timeline

2002 Base

2002 Base

AYP Report

AYP Report

Dec

embe

r

Dec

embe

r

2003 Final

2003 Final

AYP Report

AYP Report

Augus

t 15

Augus

t 15

July

22

July

22

Oct

ober

Oct

ober

2003 Growth API

2003 Growth API Report

Report

2003 AYP

2003 AYP Report

Report

Septe

mbe

r

Septe

mbe

r

Novem

ber

Novem

ber

72

Notification of AYP Status

Base 2002 AYP Report– Early summer– Districts can identify schools at risk for failing AYP in

2003– Districts can plan and prepare for possible PI

identification and implementation

Phase 1 AYP Report: – August 15, 2003– Attainment of AMO’s (i.e. percent proficient) and

participation rate

73

Notification of AYP Status

Phase 2 AYP Report: – October 2003– 2003 Growth API’s and high school graduation

rates

Phase 3 AYP Report (Final): – December 2003– Final AMO’s and participation rates, APIs for

schools making data corrections, and application of “safe harbor” to all schools and subgroups

2002 Base AYP Results

75

Schools*# % # % # % # %

Made AMOs and Part. Rates 1,911 37.2% 233 19.7% 194 19.7% 2,338 32.0%

Failed only Part. Rates 977 19.0% 174 14.7% 422 42.8% 1,573 21.5%

Below AMOs Only 1,398 27.2% 272 23.0% 53 5.4% 1,723 23.6%

Below AMO & Part. Rates 855 16.6% 504 42.6% 318 32.2% 1,677 22.9%

All Schools 5,141 100.0% 1,183 100.0% 987 100.0% 7,311 100.0%

# % # % # % # %

Made AMOs and Part. Rates 885 25.5% 71 10.1% 45 12.6% 1,001 22.1%

Failed only Part. Rates 543 15.7% 74 10.5% 107 29.9% 724 16.0%

Below AMOs Only 1,289 37.2% 199 28.3% 29 8.1% 1,517 33.5%

Below AMO & Part. Rates 749 21.6% 360 51.1% 177 49.4% 1,286 28.4%

All Schools 3,466 100.0% 704 100.0% 358 100.0% 4,528 100.0%

High All Schools

2002 Base AYP Results

Elementary Middle High

*Includes schools with 100 or more enrolled in grades 2-8 and 10

Title I Schools*

All Schools

Elementary Middle

Effects on Current Statewide Accountability System

77

How Will the API be Affected?

Remain the same:– Statewide target

(800)– Base-growth cycle– Calculation of the

index and targets– Schedule of

reporting– Timeline for inclusion

of new assessments

Changes*:– Addition of two new

subgroups (ELs and students with disabilities)

– Change in subgroup size

– Increase in participation rate for high schools to 95%

– Mobility rule

*Subject to legislation

78

Are API Growth Targets Still Important?

Attainment of API growth targets will affect eligibility for awards

Still criteria for exiting state intervention programs (e.g., II/USP)

YES!!!!

79

Importance of the CAHSEE

Contributes 15% of the API weight for high schools

The grade 10 census administration is the basis of AYP for high schools

Need 95% minimum participation rate for AYP

80

How Will State Awards and Interventions be Affected?

Legislation has been introduced to align the API methodology with the AYP requirements

The CDE is in the early stages of planning to align state and federal interventions and sanctions

Eligibility and priority for awards and interventions/sanctions will be based on making both AYP and API

81

For More Information

Questions related to AYP– Evaluation Unit at (916) 319-0875 or e-mail at epic@cde.ca.gov

Questions related to API or the AYP Reports– EPIC Unit at (916) 319-0863 or e-mail at epic@cde.ca.gov

Questions related to Program Improvement– Title I Policy and Partnerships Office at (916) 319-

0854 or pi@cde.ca.gov

82

New AYP Internet Sites

AYP Reports

http://ayp.cde.ca.gov

Letters, Memos, Informational Materials

http://www.cde.ca.gov/ayp

83

This presentation is available on-line at:

http://www.cde.ca.gov/ayp

under “Presentations”