Exploring Demographic and Selected State Policy Correlates of State Level

Educational Attainment and Achievement Indicators

Paper prepared for:

The American Educational Research Association (AERA)Annual Meeting

Prepared by:Margaret Cahalan

Jim Maxwell

Draft

April 10, 2007Chicago, Ill.

Note: All tabulations and views reported in this paper are the responsibility of the authors and do not reflect any review, authorization, or clearance by the Department of Education.

2

1.Introduction As a nation, we are fascinated by state-by-state comparisons on almost any topic. In education, increasingly, researchers and policy makers are preparing indicators, often with rankings and scores assigned to the states. ED Week’s Quality Counts (QC), grades states and is dedicated to tracking “state efforts to creating a seamless education system from early childhood through the world of work,” and the National Center for Higher Education Managers Systems (NCHEMS) provides policy makers with a “State Report Card” system to help managers make decisions. Similarly, major government surveys and assessment tools are increasingly designed to provide state- by-state estimates.

In recent years, education policy reform discussion has moved from an emphasis on understanding the importance of student background characteristics in explaining differences in outcomes to a focus on the importance of state, district, school and teacher controlled factors. This has shifted some of the focus from attainment to achievement test scores, and from compensatory programs to state, district, school, and teacher accountability. This has been accompanied by an increased emphasis on identifying practices and policies that all other things being equal, are more effective than others in providing effective schooling. At the state level, the “state standards movement” and reform has resulted in state level efforts to promote higher achievement through such things as increased core curricular requirements, exit exams, higher compulsory school attendance age, school size reform, requiring teachers to have a major in field taught, increased technology use, advanced and honors diplomas, and content standards.

In this paper we explore relationship of state aggregated student and family related background characteristics, and selected state policy variation to aggregated measures of both student attainment and achievement outcome indicators. We first explore the basic question of how much of the measured differences in educational outcomes between the states can be attributed to demographic differences in the composition of the populations of the states. Second, taking these compositional differences into account, we explore the extent to which differences in selected state policies are statistically related to differences in observed outcomes aggregated at the state level. To do this we use aggregated state level data from the Census Bureau merged with Department of Education data from the Common Core of Data (CCD), Integrated Postsecondary Data

3

Systems (IPEDS), and the National Assessment of Education Progress (NAEP), and various other sources to increase our understanding of what these state-by-state comparisons represent. In addition we provide some state level descriptive historical data on some of the major outcomes of interest.

1.1 Research Questions

Specifically we address the following questions.

1. How much variation by state is there in state high school and postsecondary completion rate indicators; and NAEP and SAT/ACT achievement indicators?

2. How much of the variation is associated with variation in state population demographics? What demographic variables are most related to the outcomes of interest?

3. Are there states that have higher or lower than expected outcomes based on demographics?

4. How much of the variation is related to differences in selected state policies?

To a limited extent, we also descriptively address trends over time and the extent of the gap between race and ethnic minority statistics with regard to high school and postsecondary completion.

Figure 1 summarizes the state level statistics examined descriptively and in the regression models. We discuss these measures in more detail as we proceed and appendices provide additional information on the distribution by state for several of these variables.

Figure 1. Summary of demographic, selected state policy/education statistics, and student outcomes variables included in models

4

Selected State Policy/Ed System StatisticsExit exams, Compulsory school age, Course requirements, Technology score, School size,

Teacher salary, Advanced diploma, Algebra 8th grade

State DemographicsEducation levels, Income/poverty, Employment, Race, Ethnicity/immigration, Mobility, Population

1. 2 Paper StructureThe remainder of this paper proceeds as follows: 2) Procedure data and methods; 3) Descriptive data on model outcome variables with some historical perspective 4) Regression models results for attainment 5) Regression models results for achievement 6) Conclusion/discussion.

2. Procedure, Data and MethodsWe address the questions posed above by a series of descriptive graphing and building exploratory regression models. Our first step was to build a state database that consists of state demographic variables, state education policy variables, and state outcome variables. The primary data source for most of the data is the Census Bureau (Decennial Census, American Community Survey and Current Population Reports on Educational Attainment) and the US Department of Education ((Common Core of Data (CCD), and the Integrated Postsecondary Education Data System (IPEDS)). In addition, data on college entrance scores comes from the College Board and the ACT. Many of the derived variables/indicators used were directly taken from compilations of state aggregated data published by the Council of Chief States School Officers (CCSSO) State Indicator Reports, ED-Week Quality Counts, and NCHEMS web based Information Center. All of the data used in this paper are aggregated at the State level. Graphs typically include the 50 US states and the District of Columbia; however DC was removed from regressions due to its unique demographics. Using these data sources, we first built a database containing about 300 state level variables. From this database we selected the variables included in Table 1(a-c) to include in our model building. These are organized conceptually into three groupings (state demographics, selected state policy and education system statistics, and state level outcomes on attainment and achievement). Our focus is on

5

Student OutcomesAttainment

Public school high school cohort survival ratePostsecondary entrance and completion indicator

Achievement8th grade NAEP math score

Number per 1000 high school graduates scoring 1200 or 26 an above on SAT/ACT

educational measures most applicable to the secondary/postsecondary level.In the next section, we present descriptive information by state on the outcome variables as a way of observing the range of differences among the states. We also include some historical information on the outcomes of interest in the form of graphing historical trends by state. We then proceed to look at the relationships among the variables and present results of regression models and examination of the expected vs. the actual rates based on state demographics. Finally we look at the extent to which the introduction of selected state policy variables changes the amount of variation explained controlling for the demographic differences. To assist in the exploratory analysis, we used the SAS proc regression grouped option, which allows for selected variables to enter into the model together in logical groupings. We used a grouped Forward selection option, which starts with no variables in the model and adds variable groups one by one that maximize the fit of the model. We use selection criteria of .15 for entrance into the model. Predicted and residual values from the estimated regression equation were also tabulated. Observing partial regression results, we also observe the percent of the variation attributed to each of the groups in the model. In forming the groups, exploratory factor analysis of the variables was performed and correlations between the independent variables were observed. These identified factors contributed to decisions about the groupings used in the models.

6

Table 1-a. State aggregated demographic variables included in various models

Name Label Source MeanStandard Deviation

Income/poverty

pu18po99Percent under 18 in poverty Census 15.8 4.7

mefain05Median family income 2005 Census 55834.0 8727.8

Employment Census

parempl

Percent of children in families in which one parent is working full time for year Census 71.3 4.2

Education

onparpst

Percent of children in families in which one parent has 2 or 4 year postsecondary degree Census 43.9 7.1

alhsd20

Percent of population age 25-and older who have high school diploma or credential Census 82.0 4.4

Race/ethnicity

pblk05Percent Black in population 2005 Census 10.4 9.7

Ethnicity/Immigration

phispa05Percent Hispanic in population 2005 Census 9.0 9.5

pforbo04Percent foreign born in 2004 Census 7.9 6.0

parengsk

Percent parents who are native English speakers Census 90.1 7.8

Population

repo02Resident population 2002 Census 5756.0 6386.8

posqm05Population density per square mile Census 189.3 257.7

Mobility

mobil05

Percent of population that lived in another state one year earlier Census 3.1 1.1

Source: US Census Bureau, Decennial Census and American Community Survey. <http://www.census.gov/popest/states/asrh/SC-EST2005-04.html

7

Table 1-b. Selected State education policy or practice variables included in various models

Name Content Source MeanStandard Deviation

HSEXIT2Had exit exam by 2004 CCSSO 0.4 0.5

Comsch05Compulsory school age

National Education Association 16.9 .9

Tecindx5QC state indicators technology score ED-Week 76.6 6.6

Ntesal

Ratio of teacher salary to per capita income

NCES/Census 1.5 0.1

Asssr03

Average school size for regular secondary schools NCES 772.9 310.8

Mcourreq

Number math courses required for graduation CCSSO 2.8 0.7

MajsteacMajor in field required for teachers

ED-Week QC 80.9 .40

Table 1-c. State outcome variables explored

Name Content Source Mean

Standard

Deviation

PCSR04

Public 9th grade school cohort survival rate

CCD/NCEHMS web site/Mortenson 71.7 9.15

PG9DCG04

Percent 9th grade graduating high school, entering postsecondary and obtaining program completion in 150 percent of time

CCD/IPEDS/ACTNCES/NCHEMS/Mortenson 18.3 14.97

Avmatsc5Average 8th grade math score NCES/NAEP 278 7.14

HISCRT04

Number per 1000 with SAT above 1200 or ACT above 26 ACT/SAT 173 36.1Gap between black and non-hispanic white high school

Census

8

completion

Source: NCHEMS Higher Information Center http://higheredinfo.org/ and Tom Mortenson—Postsecondary Education Opportunity http://www.postsecondary.org; SAT. The College Board. "2001 SAT V+M Score Bands Report," unpublished data; ACT. "Number of 2001 High School Graduates with ACT Composite Scores of 26 or Higher," unpublished analysis, Iowa City, Iowa

3. Descriptive Graphing Information on State Variation on the Outcomes of Interest

In this section, we present descriptive state data on the outcome variables included in the models. Appendix A contains additional graphs of some of demographic and state policy/system variables also included in the model. By way of introduction, we also include some historical data on decennial census data by state on high school and college educational attainment.

3.1 Education Attainment Statistics

The publication of reports such as One-Third of A Nation (Barton 2005) and Losing Our Future (Orfield et al. 2004), reflect the refocusing of attention on high school completion rates as a national problem. Trend lines and yearly rates differ depending on what measure of dropping out one chooses. As illustrated in appendix A table 1, recent estimates nationwide of public school high school completion rates range from 68-70 percent (and around 50 percent for underrepresented minorities) based on ratios of entering public school cohort size to diplomas awarded four years later --- to 86 percent as reported by 18-24 year olds in the Current Population Survey and including public and private school students, alternative completions, and out of grade completions.

3.1.1 Decennial Census Data on Attainment 1940-2000

Figure 2 gives decennial census data on the percent of the total US population 25 years of age and older that have a high school diploma or equivalent from 1940 to 2000 by race/ethnicity; and figure 3 gives

9

similar information for those who have a BA degree. These data document the dramatic increase in the percent of the population with high school diploma or equivalent, and especially among blacks, narrowing the black-white gap, over the last 60 years. The figures also document the slowing of gains in the last decade. Gains for a BA have also occurred over the period with a slowing of rate of increase in recent years (figure 3).

Figures 4 and 5 plot this same information by state (without state labels) for high school or higher and BA or higher, respectively. In 1940 the high school completion distribution ranged from 15 percent in Arkansas to 41 percent in the District of Columbia and 37 percent in California. By 2000, the high school distribution ranged from 73 percent in Mississippi to 88 percent in 4 states---Utah, Wyoming, Minnesota, and Alaska.1 Figure 4, shows that the variation among states in rates of high school credential attainment has narrowed over the period since 1940.

In 1940 the distribution for BA or higher ranged from 2 percent in Arkansas and 3 percent in Alabama to 11 percent in the District of Columbia and 7 percent in California and Nevada. One can see that the range of difference between states for the BA or higher has appears to have grown over the period since 1940.

Figure 2. Percent of population 25 years of age and older who have a high school diploma or equivalent by race/ethnicity: Decennial Census Data 1940-2000

1 This decennial census figure of 88 percent for Alaska is surprising given the relatively lower figure on the cohort survival rate.

10

Note: Based on Decennial census. White category does not exclude those of Hispanic Origin. Hispanic Origin can be of any race. White non-Hispanic is available from 1980-2000 only.SOURCE: U.S. Census Population Division Census 2000 PHC-T-41. A Half-Century of Learning: Historical Statistics on Educational Attainment in the United States, 1940 to 2000

11

Figure 3. Percent of population 25 years of age and older who have a BA degree: Decennial Census Data: 1940-2000

Note: Based on Decennial census. White category does not exclude those of Hispanic Origin. Hispanic Origin can be of any race. White non-Hispanic is available from 1980-2000 onlySOURCE: U.S. Census Population Division Census 2000 PHC-T-41. A Half-Century of Learning: Historical Statistics on Educational Attainment in the United States, 1940 to 2000

12

Figure 4. Percent of total population 25 and older with high school diploma or equivalent by state: 1940-2000

NOTE: This distribution ranged from 15 percent in Arkansas to 41 percent in the District of Columbia and 37 percent in California in 1940; and ranged from 73 percent in Mississippi to 88 percent in 4 states, Utah, Wyoming, Minnesota, and Alaska in the year 2000.

SOURCE: U.S. Census Population Division Census 2000 PHC-T-41. A Half-Century of Learning: Historical Statistics on Educational Attainment in the United States, 1940 to 2000

13

Figure 5. Percent of total population 25 and older with BA degree or higher by state: 1940-2000

NOTE: This distribution ranged from 2 percent in Arkansas to 11 percent in the District of Columbia and 7 percent in California and Nevada in 1940; and ranged from 15 percent in West Virginia to 39 percent in District of Columbia and 33 percent in Massachusetts in 2000.

SOURCE: U.S. Census Population Division Census 2000 PHC-T-41. A Half-Century of Learning: Historical Statistics on Educational Attainment in the United States, 1940 to 2000

14

Figure 6, also using the decennial-census-data, plots by state the gap between the percent of white and black persons 25 years of age and older having a high school diploma or higher from 1940 to 2000; and figure 7 shows similar information for the BA or higher attainment statistic. Figure 6 shows the increase in the high school gap, up to 1960 followed by a decline in most states. In 2000, there were 4 states where the percent of blacks having this credential was higher than that of the white population. In 2000, the high school gap nationwide was 12 percentage points (84 compared to 72) and the BA gap representing a much higher percentage difference was similar (11/12 percentage points--26 compared to 14). Figure 8 based on figures 2 and 3 plots the national gap at each period 1940-2000 and suggests that in periods of majority population rapid growth in educational attainment, the black-white gap seems to grow, (such as the period between 1950 and 1970 for high schools and between 1970 and 2000 for BA attainment).

Figure 6. Plot of gap between percent of white and black population over 25 with high school diploma or equivalent by state: 1940-2000

NOTE. The gap ranged from 8 in West Virginia in 1940 to 38 percentage points in California in 1940. In 2000 the gap ranged from –8 in North Dakota one of 4 states to have a negative gap to 24 in the District of Columbia and 19 in Mississippi and 18 in Wisconsin.

15

SOURCE: U.S. Census Population Division Census 2000 PHC-T-41. A Half-Century of Learning: Historical Statistics on Educational Attainment in the United States, 1940 to 2000

16

Figure 7. Plot of gap between percent of white and black population over 25 with a BA or higher by state: 1940-2000

NOTE. The gap ranged from less than 1 in Alaska and Hawaii and 1 in West Virginia in 1940 to –8 in Montana and –4 in Vermont and –1 in Idaho to 59 percentage point gap in DC and 20 point gap in Connecticut and 17 percentage gap in Virginia in 2000.

SOURCE: U.S. Census Population Division Census 2000 PHC-T-41. A Half-Century of Learning: Historical Statistics on Educational Attainment in the United States, 1940 to 2000

17

Figure 8. Plot of gap between percent of white and black population over 25 with high school diploma or equivalent and percent with BA or higher: 1940-2000

NOTE. This chart based on figures 2 and 3 illustrates that in periods of rapid growth in majority population educational attainment the gap seems to grow, (such as the period between 1950 and 1970 for high schools and between 1970 and 2000 for BA attainment).SOURCE: U.S. Census Population Division Census 2000 PHC-T-41. A Half-Century of Learning: Historical Statistics on Educational Attainment in the United States, 1940 to 2000

3.1.2 Public School Cohort Survival Rate

The outcome indicator representing high school completion that we used in the regressions in this paper is the public school cohort survival rate (CSR) for 2004 as published on the NCHEMS website and developed by Tom Mortenson. This statistic represents the ratio of the total 9th grade public school enrollment to public school diplomas awarded 4 years later. It is derived from NCES/CCD data on enrollment and diplomas awarded and provides a standardized measure across states. It is similar to other CCD based completion statistics such as those noted in appendix table A-1 and the NCES averaged freshman cohort completion rate. We used this version due

18

to its’ availability back to 1990 and ease of merging into our database. The CSR rate by state for 2004 is graphed below in figure 9. Figure 10 graphs the rate by state for 1990-2004. Apart from some outliers, there appears to be little change with a slight downward trend. Nationally, the CSR rate was 71.2 in 1990, 67.1 in 2000, and 69.7 in 2004.

Figure 9. Public high school cohort survival rate by state: 2004

NOTE: Calculated based on number of 9th graders/High school graduates four years later (public high schools). Doesn’t account for students who are still enrolled or transfers to private high schools or out-of-state SOURCE National Center for Higher Education Managers Systems (NCHEMS), Higher Education Information System; Tom Mortenson—Postsecondary Education Opportunity http://www.postsecondary.org

19

Figure 10. Public School Cohort Survival Rate by State 1990-2004

NOTE: Calculated based on number of 9th graders/High school graduates four years later (public high schools) SOURCE National Center for Higher Education Managers Systems (NCHEMS), Higher Education Information System; Tom Mortenson—Postsecondary Education Opportunity http://www.postsecondary.org

20

3.1.2 Postsecondary Pipeline/Completion Indicator

Figure 11 graphs state differences in the postsecondary pipeline/completion indicator statistic used as the outcome variable in the regression models. This statistics is also Tom Mortenson’s calculation as included on the NCHEMS web site. It is based on CCD enrollment figures for 9th graders, estimating the number who graduate from high school within 4 years (based on the public HS graduation rates), the number who go directly to college (based on the college going rates of recent HS graduates), the number who return for their second year of college (based on the first-year retention rates), and the number who graduate from postsecondary program within 150% of program time (based on the IPEDS graduation rates). The calculation for high school graduation doesn’t account for transfers to private high schools and out-of-state. The calculation for college graduation doesn’t account for transfers across institutions. By state, rates range from 5.8 in Alaska to 27.9 in South Dakota.

Figure 11. Postsecondary pipeline/completion indicator, percent of 9th grade high school cohort estimated to graduate high school, enter postsecondary directly and obtain postsecondary degree within 150 percent of program time by state: 2004

NOTE: This statistics is calculated based on CCD enrollment figures for 9th graders, estimating the number who graduate from high school within 4 years (based on the public HS graduation rates), the number who go directly to college (based on the college going rates of recent HS graduates), the number who return for their second year of college (based on the first-year retention rates), and the

21

number who graduate from postsecondary program within 150% of program time (based on the IPEDS graduation rates).The calculation for high school graduation doesn’t account for transfers to private high schools and out-of-state. The calculation for college graduation doesn’t account for transfers across institutions.

SOURCE: NCHEMS Information System web site; Tom Mortenson—Public high school graduation rates, Tom Mortenson—College-going rates of students directly from HS, ACT Institutional Survey—Freshmen to sophomore retention rates, NCES-IPEDS Graduation Rate Survey—Graduation Rates

3.2 Selected Achievement Outcome Variables Figures 12a to 15 present statistics on the achievement variables included in the regression models. Our historical information is much more limited than with attainment.

3.2.1 NAEP 8th Grade Math Scores

We used state 8th grade NAEP math scores for our achievement indicator outcome variable. Unfortunately, 12th grade NAEP is not state representative. By state, NAEP 8th grade math average scores range from 262 in Alabama and Mississippi to 292 in Massachusetts and 290 in Minnesota (figure 12a).

Figure 12b shows another NAEP statistic, the percent categorized as at or above proficient in 8th grade math by state. We use this variable in the regression models discussed in section 5. Figure 12b shows much the same state line up as in figure 12a, with a few differences.

Looking at Figure 13, which graphs the state average score for 1990, 2000, and 2005, we see the trend upward in the period graphed, continuing a trend that was also apparent nationally between 1980 and 1990.

Figure 12a. NAEP average 8th grade math score by state: 2005

22

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2005 data

Figure 12b. NAEP percent at or above proficient in 8th grade math by state: 2005

23

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2005 data

24

Figure 13. NAEP average 8th grade math score by state: 1990, 2000, 2005

NOTE: Nationwide NAEP 8th grade math scores were 262 in 1990; 270 in 2000; and 274 in 2005. Among states included in 1990, the highest score was North Dakota and the lowest was Louisiana with 246. By 2005 the highest score was obtained by Massachusetts 292 and the lowest by Alabama, 262 and Mississippi. In 1990 state estimates were available only for 32 states and did not include Massachusetts.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 1990, 2000, 2005 data

25

3.2.2 Rate per 1000 High School Graduates who Score above 1200 on SAT or 26 on ACT

The other achievement outcome indicator we included in our regressions is another statistic from the NCHEMS web site—the rate per 1000 high school graduates who score above 1200 on the SAT or above 26 on the ACT (figure 14). This statistic is limited in that it does not take into account those who might have taken both tests—and differences in states in the percent taking two of the tests may affect these tabulations. It should also be noted that this is the rate per 1000 high school graduates and differential rates of high school graduation would also affect comparisons by state. Rates range from 98 in Mississippi to 259 in Colorado. Figure 15, graphing results between 1999 and 2004 show the increase in this rate for most states, and also a little more spread among the states in 2004 than in 1999.

Figure 14. Rate per 1000 high school graduates who scored 1200 or above on combined SAT or 26 or above on ACT: 2004

NOTE: The number of SAT Math and Verbal Scores 1200 and above + ACT Composite Scores 26 and above) per 1,000 high school graduatesSOURCE: SAT. The College Board. "2001 SAT V+M Score Bands Report," unpublished data ACT. "Number of 2001 High School Graduates with ACT Composite Scores of 26 or Higher," unpublished analysis, Iowa City, Iowa. High School Graduates. Western Interstate Commission for Higher Education. Knocking

26

at the College Door: Projections of High School Graduates by State and Race/Ethnicity 1996-2012. Boulder, C0

27

Figure 15. Rate per 1000 high school graduates who scored 1200 or above on combined SAT or 26 or above on ACT: 1999, 2001, 2004

NOTE: The number of SAT Math and Verbal Scores 1200 and above + ACT Composite Scores 26 and above per 1,000 high school graduatesSOURCE: SAT. The College Board. " SAT V+M Score Bands Report," unpublished data ACT. "Number of High School Graduates with ACT Composite Scores of 26 or Higher," unpublished analysis, Iowa City, Iowa. High School Graduates. Western Interstate Commission for Higher Education. Knocking at the College Door: Projections of High School Graduates by State and Race/Ethnicity 1996-2012. Boulder, C0

4. Results of the Regression Runs on Attainment

4.1 High School Cohort Survival Rate4.1.1 Demographic Predictors of High School Cohort Survival RateTable 1 and figures 16 and 17 summarize results from a forward selection regression model for the outcome variable public high school cohort survival rate (CSR) in 2004. The demographic model “explains” 72 percent of the variation, with the group’s entitled “parent education”, “ parent employment”, and “population density” having a positive sign and “mobility”, “race” and

28

“ethnicity/immigration” having a negative sign. In this model, the “race” group only includes percent black. Based on a factor analysis, we included the Hispanic percentage variable with the “ethnicity/immigration” group that also includes percent foreign born and percent speaking English as first language. The group “population density” is the number per square mile and “mobility” is the percent of state population that lived in a different state 1 year earlier. The group “parent education” accounts for 40 percent of the variation, with “mobility” adding another 9 percent. The model groups “race” and “ethnicity/immigration” each contribute 7 percent and the “parent employment” variable adds another 3 percent and “population density” 2 percent (figure 15). Differences by state between actual and predicted rates, (figure 17) range from +14 in New Jersey and +11 in Arkansas to –8 in Indiana, South Carolina, and Nevada. We note here that the models we ran initially included variables representing poverty and also income directly; however, the income variables were highly related to education levels and so did not enter the models. The poverty variable did enter the model at the last step, and controlling for the other SES variables already in the model its sign was positive and it explained an additional 3 percent of the variation. We did not include it in the model presented here.

Table 1. Summary of forward selection regression model using grouped option explaining variation in state differences in public school high school cohort survival rate: demographic variables only

Step

Group entered

Direction

Number of

variables

Partial R-Squa

re

Model R-

Square

F Value

Pr> F

1Parent Education+

+2 0.4022 0.4022 15.81

<.0001

2 Mobility--

3 0.0916 0.4938 8.320.005

9

3 Race--

4 0.0732 0.567 7.610.008

4

4Ethnicity/imm-

-6 0.0735 0.6405 4.39

0.0184

5

Parent Employment+

+

7 0.0276 0.6681 3.490.068

9

6Population density+

+8 0.024 0.6921 3.2 0.081

29

NOTE: Calculated based on number of 9th graders/High school graduates four years later (public high schools). Doesn’t account for students who are still enrolled or transfers to private high schools or out-of-state SOURCE National Center for Higher Education Managers Systems (NCHEMS), Higher Education Information System; Tom Mortenson—Postsecondary Education Opportunity http://www.postsecondary.orgRepresents residuals tabulated based on SAS PROC REG SIMPLE;PROC REG SIMPLE; ID State1; MODEL PCSR04 = {alhsd20 onparpst} {pblk05} {phispa05 pforbo04}{parempl} {posqm05} {mobil05} / p r cli clm sle = .15 SELECTION = Forward GROUPNAMES = 'education' 'race' 'ethnicity/immigration' 'employment' 'pop density' 'mobility' ;

30

Figure 16. Distribution of variance among “groups” in the demographic model examining state variation in public school high school cohort survival: 2004

NOTE: Model R-square =.69 Allocation of variance based on Partial R-Squares. PCSR04 = Dependent variable public high school cohort survival rate: 2004

SOURCE: See table 1 above

31

Figure 17. Difference between actual and predicted (residuals) high school cohort survival rate (CSR) from model with demographic variables only: 2004

NOTE: Model R-square = .69. Allocation of variance based on Partial R-Squares. PCSR04 = Dependent variable public high school cohort survival rate: 2004SOURCE: see table 1 above

32

4.1.2 Adding Selected State Policy and Education Related Statistics to the Demographic Model of High School Cohort Survival Rate

Table 2 and figure 18 summarize the change in the model when selected state policy and system statistics are entered into the model using the same forward selection procedure. The total R-squared is not much increased from the demographic model; however, several of the state policy variables enter into the model—demonstrating what simple correlations revealed that some of the policies are highly related to the demographic differences. The “Parent education” group remains the major explanatory variable with a Partial R-squared of .36 “School size” is negative and adds 16 percent explanation. “Mobility” adds 9 percent and exit exams 5 percent—both with a negative sign. “Parent employment” contributes an additional 2 percent and “technology” marginally significant in the model also contributes 2 percent. Note that “race” and “ethnicity/immigration” did not enter the model with this configuration. “Exit exams” is highly correlated (.40) with race/ethnicity variables, so it is not clear if the apparent negative effect is related to the exit exams themselves or to other variables with which it is correlated. The findings for school size are consistent with other research that has found a relationship to high school completion to this variable (Garrett Z, Newman, Elbourne, Bradley, Noden, Taylor, West 2004). The variables representing “course requirements”, “teacher salaries”, and “teaching requirements” did not reach the significance levels needed to enter the model and were not included. As these variables were missing for 9 states, after testing the model with these variables and without, we removed them from the model.

33

Table 2. Summary of forward selection regression model using grouped option explaining variation in state differences in public school high school cohort survival rate: state policy and system statistics added to demographic model

Step

Group entered

Direction

Number of

variables

Partial R-Squa

re

Model R-

Square

F Value

Pr> F

1Parent education

+2 0.3639 0.3639 11.16

0.0001

2 school size-

3 0.1629 0.5268 13.080.000

9

3 mobility-

4 0.09 0.6168 8.690.005

5

4 exit exam-

5 0.0478 0.6646 5.130.029

6

5 pop density+

6 0.032 0.6966 3.70.062

7

6Parent employment

+7 0.0244 0.721 2.98

0.0936

7 technology+

8 0.0207 0.7417 2.640.113

8

NOTE: Calculated based on number of 9th graders/High school graduates four years later (public high schools). Doesn’t account for students who are still enrolled or transfers to private high schools or out-of-state SOURCE National Center for Higher Education Managers Systems (NCHEMS), Higher Education Information System; Tom Mortenson—Postsecondary Education Opportunity http://www.postsecondary.org

SOURCE: Represents residuals tabulated based on SAS PROC REG SIMPLE;ID State1;MODEL PCSR04 ={pu18po99}{alhsd20 onparpst}{pblk05}{phispa05 pforbo04 parengsk} {parempl} {posqm05} {mobil05}{Tecindx5} {HSEXIT2} {comsch05} {ASSSr03} {ntesal} {Majsteac} {Mcourreq} / p r cli clm sle = .15 SELECTION = Forward GROUPNAMES = 'Poverty' 'Education' 'Race' 'Ethnicity' 'Employment''Popdens' 'Mobility' 'technology' 'Exit exam''Comp age' 'Schoolsize' 'teachsal' 'majteach' 'math course';

34

Figure 18. Distribution of variance among “groups” in the model examining state variation in public school high school cohort survival: demographic and state policy/system variables 2004

NOTE: Model R-square = .74. Allocation of variance based on Partial R-Squares. PCSR04 = Dependent variable public high school cohort survival rate: 2004

SOURCE: See table 1 above

35

4.2 A Postsecondary Pipeline/Completion Indicator—(Rate of Graduation from High School, Entering Postsecondary the Next Year and Completing a Postsecondary Program in 150 percent of Program Time)

4.2.1 Demographic Predictors of Postsecondary Pipeline/Completion IndicatorTable 3 and figures 19 and 20 summarize results from a forward selection regression model for the outcome variable representing the postsecondary pipeline/completion indicator. Results show that the demographic variables account for .78 percent of the variation. “Parent education” has a partial r square of .53, followed by “parent employment” and “mobility.” Comparing the postsecondary pipeline/completion results with those from the high school cohort survival model, we see that “parent education” and “parent employment” both explain a relatively higher proportion of the variation; and ethnicity/immigration explains less. The “race” variable does not meet the .15 threshold for entrance into this model. Wyoming, Pennsylvania and South Dakota have the largest positive difference between actual and predicated results and Utah, Alaska, and Maryland the largest negative differences. As this measure may be subject to bias related to differences in in-state and out of state postsecondary attendance rates it is difficult to interpret these results for individual states.

Table 3. Postsecondary pipeline/completion indicator, summary of forward selection regression model: demographic variables only

Step

Group entered

Direction

Number of

variables

Partial R-Squa

re

Model R-

Square

F Value

Pr> F

1 Parent Education++

2 0.5384 0.5384 27.41<.0001

2Parent Employment+

+3 0.0783 0.6167 9.4

0.0036

3 Mobility--

4 0.0888 0.7055 13.570.000

6

4Ethnicity/immigration-

-6 0.0418 0.7473 3.56

0.0371

5 Pop density++

7 0.0285 0.7759 5.340.025

8

36

NOTE: This statistic is calculated based on CCD enrollment figures for 9th graders, estimating the number who graduate from high school within 4 years (based on the public HS graduation rates), the number who go directly to college (based on the college going rates of recent HS graduates), the number who return for their second year of college (based on the first-year retention rates), and the number who graduate from postsecondary program within 150% of program time (based on the IPEDS graduation rates).

SOURCE: NCHEMS Information System web site; Tom Mortenson—Public high school graduation rates, Tom Mortenson—College-going rates of students directly from HS, ACT Institutional Survey—Freshmen to sophomore retention rates, NCES-IPEDS Graduation Rate Survey—Graduation Rates

SOURCE: Represents residuals tabulated based on SAS PROC REG SIMPLE; ID State1; MODEL PG9DCG04 = {pu18po99}{alhsd20 onparpst} {pblk05} {phispa05 pforbo04}{parempl}{posqm05}{mobil05} / p r cli clm sle = .15 SELECTION = Forward GROUPNAMES = 'education' 'race' 'ethnicity/immigration' 'employment' 'pop density' 'mobility' ;

37

Figure 19. Distribution of variance among “groups” in the demographic model examining state variation in postsecondary pipeline/completion indicator): 2004

NOTE: Model R-square =. 78 Allocation of variance based on Partial R-Squares. Postsecondary pipeline/completion indicator is calculated based on chance of graduation from high school, enter postsecondary and complete a postsecondary program in 150 percent of program time

SOURCE: See table 3 above.

38

Figure 20. Difference between actual and predicted (residuals) for postsecondary pipeline/completion indicator from model with demographic variables only: 2004

NOTE: Model R-square = .77. Allocation of variance based on Partial R-Squares. = SOURCE: See table 3 above.

39

4.2.2 Adding Selected State Policy and Education Related Statistics to the Demographic Model of the Postsecondary Pipeline/Completion Indicator

Table 4 and figure 21 summarize the change in the model when selected state policy and system statistics are entered into the model using the same forward selection procedure. The total R-squared is increased to .83. “Parent education” is highly related to the postsecondary pipeline/completion statistic accounting for 57 percent of the variation. Mobility (percent of population who lived out of the state one year earlier) is persistently negative. Of the state education system variables (advanced diploma, teacher salary, math course requirements, technology score, compulsory school age, and exit exam) only school size entered this model.

Table 4. Summary of forward selection regression model using grouped option explaining variation in state differences in postsecondary pipeline/completion indicator: state policy and system statistics added to demographic model

Step

Group entered

Direction

Number of

variables

Partial R-Squa

re

Model R-

Square

F Value

Pr> F

1Parent Education+

+2 0.5734 0.5734 30.91

<.0001

2Parent Mobility-

-3 0.0777 0.6511 10.02

0.0028

3

Parent Employment+

+

4 0.1041 0.7552 18.72<.0001

4 School size-

5 0.0499 0.8051 11.010.001

9

5 Pop Density+

6 0.0224 0.8275 5.450.024

4

40

SOURCE: see table 3 above Represents residuals tabulated based on SAS PROC REG SIMPLE; PROC REG SIMPLE;ID State1;MODEL PG9DCG04 ={alhsd20 onparpst} {pblk05}{phispa05 pforbo04 parengsk} {parempl} {posqm05} {mobil05}{Tecindx5} {HSEXIT2} {comsch05} {ASSSr03} {advdiplo} / p r cli clm sle = .15 SELECTION = Forward GROUPNAMES = 'Education' 'Race' 'Ethnicity' 'Employment''Popdens' 'Mobility' 'technology' 'Exit exam' 'Comp age' 'Schoolsize' 'advdiploma' /*'teachsal' 'majteach' 'math course'*/;

41

Figure 21. Distribution of variance among “groups” in model examining state variation in postsecondary pipeline/completion indicator: demographic and state policy and system variables: 2004

NOTE: Model R-square = .85 Allocation of variance based on Partial R-Squares.

SOURCE: See table 3 above

42

5. Exploring Selected Achievement MeasuresStandardized achievement measures aggregated at the state level for secondary school and above are much harder to obtain. In this section we follow the same regression procedures as with the attainment indicators using the two measures of achievement: NAEP 8th grade math scores (using the percent proficient or above measure for 2005) and rate per 1000 high school graduates scoring at 1200 on SAT combined or 26 on ACT or above for 2004.

5.1 NAEP 8th Grade Math Scores5.1.1 Demographic Predictors of NAEP 8th Grade Math ScoresTable 5 and figures 22 and 23 summarize results from running forward regression models. Results indicate that fewer of the demographic variables were significant and entered the model. Parent education accounts for 67 percent of the variation and mobility barely enters the model with 2 percent of the variation. “Ethnicity” and “race” variables do not enter the model once parent education is taken into account. As shown in figure 23, the states with the greatest positive and negative differences between actual and predicted based on the model taking into account education levels within the state are quite different than the ones identified looking at the attainment variables. Texas, South Carolina, North Carolina and Ohio had the largest positive differences and Hawaii, New Mexico, Rhode Island, and Alabama the largest negative differences.

Table 5. Percent at or above proficient on 8th grade NAEP math, summary of forward selection regression model: demographic variables only

Step

Group entered

Direction

Number of

variables

Partial R-Squa

re

Model R-

Square

F Value

Pr> F

1 Parent education++

2 0.6689 0.6689 47.48<.0001

2 Mobility- - 3 0.0178 0.6867 2.61 0.113

43

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) 2005 dataResults of SAS tabulation as specified below. PROC REG SIMPLE; ID State1; MODEL promat5 = {pu18po99} {mefain05} {alhsd20 onparpst} {pblk05} {phispa05 pforbo04} {parempl} {posqm05} {mobil05}

/ p r cli clm sle = .15 SELECTION = Forward GROUPNAMES = 'poverty' 'median income' 'education' 'race' 'ethnicity/immigration' 'employment' 'pop density' 'mobility';

44

Figure 22. Distribution of variance among “groups” in the demographic model examining state variation in percent at or above proficient in 8th grade NAEP: 2005

NOTE: Model R-square =. 69 Allocation of variance based on Partial R-Squares.

SOURCE: See table 5 above.

45

Figure 23. Difference between actual and predicted (residuals) for 8th grade NAEP percent proficient or above from model with demographic variables only: 2004

NOTE: Model R-square = .69 SOURCE: See table 5 above.

46

5.1.2 Adding Selected State Policy and Education Related Statistics to the Demographic Model of 8th Grade NAEP Score

For the NAEP analysis, we did not have the same number of applicable state level policy variables to include in our models as with the high school cohort survival rate or the postsecondary pipeline/completion indicator. We did test several models with a few applicable variables. We maintained the technology score, but excluded exit exams, high school course requirements, requiring a major in field to teach. We included a new variable on a score assigned by Quality Counts based on percent of 8th grade Algebra course taking. None of the variables achieved enough significance using the forward procedure to enter into the model. No conclusions can be reached about state level variables given the limited set of applicable variables.

5.2 College Entrance Achievement Scores5.2.1 Demographic Predictors of College Entrance Achievement ScoresTable 6 and figures 24 and 25 summarize results from a forward selection regression model for the outcome variable representing the college entrance score indicator (the rate per 1000 high school graduates scoring 1200 or above on SAT or 26 or above on ACT). It is important to note that this is a rate per 1000 high school graduates and would be impacted by differences in high school graduation rates. Among the demographic variables included in the specified model, only “education” and “mobility” met the criteria for inclusion. Education accounted for 54 percent of the variation and mobility for an additional 6 percent.States that have higher than predicted rates based on this model that includes only parent education and mobility as predictors were Tennessee, Colorado, Illinois, New York, and Ohio. Those with lower rates than predicted were North Dakota, Utah, Pennsylvania, and Mississippi and Rhode Island. It is difficult to know how to interpret these results and it is probable that they are influenced by the percent of students who take both ACT and SAT tests within the states. Additional work is needed to correct for this source of double counting.

5.2.2 Adding Selected State Policy and Education Related Statistics to the Demographic Model of the college

47

entrance exam scores indicator (rate per 1000 high school graduates scoring 1200 or above on SAT or 26 or above on ACT

Models were also run that added state policy variables to the regression equations; however, none of the policy related variables met the criteria for entrance into the model that included the demographic variables. State policy/system variables included: technology score; math requirements; school size; exit exam; major required for teachers; relative teacher salary; and algebra in 8th

grade score.

Table 6. College entrance scores indicator, summary of

forward selection regression model: demographic variables only

Step

Group entered

Direction

Number of

variables

Partial R-Squa

re

Model R-

Square

F Value

Pr> F

1 education+

2 0.5447 0.5447 28.12

<.0001

2 mobility_

1 0.0550 0.5997 6.32 0.015

NOTE: The number of SAT Math and Verbal Scores 1200 and above + ACT Composite Scores 26 and above per 1,000 high school graduates

SOURCE: SAT. The College Board. " SAT V+M Score Bands Report," unpublished data ACT. "Number of High School Graduates with ACT Composite Scores of 26 or Higher," unpublished analysis, Iowa City, Iowa. High School Graduates. Western Interstate Commission for Higher Education. Knocking at the College Door: Projections of High School Graduates by State and Race/Ethnicity 1996-2012. Boulder, C0

SAS PROCPROC REG SIMPLE; ID State1; MODEL HISCRT04 = {alhsd20 onparpst} {pblk05} {phispa05 pforbo04}{parempl} {posqm05} {mobil05}

{Preenrol} {chhins} / p r cli clm sle = .15 SELECTION = Forward GROUPNAMES = 'education' 'race' 'ethnicity/immigration'

'employment' 'pop density' 'mobility' 'percent 3-4enrol' 'heathinsur';

48

49

Figure 24. Distribution of variance among “groups” in the demographic model examining state variation in college entrance indicator (rate per 1000 high school scoring 1200 or above on SAT or 26 or above on ACT: 2004)

NOTE: Model R-square =. 59 Allocation of variance based on Partial R-Squares.

SOURCE: See table 6 above.

50

Figure 25. Difference between actual and predicted (residuals) for college entrance exam indicator (rate per 1000 graduates scoring 1200 or above on SAT or 26 or above on ACT) from model with demographic variables only: 2004

NOTE: Model R-square = .59. Allocation of variance based on Partial R-Squares SOURCE: See table 6 above.

51

DiscussionFocusing on data aggregated at the state level, these exploratory analyses--- present a descriptive summary of the extent of differences among states on key educational attainment and achievement outcomes. We examined the relationship of state demographic characteristics to these outcomes and observe the extent to which selected state policy measures were related to differences in the outcome indicators. We also looked at the differences in expected and actual outcomes taking into account the demographic or compositional differences. Our general conclusion is that state-by-state comparisons are very complex—both those that are simple with no analyses of demographics—and those that attempt to take into account or “correct” for demographic differences.Attainment MeasuresThe review of census data on high school completion documents the large increase in percent of the population completing high school over the last 60 years—going from on-quarter of the population to around 80 percent by 2000, and the large increase for black population going from 8 percent in 1940 to 72 percent in 2000. Reviewing the data by state, since 1940, we observed a small trend for a narrowing the spread among states (figure 4). We also observed a decline in the gap between black and white completion rates starting after 1960 (figure 6). Figures for BA completion seem to show some increase in the spread among states over the period (figure 7). In 2004 the public school cohort survival rate ranged from 52 percent in Nevada to 91 percent in New Jersey (figure 9). The postsecondary pipeline/completion indicator rates ranged from 5 in Alaska to 27 in South Dakota. In general we found that the attainment measures we studied were highly correlated with the demographic factors, explaining between 69 and 77 percent of the variation depending on the measure. Several of the state demographic measures we originally included were found to be highly correlated with each other, and state income measures such as per capita income, poverty, and median income, were especially related to parent education measures (the strongest predictor in the models) and did not enter most of the models once parent education had entered. Parent steady employment was a better economic measure in these models. Mobility (percent of persons living in another state one year earlier) entered all of the models and was negatively associated with outcomes. Race and ethnicity/immigration entered into the demographic models for the

52

high school cohort survival but not the postsecondary pipeline/completion model. Among the state policy/state system indicators included in the two attainment models (technology score, high school exit exam, compulsory school age, secondary school size, teacher salary relative to per capita income, major in field required to teach, and math course requirements for graduation), we found that only school size (negative) and exit exams (negative) and technology (positive) were significant in either of the attainment models tested. As exit exam is highly correlated with race (.40) we do not know the meaning of this negative relationship. For the high school cohort survival rate indicator, once the state policy variables were included in the models, race and ethnicity variables failed to achieve significance and enter the models. This may be related to the fact that school exit exams and school size—are highly correlated with race and ethnicity. Technology score had a small positive relationship to the cohort survival rate but not the postsecondary pipeline/completion indicator.

Achievement MeasuresNAEP 8th grade math percent proficient or above by state ranged from 13 percent in Mississippi to 43 percent in Minnesota and Massachusetts. Average math scores have been trending upward since 1990 (figure 13). Rates per 1000 for high scores on the SAT or ACT ranged from 98 per 1000 in Mississippi to 259 in Colorado and 253 in Massachusetts. The achievement measure demographic models explained only slightly less of the variation than the attainment models, but fewer variables achieved significance and entered into the model. For NAEP 8th grade proficiency only parent education and mobility entered in the model, explaining 69 percent of the variation. None of the applicable state policy/system variables entered into the model, including a variable indicating extent of Algebra 1 participation in 8th grade in the state. The College Entrance exam indicator had the lowest R-square of .60 of any of the models. Once again only parent education and mobility were significant. Predicted Compared to ActualExamination of the predicted outcomes in relation to actual outcomes is interesting, but should be used with caution given the limitations of the measures in failing to take into account cross state transfers especially in the postsecondary pipeline/completion

53

indicator and also the double counting ACT/SAT in the college entrance indicator.

54

Selected Related Bibliography Allen, L. with Almeida, C. & Steinberg, A. (2001, August). Wall to wall:

Implementing small learning communities in five Boston high schools. LAB Working Paper No. 3. Providence, RI: Northeast and Islands Regional Educational Laboratory a program of The Education Alliance at Brown University. Available: http://www.lab.brown.edu/public/pubs/LABWorkPaper/Wall2Wall.pdf

Amrein Audrey L. and David C. Berliner (A), (2002). High-Stakes Testing, Uncertainty, and Student Learning. Education Policy Analysis Archives, Volume 10 Number 18, March 28, 2002.

Anderson J, Rogers WT, Klinger D, Ungerleider C, Glickman V Anderson B, (2005). Student and School Correlates of Mathematics Achievement: Models of School Performance Based on Pan-Canadian Student Assessment

Balfanz, R., Legters N., Jordan W, (2004). Catching up Impact of the Talent Development on Ninth Grade Instructional Interventions in Reading and Mathematics in High Poverty High Schools, Center for Research on the Education of Students Placed At Risk (CRESPAR). Report 69.

Berry C (2004) Small Wonders In Schooling, Bigger is Not Necessarily Better , Education Next.

Bill & Melinda Gates Foundation. (2003). Making the case for small schools. Available: http://www.gatesfoundation.org/NR/Downloads/ed/evaluation/BMG911SmallSchoolsBrochure.pdf

Barton, P. 2003. Parsing the Achievement Gap: Baselines for Tracking Progress. Policy Information Report. Princeton, NJ: Educational Testing Service, p. 4. Retrieved January 3, 2005 from http://www.ets.org/research/pic/parsing.pdf.

Barton, P. 2005. One Third of a Nation. Education Testing Service (ETS) Policy Information Report, February 2005. New small learning communities: Findings from recent research. Portland, Ore.: Northwest Regional Educational Laboratory. Retrieved 07/03/02 from http://www.nwrel.org/scpd/sirs/nslc.pdf

Bickel R, Howley C (2000) The influence of scale on school performance: a multi-level extension of the Matthew principle. Education Policy Analysis Archives 8. Available from: http://epaa.asu.edu/epaa/v8n22/ (Linked to Bickel and Howley, 2000; Howley and Bickel, 1999)

Bickel R, Howley C, Williams T, Glascock C (2000) Will the real ‘Texas miracle in education’ please stand up? Grade span configuration, achievement, and expenditure per pupil. Unpublished. Vermont: Rural School and Community Trust. (Linked to Bickel et al., 2001; Bickel 1999b; Howley and Bickel, 1999)

55

Bickel R, Howley C, Williams T, Glascock C (2001) High school size, achievement equity and cost: robust interaction effects and tentative results. Education Policy Analysis Archives 9. Available from: http://epaa.asu.edu/epaa/v9n40.html (Linked to Bickel 1999b; Bickel et al., 2000; Howley and Bickel, 1999)

Bishop, John H.; Mane, Ferran (2001) “The impacts of minimum competency exam graduation requirements on high school graduation, college attendance and early labor market success.” Labour Economics, 8 (2), pp. 203-222.

Bishop, John H.; Mane, Ferran (2005) “Raising academic standards and vocational concentrators: Are they better off or worse off?” Education Economics, 13 (2), pp. 171-187.

Borman, G., Hewes G., Overman, L Brown S. (2002) Comprehensive School Reform and Student Achievement: A Meta-Analysis Report No. 59, Center for Research on the Education of Students Placed At Risk (CRESPAR), a national research and development center supported by a grant (No. R-117-D40005).

Borman, G.D., Rachuba, L., Datnow, A., Alberg, M., MacIver, M., Stringfield, S., & Ross, S. (2000). Four models of school improvement: Successes and challenges in reforming low-performing, high-poverty Title I schools. CRESPAR Report #48. Baltimore, MD: Johns Hopkins University, Center for Research on the Education of Students Placed At Risk.

Carnoy M. and Loeb S (2002). Does External Accountability Affect Student Outcomes? A Cross-State Analysis, Education Evaluation and Policy Analysis, Volume 24, Number 4.

Chaney, Bradford; Kenneth Burgdorf; Nadir Atash (1997). “Influencing achievement through high school graduation requirements.” Educational Evaluation and Policy Analysis, 19(3), pp. 229-244.

Dee, Thomas S. (2002) “Standards and student outcomes: lessons from the ‘first wave’ of education reform.” Paper presented at “Taking Account of Accountability: Assessing Politics and Policy,” Kennedy School of Government, Harvard University, Cambridge, MA, June 10-11.

Darling-Hammond, L., Ancess, J., & Wichterle Ort, S. (2002, Fall). Reinventing high school: Outcomes of the Coalition Campus Schools Project. American Educational Research Journal, 39(3). pp. 639-73. Available: http://www.schoolredesign.net/srn/binary/Reinventing%20HighSchool%20-%20LDH%20et%20al.pdf

Darling-Hammond, L. with Alexander, M., & Prince, D. (2002). Redesigning schools: What matters and what works - 10 features of good small schools. School Redesign Network at Stanford University. Available: http://www.schoolredesign.com/srn/binary/SchoolsBook.pdf

Dee, Thomas S. (2002) “Standards and student outcomes: lessons from the ‘first wave’ of education reform.” Paper presented at “Taking Account of Accountability: Assessing Politics and Policy,” Kennedy School of Government, Harvard University, Cambridge, MA., June 10-11.

56

Driscoll D, Halcoussis D, Svorny S (2003) School district size and student performance. Economics of Education Review 22: 193-201.

The Education Trust. (2005). Gaining Traction, Gaining Ground How Some High Schools Accelerate Learning for Struggling Students November 2005.

Education Trust (2001). Dispelling the Myth: High Poverty Schools Exceeding

Expectations” in cooperation with the Council of Chief State School Washington, DC. (Website: www.edtrust.org)

Fort Worth Independent School District. (2005). A Study of the Effective and Efficient Size of Schools. (conducted by MGT of America, Inc).

Garrett Z, Newman M, Elbourne D, Bradley S, Noden P, Taylor J, West A (2004) Secondary School Size: A Systematic Review. In: Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London

Gregory, Tom (2000). School Reform and the No-Man's-Land of High School Size. (University of Washington, Small Schools Project at the Center for Reinventing Public Education, Seattle, WA ) http://www.smallschoolsproject.org/PDFS/gregory.pdf

Greene, Jay P.; Marcus A. Winters (2004). “Pushed out or pulled up? Exit exams and dropout rates in public high schools.” Center for Civic Innovation, Manhattan Institute, Education Working Paper 5.

Greene J and Winters M, (2004). Pushed Out or Pulled Up? Exit Exams and Dropout Rates in Public High Schools, Manhattan Institute.

Hoffer, Thomas B.. (1997) “High school graduation requirements: Effects on Dropping Out and Student Achievement..” Teachers College Record, 23 (2), pp.99-121.

Howley C and Howley Aimeee. School Size and the Influence of Socioeconomic Status on Student Achievement: Confronting the Threat of Size Bias in National Data Sets, Educational Policy Analysis Archives .12 (52) Retrieved 2/2006 from http://epaa.asu.edu/epaa/v12n52/.

Howley, C., M. Strange, and R. Bickel. 2000. Research about school size and school performance in impoverished communities. ERIC digest. Charleston, W. Va.: ERIC Clearinghouse on Rural Education and Small Schools. Retrieved 07/19/02 from http://www.ael.org/eric/digests/edorc0010.htm

Husbands J. and. Beese S (2001) “Review of Selected High School Reform Strategies,” Aspen Program on Education, 2001.

High School Graduates. Western Interstate Commission for Higher Education. Knocking at the College Door: Projections of High School Graduates by State and Race/Ethnicity 1996-2012. Boulder, C0

57

Jacob, Brian A. (2001) “Getting tough? The impact of high school graduation exams.” Educational Evaluation and Policy Analysis, 23 (2), pp.99-121.

Lee J (1998) State Policy Correlates of the Acheivement Gap Among Racial and Social Groups. Studies in Educational Evaluatuon, 24, 137-152.

Lee, V.E. (2004) Effects of High School Size on Student Outcomes: Response to Howley and Howley, Educational Policy Analysis Archives. 12 (53) Retrieved 2/2006 from http://epaa.asu.edu/epaa/v12n53/.

Lee, V.E. Restructuring High Schools for Equity and Excellence: What Works. New York: Teachers College Press. 2001.

Lee, V., & Smith, J. (1997). High school size: Which works best and for whom? Educational Evaluation and Policy Analysis, 19(3), 205-27.

Lillard, Dean R.; DeCicca, Philip P. (2001) “Higher standards, more dropouts? Evidence within and across time.” Economics of Education Review, 20 (5), pp. 459-473.

Lezotte, Lawrence W. and Jo-Ann Cipriano Pepperl. The Effective Schools Process: A Proven Path to Learning for All. Effective Schools Products, Ltd., Okemos, MI, 1990.

Ma L. and Ma X, (2005) Estimating Correlates of Growth Between Mathematics and Science Achievement via a Multivariate Multilevel Design With Latent Variables

Muller, Chandra. (1998) “The minimum competency exam requirement, teachers' and students’ expectations and academic performance.” Social Psychology of Education, 2 (2), pp. 199-216.

McLaughlin D, Drori G, Ross M (2000) School-Level Correlates of Academic Achievement: Student Assessment Scores in SASS Public Schools. NCES 2000-303. Washington DC: Office of Education Research and Improvement, US Department of Education. Available from: http://nces.ed.gov/pubs2000/2000303.pdf

Measuring Up: The State-By-State Higher Education Report Card : http://highereducation.org

The National Association of Secondary School Principals (NASSP) (2000) Breaking Ranks II: Strategies of Leading School Reform.

Overbay, Amy (2003). Schools Size: A Review of the Literature, E&R Report, No. 03.03 February 2003.

Orfield, Gary, Losen, Daniel, Wald Johanna, Swanson, Christopher (2004). Losing Our Future: How Minority Youth are Being Left Behind by the Graduation Rate Crisis. Civil Rights Project a Harvard University, Urban Institute, Advocates for Children of New York, Civil Society Institute

58

Potts, A, Blank R, and Williams A (2002)., Key State Education Policies on PK-12 Education: 2002, Results from the 2002 CCSSO Policies and Practices Survey.

Reardon S and Galindo C (2002), Do High-Stakes Tests Affect Students’ Decisions to Drop Out of School? Evidence from NELS, Paper presented for presentation at the Annual Meeting of the American Educational Research Association.

Raudenbush, Stephen, (2004) Schooling, Statistics, and Poverty: Can We Measure School Improvement, ETS.

Raudenbush, S. W. (2001). Comparing personal trajectories and drawing causal inferences from longitudinal data. Annual Review of Psychology, 52, 501-525.

Raudenbush, S. W., & Willms, J. D. (1995). The estimation of school effects, Journal of Educational and Behavioral Statistics, 20 (4), pp. 307-35.

Slavin, R.E. (2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31(7), 15-21.

Slavin, R.E., Madden, N.A., Dolan, L.J., Wasik, B.A., Ross, S., Smith, L., & Dianda, M. (1996).Success for All: A summary of research. Journal of Education for Students Placed At Risk,1 (1), 41-76.

Stone, C.A., & Lane, S. (2003). Consequences of a state accountability program: Examining relationships between school performance gains and teacher, student, and school variables. Applied Measurement in Education, 16, 1-26

Strizek, G.A., Pittsonberger, J.L., Riordan, K.E., Lyter, D.M., and Orlofsky, G.F. (2006). Characteristics of Schools, Districts, Teachers, Principals, and School Libraries in the United States: 2003-04 Schools and Staffing Survey (NCES 2006-313 Revised). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

U.S. Department of Education, National Assessment Governing Board, (2003) Background Information Framework for the National Assessment of Educational Progress National Assessment Governing Board Adopted August 1, 2003

Warren, John Robert; Melanie R. Edwards (2005). “High school exit examinations and high school completion: evidence from the early 1990s.” Educational Evaluation and Policy Analysis, 27(1), pp. 53-74.

Warren, John Robert; Krista N. Jenkins; Rachael B. Kulick (2006). “High school exit examinations and state-level completion and GED rates, 1975 through 2002.” Educational Evaluation and Policy Analysis, 28(3), pp. 131-152.

Zvoch, K., & Stevens, J. J. (2003). A multilevel, longitudinal analysis of middle school math and language achievement. Educational Policy Analysis Archives, 11 (20). (Available at: http://epaa.asu.edu/epaa/v11n20/).

59

Zvoch, K., & Stevens, J. J. (2005). Sample exclusion and student attrition effects in the longitudinal study of middle school mathematics performance. Educational Assessment, 10(2), 105-123

SAT. The College Board. "2001 SAT V+M Score Bands Report," unpublished data

ACT. "Number of 2001 High School Graduates with ACT Composite Scores of 26 or Higher," unpublished analysis, Iowa City, Iowa

60

61