35th Annual National Conference on Large-Scale Assessment June 18, 2005 How to compare NAEP and State Assessment Results NAEP State Analysis Project Don

35th Annual National Conference on Large-Scale AssessmentJune 18, 2005

How to compare NAEP and State Assessment Results

How to compare NAEP and State Assessment Results

NAEP State Analysis ProjectNAEP State Analysis Project

Don McLaughlinVictor Bandeira de Mello

how do NAEP and state assessment trend results compare to each other?

how do NAEP and state assessment gap results compare to each other?

do NAEP and state assessments identify the same schools as high-performing and low-performing?

how do NAEP and state assessment trend results compare to each other?

how do NAEP and state assessment gap results compare to each other?

do NAEP and state assessments identify the same schools as high-performing and low-performing?

overview: the questionsoverview: the questions

results are different because standards are different students are different time of testing is different motivation is different manner of administration is different item formats are different test content is different tests have measurement error


overview: the differencesoverview: the differences



overview: the differencesoverview: the differences

the problem of different standards how we addressed it

the problem of different students how we addressed it

factors that affect validation

the problem of different standards how we addressed it

the problem of different students how we addressed it

factors that affect validation

overview: the focusoverview: the focus

the problem of different standards the problem of different standards

trends and gaps are being reported in terms of percentages of students meeting standards

the standards are different in every state and in NAEP

comparisons of percentages meeting different standards are not valid




the different standardsthe different standards

concept of population profile

a population profile is a graph of the achievement of each percentile of a population

concept of population profile

a population profile is a graph of the achievement of each percentile of a population


a population achievement profile a population achievement profile


100

150

200

250

300

350

400

0 10 20 30 40 50 60 70 80 90 100

Percentile

NAEP scale

Advanced

Proficient

Basic

500

0

a population achievement profile a population achievement profile


100

150

200

250

300

350

400

0 10 20 30 40 50 60 70 80 90 100

Percentile

NAEP scale

Advanced

Proficient

Basic

500

0

76%

32%

5%

a population trend profile a population trend profile


100

150

200

250

300

350

400

0 10 20 30 40 50 60 70 80 90 100

Percentile

NAEP scale

Future NAEP achievement ( 2003 + 10 Points )

Average NAEP achievement in 2003

Advanced

Proficient

Basic

500

0

a population trend profile a population trend profile


100

150

200

250

300

350

400

0 10 20 30 40 50 60 70 80 90 100

Percentile

NAEP scale

Future NAEP achievement ( 2003 + 10 Points )

Average NAEP achievement in 2003

Advanced

Proficient

Basic

500

0

+9% +13% +5%

gains

a population gap profile a population gap profile


100

150

200

250

300

350

400

0 10 20 30 40 50 60 70 80 90 100

Percentile

NAEPscale

NAEP Achievement of non-disadvantaged students

NAEP Achievement of disadvantaged students

Advanced

Proficient

Basic

500

0

2%

13%

13%

gaps



100

150

200

250

300

350

400

0 10 20 30 40 50 60 70 80 90 100

Percentile

NAEPscale

Basic

NAEP Achievement of non-disadvantaged studentsNAEP Achievement of disadvantaged students

Advanced

Proficient

500

0

8%

5%

13%

gaps after a 20-point gain



100

150

200

250

300

350

400

0 10 20 30 40 50 60 70 80 90 100

Percentile

NAEPscale

Basic

NAEP Achievement of non-disadvantaged studentsNAEP Achievement of disadvantaged students

Advanced

Proficient

500

0

8%

5%

13%

8 points smaller

the same 6 points larger

gap changes








the solution to the problem is to compare results at comparable standards

for comparing NAEP and state assessment gains and gaps in a state, score NAEP at the state’s standard

the solution to the problem is to compare results at comparable standards

for comparing NAEP and state assessment gains and gaps in a state, score NAEP at the state’s standard


NAEP

individual plausible values for 4th and 8th grade reading in 1998, 2002, and 2003 and mathematics in 2000 and 2003

state assessment scores

school percentages meeting standards linked to NCES school codes, in 2003 and some earlier years

www.schooldata.org

NAEP

individual plausible values for 4th and 8th grade reading in 1998, 2002, and 2003 and mathematics in 2000 and 2003

state assessment scores

school percentages meeting standards linked to NCES school codes, in 2003 and some earlier years

www.schooldata.org


a school-level population gap profile a school-level population gap profile


0

20

40

60

80

100

0 10 20 30 40 50 60 70 80 90 100

Percentile in group

Percent meetingstates' primary

standards

Not disadvantaged

Disadvantaged

Achievement measured by state assessments

comparing school-level population gap profiles comparing school-level population gap profiles


Achievement measured by NAEP (thin) and state assessments (thick)

0

20

40

60

80

100

0 10 20 30 40 50 60 70 80 90 100Percentile in group

Percent meetingstates' primary

standards

Not disadvantaged

Disadvantaged

comparing school-level population gap profiles comparing school-level population gap profiles


Achievement measured by NAEP (thin)

and state assessments (thick)

-60

-40

-20

0

20

40

0 10 20 30 40 50 60 70 80 90 100Percentile in group

Gap in percent meeting states'

primary standards

Not disadvantaged

Disadvantaged

Gap

scoring NAEP at the state assessment standard

determine the cutpoint on the NAEP scale that best matches the percentages of students meeting the state’s standard

compute the percentage of the NAEP plausible value distribution that is above that cutpoint

scoring NAEP at the state assessment standard

determine the cutpoint on the NAEP scale that best matches the percentages of students meeting the state’s standard

compute the percentage of the NAEP plausible value distribution that is above that cutpoint


equipercentile equating equipercentile equating


Setting NAEP Scale Score for State Performance Standard

0

9

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45

60% 60%

State Assessment Scores

NAEP Scores

Standard



A B C D average

205 215 225 235 220 average NAEP scale score

hypothetical NAEP results in four schools in a state (actual samples have about 100 schools)



A B C D average

205 215 225 235 220

10% 20% 40% 50% 30%

average NAEP scale score

percent meeting state standard

in school A, the state reported that 10% of the students met the standards



A B C D average

205 215 225 235 220

10% 20% 40% 50% 30%

225 225 235 235 230



NAEP scale score corresponding to percent meeting state standard

in school A, 10% of the NAEP plausible value distribution was above 225



A B C D average

205 215 225 235 220

10% 20% 40% 50% 30%

225 225 235 235 230

5% 10% 45% 60% 30%




percent above 230 on NAEP

If the equating is accurate, we should be able to reproduce the percentages meeting the state’s standard from the NAEP sample



A B C D average

205 215 225 235 220

10% 20% 40% 50% 30%

225 225 235 235 230

5% 10% 45% 60% 30%

error -5% -10% +5% +10%




percent above 230 on NAEP

relative error

in estimating cutpoints for state standards, relative error is the ratio of the observed error in reproducing school-level percentages meeting standards to that expected due to sampling and measurement error

relative error

in estimating cutpoints for state standards, relative error is the ratio of the observed error in reproducing school-level percentages meeting standards to that expected due to sampling and measurement error


mapping of primary state standards on the NAEP scale: math grade 8 in 2003



WY

WI

WA

VA

VT

TX

SD

SC

RI

PA

OR

OK

ND

NC

NY

NM

NJNV

MT

MO

MS

MI

MA

MD

MELA

KY

KS

IA

IN

IL

ID

HI

GA

FL

DC

DECT

CO

CA

AR

AZ

AK

225

250

275

300

325

1 2 3

Relative error

NAEP scale

NAEP Basic

NAEP Proficient

500

0




WY

WI

WA

VA

VT

TX

SD

SC

RI

PA

OR

OK

ND

NC

NY

NM

NJNV

MT

MO

MS

MI

MA

MD

MELA

KY

KS

IA

IN

IL

ID

HI

GA

FL

DC

DECT

CO

CA

AR

AZ

AK

225

250

275

300

325

1 2 3

Relative error

NAEP scale

NAEP Basic

NAEP Proficient

500

0




WY

WI

WA

VA

VT

TX

SD

SC

RI

PA

OR

OK

ND

NC

NY

NM

NJNV

MT

MO

MS

MI

MA

MD

MELA

KY

KS

IA

IN

IL

ID

HI

GA

FL

DC

DECT

CO

CA

AR

AZ

AK

225

250

275

300

325

1 2 3

NAEP Basic

NAEP Proficient

500

0

national percentile ranks corresponding to state grade 4 reading standardsin 2003

national percentile ranks corresponding to state grade 4 reading standardsin 2003


WY

WI

WA

VA

VT

SD

SC

RI

PA

OK

OH

ND

NC

NYNM

NJ

NHNV

MT

MO

MS

MN

MI

MA

MD

ME

LA

KYKS

IA

IL

ID

HI

GA

FL DC

CT

CO

CA

ARAZ

AK

(wv)

(tx)

(or) (ne)

(in)

(de)

0.0

20.0

40.0

60.0

80.0

100.0

1 2 3

Relative error in determining the placement of the standard

National percentile rank corresponding

to the standard

NAEP Basic Standard

NAEP Proficient Standard

HighStandards

LowStandards

(These state's standards are less accurately determined.)

states have set widely varying standards

does it matter?

standards should be set where they will motivate increased achievement

surely some are too high and some are too low

states have set widely varying standards

does it matter?

standards should be set where they will motivate increased achievement

surely some are too high and some are too low


states with higher standards have lower percentages of students meeting them

states with higher standards have lower percentages of students meeting them


PCT = 339 - 1.04 (NSE)

R2 = 0.78

0

20

40

60

80

100

225 250 275 300 325

NAEP scale equivalent (NSE) of primary state standard

Percent of students achieving primary standard (PCT)

0 500

on NAEP, states with higher standards do about the as well as other states

on NAEP, states with higher standards do about the as well as other states


PCT = 41.5 + 0.02 (NSE)

R2 = 0.001

0

20

40

60

80

100

225 250 275 300 325

NAEP scale equivalent (NSE) of primary state standard

Percent proficient on NAEP (PCT)

0 500

the problem of different students the problem of different students

the problem of different students

different school coverage

different grade tested

absent students

excluded SD/ELLs

the problem of different students


different grade tested

absent students

excluded SD/ELLs

the different studentsthe different students


our comparisons between NAEP and state assessment results are for the same schools. NAEP weights these schools to represent the public school population in each state

we matched schools serving more than 99 percent of the public school population. However, especially for gap comparisons, we were missing state assessment results for small groups whose scores were suppressed for confidentiality reasons

the median percentage of the NAEP student population included in the analyses was about 96 percent


our comparisons between NAEP and state assessment results are for the same schools. NAEP weights these schools to represent the public school population in each state

we matched schools serving more than 99 percent of the public school population. However, especially for gap comparisons, we were missing state assessment results for small groups whose scores were suppressed for confidentiality reasons

the median percentage of the NAEP student population included in the analyses was about 96 percent


different grades tested

in some states, assessments were administered in grades 3, 5, or 7, and we compared these results to NAEP results in grades 4 and 8

the difference in grades involved a different cohort of students, as well as a difference in curriculum content. These effects combined to reduce NAEP-state assessment correlations in some states by about 0.05 to 0.10

different grades tested

in some states, assessments were administered in grades 3, 5, or 7, and we compared these results to NAEP results in grades 4 and 8

the difference in grades involved a different cohort of students, as well as a difference in curriculum content. These effects combined to reduce NAEP-state assessment correlations in some states by about 0.05 to 0.10


absent students

some students are absent from NAEP sessions, and some of these are not made-up in extra sessions.

NAEP imputes the achievement of the absent students to be similar to that of similar students who were not absent

a study by the NAEP Validity Studies Panel has found that these imputations leave negligible (if any) bias in NAEP results due to absences

that study compared the state assessment scores of students absent from NAEP to the scores of students not absent

absent students

some students are absent from NAEP sessions, and some of these are not made-up in extra sessions.

NAEP imputes the achievement of the absent students to be similar to that of similar students who were not absent

a study by the NAEP Validity Studies Panel has found that these imputations leave negligible (if any) bias in NAEP results due to absences

that study compared the state assessment scores of students absent from NAEP to the scores of students not absent


excluded SD/ELLs

some students with disabilities and English language learners are excluded from NAEP and others are included. A teacher questionnaire is completed for each SD/ELL selected for NAEP

in the past, NAEP has ignored this exclusion, and there is clear evidence that as a result, states in which NAEP exclusions increased had corresponding reports of larger NAEP achievement gains (and vice versa)

excluded SD/ELLs

some students with disabilities and English language learners are excluded from NAEP and others are included. A teacher questionnaire is completed for each SD/ELL selected for NAEP

in the past, NAEP has ignored this exclusion, and there is clear evidence that as a result, states in which NAEP exclusions increased had corresponding reports of larger NAEP achievement gains (and vice versa)


full population estimates

the trend distortions caused by changing exclusion rates can be minimized by imputing the achievement of excluded students.

in this project, comparisons between NAEP and state assessment results are based on the NAEP full population estimates [1]

imputations for excluded SD/ELLs are based on the achievement of included SD/ELLs with similar questionnaire and demographic profiles in the same state

[1] an appendix includes comparisons using standard NAEP estimates


the trend distortions caused by changing exclusion rates can be minimized by imputing the achievement of excluded students.

in this project, comparisons between NAEP and state assessment results are based on the NAEP full population estimates [1]

imputations for excluded SD/ELLs are based on the achievement of included SD/ELLs with similar questionnaire and demographic profiles in the same state

[1] an appendix includes comparisons using standard NAEP estimates


statistically significant state NAEP gains from 1996 to 2000

statistically significant state NAEP gains from 1996 to 2000

grade 4 grade 8

17 of 37 16 of 35

12 of 37 7 of 35

ignoring excluded students



statistically significant state NAEP gains and losses from 1998 to 2002

statistically significant state NAEP gains and losses from 1998 to 2002

grade 4 grade 8

gains losses gains losses

18 1 8 6

23 0 7 2

ignoring excluded students



factors that affect validation factors that affect validation

the question

do state assessments and NAEP agree on which schools are doing better than others?

the measure

correlation between state assessment and NAEP school-level results

the question

do state assessments and NAEP agree on which schools are doing better than others?

the measure

correlation between state assessment and NAEP school-level results

validationvalidation

factors that specifically affect NAEP-state assessment correlations of school-level statistics

size of school NAEP samples

grade level the same or different

extremeness of the standard

factors that specifically affect NAEP-state assessment correlations of school-level statistics

size of school NAEP samples

grade level the same or different

extremeness of the standard


median school-level correlations between NAEP and state assessment results

median school-level correlations between NAEP and state assessment results

grade 4 grade 8

math reading math reading

original 0.76 0.72 0.81 0.73

adjusted 0.84 0.82 0.86 0.81


NAEP and state assessment school means NAEP and state assessment school means


two reports have been produced on 2003 NAEP-state assessment comparisons, one for mathematics and one for reading

each report has an appendix with multi-page comparison profiles for all of the states

the following are examples of the kinds of information included

two reports have been produced on 2003 NAEP-state assessment comparisons, one for mathematics and one for reading

each report has an appendix with multi-page comparison profiles for all of the states

the following are examples of the kinds of information included

summarysummary

state profiles of NAEP-state assessment comparisons

test score descriptions and results summary standards relative to NAEP correlations with NAEP changes in NAEP exclusion/accommodation rates trends (NAEP vs. state assessment) gaps (NAEP vs. state assessment) gap trends (NAEP vs. state assessment)

state profiles of NAEP-state assessment comparisons

test score descriptions and results summary standards relative to NAEP correlations with NAEP changes in NAEP exclusion/accommodation rates trends (NAEP vs. state assessment) gaps (NAEP vs. state assessment) gap trends (NAEP vs. state assessment)

state profilesstate profiles

a state’s standards relative to its achievement distribution

a state’s standards relative to its achievement distribution


a state’s math trends comparison a state’s math trends comparison


poverty gap comparison




state assessment results


state assessment results



NAEP results


NAEP results



NAEP - state assessment


NAEP - state assessment


a state’s poverty gap comparison a state’s poverty gap comparison


trends gaps overall coverage subpopulation coverage school analyses sample

trends gaps overall coverage subpopulation coverage school analyses sample

other resultsother results

comparison of trends reported by NAEP and state assessments (number of states)

comparison of trends reported by NAEP and state assessments (number of states)

other results: trendsother results: trends

grade 4 grade 8

math 00-03 read 98-03 math 00-03 read 98-03

3 5 5 5

10 3 11 1

11 0 6 0

state assessment reported greater gains

no significant difference

NAEP reported greater gains

reading 2003 NAEP and state assessments tended to find similar

achievement gaps

math 2003 NAEP tended to find slightly larger gaps than state

assessments did

reading 2003 NAEP and state assessments tended to find similar

achievement gaps

math 2003 NAEP tended to find slightly larger gaps than state

assessments did

other results: gapsother results: gaps

median state percentages of NAEP schools and student population matched and included in analyses

median state percentages of NAEP schools and student population matched and included in analyses

other results: coverageother results: coverage

grade 4 grade 8

math read math read

99.1 99.1 99.2 99.2

99.5 99.6 99.8 99.8

94.9 94.4 95.3 94.2

95.8 95.4 96.8 96.1

percent of schools matched

percent of student population matched

percent of schools included in analyses

percent of students included in analyses

number of states and percent minority students included in the 2003 reading gap analyses

number of states and percent minority students included in the 2003 reading gap analyses

other results: coverageother results: coverage

grade 4 grade 8

number of states 26 20

students included (%) 88.0 99.2





black

hispanic

disadvantage

percent meeting standards from state tests in NAEP schools and from state reports, 2003

percent meeting standards from state tests in NAEP schools and from state reports, 2003

other results: school sampleother results: school sample

0

20

40

60

80

100

0 20 40 60 80 100

State aggregate from NAEP sample

State aggregate from website

producing the report producing the report

SAS programsSAS programs

the process find state scores for NAEP sample score NAEP in terms of state standards compute inverse CDF pair for subpopulation profiles compute mean NAEP-state gap differences and standard

errors compute trends and gains compute smoothed frequency distribution of plausible values compute NAEP-state correlations








data setup






population profiles

















programs makefiles.sas standards.sas gaps.sas gaps_g.sas trends.sas trends_r.sas trends_g.sas distribution.sas correlation.sas





http://www.schooldata.org/reports.asp

SAS programs: setupSAS programs: setup

makefiles.sas for state st get NAEP plausible values for subject s, grade g,

and year y get state assessment data for NAEP schools (from NLSLASD) merge files to get example02.sas7bdat and

example03.sas7bdat

makefiles.sas for state st get NAEP plausible values for subject s, grade g,

and year y get state assessment data for NAEP schools (from NLSLASD) merge files to get example02.sas7bdat and

example03.sas7bdat


makefiles.sas

*******************************************************************************;* Project : NAEP State Analysis *;* Program : MakeFiles.SAS *;* Purpose : make source file for workshop at LSAC 2005 *;* *;* input : naep_r403 NAEP Reading grade 4 2003 data *;* naep_r402 NAEP Reading grade 4 2002 data *;* XX state XX assessment data *;* YY state YY assessment data *;* *;* output : example02 - 2002 data *;* example03 - 2003 data *;* *;* Author : NAEP State Analysis Project Staff *;* American Institutes for Research *;* *;*******************************************************************************;

makefiles.sas

*******************************************************************************;* Project : NAEP State Analysis *;* Program : MakeFiles.SAS *;* Purpose : make source file for workshop at LSAC 2005 *;* *;* input : naep_r403 NAEP Reading grade 4 2003 data *;* naep_r402 NAEP Reading grade 4 2002 data *;* XX state XX assessment data *;* YY state YY assessment data *;* *;* output : example02 - 2002 data *;* example03 - 2003 data *;* *;* Author : NAEP State Analysis Project Staff *;* American Institutes for Research *;* *;*******************************************************************************;


standards.sas compute NAEP equivalents of state standards based on

school level state assessment scores in NAEP schools macro %stan(s,g,y,nlevs)

output Stansgy file with state standard cutpoints on NAEP sample

standards.sas compute NAEP equivalents of state standards based on

school level state assessment scores in NAEP schools macro %stan(s,g,y,nlevs)

output Stansgy file with state standard cutpoints on NAEP sample

s g y varname level cut stderror percent

R 4 03 Rs5t0403 2 164.2 3.2 91.9

R 4 03 Rs5t0403 3 205.8 1.1 68.9

R 4 03 Rs5t0403 4 264.9 1.8 9.4

StanR403


standards.sas generate school level file with percentages meeting levels by

reporting category, with jackknife statistics macro %StateLev(file,s,g,y) macro %NAEP_State_Pcts(s,g,y,group) macro %Sch_State_Pcts(standard,s,g,y)

output StPcts_standard_sgy with school stats for first/recent standard, by category

macro %Criterion(standard,s,g,y)

output criterion_ standard_sgy with criterion values for cutpoints

standards.sas generate school level file with percentages meeting levels by

reporting category, with jackknife statistics macro %StateLev(file,s,g,y) macro %NAEP_State_Pcts(s,g,y,group) macro %Sch_State_Pcts(standard,s,g,y)

output StPcts_standard_sgy with school stats for first/recent standard, by category

macro %Criterion(standard,s,g,y)

output criterion_ standard_sgy with criterion values for cutpoints

SAS programs: gapsSAS programs: gaps

gaps.sas compute and plot subpopulation profiles (inverse CDF) and

compute mean NAEP-state gap differences and respective standard errors, by regions of the percentile distribution

macro %gap(s,g,lev,y1,y2,group1,group2) where

y1 is the earlier years (need not be present)y2 is the later yearlev is the standard for which the gap is being comparedgroup1 is the 5-char name of the focal groupgroup2 is the 5-char name of the comparison group

gaps.sas compute and plot subpopulation profiles (inverse CDF) and

compute mean NAEP-state gap differences and respective standard errors, by regions of the percentile distribution

macro %gap(s,g,lev,y1,y2,group1,group2) where

y1 is the earlier years (need not be present)y2 is the later yearlev is the standard for which the gap is being comparedgroup1 is the 5-char name of the focal groupgroup2 is the 5-char name of the comparison group


gaps.sas output: inverse CDF for comparison pairs

ICDFr4__03group1group2

mean NAEP-State gap differences and SEs by regions of the percentile distribution

DiffGapsMINtoMAXgroup1group2R4__03.XLSDiffGapsMINtoMEDgroup1group2R4__03.XLSDiffGapsMEDtoMAXgroup1group2R4__03.XLSDiffGapsMINtoQ1_group1group2R4__03.XLSDiffGapsQ1_toQ3_group1group2R4__03.XLSDiffGapsQ3_toMAXgroup1group2R4__03.XLS

gaps.sas output: inverse CDF for comparison pairs

ICDFr4__03group1group2

mean NAEP-State gap differences and SEs by regions of the percentile distribution

DiffGapsMINtoMAXgroup1group2R4__03.XLSDiffGapsMINtoMEDgroup1group2R4__03.XLSDiffGapsMEDtoMAXgroup1group2R4__03.XLSDiffGapsMINtoQ1_group1group2R4__03.XLSDiffGapsQ1_toQ3_group1group2R4__03.XLSDiffGapsQ3_toMAXgroup1group2R4__03.XLS


gaps.sas output: population profiles

STATE_PV_03.gif state achievement profileSTATE_BW_03.gif state achievement profileNAEP_PV_03.gif NAEP achievement profileNAEP_BW_03.gif NAEP achievement profileNAEP_STATE_PV_03.gif NAEP/state gap profileNAEP_STATE_BW_03.gif NAEP/state gap profile d


STATE_PV_03.gif state achievement profileSTATE_BW_03.gif state achievement profileNAEP_PV_03.gif NAEP achievement profileNAEP_BW_03.gif NAEP achievement profileNAEP_STATE_PV_03.gif NAEP/state gap profileNAEP_STATE_BW_03.gif NAEP/state gap profile d



NAEP_PV_03.gif NAEP achievement profile


NAEP_PV_03.gif NAEP achievement profile


gaps_g.sas plot subpopulation profiles and place them on a four-panel

template to include in report macro %pop_profile set SAS/Graph options macro %plot_gaps plot graphs using options macro %createtemplate create four-panel template macro %replaygaps place graphs in template

gaps_g.sas plot subpopulation profiles and place them on a four-panel

template to include in report macro %pop_profile set SAS/Graph options macro %plot_gaps plot graphs using options macro %createtemplate create four-panel template macro %replaygaps place graphs in template


gaps_g.sas gaps_g.sas

SAS programs: trendsSAS programs: trends

trends.sas compute difference between state and NAEP and respective

standard errors output data file trends_sy including both NAEP and NAEP

state standard measures

trends_r.sas compute gains and respective standard errors output data file summary_s

trends.sas compute difference between state and NAEP and respective

standard errors output data file trends_sy including both NAEP and NAEP

state standard measures

trends_r.sas compute gains and respective standard errors output data file summary_s

SAS programs: trendsSAS programs: trends

trends_g.sas plot NAEP and state assessment trends by grade and place

them on a two-panel template to include in report compute t for testing significance of differences in gains

between NAEP and state assessment

trends_g.sas plot NAEP and state assessment trends by grade and place

them on a two-panel template to include in report compute t for testing significance of differences in gains

between NAEP and state assessment

SAS programs: correlationsSAS programs: correlations

correlation.sas compute NAEP-state correlations and standard errors macro %corrs(standard,s,g,y,group)

output CorrsY_standard_groupsgy file with state standard

correlation.sas compute NAEP-state correlations and standard errors macro %corrs(standard,s,g,y,group)

output CorrsY_standard_groupsgy file with state standard

RtR4032 RtR4032 RtR4032

correlation 0.60 0.73 0.43

standard error 0.11 0.06 0.10

SAS programs: distributionSAS programs: distribution

distribution.sas create file with plausible value frequency distribution

output distribution_sgy file

distribution.sas create file with plausible value frequency distribution

output distribution_sgy file

0

0.2

0.4

0.6

0.8

1

1.2

1.4

50 100 150 200 250 300 350


all programs and data files are available for download at


including files with the imputed scale scores for excluded students we used in the report

all programs and data files are available for download at


including files with the imputed scale scores for excluded students we used in the report

NAEP State Analysis Project NAEP State Analysis Project

American Institutes for Research

Victor Bandeira de Mello [email protected] Don McLaughlin [email protected]

National Center for Education Statistics

Taslima Rahman [email protected]

American Institutes for Research

Victor Bandeira de Mello [email protected] Don McLaughlin [email protected]

National Center for Education Statistics

Taslima Rahman [email protected]

Documents

35th Annual National Conference on Large-Scale Assessment June 18, 2005 How to compare NAEP and State Assessment Results NAEP State Analysis Project Don