Data Reporting and Evaluation: Back to the BasicsLiann Seiter and Dory Seidel, NDTAC

2

Agenda

• Collection of Data– Federal collection requirements– State collection processes– Data quality– Resources available for collection

• Reporting of Data– Federal data collection systems– Federal reporting timeline– Resources available for reporting

• Evaluation– Federal use of data– State use of data

• Beyond the Basics ND Coordinator responsibilities related to data collection and

reporting are highlighted in text boxes

3

Collection of Data

Federal collection requirementsState collection processesData qualityResources available for collection

4

Collection: Statutory Requirements

Each State agency and local educational agency shall—

(1) submit evaluation results to the State educational agency and the Secretary; and

(2) use the results of evaluations under this section to plan and improve subsequent programs for participating children and youth.

State and local agencies receiving Title I, Part D, funds must evaluate their programs’ impact on the ability of students:

(1) to maintain and improve educational achievement;

(2) to accrue school credits that meet State requirements for grade promotion and secondary school graduation;

(3) to make the transition to a regular program or other education program operated by a local educational agency;

(4) to complete secondary school (or secondary school equivalency requirements) and obtain employment after leaving the correctional facility or institution for neglected or delinquent children and youth; and

(5) as appropriate, to participate in postsecondary education and job training programs.

Source: Title I, Part D, Statute, Subpart 3Become familiar with

statutory requirements

5

Collection: Two Federal Data Collections

Major Federal data collections for Title I, Part D:• Annual Child Count

– Used by U.S. Department of Education (ED) to determine Title I, Part A, and Title I, Part D, funding allocations

• Consolidated State Performance Report (CSPR) – Provides demographic and outcome data for

programs receiving Title I, Part D, funds

Understand the types of data requested by ED

6

Collection: Federal Collection Requirements

All Title I, Part D-funded programs must collect data to report to ED. Program categories:

• At-Risk Programs—Subpart 2 only

• Other Programs• Neglected Programs• Juvenile Detention• Juvenile Corrections• Adult Corrections—Subpart 1 only

Resources: CSPR Forms, CSPR Guide

Identify the programs receiving funds

7

Collection: Federal Collection Requirements

Each State agency and local agency program submits the following data to ED:

• Student and facility counts

• Student demographics

• Transition services

• Academic and vocational outcomes

• Academic performance in reading and mathematics

Be familiar with the CSPR measures

8

Collection: State Collection Processes

• State and Federal reporting requirements may differ.

• Each State has its own collection tool and processes (i.e., paper and pencil, online tool, electronic files).

• Data contacts at State, subgrantee, and program levels are responsible for certain aspects of data collection and reporting.

Know State reporting

requirements

Develop systems to assist and improve the data collection

process

Build relationships with the data

contacts in the State

9

Collection: Data Quality Review

• Data need to be trustworthy—high-quality data are accurate, consistent, unbiased, understandable, transparent.

• Data quality is best achieved when timely reviews are done at the program level.

• Educating subgrantees/programs on data collection can improve data quality.

Assess the quality of the data received

Provide training to improve data quality

10

Collection: Data Quality

ED

B A

SEA

State Agency

S1

LEA 1S2

LEA 2S2

1A 1B 2A

11

Collection Resource: CSPR Forms

http://www2.ed.gov/admins/lead/account/consolidated/index.html

12

Collection Resource: CSPR Guide

“Instructional Guide to Reporting Title I, Part D, Data in the CSPR”

• Overview of data submission• Table-by-table instructions• Data quality checklists• List of CSPR and EDFacts

coordinators by State

12

http://www.neglected-delinquent.org/nd/topics/index2.php?id=10

13

Collection Resource: NDTAC

NDTAC provides multiple products and services to help State coordinators with data collection:

• ND community calls• Annual CSPR Webinar• Data collection list• NDTAC presentations

– Better Data, Better Decisions by Bi Vuong– Taking the Fast Lane to High-Quality Data by

Stephanie Lampron and Sarah Bardack• Individual technical assistance

14

Federal data collection systemsFederal reporting timelineResources available for reporting

Reporting the Data

15

Reporting: Federal Reporting Systems, CSPR

• CSPR is a data collection instrument administered annually by ED’s Office of Elementary and Secondary Education (OESE).

• The required measures for Title I, Part D, can be found in CSPR section 2.4.

16

Reporting: Federal Reporting Systems, EDFacts

• EDFacts is an ED initiative to collect, analyze, report on, and promote the use of high-quality performance data.

• Most of the Title I, Part D, data are now reported through EDFacts. Eventually, all data will be reported through the EDFacts’ online EDEN Submission System (ESS).

17

Reporting: Federal Reporting Systems

Category/Area CSPR Tables EDFacts Specification Files

Facility Counts/Programming

S1=2.4.1.1 & 2.4.1.1.1S2=2.4.2.1 & 2.4.2.1.1 Not included

Student Count/Demographics

S1=2.4.1.2S2=2.4.2.2

S1=C119 & C135S2=C127 & C135

Transition Services S1=2.4.1.3.1S2=2.4.2.3.1 S1 & S2=C182

Academic/Vocational Outcomes

S1=2.4.1.3.2S2=2.4.2.3.2

In Program=C180Exited Program=C181

Pre- and Posttesting in Reading and Math S1=2.4.1.6.1 & 2.4.1.6.2

S2=2.4.2.6.1 & 2.4.2.6.2S1=C113 & C135S2=C125 & C135

18

Reporting: Federal CSPR Reporting Timeline

Establish reporting timelines

19

Reporting Resource: EDFacts Coordinator List

Identify who is responsible for data submission in your

State and who has access to EDEN

http://www2.ed.gov/about/inits/ed/edfacts/eden/ess/edfacts-coordinators.pdf

20

Reporting Resources: Federal Reporting Systems

• CSPR– CSPR forms from ED

– State CSPR coordinators

– NDTAC’s CSPR guide, collection list, and annual CSPR Webinar

– NDTAC’s data team individualized technical assistance about data quality during CSPR collection

• EDFacts– File specifications

– State EDFacts coordinators

– Partner Support Center (PSC)• Phone: 877-HLP-EDEN

• Email: EDEN_SS@ed.gov

21

Federal use of dataState use of data

Evaluation

22

Evaluation: Federal Use of Data

• Program evaluation and performance assessments– Government Performance Results Act of 1993 (GPRA)

Performance Reports– Federal Budget Requests to Congress– In response to Congressional Requests (e.g.,

Elementary and Secondary Education Act reauthorization)

• Public dissemination Understand how ED will use the CSPR data

23

Public Dissemination of Data

ED Data Express

http://www.eddataexpress.ed.gov/

24

Public Dissemination of Data

National and State Fast Facts

http://data.neglected-delinquent.org/index.php?id=01

25

Public Dissemination of Data

NDTAC Annual Reports

http://www.neglected-delinquent.org/nd/topics/index2.php?id=12

26

Evaluation: State Use of Data

• States must use data to improve Title I, Part D, programs, as directed by Subpart 3.

• Effective uses of data at the State level for program improvement may include:– Program evaluation– Decision making– Goal setting

• Data use at the subgrantee level also supports program improvement.

Use data for program quality improvement

27

Evaluation: State Use of Data

28

Evaluation: State Use of Data

Suggested areas for program improvement:• Federal performance measures

– High school diploma/GED

– High school course credits

– Math and reading

– Cost per diploma• State priorities• Specific issues related to student population

29

Evaluation Resources: State Use of Data

• NDTAC publications– Title I, Part D, Program Administration Planning Toolkit

Application Review and Needs Assessment tools– State and local Report Card resources

• Data and decision-making presentations– Better Data, Better Decisions– Making the Most of Your Data: Strategies for Evaluating Yo

ur Program

– Using Pre- and Posttesting To Improve Programming and Student Achievement

– Using Data To Identify and Promote the Implementation of Effective Programming for Children and Youth Who Are At-Risk

30

Beyond the Basics

31

Beyond the Basics

• Establish goals for Part D programs– Using the data, establish long- and short-term goals for

programs• Expand upon the data collected• Use all available data

– Use access you may have to additional data from these programs (i.e., student-level data)

• Share the data– Use data collected to promote programs

• Use data in program administration– Needs assessment, subgrantee monitoring, application reviews