Why Data Integrity? The Importance of Processes and Procedures TASSP Conference, June 2014 Texas Education Agency, Program Monitoring and Interventions

Why Data Integrity?

The Importance of Processes and Procedures

TASSP Conference, June 2014

Texas Education Agency, Program Monitoring and Interventions Michael Benedict, Heather Christie, Gordon

Franzen, Lizette Ramos

Copyright © Texas Education Agency 2014. All rights reserved.

Today’s Purpose

• The background of and differences between the three areas monitored by the Texas Education Agency.

• An overview of TEA's data validation monitoring processes, interventions, and supports.

• Data integrity tips for secondary schools based on issues and causal factors identified over the years.


Performance-Based Monitoring (PBM) Background

• The Performance-Based Monitoring (PBM) system was developed in 2003.

• Performance-based Monitoring Analysis System (PBMAS)

• Data Validation Monitoring (DVM).

• DVM is a data-driven system that relies on data submitted by districts.

• Includes annual data validation analyses that examine districts’:• Leaver and dropout data;

• Discipline data; and

• Student Assessment data.


Differences between DVM & PBMAS


Differences between DVM & PBMAS


Differences between Data Validation Indicators and PBMAS Indicators

Indicator Type ResultPublicly Posted by

TEA Standards District Response

Data Validation Suggests an anomaly No Based on annual review of data to identify

anomalous data and trends observed over

time

Validate accuracy of data locally and, as necessary,

improve local data collection and

submission procedures or address program implementation

concerns

PBMAS Yields a definitive result

Yes Based on standards established in advance

Improve performance or program effectiveness or if identification occurred

because of inaccurate data, improve data

collection and submission procedures.

Leavers - Legal

• TEC §39.308

• Amended to require annual electronic audit of drop out records

• Designed to identify districts that are at a high risk of having inaccurate dropout records, potentially leading to on-site monitoring of drop-out records.


2013 Leavers - Eight Data Validation Indicators

1. Leaver Data Analysis

2. Underreported Students

3. Use of leaver Reason Codes with No Dropouts

4. Use of One or More Leaver Reason Codes

5. Use of Certain Leaver Reason Dropout Codes

6. Missing PET* Submission (August 19, 2013-September 20, 2013)

7. Missing PET Submission (2012-2013 Reporting Year)

8. Continuing Students’ Dropout Rate (Class 2011), as of Fall 2012*PET is the Person Identification Database (PID) Enrollment Tracking extension.


Leavers – 2013 DVM Data Sources

• #1-5 and #8 reflect the 2011-2012 leaver data submitted by districts in the fall of 2012.

• #1 and #8 also include data submitted in the fall of 2011

• #1 also includes data submitted in the fall of 2010.

• #6 and #7 – PID* Enrollment Tracking reports for August 20, 2012 through September 20, 2013.

*Person Identification Database


2013 DVM LeaversIndicator 4 – Use of One or More Leaver Codes

• Code 60 – Home School• Parent must verify that student IS being home schooled

• Federal requirement - not enough to document that student intends to be home schooled

• Proper Documentation• Letter, signed and dated by parent/guardian


2013 DVM LeaversIndicator 4 – Use of One or More Leaver Codes

• Code 81 – Private School in Texas/Texas Job Corps Diploma Program

• Campus must document that student has actually enrolled in a private school in Texas or the Texas Job Corp Diploma Program

• Federal requirement - not enough to document that student intends to be home schooled

• Proper Documentation• Transcript request

• Verification by the superintendent or authorized campus/district administrator of the receiving district

• Verification by the parent/guardian or qualified studentCopyright © Texas Education Agency 2014. All rights reserved.

DVM LeaversIndicator 5-Use of Certain Leaver Reason

Dropout Codes

• Code 88 – Student was ordered by a court to attend a GED program and has not earned a GED certificate

• Students in this category used to be coded 98 – drop-out

• Required Documentation• Copy of court order stating that the student has been ordered to attend a

high school equivalency or GED program


Leavers - Documentation Requirements and Additional Resources

• Appendix D of the 2012-2013 PEIMS Data Standards provides definitions and specific guidelines on acceptable documentation for each of the leaver reason codes.

• PBM contacts at each ESC are available to provide districts with technical assistance concerning the 2013 leaver records data validation indicators (See Appendix C of Leaver PBM Manual).

• The PEIMS Data Standards, which describe the PEIMS data reporting requirements and provide descriptions of data elements and the codes used to report them.


Leavers - Documentation Requirements and Additional Resources

• Five PEIMS EDIT+ reports in particular that districts may find helpful as part of a local review of leaver coding. • PRF8D002: School Leaver Roster

• PRF8D003: School Leaver Summary

• PRF8D004: Non-Dropout Non-Graduate Leaver Roster

• PRF6D002: Dropout Roster

• PRF0B032: Presumed Underreported Students ListCopyright © Texas Education Agency 2014. All rights reserved.

DVM – Leavers Next Steps

• Use Appendix D for documentation requirements

• Written Procedures

• Checks and balance verification system

• Training• Administrative staff

• Office staff

DVM-Discipline Legal

• TEC Chapter 7.028• Allows TEA to monitor data integrity for the purposes of

• The Public Education Information Management System (PEIMS) and Accountability under TEC Chapter 39

• TEC Chapter 37.008• Requires an electronic evaluation of discipline data

• TEC Chapter 39.057• Authorizes special accreditation investigations when extraordinary numbers of

student placements in disciplinary alternative education programs, other than placements under Section 37.006 and 37.007 are determined


List of 2013 Discipline Data Validation Indicators

1. Length of Out-of-School Suspension

2. Length of In-School Suspension (Report Only)

3. Unauthorized Expulsion: Students Age 10 and Older

4. Unauthorized Expulsion: Students under Age 10

5. Unauthorized DAEP Placement: Students under Age 6

6. High Number of Discretionary DAEP Placements

7. African American (Not Hispanic/Latino) Discretionary DAEP Placements

8. Hispanic Discretionary DAEP Placements

9. No Mandatory Expellable Incidents Reported for Multiple Years

Discipline – 2013 DVM Data Sources

• Current data validation is for school year 2012-2013 discipline data, which was submitted in June 2013.

• Indicator 9 also includes discipline data from the 2010-2011 and 2011-2012 school years.


DVM Discipline - Indicator 1Length of Out of School (OSS) Suspension

• TEC Chapter 37.005 regulates that no out-of-school suspension can last longer than 3 days

• A partial day counts as a day

• Check appropriate coding• OSS is 05 and 25

• Was it OSS or in-school suspension (06 and 26) or a DAEP placement


DVM Discipline – Indicator 3Unauthorized Expulsion – Age 10 and Older

• A reason code of 21 – violation of student code of conduct is not an allowable expulsion offense

• Incorrect action codes entered• The disciplinary action was actually a DAEP placement, but was coded as an

expulsion to DAEP (codes 03 and/or 04)

• Documentation reviewed• Referrals

• Student code of conduct

• Due process documentation


DVM Discipline - Indicator 6High Number of Discretionary DAEP Placements

• Action codes monitored 07, 08, 10

• For reason code 21 – violation of student code of conduct- to be appropriate, DAEP has to be listed in published district student code of conduct as applicable consequence for specific incident listed on referral

• Appropriate conference has to be held according to TEC 37.009


Due Process ConferencesTEC 37.009

DAEP

• Student given written or oral notice of reasons for removal

• Explanation for basis of removal

• Opportunity to respond to the reasons

Expulsion

• Student given written notice of reasons for removal

• Explanation for basis of removal

• Opportunity to respond to the reasons

Due Process Conference Documentation

• Notice of conference

• Date, time, and who was in attendance

• Content of conversation

• Final decision of placement, including reason and length

• Ensure students receiving special education services have appropriate ARD committee meetings prior to final placement decisions


DVM Discipline – Next Steps

• Written policies and procedures


• Training • Administrative staff

• Staff entering discipline codes

• Review/revise student code of conduct

DVM Student Assessment - Legal

• TEC §39.057• Authorizes special accreditation investigations when excessive

numbers of absences of students eligible to be tested on state assessment instruments are determined.

• TEC §7.028• Allows TEA to monitor data integrity for the purposes of

• The Public Education Information Management System (PEIMS) and

• Accountability under TEC Chapter 39


DVM – Student Assessment Indicators

1. STAAR 3-8 Absent Rate Mathematics*

2. STAAR 3-8 Absent Rate Reading*

3. STAAR 3-8 Absent Rate Science*

4. STAAR 3-8 Absent Rate Social Studies*

5. STAAR 3-8 Absent Rate Writing*

* Indicator is calculated for 11 student groups


6. STAAR 3-8 Absent Other Mathematics*

7. STAAR 3-8 Absent Other Reading*

8. STAAR 3-8 Absent Other Science*

9. STAAR 3-8 Absent Other Social Studies*

10.STAAR 3-8 Absent Other Writing*

* Indicator is calculated for 11 student groups


11.TELPAS Reading Absent Rate

12.TELPAS Reading Other Rate

13.STAAR End-of-Course (EOC) Test Participation Rate*

14.Discrepancy between PEIMS** Career and Technical Education (CTE) Status and STAAR EOC Answer Documents

* Calculated for 6 subject area tests

**Public Education Information Management System

Student Assessment – 2013 DVM Data Sources

• Student 3-8 STAAR and EOC answer documents (latest spring 2013)

• Indicator 13 (EOC Test Participation Rate)• EOC answer documents

• Course completion data summer submission

• Indicator 14 (Discrepancy between PEIMS CTE status and STAAR EOC answer documents

• EOC answer documents

• Fall PEIMS snapshot submission (2012)


Student Assessment – Indicator 6STAAR 3-8 Other Rate Mathematics

• Students testing above grade level• The “O” code goes on the enrolled grade level answer

document

• The actual test taken document is marked “S” for score

• Possible next steps• DCCM* coding reference

• System for tracking students issued an “O” score code

*District Campus Coordinator ManualCopyright © Texas Education Agency 2014. All rights reserved.

Student Assessment – Indicator 13EOC Test Participation Rate

• Percentage of students who completed a course for which an EOC assessment is required (according to PEIMS record 415) that are• Missing an EOC answer document,

• Coded absent on an EOC answer document, or

• Coded other on an EOC answer document

• It is the “missing” component that is causing districts to trigger this indicator


Student Assessment – Indicator 13EOC Test Participation Rate

• Possible causes• TAKS students

• PEIMS course completion submission (415 record) not correct

• Possible next steps• Verify PEIMS 415 record before submission

• System to verify all required students are scheduled for testing


Student Assessment – Indicator 14Discrepancy Between PEIMS and Answer Document Coding for

CTE

• Validates students with a CTE course code of 2 or 3 on fall PEIMS snapshot to students with CTE course code of 2 or 3 on EOC answer documents

• Two possible causes

• Not clearly defining CTE programs in fall

• Not following answer document precode verification protocols

• Possible next steps

• Have system for verifying precode data



Today’s Key Take Aways

• Establish written district-wide processes and procedures

• Develop and follow through on annual training

• Design second verification before PEIMS submissions


2014 DVM – Tentative Release Dates

• Leavers• October 31

• Discipline• November 21

• State Assessment• December 19


Resources

• Leaver DVM Manual

• Discipline DVM Manual

• Student Assessment DVM Manual

• Appendix D – Leaver codes and required documentation

• Appendix E – Discipline Reason Codes and Action Codes

• District and Campus Coordinator Manual


http://www.tea.state.tx.us/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=25769807419&libID=25769807422







http://www.tea.state.tx.us/student.assessment/manuals/dccm/

Questions

Contact Information

Division of Program Monitoring and Interventions

Phone: (512) 463-5226

Email: [email protected]


mailto:[email protected]

• Copyright © Notice The materials are copyrighted © and trademarked ™ as the property of the Texas Education Agency (TEA) and may not be reproduced without the express written permission of TEA, except under the following conditions:

1. Texas public school districts, charter schools, and Education Service Centers may reproduce and use copies of the Materials and Related Materials for the districts’ and schools’ educational use without obtaining permission from TEA.

2. Residents of the state of Texas may reproduce and use copies of the Materials and Related Materials for individual personal use only without obtaining written permission of TEA.

3. Any portion reproduced must be reproduced in its entirety and remain unedited, unaltered and unchanged in any way.

4. No monetary charge can be made for the reproduced materials or any document containing them; however, a reasonable charge to cover only the cost of reproduction and distribution may be charged.

• Private entities or persons located in Texas that are not Texas public school districts, Texas Education Service Centers, or Texas charter schools or any entity, whether public or private, educational or non-educational, located outside the state of Texas MUST obtain written approval from TEA and will be required to enter into a license agreement that may involve the payment of a licensing fee or a royalty.

• For information contact: Office of Copyrights, Trademarks, License Agreements, and Royalties, Texas Education Agency, 1701 N. Congress Ave., Austin, TX 78701-1494; phone 512-463-9270 or 512-463-9713; email: [email protected]
