IDEA Part C and Part B 619 State Data Systems: Current … · IDEA Part C and Part B 619 State Data Systems: Current Status and Future Priorities September 2013 Taletha Derrington

IDEA Part C and Part B 619 State Data Systems: Current Status and Future Priorities

September 2013

Taletha Derrington Donna Spiker Kathleen Hebbeler SRI International

Martha Diefendorf Frank Porter Graham Child Development Institute (FPG)

The contents of this report were developed under a grant from the U.S. Department of Education, #H373Z120002. However, those contents do not necessarily represent the policy of the U.S. Department of Education, and you should not assume endorsement by the Federal Government. Project Officers, Meredith Miceli and Richelle Davis.

September 2013

Suggested citation: Derrington, T., Spiker, D., Hebbeler, K., & Diefendorf, M. (2013). IDEA Part C and Part B 619 state data systems: Current status and future priorities. Menlo Park, CA: SRI International.

Acknowledgments This report is based on the information provided by state Part C and Part B 619

coordinators, data managers, and other state staff. We are grateful to them for their willingness to gather and share this information and their time in completing the needs assessment. The high response rate is testimony to their commitment to IDEA early childhood programs in general and data systems in particular.

We want to thank the members of the Infant Toddlers Coordinators Association (ITCA) for sharing the data from their recently completely survey on data systems. We received invaluable guidance and instrumental assistance during the development of the survey from Maureen Greer (ITCA); Carlise King Elizabeth Groginsky, and Fran Kipnis (Early Childhood Data Collaborative); and from the following former Part C and Part B 619 state staff: Sue Brown, Kim Carlson, Debbie Cate, Jim Lesko, Anne Lucas, and Carol Noddings Eichinger. Many members of the Early Childhood Technical Assistance Center (ECTA) helped us achieve our high response rate through reminders to state representatives to complete the survey. Finally, we wish to thank the following Part C and Part B 619 state staff who reviewed drafts of the report: Lisa Backer, Ron Benham, Bonnie Dye, Patti Fougere, and Christine Payne Robin.

We also want to acknowledge the contribution of the many DaSy Center staff members and consultants who worked on the survey development, dissemination, analysis, and report preparation: Siobhan Colgan, Kathi Gillaspy, Lynne Kahn, Mary Peters, and Kellen Reid (Frank Porter Graham Child Development Institute); Missy Cochenour and Denise Mauzy (Applied Engineering and Management); Haidee Bernstein, Danielle Crain, and Joy Markowitz (Westat); Amy Nicholas (Center for Technology in Education at Johns Hopkins University); Bruce Bull, Jamie Kilpatrick, Robin Nelson, and Sharon Walsh (consultants to the DaSy Center); and Mimi Campbell, Jessica Gonzalez, Roxanne Jones, Klaus Krause, Mary McCracken, Ron Orpitelli, Suzanne Raber, Nai Saetern, and Cornelia Taylor (SRI International).

Contents Executive Summary ................................................................................................................ vii

Introduction ............................................................................................................................. 1

Findings .................................................................................................................................. 3

State of the States’ IDEA Early Childhood Data Systems .................................................... 3

State Data Systems and Data Elements .......................................................................... 3

Statewide Data Systems with Identifiable Data............................................................. 3

Child-Level Data Systems: Child and Family Data Elements........................................ 4

Child-Level Data Systems: Data Elements about Services........................................... 5

Program-Level Data Systems: Data Elements.............................................................. 7

Workforce Data Systems: Data Elements..................................................................... 8

Linking Data within Part C and within Part B 619 Data Systems................................... 10

Use of Unique Identifiers .............................................................................................. 11

Linkages between Different State Data Systems ............................................................. 13

Linkages between Part C and Part B 619 Data Systems .............................................. 13

Linkages with K-12 Education Data Systems ............................................................... 13

Linkages with Other Early Childhood, Social Service, and Health Data Systems.......... 14

Data System Administration and Use............................................................................... 16

Data Governance ......................................................................................................... 16

Managing Data Quality ................................................................................................. 17

Access to Data ............................................................................................................. 17

Using Data ................................................................................................................... 19

Knowledge and Use of Common Education Data Standards........................................ 20

State Priorities and Needs for Technical Assistance for IDEA Data Systems....................... 21

Types and Quality of Data Elements................................................................................ 21

Linkages within and between Different State Data Systems............................................. 21

Data System Administration and Use............................................................................... 22

Technical Aspects of State Data Systems........................................................................ 22

Conclusion .............................................................................................................................. 24

v

List of Tables 1. Connections between Part C/Part B 619 Child Data and Other State Programs’

Child Data ................................................................................................................... 15

List of Figures 1. States that Have Statewide Data Systems with Different Kinds of Identifiable Data ....... 4 2. Data Elements Contained in Child-Level Data Systems: Child, Family, and Eligibility .... 5 3. Data Elements Contained in Child-Level Data Systems: Services ................................. 6 4. Data Elements Contained in Program-Level Data Systems ........................................... 7 5. Workforce Data Systems: Data Elements for Part C ...................................................... 8 6. Workforce Data Systems: Data Elements for Part B 619 ............................................... 9 7. Linkages within Child, Program, and Workforce State Data Systems............................. 10 8. Linkages between Child, Program, and Workforce Data Systems ................................. 11 9. Use of Unique Identifiers within State Data Systems ..................................................... 12 10. Use of Common Unique Identifiers across Part C and Part B 619 State Data Systems . 12 11. Linkages between Part C and Part B 619 State Data Systems ...................................... 13 12. Linkages between Part C/Part B 619 and K-12 Special and General Education

Data Systems.............................................................................................................. 14 13. Other Programs Included in Responsibility of the Data Governance Body with

Part C/Part B 619 ........................................................................................................ 16 14. Tools and Procedures Used to Verify the Reliability and Validity of Part C/

Part B 619 Data........................................................................................................... 17 15. Access to Part C/Part B 619 Child-Level Data, by Level of Data and Type of Staff ........ 18 16. State Use of Data about Children Receiving Part C/Part B 619 Services....................... 19 17. Part C/Part B 619 Coordinators’ Knowledge of CEDS and Use of CEDS Tools ............. 20 18. State Priorities and TA Needs for IDEA Early Childhood Data Systems......................... 23

vi

IDEA PART C AND PART B 619 STATE DATA SYSTEMS: CURRENT STATUS AND FUTURE PRIORITIES

Executive Summary The Center for IDEA Early Childhood Data Systems, the DaSy Center, was funded by the Office of

Special Education Programs (OSEP) to provide technical assistance (TA) to states to support them in developing or enhancing Part C and Part B Section 619 (Part B 619) data systems. TA also will assist Part C and Part B 619 state agencies in participating in the development of integrated early childhood data systems and longitudinal data systems in their states. To inform the DaSy Center’s work, the Center collected information about the current status of Part C and Part B 619 state data systems, priorities for improvement, and areas where the states would like TA. State Part C and Part B 619 coordinators, their respective data managers, and other state staff completed an online survey over the summer of 2013. Responses were obtained from 94% of the 50 states, DC, and Puerto Rico for Part C and from 96% for Part B 619. This report summarizes what was learned about the current status of Part C and Part B 619 data systems and where states are in moving to improve their data systems.

All Part C and Part B 619 programs that provided data indicated that they have state child-level data systems with personally identifiable information. Variation was found across states both within and between Part C and Part B 619 data systems with regard to the specific data elements contained in states’ child-level data systems. For example, all systems contain data elements on child demographics, but only 83% of Part C data systems and 56% of Part B 619 systems contain data on services received. One challenge for more than one-third of state Part B 619 programs in making full use of the data elements in child-level data systems is that not all these data elements reside in the same data system or have ever been linked. Only 8% of the Part C programs face this challenge.

Compared with child-level data systems, data systems with workforce data are less frequent (65% of states for Part C and 83% of states for Part B 619), and even fewer states have data systems with program/school data (29% of states for Part C and 40% of states for Part B 619). To be able to answer key questions about factors that affect results, states will need information about the workforce and the key features of children’s programs and be able to link these data elements to the child-level data, which less than half of the states can do now.

Addressing critical policy and programmatic questions about outcomes and their contributing factors for young children with disabilities also requires linking Part C or Part B 619 data elements with data outside of these programs. One of the most fundamental linkages for IDEA’s early childhood programs is the linkage between the Part C and Part B 619 child-level data systems; 29% of states reported that these data reside in the same data system or have been linked. Part C data can be linked to general education K-12 data in 14% of the states, while 79% of states reported Part B 619 to K-12 linkages, a difference that is not surprising given Part B’s connection to departments of education. A minority of Part C coordinators reported that child-level data have been linked to other early childhood, social service, or health data systems, the highest being Medicaid/State Children’s Health Insurance Program (42%) and Early Hearing Detection and Intervention (37%). The most frequent linkages for Part B 619 are with state pre-K (46%) and Head Start (22%) programs.

There are both similarities and differences across states in data system governance, data quality verification, who is granted access to the data, and how they have used their data.

Few Part C state programs have data governance bodies, whereas many Part B 619 state programs do. For Part B 619, more than half have data governance bodies whose responsibility includes K-12 education data along with Part B 619 data. Part C and Part B 619 programs reported using a variety of tools and procedures to check data quality, including data audits built into the data system and user manuals.

vii


States reported using their data for a variety of purposes, the most commonly reported uses being checking data quality; program improvement; identifying professional development and technical assistance needs; and reporting to the public, the governor, and the legislature.

Part C and Part B 619 reported a similar pattern of access to aggregate and individual child-level data, with more state coordinators having access than regional or district personnel, even fewer states providing access to school or program personnel, and fewer yet to teachers and service providers. Using data to inform decision-making at any level requires access to information, so it is surprising that 19% of Part B 619 coordinators do not have access to aggregated child-level data on the programs they oversee.

Much was learned about where states are in building high-quality data systems; equally interesting is where states are trying to go. For Part C, the most frequent priorities for improvement were data use; data quality, verification, and audit systems; increasing the type or quality of child and family outcome data; and data sharing permissions and privacy. The last three areas were identified most frequently for DaSy Center technical assistance, along with increasing the type or quality of finance data. For Part B 619, the priorities reported most frequently were data use; data quality, verification, and audit systems; child and family outcome data; and increasing the type or quality of data for the Annual Performance Report (APR) indicators or 618 data. Technical assistance needs identified most frequently were child and family outcomes, APR or 618 data, linkages between Part C and Part B 619 data, and data use. For Part C and Part B 619, most of the areas asked about were identified as TA needs by one-third or more of the states. Even the areas least frequently indicated as TA needs were still identified as needs by more than one in five states.

As momentum builds across the country to develop coordinated services for young children and to integrate the data across the various programs serving these children, the Part C and Part B 619 programs will have much to contribute to the design and implementation of integrated data systems in their states. The Part C and Part B 619 programs in many states have made substantial strides in building data systems that can be used for numerous purposes. Nevertheless, much work remains to be done to continue increasing the capacity of the Part C and Part B 619 programs to improve the quality of the data and to use data to examine results and drive meaningful program improvement.

viii


Introduction Across education and human services, there is increasing recognition of the importance of program

accountability and the centrality of good data to meaningful accountability. The United States Department of Education, Office of Special Education Programs (OSEP), oversees state implementation of the Individuals with Disabilities Education Act (IDEA), including IDEA’s two early childhood programs: Part C Infants and Toddlers With Disabilities and Part B 619 Preschool Special Education for 3- to 5-year-olds. For many years, states have been required to submit data to OSEP under Section 618 of IDEA. In addition, OSEP monitors state implementation of IDEA through State Performance Plans (SPP) and Annual Performance Reports (APR). OSEP is currently in the process of redesigning its approach to federal monitoring in accordance with the 2004 reauthorization of IDEA, which places a strong emphasis on results. OSEP’s new approach, grounded in Results-Driven Accountability (RDA), is intended to provide both the states and OSEP with the tools to monitor outcomes for children and youth with disabilities and their families in addition to compliance with the procedural requirements of the statute (see http://www2.ed.gov/about/offices/list/osers/osep/rda/index.html).

IDEA’s emphasis on results as operationalized through OSEP’s new RDA efforts is occurring simultaneously with two other developments in data systems across the nation that could enhance both the quality and power of the early childhood data exponentially. One of these is increasing recognition of the need for integrated data systems that include information across all programs serving young children in a state. The other is the move to build K-12 state longitudinal data systems that include early childhood (EC) data. The converging of these three forces—that is, the emphasis on monitoring results under IDEA, the recognition of the need to build integrated EC data systems, and the focus on linking data from EC programs to state longitudinal data systems—constitute an unprecedented opportunity for Part C and Part B 619 state agencies to improve the quality of the data submitted to OSEP and more effectively use data for program improvement.

The Center for IDEA Early Childhood Data Systems, the DaSy Center, was funded by OSEP to provide technical assistance (TA) to states to support them in developing or enhancing Part C and Part B Section 619 data systems in coordination with other statewide efforts to build integrated early childhood data systems.

Improvements in states’ IDEA early childhood data systems will increase state capacity to collect, analyze, and report data that are required under IDEA for 618 and APR/SPP reporting. Improved data systems also will provide information states need to plan for program improvements and to demonstrate that public investments are making meaningful differences for young children with disabilities and their families.

To inform the DaSy Center’s work with states, the Center collected information about the current status of Part C and Part B 619 state data systems. Information also was collected about state priorities regarding improvements and areas where the states would like TA. This report summarizes what was learned about the current status of Part C and Part B 619 data systems.

An online survey was developed in coordination with the IDEA Infant and Toddler Coordinators Association (ITCA) and the Early Childhood Data Collaborative (ECDC). Surveys were sent to Part C and Section 619 coordinators in each state, the District of Columbia, and the territories in May 2013, with survey completion occurring between May and August 2013. Strategies used to increase responses included sending an introductory e-mail to let state representatives know that the survey would be coming, first distribution via e-mail, and formal and informal follow-up via e-mail and phone, with assistance from the Early Childhood Technical Assistance (ECTA) Center.

The coordinators were invited to complete the survey with Part C/Part B 619 data managers or other appropriate individuals. Slightly different surveys were sent for Part C and Part B 619. Only the results of

1

http://www2.ed.gov/about/offices/list/osers/osep/rda/index.html


surveys from the 50 states, District of Columbia, and Puerto Rico (N=52) are included in this report (hereafter referred to as states).

The response rate for Part C was 94% and for Part B 619 was 96%. For Part C, the coordinators contributed to survey completion in 77% of states, with data managers contributing in 44% of states and other state staff contributing in 8%. For Part B 619, the coordinators contributed to survey completion in 79% of states, with data managers contributing in 46% of states and other state staff contributing in 29%. In the interest of brevity, the person who completed the survey will be referred to as the state coordinator. Additional details on the methods will be provided in a separate Technical Report, which will be disseminated on the Center’s website (http://www.dasycenter.org).

The remainder of this report presents the findings in two sections: the state of the states’ IDEA data systems, and state priorities and needs for TA. The current status of state data systems is described with regard to:

State data systems and data elements Linkages between different state data systems Data system administration and use

A concluding section summarizes the findings and describes implications for IDEA data systems and DaSy technical assistance.

2

http://www.dasycenter.org/


Findings

State of the States’ IDEA Early Childhood Data Systems

State Data Systems and Data Elements States were asked whether they have state-level data systems that contain identifiable data about

children, programs, and workforce members. For Part C, program was defined as the “administrative entity which receives referrals, provides evaluation services, provides service coordination, and/or provides direct services to children and families.” For Part B 619, program/school was defined as the “specific location where the child receives preschool special education services through Part B 619, including child care, Head Start, Title I, etc.”

The questions on workforce data systems differed for the two programs. For Part C, one question asked about data on early intervention personnel (defined as those individuals who provide Part C services to children and families, such as a developmental specialist, speech-language pathologist, physical therapist, or service coordinator). The Part B 619 survey had separate questions about data on preschool special education teachers, related services personnel (e.g., physical therapists, school psychologists), and general education teachers.

Throughout the report, percentages are presented for all states, DC, and Puerto Rico (N=52).

Statewide Data Systems with Identifiable Data

For both Part C and Part B 619, all responding states reported that they have a child-level state data system that contains personally identifiable information for all or nearly all individual children receiving services (Figure 1). The remaining states did not provide any information. Less than half of states have data systems containing program-level data (29% for Part C, 40% for Part B 619). Nearly two-thirds of states have state data systems containing Part C workforce data (65%). A majority of states have Part B 619 data systems containing data about preschool special education teachers (83%), related services personnel (71%), and general education personnel who work with children receiving preschool special education (71%).

The next section describes the types of data elements that states have in their child-level data systems.

3


Figure 1. States that Have Statewide Data Systems with Different Kinds of Identifiable Data

Child-Level Data Systems: Child and Family Data Elements

States reported having a variety of data elements about children and families in state-level child data systems (Figure 2).

For both Part C and Part B 619 data systems, almost all states (more than 90%) have child-level data on the child’s name or identification number, birth date, and demographic information (e.g., gender, race/ethnicity). Child disability category is contained in most Part C and almost all Part B 619 data systems. Other types of child disability information (e.g., severity, diagnostic codes) are contained in just over half of Part C data systems (56%) and in about one-fifth of Part B 619 data systems (21%). Family demographic information (e.g., race/ethnicity, education level, household income) is contained in a majority of Part C and about half of Part B 619 data systems (83% and 58%). Child assessment scores are contained in almost three-fourths of Part C data systems (71%) and about half of Part B 619 data systems (48%). A majority of Part C and Part B 619 data systems contain data about child outcomes at entry to and exit from the program. Almost all Part C data systems contain information about referral evaluation and assessments and children’s eligibility status. Almost all Part B 619 data systems contain information on child’s eligibility status, and most contain information about evaluation and assessments; fewer contain referral information.

4


Figure 2. Data Elements Contained in Child-Level Data Systems: Child, Family, and Eligibility

Child-Level Data Systems: Data Elements about Services

Most Part C and Part B 619 data systems contain a variety of data elements about services (Figure 3).

A majority of both Part C and Part B 619 data systems contain information about children’s entry into services, such as initial service planning meetings, parental consent for evaluation and/or services, initiation of services, and service settings. Services authorized are contained in almost all Part C data systems but in only about half of Part B 619 data systems. Most Part C data systems contain information about service coordinators and other service providers working with the child, the EI program providing services, and types of services received. More than three-fourths of Part B 619 data systems contain information about the school or program serving the children, but fewer contain information about the special education teacher or the types of services received.

5


Only about one-fourth of Part C data systems contain information about attendance, which may reflect the fact that EI services are typically delivered in the home rather than in center-based programs, but nearly half of Part B 619 data systems contain attendance data, reflecting greater use of center-based programs for preschool. Almost all Part C data systems and a majority of Part B 619 data systems contain information about the reason the child exited the program, and a majority of both data systems also contain information about when the child received the last service. More than half of both Part C and Part B 619 data systems contain information about referral from Part C to Part B 619; the date of the Part C to Part B 619 transition conference is contained in almost all Part C data systems and in less than half of Part B 619 data systems.

Figure 3. Data Elements Contained in Child-Level Data Systems: Services

6


Program-Level Data Systems: Data Elements

Far fewer states have Part C and Part B 619 data systems containing program data (29% and 40%, respectively, shown in Figure 1), compared with child-level data. Figure 4 presents the types of data contained in state Part C and Part B 619 program-level data systems. Percentages are for all 52 states and indicate those that do have the program data element.1

About one-fourth of state Part C data systems contain data about program structure (e.g., service model, number of regular or contracted staff, administrative agency), which represents all of the states that have program data systems (29%, shown in Figure 1). None of the Part C program data systems contain data about program costs or quality (e.g., program quality measures such as the Infant/Toddler Environment Rating Scale, Quality Rating and Improvement System [QRIS] rating, accreditation). About one-third of Part B 619 data systems contain information about program structure and whether children with IEPs are served in a program or classroom with children without IEPs (shown as Inclusion in Figure 4), which represents most of the states that have Part B 619 data systems with any program data (40%, shown in Figure 1). Only a few Part B 619 data systems contain information about program quality (e.g., Early Childhood Environment Rating Scale, QRIS rating, accreditation) or characteristics of the programming (e.g., curriculum).

Figure 4. Data Elements Contained in Program-Level Data Systems

1 Different types of program/school data elements were listed on the Part C and Part B 619 surveys.

7


Workforce Data Systems: Data Elements

As discussed earlier, most state Part C and Part B 619 programs have data systems with information about their workforce that contain a variety of workforce data elements (Figures 5 and 6).

About half of state Part C data systems Figure 5. Workforce Data Systems: Data contain information about early Elements for Part C intervention personnel such as employment status (e.g., program where employed, position title, years of employment), education (e.g., degree, field of study), and licenses or certification (Figure 5). Only about one-third of state Part C data systems contain information about Part C personnel’s demographic characteristics and professional development participation, and only a few contain information about wages (Figure 5).

8


Two-thirds or more of Part B 619 data Figure 6. Workforce Data Systems: Data systems contain information about special Elements for Part B 619 education teachers such as demographic characteristics, employment information, education, and licenses or certifications (Figure 6). Similar to data for Part C providers, only about one-third of Part B 619 data systems contain information about special education teachers’ professional development. Almost half contain information about their wages (Figure 6). Part B 619 data systems also contain data about related services personnel and general education teachers. The percentages of states with various types of data for these other personnel are similar to the percentages for special education teachers but slightly lower for some types of information (Figure 6).

9


Linking Data within Part C and within Part B 619 Data Systems

Data on children and families, programs and schools, and the workforce may reside in one or more data systems, and the state may or may not be able to link data across different data systems. As defined on the survey, linking refers to the process of joining or connecting records about one individual or entity in one data system or dataset with those in another data system or dataset. Records can be linked through a common identifier (see below) or by some other method (e.g., matching algorithms using multiple data elements).

For about half of Part C data systems, the child and family and service data elements are in one data system and for almost one-third of states they are in separate data systems that have been linked at least once (Figure 7). For more than one-fourth of state Part B 619 systems, the child and family and service data elements are in one data system, and for more than one-third of states, they are in separate data systems that have been linked at least once (Figure 7). Few states collect Part C or Part B 619 program data, and only 8% of Part C and 10% of Part B 619 systems have all their program data in the same system. For an additional 12% of states for both Part C and Part B 619, program data elements are in separate data systems that have been linked at least once (Figure 7). For one-third of states, Part C and Part B 619 workforce data are contained within one data system; an additional 25% of states have Part B 619 workforce data in more than one system but have linked their data, compared with only 12% of states for Part C (Figure 7).

Figure 7. Linkages within Child, Program, and Workforce State Data Systems

Linkages between child, program, and/or workforce data systems are not common, largely because states do not have program data systems (Figure 8).

Only 19% of states have linked child and program data systems for Part C, and 31% have such linkages for Part B 619. In addition, for Part B 619, 19% of states have linked child and classroom data. Child-to-workforce linkages are more common for Part C (40%) than for Part B 619 (31%), and 17% of both Part C and Part B 619 programs have workforce-to-program linkages.

10


Figure 8. Linkages between Child, Program, and Workforce Data Systems

Use of Unique Identifiers

A unique identifier is assigned to one individual or entity, is associated with all the data for that individual or entity in a data system or dataset, and remains the same over time; in other words, it “follows” that individual or entity. Unique identifiers can be used to link data within a data system or dataset or across more than one data system or dataset. Linking data is easier to do if the same unique identifiers are used in different data systems or datasets.

Almost all states use unique child identifiers in Part C and in Part B 619 data systems (Figure 9). A majority of Part C data systems use unique identifiers for programs, and almost all Part B 619 data systems use unique identifiers for programs and schools and for school districts (Figure 9). Unique identifiers for workforce data are contained in more than half of Part C and two-thirds of Part B 619 data systems (Figure 9).

Part C and Part B 619 coordinators also were asked whether the same unique child, school/program, and workforce identifiers are used across Part C and Part B 619 data systems. Given that 12 states have Education as the Part C lead or co-lead agency, those states are more likely to have common identifiers across Part C and Part B 619. Information from both surveys was analyzed together (Figure 10).

Most states do not have common unique identifiers that are used across both Part C and Part B 619, with only 21% of states reporting that there are common unique child identifiers used across both Part C and Part B 619 data systems.2

Only a small number of states use common unique school or program identifiers across both Part C and Part B 619 (12%). Similarly, very few states reported the use of common workforce unique identifiers across Part C and Part B 619 data systems (6%).

2 In some cases, the responses for Part C and Part B 619 were discrepant so these were noted as disagreements in Figure 10.

11


Figure 9. Use of Unique Identifiers within State Data Systems

Figure 10. Use of Common Unique Identifiers across Part C and Part B 619 State Data Systems

12


Linkages between Different State Data Systems Different early childhood programs generally have their own data systems, and the state may or may

not be able to link data contained in these different data systems.

Linkages between Part C and Part B 619 Data Systems

One critical linkage for IDEA data systems is a linkage between Part C and Part B 619 data. Being able to link Part C and Part B 619 data allows a state to address many important program and policy issues, such as planning and implementing smooth transitions and examining outcomes for former Part C recipients who transition into Part B. State representatives from both programs were asked about whether linkages exist between their child data system and the other program’s child data system. Information from both surveys was analyzed together.

Almost one-third of states reported that their state’s Part C and Part B 619 data systems were in the same system or have been linked at least once, while nearly half reported that the two data systems have not been linked. Results for 23% of states were ambiguous because the answers given by Part C and Part B 619 coordinators did not agree.

Figure 11. Linkages between Part C and Part B 619 State Data Systems

Linkages with K-12 Education Data Systems

Linkages of data from Part C or Part B 619 data systems to K-12 special and general education data systems allow states to follow children’s progress longitudinally from early childhood program participation into their later school career and address questions about long-term outcomes of Part C and Part B 619 program participants.

Less than half of states have connected Part C child data to K-12 special education data: data are in the same data system in 12% of the states, and data are in separate systems that have been linked in 29% of the states (Figure 12). Most states have linked Part B 619 child data to K-12 special education data (85%, with 73% in the same data system and an additional 14% that have linked the data at least once) (Figure 12).

13


Only 14% of states have connected Part C child data with K-12 general education data (4% in the same system, 10% in separate systems have been linked). In contrast, more than three-fourths of states have connected Part B 619 child data to K-12 general education data (46% in the same data system, 33% in separate data systems that have been linked).

Figure 12. Linkages between Part C/Part B 619 and K-12 Special and General Education Data Systems

Linkages with Other Early Childhood, Social Service, and Health Data Systems

Many children participating in Part C or Part B 619 also participate in other early childhood (e.g., Head Start, state pre-K) and social service (e.g., child welfare, income assistance) programs, and all states collect at least some types of health data on children (e.g., vital records, hospital data). Linkages to early childhood, social service, and health data systems allow states to track children’s participation, and their outcomes as a result of participation, in these different programs. For example, linkages provide information on how many and which children are served in multiple EC programs, including whether children with disabilities are being served in settings with typically developing peers; improve coordination across programs; and allow for the examination of developmental progress and outcomes following services in different types of settings. Linkages to social services data could assist states in responding to federal requirements, such as facilitating referrals from child welfare to Part C. Linkages to vital statistics records, such as birth or death certificates, and health records, such as hospital discharge data, can be useful for states to be able to conduct population-based public health monitoring and research. Linkages to other types of health data, such as information about use of Medicaid or the State Children’s Health Insurance Program (SCHIP), the Early Hearing Detection and Intervention (EHDI) program (newborn hearing screening), nutrition programs such as WIC (the Special Supplemental Nutrition Program for Women, Infants, and Children) or SNAP (the Supplemental Nutrition Assistance Program), and other health insurance can allow states to learn about how many families served in Part C and Part B 619 programs are accessing and engaging in those services.

Table 1 shows the percentage of states that can connect Part C and Part B 619 data with these other data systems, either because they are in the same system or because they are in separate systems that have been linked at least once.

With a few exceptions, few states have linked Part C data to other data systems. • The most common connections for Part C are to health data such as Medicaid and SCHIP

(42%), probably due to needs for billing and reimbursement to pay for services, and to newborn hearing screening EDHI data (37%) for early identification and referral for hearing impairments. In addition, 13% of states have linked Part C data and state all-payer claims data systems, which include data for all insurance companies operating in each state.

14


• The most frequently reported linkage for Part C to social service data is to child welfare data, with 21% of states having linkages between Part C and child welfare data systems.

• About one-fifth of states have linkages between Part C and vital records and birth defects registries. Less than 10% of states have Part C linkages to other early childhood programs, with linkages to state pre-K program data being the most common (12%).

For Part B 619, linkages to other data systems are not common. Compared to Part C, more states have linkages to early childhood program data and fewer have linkages to health data. • Almost half of the states have linkages between Part B 619 data and state pre-K data, and

almost one-fourth of states have linkages to Head Start data. • Only 10% of states have linkages between Part B 619 and child welfare data. The most

common linkages for Part B 619 to social service data are to Temporary Assistance for Needy Families (TANF) and homeless services data (14% for each).

• Few states have Part B 619 linkages to any kind of health data (most less than 10%, with the highest for Medicaid/SCHIP in 12% of states).

Table 1. Connections between Part C/Part B 619 Child Data and Other State Programs’ Child Data

Part C Part B 619 Early childhood program data

State pre-K 12% 46% Head Start 6% 22% Early Head Start 2% 10% Child care 6% 8% Home visiting 8% 8%

Social service data Child welfare 21% 10% Foster care 12% 8% Temporary Assistance for Needy Families (TANF) 10% 14% Homeless services 6% 14%

Vital records and health data Medicaid/SCHIP 42% 12% EHDI 37% 8% Vital records 21% 0% Birth defects registry 21% 2% All-payer claims (insurance) 13% 0% WIC/SNAP 8% 6% Hospital 6% 2% Behavioral health 4% 2%

Note: Percentages are percent of states that reported the data are in the same system with Part C/Part B 619 or have been linked at least once. SCHIP = State Children’s Health Insurance Program; EHDI = Early Hearing Detection and Intervention; WIC/SNAP = Women, Infants, and Children/Supplemental Nutrition Assistance Program.

15


Data System Administration and Use Many states have been developing formal processes and procedures to ensure that data collected by

state agencies and programs are well managed, meet high standards for reliability and validity, and are used in effective ways by a variety of users to address their critical questions.

Data Governance

Part C and Part B 619 coordinators reported different experiences with regard to data governance bodies, defined in the survey as entities that establish policy and procedures for the overall management of the availability, usability, integrity, quality, and security of the data.

About two-thirds of states do not have a data governance body with responsibility for the Part C data system. Less than one-fifth of states have a Part C data governance body that also has responsibility for Part B 619 data, other early childhood program data, and K-12 special and general education data (Figure 13). In contrast, about three-fourths of states have a data governance body with responsibility for Part B 619 data system. About one-third of these data governance bodies also have responsibility for Part C data and other early childhood program data. For more than half of states, they also include K-12 special and general education data (Figure 13).

Figure 13. Other Programs Included in Responsibility of the Data Governance Body with Part C/Part B 619

16


Managing Data Quality

One critical aspect of managing data effectively requires having processes and procedures to check and verify that the data are accurate, complete, and internally consistent—that is, ways to ensure and check that data are trustworthy in their collection, entry into the data system, and consistency with other data in the data system.

Many states use various tools and procedures to ensure, check, and verify the quality of data in their Part C and Part B 619 data (Figure 14). • Most states use audits built into the data system for Part C and Part B 619 to verify the

reliability and validity of data (e.g., validity checks on dates, flagging missing data, or forced proper data entry format for a given data field) and user manuals.

• A majority of states for both Part C and Part B 619 use verification visits in which staff randomly check and verify child data, have data dictionaries, and have annual or regularly scheduled trainings related to the data.

• Comparison of paper forms to electronic data is a common data check used by a majority of Part C programs and more than half of Part B 619 programs.

Figure 14. Tools and Procedures Used to Verify the Reliability and Validity of Part C/Part B 619 Data

Access to Data

Data systems can be configured with access and security settings to give different types of individuals working in Part C and Part B 619 systems access to aggregated data (summaries of data for groups of children without individual identifying information included) and/or individual-level data (with individual identifying information included). Users across different levels of the Part C and Part B 619 systems may have different levels of access to the data, depending on their role and the kinds of data security policies and procedures the state has in place (e.g., a teacher may have access to individual child data only for children in his/her classroom, but also have access to aggregated data for his/her district or state). For data-driven decision-making to be most effective, it is critical that users across all levels of the service system have appropriate access to data.

17


State Part C and Part B 619 coordinators have the greatest access to aggregate data, followed by regional and school district staff and by school/program staff. Teachers and service providers least frequently have access to data (Figure 15). This pattern of access being greatest for Part C and Part B 619 coordinators and least for teachers and service providers is the same for access to individual data (Figure 15). Greater percentages of state Part C coordinators than of state Part B 619 coordinators have access to both aggregate and individual child data (Figure 15).

Figure 15. Access to Part C/Part B 619 Child-Level Data, by Level of Data and Type of Staff

18


Using Data

One of the ultimate goals of collecting data and having good state data systems is to have available high-quality information that can be used for different purposes.

For both Part C and Part B 619, states are using their data in many different ways (Figure 16). • Many states use their Part C and Part B 619 data to check or review data quality and identify

and plan for program improvement activities. • A majority of states also use their Part C and Part B 619 data to produce general reports for

the public or for the governor or state legislature (slightly less frequent for Part B 619). • For Part C, a majority of states use their data to identify systemic issues with service delivery

(e.g., underserved or overserved populations), to monitor program improvement efforts, or to examine program effectiveness. These uses are also common but less frequent for Part B 619.

• About three-fourths of states use their Part C and Part B 619 data to identify professional development or technical assistance needs.

• Part C data are used by more states than are Part B 619 data to make resource allocations.

Figure 16. State Use of Data about Children Receiving Part C/Part B 619 Services

19


Knowledge and Use of Common Education Data Standards

Common Education Data Standards (CEDS) is a national collaborative effort to develop voluntary, common data standards for a key set of education data elements to streamline the exchange, comparison, and understanding of data within and across data systems from different sectors (i.e., early learning through K-12 through postsecondary and workforce). Early childhood is one of the newer sectors being included in this national effort.

Part C and Part B 619 coordinators were asked whether they know about CEDS and, if so, whether they are using any of the CEDS tools, such as Align or Connect. Align is an online tool that states can use to see how well the data dictionaries for their data systems match the CEDS data elements. Connect is an online tool that states can use to determine whether their state data systems have the data elements to answer specific policy questions.

About two-thirds of Part C and Part B 619 coordinators know about CEDS (69% and 67%, respectively) (Figure 17). However, very few Part C coordinators use the CEDS tools (6%), and only about one-fourth of Part B 619 coordinators use them (23%) (Figure 17).

Figure 17. Part C/Part B 619 Coordinators’ Knowledge of CEDS and Use of CEDS Tools

20


State Priorities and Needs for Technical Assistance for IDEA Data Systems The purpose of the DaSy Center is to provide technical assistance to states to support their

development or enhancement of IDEA data systems. Therefore, in addition to learning about the status of states’ data systems, the survey asked about the kinds of data system issues and topics that Part C and Part B 619 state staff see as priorities for improvement in the next two years, as well as whether states would like the DaSy Center to provide them with TA to address those priorities (Figure 18).

Types and Quality of Data Elements Many states identified priorities related to increasing the types of data elements collected or the

quality of their Part C and Part B 619 data or data systems.

For both Part C and Part B 619, a majority of states indicated that improving data on child and family outcomes is a priority, with about two-thirds of states reporting that this is an area for which they would like to receive TA. Data for the Annual Performance Report indicators/Individuals with Disabilities Education Act Section 618 data also is a priority area for a majority of states for both Part C and Part B 619. Almost half of states would like TA for this area for Part C, and almost two-thirds of states for Part B 619. Improving data on children and families and on services also is a priority for a majority of states for both Part C and Part B 619, with more than one-third of Part C and Part B 619 wanting TA for these areas. About half or more of states identified improving workforce and program quality data as a priority for both Part C and Part B 619, and about one-third of states want TA for this area for both Part C and Part B 619. Finance and cost data is an area of priority for more states for Part C than for Part B 619, with more than half of states wanting TA for this area for Part C.

Linkages within and between Different State Data Systems Establishing or enhancing linkages within the data elements contained in Part C and Part B 619 data

systems, as well as linkages with other state data systems, is the hallmark of integrated data systems. States reported having a number of priorities and needs for TA in these areas.

Linkages within Part C and Part B 619 data systems are priorities for the majority of states for both Part C and Part B 619. More than half of states indicated a need for TA about improving linkages within their data systems for Part C and Part B 619. Developing linkages between Part C and Part B 619 also is a priority for a majority of both systems in states, with almost half of Part C and Part B 619 programs wanting TA in this area. Almost two-thirds of the states for Part C and Part B 619 indicated that linkages to other early childhood programs are a priority, but more Part C programs want TA for this area than do Part B 619 (46% versus 37%). Linkages to K-12 education data are more often a priority for Part B 619 (for a majority of states) than for Part C, while the reverse is true for linkages to social service or health data. Matching this pattern, linkages to K-12 data are more often identified as an area for TA for Part B 619 (for almost half of states) than for Part C (one-fourth of states), while more states indicated a need for TA related to linkages to social service/health data for Part C (half of states) than for Part B 619 (about one-fourth of states).

21


Data System Administration and Use Although a focus on the types and quality of data and linkages within and between systems is

important, development of a high-quality functional data system also requires attention to how the system is administered, including data system planning and management, as well as technical features of the data system design.

For a majority of states, data use (e.g., analyzing data and using data for program improvement) is the most frequent priority area for Part C and Part B 619, and more than half of Part C and Part B 619 programs want TA in this area. Data sharing permissions and/or privacy issues (e.g., confidentiality policies, data access decisions, security models, federal privacy laws) are a priority for more than four-fifths of states for Part C and for two-thirds of states for Part B 619. More states want TA for this area for Part C than for Part B 619, but this is still a topic for TA for many states (67% and 48%, respectively). A majority of both programs identified inclusion of Part C/Part B 619 representatives in broader statewide coordinated data system planning efforts and communicating with and engaging stakeholders as top priorities, with about half of the Part C and Part B 619 programs indicating that these are areas for which they would like to receive TA. Data system planning and management (e.g., memoranda of agreement/understanding, evaluation, strategic planning, setting priorities) is a priority area for two-thirds of states for Part C and three-fourths of states for Part B 619, with about one-third of Part C and Part B 619 programs wanting TA. Data governance (e.g., organizational structure, ownership, cross-system coordination) is a priority for more states for Part B 619 than for Part C, but for both Part C and Part B 619, about one-third of states would like TA for this area.

Technical Aspects of State Data Systems Attention to the technical aspects of data system design is important to the creation of a functional

and usable system. A data system needs to be designed and built to store, retrieve, integrate, and analyze data effectively and efficiently, to ensure high standards for the security and quality of the data in the system, and to provide meaningful data to those accessing it for a variety of uses in an efficient, timely, and accurate manner.

A majority of states reported that improvements in the area of data quality tools, including verification and audit systems, are a priority for both Part C and Part B 619, and about half of Part C and Part B 619 identified this as an area for which they would like to receive TA. Creating reports for different audiences also is a priority for a majority of states for both Part C and Part B 619, more so for Part C; more than one-third of states would like TA on this topic. Data collection and storage (e.g., warehousing, consolidation) is a priority for more than two-thirds of states for both Part C and Part B 619, and this is an area for TA for about one-third of states for Part C and one-fourth of states for Part B 619. More than half of Part C and Part B 619 programs identified a variety of technical aspects of data system design (e.g., data models, technical architecture, metadata, requirements) and data integration/exchange (e.g., data exchange, data matching, interoperability, unique IDs) as priorities. Both areas were identified for TA more often for Part C (about one-third of states) than for Part B 619 (about one-fourth of states). Implementing the use of unique identifiers is a priority for about half of states for Part B 619 and one-third of states for Part C, with a minority of states wanting TA on this topic. Similarly, although about half of states identified CEDS as a priority area for both Part C and Part B 619, a minority of states want TA for this topic.

22


Figure 18. State Priorities and TA Needs for IDEA Early Childhood Data Systems

23


Conclusion This report presents a comprehensive look at the status of Part C and Part B 619 state data systems,

including what data states are collecting, what data have been linked, how states are using data, and what their priorities are for the coming years regarding enhancing and developing their data systems. The information, which was provided by Part C and Part B 619 coordinators, their respective data managers, and other state staff in the summer of 2013, provides a national picture of where states are in moving to fully being able to use data to examine results and improve services. These important goals require that states have ready access to a variety of different types of information about children, programs, and the workforce, and be able to combine this information in various ways to examine what is making a difference for children and families. States also need ready access to high-quality data to operate and oversee the programs statewide—for example, to monitor costs and maximize their resources for services.

All Part C and Part B 619 programs that provided data indicated that they have child-level state data systems with identifiable information. Data elements commonly found (i.e., by 85% or more of states) in Part C and Part B 619 systems include child name, date of birth, child demographics, eligibility status, and service setting. Elements that were more common in Part C than in Part B 619 child-level data systems are date of referral, referral source, date of evaluation, date of initial IFSP meeting, services authorized, program name, exit reason, and date of the C to B transition conference. Common data elements found in many Part B 619 child-level data systems that were not as frequently reported for Part C systems are child outcomes and date of last service. Part C and Part B 619 data systems are likely to have some elements in common because of similar federal reporting requirements, whereas some of the differences may be due to reporting requirements unique to each program. A challenge for Part B 619 programs to make full use of the child-level data elements is that for more than one-third of the states, not all these data elements reside in the same data system or have ever been linked. Only 8% of the Part C programs face this challenge.

States were less likely to report having data systems with workforce data, and even fewer have program data. States with Part C workforce data systems are most likely to have data on licenses/certifications and employment information. Many states reported having Part B 619 workforce data on demographics, employment, education, and licenses/certification. A minority of states for Part C and Part B 619 reported having data on program characteristics such as cost, structure, or quality, and for Part B, programming (i.e., curriculum) and whether the child’s program serves typically developing peers (i.e., inclusion). Both workforce and program factors, such as personnel qualifications and program quality, are likely to have a substantial impact on child outcomes, so states will need to include this kind of information in their data systems in the future to be able to answer key questions about factors that are affecting results. Having these data will not be enough; answering these kinds of questions also requires that states be able to link workforce and program characteristics to the child-level data, which less than half the states can do now.

Addressing many critical policy and programmatic questions about IDEA’s early childhood programs requires linking data elements with data outside of the Part C or Part B 619 programs. To know whether former early intervention recipients require special education in kindergarten or how preschool special education graduates are doing in third grade are examples of questions that require longitudinal linkages. For states to understand the full impact of their investment in early childhood programs, they need to know how many children are served by multiple programs and the characteristics of these programs. Among the most fundamental linkages for IDEA’s early childhood programs are the linkages between Part C and Part B 619 child-level data systems. Twenty-nine percent of states reported that the Part C and Part B 619 child-level data reside in the same system or have been linked, although in another 23% of the states, the Part C and Part B 619 coordinators provided conflicting answers to this question. Part C data can be linked to general education K-12 data in 14% of the states. Not surprisingly, given Part B’s connection to departments of education in many states, the numbers for Part B 619 are much higher, with 79% of the states reporting Part B 619 data in the same system as or linked to K-12 data. A minority of

24


Part C programs reported that child-level data have been linked to other early childhood, social service, or health data systems, the highest being 42% of states for Medicaid/SCHIP and 37% for EHDI. The programs for which states have most frequently linked data for Part B 619 are state pre-K at 46% of states and Head Start at 22%. These numbers are a good start but indicate that many more linkages need to be established.

The needs assessment uncovered both similarities and differences across states in how they govern their data systems, how they verify data quality, who is granted access to the data, and how they have used their data.

Few Part C programs have data governance bodies, whereas many Part B 619 programs do, again probably a reflection of Part B 619’s connection with departments of education in many states. Part B 619 is included as part of a data governance body with special education and general education in more than half the states. Part C and Part B 619 programs reported using a variety of tools and procedures to check data quality, including data audits built into the data system and user manuals. Common data standards also support data quality, and these are available to states through CEDS. The majority of Part C and Part B 619 coordinators were familiar with CEDS, although few Part B 619 coordinators and very few Part C coordinators reported having used the CEDS online tools. With regard to who has access to data at what level, the findings show a similar pattern for Part C and Part B 619 for both aggregate and individual-level data, with more state coordinators having access than regional or district personnel, even fewer states providing access to school or program personnel, and fewer yet to teachers and service providers. For example, 96% of Part C coordinators and 77% of Part B 619 coordinators have access to aggregated child-level data, compared with 42% of states where early intervention service providers have access and 44% of states where preschool special education teachers or related service providers have access. Obviously, using data to inform decision-making at any level requires access to information. Some local providers and teachers may have access to local databases with comparable child-level information, so lack of access to the state-level data system may not be a problem in some places. On the other hand, it is surprising that 19% of Part B 619 coordinators reported that they do not have access to aggregated child-level data on the program they oversee. States reported using data for a variety of purposes, with most using data to check data quality and for program improvement for Part C and Part B 619. Other common uses for states are reporting to the public, identifying professional development and technical assistance needs, and reporting to the governor and the legislature.

Much was learned about where states are in building high-quality data systems; equally interesting is where states are trying to go. To assist the DaSy Center in planning technical assistance, states were asked to identify their priorities for improving their data systems in the next two years and whether they wanted technical assistance with those priorities. For Part C, the priorities reported most frequently were data use (i.e., analyzing data, using data for program improvement, etc.); data quality, verification, and audit systems; increasing the type or quality of data on child and family outcomes; and data sharing permissions and privacy issues. The most frequent topics for which the Part C coordinators indicated they would like assistance from the DaSy Center were child and family outcome data, data sharing permissions and privacy issues, linkages to Part B 619 data, data use, and increasing the type or quality of finance data. For Part B 619, the priorities reported most frequently were data use; data quality, verification, and audit systems; child and family outcome data; and increasing the type or quality of data for the APR indicators or 618 data. Note that Part C and Part B 619 programs indicated the same areas for three of their top four priorities. The areas for which the most Part B 619 states requested assistance from the DaSy Center were child and family outcomes, APR or 618 data, linkages between Part C and Part B 619 data, and data use. In general, a fairly high proportion of states reported wanting assistance in nearly all the areas asked about. Even the areas least frequently indicated as TA needs were still identified as needs

25


by about one in five states or more. This information will be extremely valuable in helping the DaSy Center plan upcoming TA activities and in working with individual states.

The importance of state agencies’ having good data for multiple purposes is widely recognized. The Part C and Part B 619 state agencies have been building data systems for many years. They have used their state data to address multiple purposes including monitoring, program improvement, and providing data required for federal reporting. Over time, states have built service delivery systems for young children with disabilities and delays that reflect each state’s unique context and history. State data systems have evolved as well, capturing various facets of those service delivery systems and reflecting that same unique state context. Although all Part C and Part B 619 state data systems must be capable of providing data to meet federal reporting requirements, the systems are capable of meeting a variety of state needs as well. Furthermore, many states are working to have better data and to use those data even more effectively in the future to improve service delivery. Similarly, examining how the recipients of early intervention or preschool special education services are doing a few years or many years after leaving the programs will move within the reach of more states as they expand their longitudinal state data systems to include IDEA’s early childhood programs. As momentum builds across the country to develop coordinated services for children under 5 years of age and to integrate the data across the various programs serving these children, the Part C and Part B 619 programs, with their long history of data collection and reporting, will have much to contribute to the design and implementation of integrated data system efforts in their states.

The Part C and Part B 619 agencies in many states have made substantial strides in building data systems that can be used for numerous purposes. Nevertheless, much work remains to be done to continue increasing the capacity of the Part C and Part B 619 programs to improve the quality of the data and to use data to drive meaningful program improvement. The information in this report will assist the DaSy Center and other technical assistance centers in supporting states in harnessing the power of data to track and improve results for young children and their families served under IDEA.

26

Documents

IDEA Part C and Part B 619 State Data Systems: Current … · IDEA Part C and Part B 619 State Data Systems: Current Status and Future Priorities September 2013 Taletha Derrington