Preview: State Child Outcomes Data Quality profiles

PREVIEW: STATE CHILD OUTCOMES DATA QUALITY PROFILESNational Webinar

February 2014

Objectives① Introduce the new state child outcomes data

quality profiles② Explain how to interpret the charts and graphs

in the profiles③ Review the data quality criteria used to analyze

the national child outcomes data④ Provide an opportunity for you (states) to have

specific questions regarding the profiles answered by TA providers

Overview• The profiles include information about: • State vs. national• Data quality criteria used for national analysis• Completeness of data• Progress categories patterns

• Trends over time

National vs. State: State A

National vs. State: State B

National vs. State: State C

QUESTIONS?Feel free to ask your questions via the chat box Or by unmuting your line by pressing *6

Quality Child Outcomes Data

Criteria for quality state-wide data:1. Reporting data on enough children2. Within expected patterns

8

Completeness of Data1. Reporting data on enough children

• Part B 619: 12% or more of child count• Number of children reported / Child count

• Part C: 28% or more of exiting children• Number of children reported / Total exiters

• Note: Some states have established more accurate methods for estimating the number of children receiving services

State A Part B 619: Percent of Child Count Included in Child Outcomes Data

Plotted +-1 standard deviation (SD) above and below the national average.

State B Part B 619: Percent of Child Count Included in Child Outcomes Data

State C Part C: Percent of Exiters Included in Child Outcomes Data

QUESTIONS?Feel free to ask your questions via the chat box Or by unmuting your line by pressing *6

Patterns in the Data

2. Within expected patterns in the data• Criteria for inclusion in national analysis

• Category ‘a’ not greater than 10% • (However, less than 5% recommended as more stringent data quality

criteria for states)

• Category ‘e’ not greater than 65%

Progress Category Patterns: State A

Progress Category Patterns: State B

Progress Category Patterns: State C

What Types of Change are Important?

18

Trends over Time SS1: State A

Trends over Time SS1: State B

Trends over Time SS1: State C

Trends over Time SS2: State A

Trends over Time SS2: State B

Trends over Time SS2: State C

Summary• State Child Outcomes Data Quality Profiles present data

on: • State compared to national • State data in relation to quality criteria used for national analysis

• Reporting data on enough children• Within expected patterns in the data

• Trends over time• Profiled 3 states

• 1 consistently high quality child outcomes data• 2 states with some different potential data quality issues

• Patterns in progress categories• Patterns over time

QUESTIONS?Feel free to ask your questions via the chat box Or by unmuting your line by pressing *6

How to Use the Child Outcomes Profiles

• Use to identify trends in the data and data quality issues that require further exploration

• Use graphs in other materials to share with various audiences

• Can be used as part of broad data analysis

Data Quality•Not the focus of the SSIP•But must be addressed in the SSIP• Describe data quality issues identified through data analysis

• Describe data quality efforts

Looking for Patterns in Data• What capacity do states have to do more pattern checking than nationally available?

• Some possibilities: • By entry scores• By disability subgroup• By family characteristics• By geography• By age at entry• Other child or service characteristics

SSIP Child Outcomes Broad Data Analysis Template

Tool to assist in conducting an initial broad data analysis using data you currently use for reporting in the APR to look at:

• State relative to national• Data Across years• Patterns within the state• Comparisons across programs

within the state

NEW!

http://ectacenter.org/~docs/eco/SSIP_child_outcomes_broad_data_analysis_template_FINAL.docx

Meaningful Differences Calculator• Are differences meaningful? • This tool can be used to…

• Compare current year summary statement values to the previous year for the state

• Identify local programs that are meaningfully different from the state

http://ectacenter.org/eco/pages/summary.asp

Other resources• Data quality resources

• http://www.ectacenter.org/eco/pages/quality_assurance.asp

• Child Outcomes Measurement Framework• http://www.ectacenter.org/~docs/eco/selfassessment.doc

• Guidance for looking for patterns in data• http://www.fpg.unc.edu/~eco/assets/pdfs/

Pattern_Checking_Table.pdf• Data analysis for program improvement resources

• http://www.ectacenter.org/eco/pages/usingdata.asp

32

SSIP Phase I Webinar Series for Part C and 619

• Hosted by the Regional Resource Centers Program (RRCP) Part C/619 State Accountability Priority Area, in collaboration with DaSy, ECTA and IDC. The webinar topics are as follows:

March 6, 2014,3:00pm ET: SSIP Overview• April 8,2014, 3:00pm ET: SSIP Data Analysis• May 1, 2014, 3:00pm ET: SSIP Infrastructure Analysis

Save the Date!• Early Childhood National Conference

• September 8-10, 2014: Improving Data, Improving Outcomes

• Hosted by DaSy, ECTA, and IDC in New Orleans, LA.

• DaSy will be sponsoring up to four people from each state to attend.

How ECTA Can Help!• Contact us for support for data quality analysis and quality assurance activities

• Contact us for support for program improvement planning and data analysis

Abby Winer: abby.winer@sri.com

QUESTIONS?