Embed Size (px)
Citation preview
Overview and Application of the INQUIRE Data Tools
March 20, 2013
Welcome!
Kathryn Tout, Child Trends
Ivelisse Martinez-Beck, OPRE
The diverse audience for the webinar reflects the many ways that we interact with and rely upon data in our work.
State child care/subsidy/early learning administrators
Researchers/evaluators
Data managers
Technical assistance specialists
Grant writers/coordinators
Quality assurance specialists
Operations directors
3
What is the primary early childhood data challenge you face?
Developing an integrated data
system 35%
Accessing data to answer policy
questions 24%
Ensuring data quality
19%
Knowing what data to collect
12%
Funding an integrated data
system 10%
N=304 webinar participants who responded upon registration
Purpose of the Webinar Series
Provide practical tools and guidance that can support your work in designing, linking, reporting, analyzing and acting upon data.
Today’s webinar will focus on a set of data tools that include: • Data elements and definitions
• Data matrix to facilitate sorting and selection
• Guidance document on linking data with key policy, research, and monitoring questions
5
Agenda
Background and Context for the Data Tools
Overview of Data Tools
State Perspectives & Applications • Vermont
• California
Next steps
Questions/Discussion
6
The Quality Initiatives Research and Evaluation Consortium (INQUIRE)
Consortium of primarily researchers and evaluators who are working on projects related to Quality Rating and Improvement Systems (QRIS) or other quality improvement initiatives or topics
Purpose of INQUIRE • Support high quality, policy relevant research and evaluation
• Provide guidance to policymakers on evaluation strategies, new research, interpretation of research results, and implication of new research for practice
7
Through OPRE-funded projects and in state QRIS evaluations, we heard from states and from evaluators about the need for support on data.
Need to think about data for a variety of purposes • Reporting
• Research and evaluation
• Performance management
Need guidance on what data to collect and at what level
Need to coordinate with other data initiatives/ reporting requirements
8
Background on INQUIRE’s data work
INQUIRE Data Elements Workgroup began in the spring of 2012
Purpose: • Develop a set of data elements that can guide data collection
efforts.
• Link the data elements to questions that inform monitoring, reporting, performance management and evaluation.
• Provide guidance on data governance and data integrity.
Process: As tools were developed, efforts were made to align efforts with other national data collection efforts and reporting
9
Key Partnerships
The Common Education Data Standards (CEDS) • A national collaborative effort to develop voluntary, common
data standards for a key set of education data elements for early learning through postsecondary and workforce
Worked closely with the CEDS Early Learning Quality group
Used the CEDS definitions when they overlapped with relevant data elements
10
Key Partnerships
Coordinated with other national efforts & experts • Early Childhood Data Collaborative
• National Registry Alliance
• National Early Care and Education Survey Fields
Federal reporting • ACF 801 Case-level reporting form
• Quality Performance Report
• Head Start Program Information Report
• Early Learning Challenge Grant Performance
11
Presenters
Child Trends • Carlise King, Data and Technical Assistance Director
• Sarah Friese, Senior Research Analyst
Building Bright Futures-Vermont's Early Childhood Data Reporting System (ECDRS) Project • Kathleen Eaton Paterson, BBF-ECDRS Project Co-Director
California Child Care Development Division • Cecelia Fisher-Dahms, Administrator of the Quality
Improvement Office
• Sarah Neville-Morgan, Child Development Consultant
12
Data Element Tools
Audience • State administrators and data specialists
• Researchers
• Technical assistance specialists
Data Elements Products • Data Dictionary
• Data Elements Matrix
• Linking Policy Questions Brief
13
Data Dictionary & Matrix
Organized by level • Child
• Family
• Program
• Class/Group
• Practitioner
14
Data Dictionary-Child Level
15
Data Dictionary-Family Level
16
Data Dictionary-Program Level
17
Data Dictionary-Program Level
18
Data Dictionary-Practitioner Level
19
Data Dictionary-Practitioner Level
20
Data Dictionary-Classroom Level
21
Data Matrix
“Yes” means the data element is aligned to meet the designated reporting or data collection effort listed. “No” means the field is not included in the referenced data collection.
22
Policy Questions
Provides guidance to analyze basic policy, monitoring and evaluation questions that states may ask about their early childhood systems
Organized by • ECE Programs
• Practitioners
• Children
• Parents’ Child Care Decision Making
• Quality Rating and Improvement System (QRIS)/Quality Improvement Initiatives (QII)
23
Policy Questions
The policy questions range from descriptive to analyzing changes over time • Characteristics
• Changes over time
• Outcomes for specific groups
• Impact of quality improvement initiatives
24
Child Element
Child ID
Child Language Code
Individualized Program Type
Foster
Homelessness Status
Child resides Indian Lands
Child migrant
Family Element
Family Income
Number of People in Family
Program Element
Organization ID
Quality Rating and Improvement System Participation
Quality Rating and Improvement System Score
Question: Do targeted populations with high needs have access to high quality care?
25
Analysis Recommendations
High needs populations’ access to quality care is calculated by first answering three sub-questions. • What percentage of the children in the state meet the criteria
for “high needs”?
• How many high needs children are receiving care from programs participating in the state’s QRIS?
• How many high needs children are receiving the highest-quality care from programs participating in the state’s QRIS, as defined by the state?
26
Analysis Recommendations Use the data element Quality Rating and Improvement System
Participation to isolate programs participating in the state’s QRIS.
Next, determine if a child is high needs by using the data elements related to the seven categories outlined in Race to the Top. A child is high needs if the Child’s Language Code is a language other than English; if she has an Individualized Education Program; is in foster care or is homeless; resides on “Indian Lands”; is a migrant; or comes from a family with an income of less than 200% of the poverty line. Family income is determined by using the data elements, Family Income and Number of People in Family, in comparison with the federal poverty guidelines.
27
Practitioner Elements
Practitioner ID
Practitioner position
Employment start date
Employment end date
Program Elements
Organization ID
Number of full-time staff
Number of half-time staff
Number of staff hired
Number of staff that ended their employment
Question: Does workforce stability increase from year-to-year?
28
Analysis Recommendations Workforce stability by practitioner, by year is captured by
calculating: • The average number of job changes practitioners made while retaining the
same job title;
• The average number of job changes practitioners made to new positions within the field, and;
• The total number and percentage of practitioners that left the field completely
29
Analysis Recommendations To calculate the number of times practitioners change jobs but
retain the same job title, select one of the six job type options from the data element Practitioner Position.
• For each Practitioner ID in that position, total the number of Employment Start Dates for the total number of jobs held by individual practitioners in one year.
Average the number of positions per practitioner to calculate the mean for the group. Data from previous years can be used for practitioners that report no hire or end dates in a given year. If the last reported employment date is a hire date, practitioners are assumed to be employed at the same programs, in the same positions in each subsequent year until they report a change to their employment status. If the last reported date is an end date, the practitioner is considered to have left the field and is not included in the analysis for the year of interest.
30
State Perspectives & Applications
31
Vermont - Building Bright Futures (BBF)
Early Childhood Data Reporting System (ECDRS) Project
March 20, 2013
INQUIRE
Early Childhood Data: Building a Strong Foundation Webinar Series Overview and Application of INQUIRE Data Tools to Support
High Quality Early Care and Education Data
32
End Goals and End Users
Comprehensive, longitudinal, ecological
Continued improvement and results accountability
Appreciative inquiry and collective impact
End Users: Parents-Practitioners-Policymakers
How are Vermont’s children and families?
Where are the bright spots and where are the gaps in children’s health, learning and achievement?
Use data to connect, engage and take action 33
Policy Questions to Prototype
• Expand ECDC policy questions to represent the whole early childhood system
• Unpack each policy question, what are the data elements needed to answer the question?
• Select one policy question for prototype
• Acquire data – data sharing agreement
• Design and test with end-users
• Create data catalogue – data maps and data gaps 34
VT’s Early Childhood Data Reporting System (ECDRS) http://ecdrs.buildingbrightfutures.org/
35
VT Early Childhood Data Reporting System Contact Information
VT’s State Early Childhood Council http://buildingbrightfutures.org/
Julie Coffey, Executive Director
Kathleen Paterson and Dave Lapointe, VT-ECDRS Project Co-Directors
BBF Data and Evaluation Committee Co-Chairs, Ben Allen and Jim Squires
36
CA DEPARTMENT OF EDUCATION
TOM TORLAKSON, State Superintendent of Public Instruction
37
California’s Need for Common
Data Elements & Related Tools
• Large and diverse state with many partners and
initiatives at both state and local level
• Race to the Top-Early Learning Challenge
- California implementing a unique, locally-based approach
- 17 Regional Leadership Consortia (Consortia) in 16
counties operating local Quality Rating and Improvement
Systems (QRIS)
CA DEPARTMENT OF EDUCATION
TOM TORLAKSON, State Superintendent of Public Instruction
Common Data Elements
Will Support:
• Coordination and Alignment
• Evaluation
• Policy Development
38
CA DEPARTMENT OF EDUCATION
TOM TORLAKSON, State Superintendent of Public Instruction
Coordination and Alignment
• Use of aligned and consistent data
elements across RTT-ELC Consortia
• Reporting requirements - cross-walk
data to other data systems for ease in
reporting
39
CA DEPARTMENT OF EDUCATION
TOM TORLAKSON, State Superintendent of Public Instruction
Evaluation
• Comprehensive data available for RTT-
ELC Consortia QRIS evaluation
• Alignment of local, state and cross state
evaluations
40
CA DEPARTMENT OF EDUCATION
TOM TORLAKSON, State Superintendent of Public Instruction
Policy Development
• Comprehensive data available on RTT-
ELC Consortia QRISs
• Ability to document and report key
policy questions
41
Looking Across the State Perspectives
States are at different stages in the development of data systems
The tools can support work at different stages. They can be part of a process involving: • Planning
• Coordinating
• Implementing
Technical assistance providers can facilitate the use of the tools through their work with states
As we learn more about how states are using the tools, we will share information and provide updates to the materials
42
Next Steps
Tools will be available at the end of April 2013 on Research Connections • www.researchconnections.org
Upcoming Webinars on Data Management • May 6, 2013, 2:00-3:30 EST:
Developing Data Governance Structures
• May 16, 2013, 2:00-3:30 EST:
Best Practices for Producing High-Quality Data
Webinar recording will be available on Research Connections
43
Acknowledgements of Contributors to INQUIRE’s Data Work Group Rick Brandon, Consultant
Missy Cochenour, AEM
Iheoma Iruka, FPG, University of North Carolina
Tabitha Isner, MN Department of Human Services
Fran Kipnis, Center for the Study of Child Care Employment at UC Berkeley
Lee Kreader, National Center for Children in Poverty
Minh Le, Office of Child Care, ACF
Lizabeth Malone, Mathematica Policy Research
Frances Majestic & Elizabeth Hoffman, Office of Head Start, ACF
Dawn Ramsburg, Office of Child Care, ACF
Bobbie Weber, Oregon State University
44
Questions and Discussion
45
Contact Information
Ivelisse Martinez-Beck, Office of Planning, Research and Evaluation, Administration for Children and Families • [email protected]
Kathryn Tout, Child Trends • [email protected]
46
