Early Childhood Indicators and Family Engagement · specific types of parent engagement ... Leaders in early education almost ... Instruments for Evaluating Family Involvement.”

In the following report, Hanover Research examines practices and policies pertaining to the

evaluation of state-funded family engagement initiatives in early childhood settings.

Early Childhood Indicators and Family Engagement

March 2014

Hanover Research | March 2014

© 2014 Hanover Research | District Administration Practice 2

TABLE OF CONTENTS

Executive Summary and Key Findings ............................................................................... 3

INTRODUCTION ........................................................................................................................... 3

KEY FINDINGS ............................................................................................................................. 4

Section I: Features of Family Engagement Evaluations ...................................................... 5

FLEXIBILITY ................................................................................................................................. 5

DIMENSIONALITY ......................................................................................................................... 6

INSTRUMENTS ............................................................................................................................ 7

ITEMS ..................................................................................................................................... 10

Section II: State Profiles ................................................................................................. 15

CALIFORNIA ............................................................................................................................. 15

DELAWARE .............................................................................................................................. 17

ILLINOIS................................................................................................................................... 19

GEORGIA ................................................................................................................................. 20

KANSAS ................................................................................................................................... 21

MARYLAND .............................................................................................................................. 23



EXECUTIVE SUMMARY AND KEY FINDINGS

INTRODUCTION

Decades of research have documented the vital role that family engagement plays in promoting school readiness among young children. However, the necessarily broad conception of what it means to effectively engage the families of young children has constricted evaluation procedures for family engagement initiatives. This report examines the factors that influence the evaluation of family engagement initiatives and describes state-level efforts to bring greater order and consistency to these evaluation processes across a variety of programs. This report proceeds in two sections:

Section I: Features of Family Engagement Evaluations examines core features of family engagement evaluations and describes the evaluation instruments and items common in family engagement evaluation frameworks.

Section II: State Profiles describes how six states have implemented family engagement frameworks and accompanying evaluation procedures to develop a state-level view of family engagement initiatives and/or to support growth in family engagement initiatives on the program level. Profiled states include:

o California

o Delaware

o Illinois

o Georgia

o Kansas

o Maryland



KEY FINDINGS

Family engagement initiatives may seek to achieve goals in many domains. The family and community engagement model developed by Joyce L. Epstein has formed the basis of many state-level family engagement frameworks and addresses six forms of family engagement, including parenting, communicating, volunteering, learning at home, decision making, and collaboration with the community. Many specific types of parent engagement that fall within each of these categories, such as parental warmth and the establishment of at-home learning environments, are associated with school readiness.

Family engagement initiatives are best evaluated within the context of each individual program. Individual programs vary in terms of resources and families served and, as a result, may achieve the best outcomes by tailoring family engagement initiatives to meet families’ specific needs. For this reason, state policies frequently encourage program leaders to establish family engagement goals on a local level. While this practice may improve the quality and monitoring of family engagement initiatives, it also limits the scope and specificity of statewide evaluation procedures.

Policymakers and program leaders employ many instruments to evaluate family engagement initiatives. Parent surveys, interviews, and benchmarking reports each offer opportunities to evaluate family engagement outcomes and the mechanisms driving those outcomes. States may directly develop evaluation instruments, provide guidance to individual programs in the development of program evaluations, or commission independent evaluations.



SECTION I: FEATURES OF FAMILY ENGAGEMENT

EVALUATIONS This section examines core features of family engagement evaluations and describes the evaluation instruments and items common in family engagement evaluation frameworks.

FLEXIBILITY

Leaders in early education almost universally recommend flexibility in the evaluation of family engagement initiatives. Family engagement goals, as well as families themselves, often vary substantially from program to program, and while states may provide guidance pertaining to family engagement, effective evaluations must account for the local context.1 In many cases, accounting for the local context will require accounting not only for variety in programming, but also for a range of effective evaluation strategies. A 2009 report published by the National Association for the Education of Young Children (NAEYC) cautions that a traditional conception of parental involvement, which relies heavily upon attendance at events and rates of volunteerism, “may also be perceived as insensitive to family members’ time, financial, or educational limitations.”2 Accordingly, administrators should be aware of the limitations and/or potential biases of certain evaluation items. In developing evaluation tools and procedures, the Harvard Family Research Project urges leaders to consider several factors:

Alignment of program objectives with evaluation instrument. Given its different measures, will the evaluation instrument you selected yield useful information about how well your program is meeting its own particular objectives?

Applicability to respondents. If your respondents differ from the population in which the instrument was tested for validity and/or reliability, how will this influence your interpretation of evaluation results? Is the format and language of the instrument conducive to the way you are currently engaging with parents, teachers, and others to whom you might administer the instrument?

Human and financial costs. Will you need to invest resources in building capacity—in expertise or in time—to collect, analyze, or use data that will be harvested from the instrument?3

1 “A Toolkit for Family Involvement in Education.” Oregon Department of Education, 2006, p. 148.

http://www.ode.state.or.us/opportunities/grants/nclb/fitoolkitpdf.pdf 2 Stark, D.R., S. Moodie. “Family Engagement, Diverse Families, and Early Childhood Education Programs: An

Integrated Review of the Literature.” National Association for the Education of Young Children, 2009, p. 4. http://www.naeyc.org/files/naeyc/file/ecprofessional/EDF_Literature%20Review.pdf

3 Bulleted items taken verbatim from: Westmoreland, H., S. Bouffard, K. O’Carroll, H. Rosenberg. “Data Collection

Instruments for Evaluating Family Involvement.” Harvard Family Research Project, May 2009, p. 4. http://www.hfrp.org/family-involvement/publications-resources/data-collection-instruments-for-evaluating-family-involvement



Even within a flexible evaluation framework, however, certain standards and practices may be universally implemented. For example, the Maryland Early Childhood Family Engagement Framework provides guidance to programs responsible for developing evaluations independently, while also noting that these evaluations should be developed around seven statewide goals for early childhood family engagement initiatives.4 The National Parent Teacher Association recommends that programs use the evaluation development process as an important means of family engagement, allowing “for parental input and local control in designing the specific elements of programs.”5

DIMENSIONALITY

Comprehensive family engagement initiatives are multidimensional, examining the efforts that promote family engagement and the effects or outcomes of those efforts. By assessing both effort and effect, leaders develop a better understanding not only of the impact of an initiative but also the mechanisms that influence that impact. As described below, measures of effort and effect may account for many factors but are largely dependent upon the specific goals of a given initiative:

Measures of effort, sometimes referred to as “outputs,” describe whether and to what extent activities were implemented as you had intended. These measures describe whether the district staff, school staff, and families were supported as you had planned. Such measures can be calculated by counting the number of attendees at an event, or materials that your district produced and disseminated. Measures of effort can also describe what participants thought of the activities and information you offered through participant satisfaction surveys.

Measures of effect convey whether or not you are meeting your desired outcomes. Unlike measures of effort, which demonstrate what activities your district implemented and how it implemented them, measures of effect help track whether your activities have made a difference. For example, an evaluation of a district’s professional development program for parent liaisons might include measures of effort, such as how many liaisons attended a training session about sponsoring a family-friendly walkthrough of their schools, and measures of effect, such as what percentage of participating schools showed improvements on their level of family-friendliness.6

Although many evaluations of family engagement initiatives emphasize the parent experience, some evaluations have focused on the direct and indirect impact of these initiatives on the children themselves. For instance, in 2005, the Family-School Partnership Lab at Vanderbilt University evaluated the impact of a family engagement initiative funded by the federal Office of Educational Research and Improvement. Researchers constructed the evaluation model based on research that assumes that two constructs mediate the impact of parental involvement on student achievement:

4 “The Early Childhood Family Engagement Framework: Maryland’s Vision for Engaging Families with Young Children.”

Maryland Family Engagement Coalition, 14 August 2013, p. 4. http://www.marylandpublicschools.org/MSDE/divisions/child_care/docs/MD_Fam_Engage.pdf

5 “State Laws on Family Engagement in Education.” National Parent Teacher Association, 2009, p. 113.

http://www.pta.org/files/State_Laws_Report.pdf 6 Westmoreland et al., Op. cit., p. 7.



The first construct is children’s perceptions of parents’ involvement activities (i.e., what children attend to, hear, and experience in the course of their parents’ involvement). These student perceptions are important because children learn from what they perceive and experience; absent perception and engagement, parents’ and teachers’ actions are much less likely to influence target outcomes in students.

The second construct includes a sample of student attitudes, behaviors, and skills that influence or lead to achievement, or proximal student academic outcomes. These are important not only because they often lead to achievement, but also because they are more directly amenable to parental influence than is student performance on summary measures of achievement.7

For the purposes of the 2005 study, researchers defined proximal student academic outcomes to include “self-efficacy, intrinsic motivation to learn, self-regulatory strategy use, and social self-efficacy for relating to teachers.” 8 While this study examined family engagement initiatives implemented in an elementary school setting, some studies have established procedures to examine achievement outcomes for preschool programs. For example, the parents of children enrolled in Maryland’s Pilot Preschool Programs have been asked to sign releases that permit data sharing between preschool and kindergarten programs for the purpose of gaining a more complete understanding of program impact on school readiness and other outcomes.9

INSTRUMENTS

Three general instruments are commonly employed in the evaluation of family engagement initiatives: surveys, interviews, and benchmarking reports. Each of these evaluation tools offers unique benefits and poses certain challenges, and may best be used in combination. Section II provides examples of how each of these instruments has been implemented by state-funded family engagement initiatives. Personal and group interviews afford an opportunity for a conversation about personal experiences in the program.10 This may provide a more in‐depth picture of the program or bring to light issues that were not previously considered.11 However, there are some disadvantages to this type of data gathering, most of which are centered on the difficulty of coordinating and conducting in‐depth interviews. For example, interviews involve a serious time commitment, do not lend themselves to pre‐ and post‐test comparisons, and are

7 Bulleted items taken verbatim from: Hoover-Dempsey, K.V., H.M. Sandler. “Final Performance Report for OERI Grant

# R305T010673: The Social Context of Parental Involvement: A Path to Enhanced Achievement.” The Family-School Partnership Lab, 22 March 2005, p. 33. http://www.vanderbilt.edu/peabody/family-school/Reports.html

8 Ibid., p. 34.

9 “The Judith P. Hoyer Early care and Education Enhancement Grant for Preschool Services.” Maryland State

Department of Education, July 2009, p. 9. http://www.marylandpublicschools.org/NR/rdonlyres/50E778B3-88E6-4276-8CA4-981A50B7F31D/22077/PFAPilotSites1.pdf

10 Bouffard, S. and P.M.D. Little. “Detangling Data Collection: Methods for Gathering Data.” Harvard Family Research

Project, August 2004, p. 3. http://www.hfrp.org/publications‐resources/browse‐our‐publications/detanglingdata‐ collection‐methods‐for‐gathering‐data

11 Ibid.



difficult to adapt to complex outcome measurements (e.g., it is easier to ask parents about their child’s knowledge than about their attitudes).12 Parent surveys are considered an efficient tool not only for information gathering but also for sending a “message to parents that a school cares what they think.”13 Response rates for parent surveys, however, can be quite low in settings in which parental engagement is already low, which may negatively affect the quality of evaluations in these settings. The National Center on Safe and Supportive Learning Environments, which provides guidance to many districts in the administration of school climate surveys, attests that the existing school-family relationship heavily influences parent participation in surveys. In an instructional webinar hosted in 2012, representatives of the organization indicated that the response rate for parent surveys improves when “family engagement with schools is high; trusting relationships exist between families and school personnel; [and] family perspectives influence how surveys are developed, administered, interpreted, and used.”14 Benchmarking reports permit programs to evaluate processes and outcomes against predetermined goals and expectations. States and other funders commonly provide guidance for the content and format of benchmarking reports, in some cases establishing statewide goals and in other cases offering guidance to programs as they establish goals independently. For example, Head Start programs nationwide establish internal benchmarks with the guidance of the Head Start Parent, Family, and Community Engagement Framework. This framework, which encourages programs to establish specific foundations for family engagement, focus programming in key impact areas, and work toward several broadly-defined family engagement and child outcomes, was recently adapted to create the Maryland Early Childhood Engagement Framework.15 Figure 1.1 describes the foundations, program impact areas, family engagement outcomes, and child outcomes central to the Head Start Framework.

12

Sabatelli, R., S. Anderson, and V. LaMonte. “Assessing Outcomes in Child and Youth Programs: A Practical Handbook.” Revised Edition. University of Connecticut School of Family Studies Center for Applied Research, September 2005, p. 16. http://www.uwex.edu/ces/4h/evaluation/documents/ChildYouthOutcomeHandbook2005.pdf

13 “Engaging Parents in Education: Lessons from Five Parental Information and Resource Centers.” U.S. Department of

Education, June 2007. http://www2.ed.gov/admins/comm/parents/parentinvolve/report_pg18.html 14

“Increasing Staff and Family Survey Response Rates,” National Center on Safe and Supportive Learning Environments, 18 January 2012, p. 32. http://safesupportiveschools.ed.gov/index.php?id=9&eid=1525

15 “The Early Childhood Family Engagement Framework: Maryland’s Vision for Engaging Families with Young

Children,” Op. cit., p. 2.



Figure 1.1: Head Start Parent, Family, and Community Engagement Framework

Source: U.S. Department of Health and Human Services

16

The family engagement outcomes within the Head Start framework have been found to be associated with “enhanced school readiness skills, sustained learning, and developmental gains across early childhood education into elementary school.”17 Figure 1.2 defines each of the seven family engagement outcomes central to the Head Start framework.

Figure 1.2: Family Engagement Outcomes

FAMILY ENGAGEMENT

OUTCOMES FEATURES

Family well-being Parents and families are safe, healthy, and have increased financial security.

Positive parent-child relationships

Beginning with transitions to parenthood, parents, and families develop warm relationships that nurture their child’s learning and development.

Families as lifelong educators

Parents and families observe, guide, promote, and participate in the everyday learning of their children at home, school, and in their communities.

Families as learners Parents and families advance their own learning interests through education, training and other experiences that support their parenting, careers, and life goals.

Family engagement in transitions

Parents and families support and advocate for their child’s learning and development as they transition to new learning environments, including Early Head Start to Head Start, Early Head Start/Head Start to other early learning environments, and Head Start to kindergarten through elementary school.

16

“The Head Start Parent, Family, and Community Engagement Framework.” U.S. Department of Health and Human Services, 2011, p. 1. http://eclkc.ohs.acf.hhs.gov/hslc/standards/ims/2011/pfce-framework.pdf

17 Ibid., p. 5.



FAMILY ENGAGEMENT

OUTCOMES FEATURES

Family connections to peers and community

Parents and families form connections with peers and mentors in formal or informal social networks that are supportive and/or educational and that enhance social well-being and community life.

Families as advocates and leaders

Parents and families participate in leadership development, decision making, program policy development, or in community and state organizing activities to improve children’s development and learning experiences.

Source: U.S. Depatment of Health and Human Services18

The Head Start framework provides guidance to program leaders in developing the foundations and implementing the strategies that support each family engagement outcome. Programs evaluating family engagement initiatives could easily use this guidance to establish and measure progress against specific benchmarks. For example, to achieve family outcomes in the area of family well-being, Head Start recommends the following strategies for establishing a foundation of program leadership:

Ensure that systems, supports, and resources are in place to address professional development, continuous improvement, program environment, and partnerships related to family well-being.

Incorporate goals related to family well-being into agency work plans and strategic planning.

Ensure staff members have appropriate training and supervision and manageable caseloads.

Develop relationships with community members and community organizations that support families’ interests and needs.

Promote cross-service area teamwork.

Contract or hire a mental health consultant with appropriate credentials and experience to be a resource for staff and program needs around family well-being.19

By focusing on underlying mechanisms, the Head Start framework drives improvement in key areas that are directly and indirectly related to student outcomes.

ITEMS

The relationship between family engagement and school readiness is well established and complex. A literature review published in Early Education and Development in 2010 identified three parental behaviors that are “highly predictive” of social-emotional learning and cognitive development: parental warmth and sensitivity, support for a child’s emerging autonomy, and active participation in learning.20 Notably, each of these parental behaviors recognize engagement that can occur at school events or away from the school setting,

18

Ibid. 19

Taken verbatim from: Ibid., p. 7. 20

Sheridan, S.M., L. Knoche, C.P. Edwards, J.A. Bovaird, K.A. Kupzyk. “Parent Engagement and School Readiness: Effects of the Getting Ready Intervention on Preschool Children’s Social-Emotional Competencies.” Center for Research on Children, Youth, Families, and Schools, 2010, p. 128. http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1012&context=cyfsfacpub



reinforcing findings that effective parenting can mediate the negative effects of low socioeconomic status on school readiness.21 Many specific parent behaviors have been linked to school readiness and child learning, such as attendance at parent engagement activities offered through a prekindergarten program,22 parent-child reading,23 home learning environments,24 nurturing behaviors,25 and parent conversational practices,26 just to name a few. In 2010, the Pew Center on the States conducted a literature review of family engagement in prekindergarten settings that revealed the following five “essential elements” of effective family engagement:

Early learning programs expect, welcome, and support family participation in decision making related to their child’s education. Programs that recognize and respect family members’ role as their child’s first and most important teachers can encourage families to serve as advocates for their children and participate in early learning activities at home and in the program.

Families and early learning programs engage in consistent, two-way, linguistically and culturally appropriate communication. Programs that provide multiple avenues for timely and ongoing communication can both acquire and share crucial insights about a child’s strengths and needs so that both teachers and parents can more effectively support his or her development and education.

Families’ knowledge, skills, and backgrounds are integrated into the learning experience. Programs can benefit from family members’ unique knowledge and skills by providing opportunities for volunteering and other participation. Teachers can seek out valuable information about children’s lives, families, and communities and use it to enrich and increase the relevance of their curricula and instructional practices.

Programs help families foster a home environment that enhances learning. Engagement allows programs to assist families in providing a home environment that is both reflective of their culture and supports their child’s educational experience.

Early learning programs create an ongoing and comprehensive system for promoting family engagement. Programs that ensure teachers and administrators are trained and receive needed supports, e.g. professional development, can fully engage culturally, linguistically, and economically diverse families and promote meaningful relationships.27

21

Dotterer, A., I. Iruka, E. Pungello. “Parenting, Race, and Socioeconomic Status: Links to School Readiness.” Family Relations, 61(4), October 2012, p. 657.

22 Miedel, W., A. Reynolds. “Parent Involvement in Early Intervention for Disadvantaged Children: Does It Matter?”

Journal of School Psychology, 37(4), 1999, p. 379. 23

Trivette, C., C. Dunst, E. Gorman. “Effects of Parent-Mediated Joint Book Reading on the Early Language Development of Toddlers and Preschoolers.” Center for Early Literacy Learning, CELL Reviews, 3(2), 2010, p. 1. http://www.earlyliteracylearning.org/cellreviews/cellreviews_v3_n2.pdf

24 Fantuzzo, J., C. McWayne, M.A. Perry, S. Childs. “Multiple Dimensions of Family Involvement and Their Relations to

Behavioral and Learning Competencies for Urban, Low-Income Children.” School Psychology Review, 33(4), 2004, p. 470.

25 Merlo, L.J., M. Bowman, D. Barnett. “Parental Nurturance Promotes Reading Acquisition in Low Socioeconomic

Status Children.” Early Education and Development, 18(1), 2007, p. 51. 26

Sparks, A., E. Reese. “From Reminiscing to Reading: Home Contributions to Children’s Developing Language and Literacy in Low-Income Families.” First Language, 33(1), 2012, p. 89.

27 Bulleted items taken verbatim from: Stark, D.R. “Engaged Families, Effective Pre-K: State Policies that Bolster

Student Success.” The Pew Center on the States, 2010, p. 3.



As noted at the beginning of this section, family engagement initiatives may vary substantially from program to program, based upon either funding requirements or regional needs. In 2012, the annual publication of the National Institute for Early Education Research (NIEER), “The State of Preschool,” revealed that 37 of the 40 states with public prekindergarten require providers to offer support services that engage parents in some way. Figure 1.3 describes the number of states that have required prekindergarten programs to engage parents in five specific categories. While most states generally require prekindergarten programs to offer parent involvement activities and parent conferences and/or home visits, states vary in terms of required parenting support, parent education, and parent health services. The diversity of these family engagement initiatives, many of which promote parent engagement outside of the program setting, demonstrate the opportunity for family engagement initiatives to effectively bridge the home and school contexts.28

Figure 1.3: States Requiring Specific Parent Engagement Initiatives

Source: NIEER

29

In identifying key features of family engagement initiatives, many states have relied upon the model of family engagement proposed by Joyce L. Epstein, Director of the National Network of Partnership Schools.30 The Epstein model addresses six key components of

http://www.pewtrusts.org/uploadedFiles/wwwpewtrustsorg/Reports/Pre-k_education/PkN_Family_Engagement_FINAL.pdf

28 El Nokali, N.E., H.J. Bachman, E. Votruba-Drzal. “Parent Involvement and Children’s Academic and Social

Development in Elementary School.” Child Development, 81(3), 2010, p. 2. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2973328/pdf/nihms-248792.pdf

29 “The State of Preschool.” National Institute for Early Education Research, 2012, pp. 185-186.

http://nieer.org/sites/nieer/files/yearbook2012.pdf 30

[1] “Dr. Joyce L. Epstein.” National Network of Partnership Schools. http://www.csos.jhu.edu/p2000/staff/joyce.htm

Family engagement plans influenced by the Epstein model include: [2] “Parent and Family Involvement: A Guide to Effective Parent, Family, and Community Involvement in North Carolina Schools.” Public Schools of North

10

14

23

34

34

0 10 20 30 40

Parent health services

Parent education or job training

Parenting support or training

Parent conferences and/or home visits

Parent involvement activities



family engagement: parenting, communicating, volunteering, learning at home, decision making, and collaborating with the community. A 2009 NAEYC report describes each component of the model, which broadly meet many needs:

Parenting: Help all families establish home environments to support children as students

Communicating: Design effective forms of school-to-home and home-to-school communications about school programs and their children’s progress

Volunteering: Recruit and organize parent help and support

Learning at Home: Provide information and ideas to families about how to help students at home with homework and other curriculum-related activities, decisions, and planning

Decision Making: Include parents in school decisions, developing parent leaders and representatives

Collaborating with Community: Identify and integrate resources and services from the community to strengthen school programs, family practices, and student learning and development31

Initiatives that rely upon the Epstein model of family engagement may elect to evaluate program outcomes for students, parents, and teachers.32 The table on the following page provides examples of how initiatives that promote each form of family engagement may result in differential outcomes for each group. These outcomes may be considered when assessing program goals and appropriate evaluation measures.

Carolina, 2010, pp. 18-25. http://www.dpi.state.nc.us/docs/parents/toolkit/guide.pdf [3] “Parent Engagement Information and Tools.” Michigan Department of Education, 2004, p. 20. http://www.michigan.gov/documents/Parent_Involvement_Part_1_12-16-04_111426_7.pdf [4] “Family Engagement Framework.” California Department of Education, 2011, p. 5. http://www.wested.org/online_pubs/cpei/family-engagement-framework.pdf [5] “The Missing Piece of the Proficiency Puzzle.” Commissioner’s Parents Advisory Council, June 2007, p. 11. http://education.ky.gov/CommOfEd/adv/Documents/PACtheMissingPiecev2.pdf

31 Stark et al., Op. cit., p. 5.

32 [1] “Results.” National Network of Parenting Partnership Schools.

http://www.csos.jhu.edu/p2000/RESULTS/Results.htm [2] See also: Epstein, J.L., L. Coates, L., K.C. Salinas, M.G. Sanders, B.S. Simon. School, Family, and Community

Partnerships: Your Handbook for Action. Thousand Oaks, CA: Corwin Press, 1997. http://books.google.com/books?id=pts4TnnTVGgC&printsec=frontcover&dq=Joyce+Epstein's++6+types+of+family+involvement+Johns+Hopkins&source=bl&ots=di_-i7F3gs&sig=h1ZIw2dP5cMPC8J4PrxLsEnigN8&hl=en&ei=QrDVS9gCia72BMqOsboP&sa=X&oi=book_result&ct=result&resnum=3#v=onepage&q&f=false



Figure 1.4: Student, Parent, and Teacher Outcomes of Family Engagement

Source: National Network of Partnership Schools33

33

“Results,” Op. cit.

Parenting Student outcomes: Awareness of family supervision; respect for parents; positive personal qualities, habits, beliefs, and values taught by family; balance between time spent on chores, other activities, and homework; regular attendance; awareness of importance of school

Parent outcomes: Self-confidence about parenting; knowledge of child and adolescent development; adjustments in home environment as children proceed through school; awareness of own and others’ challenges in parenting; feeling of support from school and other parents

Teacher outcomes: Understanding of families’ backgrounds, cultures, concerns, goals, needs, and views of their children; respect for families’ strengths and efforts; understanding student diversity; awareness of own skills to share information on child development

Communicating Student outcomes: Awareness of own progress in subjects and skills; knowledge of actions needed to maintain or improve grades; understanding school programs and policies; informed decisions about courses and programs; awareness of own role as courier and communicator in school-family partnerships

Parent outcomes: Understanding school programs and policies; monitoring and awareness of child’s progress in subjects and skills; responses to student problems; ease of interactions with school and teachers; high rating of school quality

Teacher outcomes: Diversity of communications with families; ability to communicate clearly; use of network of parents to communicate with all families; ability to understand family views and elicit help with children’s progress

Volunteering Student outcomes: Skills in communicating with adults; skills that are tutored or taught by volunteers; awareness of many skills, talents, occupations, and contributions of parents and other volunteers

Parent outcomes: Understanding the teacher’s job; self-confidence about ability to work in school and with children; awareness that families are welcome and valued at school; specific skills of volunteer work; use of school activities at home; enrollment in programs to improve own education

Teacher outcomes: Organization, training, and use of volunteers; readiness to involve families in new ways, including those who do not volunteer at school; awareness of parents’ talents and interests in school and children; individual attention to students because of help from volunteers

Learning at home

Student outcomes: Skills, abilities, and test scores linked to homework and classwork; homework completion; positive attitude about homework and school; view of parent as more similar to teacher and of home as more similar to school; self-confidence in ability as learner

Parent outcomes: Knowledge of how to support, encourage, and help student at home each year; discussions of school, classwork, homework, and future plans; understanding of instructional program and what child is learning in each subject; appreciation of teacher’s skill; awareness of child as a learner

Teacher outcomes: Varied designs of homework, including interactive assignments; respect of family time; recognition of helpfulness of single-parent, dual-income, and all families in motivating and reinforcing student learning; satisfaction with family involvement and support

Decision making

Student outcomes: Awareness of representation of families in school decisions; understanding that student rights are protected; specific benefits linked to policies enacted by parent organizations

Parent outcomes: Input into policies that affect children’s education; feeling of ownership of school; awareness of parents’ voices in school decisions; shared experiences and connections with other families; awareness of school, district, and state policies

Teacher outcomes: Awareness of perspectives of families in policy development and school decisions; acceptance of equality of family representatives on school committees and in leadership roles

Collaborating with the

community

Student outcomes: Skills and talents from enriched curricular and extracurricular experiences; knowledge and exploration of careers and options for future education and work; self-confidence and feeling value and belonging in the community; positive relationships with adults in the community

Parent outcomes: Knowledge and use of local resources to increase skills and talents or to obtain needed services; interactions with other families in community activities; awareness of community’s contributions to the school; participation in activities to strengthen the community

Teacher outcomes: Knowledge and use of community resources to enrich curriculum and instruction; skill in working with mentors, business partners, community volunteers, and others to assist students and teaching practice; knowledge of referral processes for families and children with needs for specific services



SECTION II: STATE PROFILES This section describes six states’ evaluation procedures for family engagement initiatives, which include establishing a state-level view of family engagement initiatives and/or supporting growth in family engagement initiatives on the program level. Profiled states include:

California

Delaware

Illinois

Georgia

Kansas

Maryland

The six states profiled in this section were identified based upon the thoroughness of their evaluation procedures or the opportunity to illustrate an alternative evaluation procedure. Although evaluation procedures differ from state to state, several processes have been implemented by multiple states. For example, most states profiled in this section permit program leaders to establish family engagement goals locally and many provide a formal framework or other guidance to support programs in establishing these goals. Five of the six states have evaluated initiatives using measures that permit families to directly report their experiences or outcomes, such as surveys and interviews. At least three states profiled in this section have also commissioned studies of family engagement initiatives that serve young children, and these efforts have provided state and local leaders with valuable, independent analysis of the structure and function of family engagement initiatives.

CALIFORNIA

The California Family Engagement Framework provides a comprehensive guide for “educators, districts, schools, families, and communities as they plan, implement, and evaluate strategies across multiple programs for effective family engagement to support student achievement and close the academic achievement gap.”34 The California framework applies across public prekindergarten programs and K-12 settings. The framework describes state and federal requirements and contains rubrics that encompass three tiers of implementation, including basic implementation, progressive implementation, and innovative implementation. Figure 2.1 describes the legal requirements and implementation rubrics associated with the requirement to integrate community resources.

34

“Family Engagement Framework.” California Department of Education, 2011, p. 5. http://www.wested.org/online_pubs/cpei/family-engagement-framework.pdf



Figure 2.1: Requirements and Implementation Rubric, Resource Integration

DISTRICT REQUIREMENT: Identify and integrate resources and services from the community to strengthen school programs, family practices, and student learning and development.

FEDERAL AND STATE ACTIVITIES:

Coordinate and integrate parent involvement activities with

public preschool;

other public educational programs;

parent resource centers.

Conduct outreach to community agencies to

inform of services;

solicit support.

IMPLEMENTATION RUBRIC

BASIC IMPLEMENTATION PROGRESSIVE IMPLEMENTATION INNOVATIVE IMPLEMENTATION

Utilize one-way communication outlets (mailers, brochures, public service announcements) to community agencies to:

provide information about educational services;

solicit support for educational programs; and

Ensure that parent involvement activities address the needs of and are available to families with children

across education programs;

in public preschool; and

work with parent resource centers (established under Title I and the Individuals with Disabilities Education Act) to implement parent involvement activities.

In addition to basic activities,

provide teachers with lists of available community resources; and/or

provide presentations for, attend functions of, and maintain communications with community organizations to cultivate relationships in support of education; and/or

collaborate with other public agencies providing services to families to support parent/family involvement in education; and/or

encourage parent resource centers to collaborate to provide services and support to all families.

Form a partnership with community-based organizations and other public agencies to

plan or coordinate parent/family involvement activities and programs;

obtain or provide technical assistance;

establish effective channels for communicating with families;

obtain or provide training for families;

disseminate resources and information on an ongoing basis; and/or

support the establishment of a comprehensive, collaborative parent resource center to meet the needs of all families.

Overall, the implementation rubrics provide guidance to leaders, but do not establish strict expectations for individual programs. The state monitoring instrument for preschool programs requires that providers submit a plan for parent involvement that includes provisions for “the sharing of program goals and structure with families” and ensures “that effective, two-way, comprehensive communication between staff and parents is carried out on a regular basis throughout the program year.”35

State-funded prekindergarten programs are also required to administer an annual parent survey.36 The parent survey assesses the following elements:

35

“Monitoring Instrument Item Report.” California Department of Education, 2013-2014, p. 1. http://www.cde.ca.gov/ta/cr/documents/cd201314b.pdf

36 “The State of Preschool,” Op. cit., p. 202.



Overall satisfaction;

Parent belief that their child is safe and happy in the program;

Availability of information on child development, parenting, program operations, and opportunities for involvement;

Program impact on parent educational attainment or employment;

Parent satisfaction with specific aspects of the program, including program operations, communication, opportunities for involvement, activities, equipment, environment, and safety; and

Parent suggestions for improvement.37

DELAWARE

A review of current state policy revealed that efforts to evaluate family engagement initiatives in Delaware programs that serve young children focus primarily upon compliance monitoring. To assess compliance with the parental engagement components of the Individuals with Disabilities Education Act (IDEA), the state education agency commissions an annual survey of parents with students with disabilities to determine whether programs have supported families in understanding their rights, communicating about their children’s needs, and supporting their child’s learning and development.38 Some years ago, however, Delaware commissioned a more in-depth study of family engagement in state-funded preschool settings that employed a unique approach to assess the diversity of program goals as well as program effectiveness. In the 2006-2007 academic year, the Delaware Interagency Resource Management Committee commissioned an outcomes study of Delaware Head Start and Early Childhood Assistance Programs. The study, conducted by the Human Capital Research Collaborative at the University of Minnesota, examined family engagement goals established in family partnership agreements developed and implemented by 12 state-funded programs.39 To examine program success, researchers examined outcomes for family in a random selection of 333 families with children enrolled in the 12 programs selected for the study. Data related to family outcomes was self-reported by families.40

37

“Desired Results for Children and Families – Parent Survey.” California Department of Education, 2003. http://www.cde.ca.gov/sp/cd/ci/drdpforms.asp

38 [1] “Part C State Annual Performance Report (APR ) for FFY2011.” Delaware Department of Health and Social

Services, 2013, p. 19. http://www.dhss.delaware.gov/dms/epqc/birth3/files/annualperformancerpt_fy11.pdf [2] “Center for Disabilities Studies: Early Childhood: Birth to Three Early Intervention System.” Center for Disabilities

Studies. http://www.cde.ca.gov/ta/cr/documents/cd201314b.pdf 39

“Delaware.” Human Capital Research Collaborative. http://humancapitalrc.org/resources/delaware.cfm 40

“Overview of Head Start/ECAP Outcomes Study: State of Delaware.” Delaware Department of Education, 2007, p. 28. http://www.doe.k12.de.us/infosuites/students_family/earlychildhood/files/ECAP06%2007%20State%20Report.pdf



Researchers determined that programs had established family engagement goals in 12 categories. By classifying goals by category and examining outcomes across programs, researchers illuminated priorities across programs and gathered evidence of the general success rates in each category. As shown in Figure 2.2, participating programs had the most success achieving family engagement goals pertaining to child education and basic needs and were most likely to struggle to meet goals pertaining to adult education and housing.

Figure 2.2: Family Engagement Goals, Delaware Prekindergarten Settings

Source: Delaware Department of Education

41

41

Ibid., p. 27.

24

20

32

42

45

49

56

60

50

91

102

173

26

26

42

58

61

61

61

61

91

99

136

175

0 40 80 120 160 200

Mental health

Relationships

Finances

Childcare

Parenting

Employment

Transportation

Basic needs

Housing

Health/nutrition

Adult education

Child education

Total Goals Goals Met



ILLINOIS

In 2012, the Illinois State Board of Education published a three-year study of the state-funded Preschool for All programs. The study was conducted by the Erikson Institute and SRI International with the purpose of providing “a picture of the school readiness outcomes of children attending [Preschool for All] programs, the characteristics and quality of the programs, and the characteristics of the children and families being served.” 42 The Preschool for All programs exist “to provide early education opportunities for children between the ages of three and five, with priority given to children from low-income families and those who are determined to be ‘at risk.’”43 While the study relied upon many measures to assess child outcomes, family engagement was primarily assessed through surveys and interviews. Between 2009 and 2012, researchers conducted online parent surveys to measure parent participation in the following support services:

Parent-child interaction activities

Parent resource library

Links to other resources

Parent skill development

Speech and language therapy

Home visits

Family support services

Behavioral management

Children’s mental health consultation44

Furthermore, in parent interviews, researchers assessed the following factors pertaining to family engagement:

Parent has met teacher

School or teacher invited parents and children to visit the school or class

School or teacher provided information about who to call to discuss concerns

School or teacher provided workshops, materials, or advice to parents to help children learn at home

School or teacher sent home information about how to prepare child to be ready for school45

Family engagement activities are also assessed during regular site visits. These formal evaluations include a review of parent involvement documentation, including agendas and attendance records for parent group meetings and family activities.46

42

“Preschool for All: Evaluation of Programs for Children Ages 3-5.” Illinois State Board of Education, 2012. http://www.isbe.net/earlychi/preschool/pfa-report/pfa-3-5/default.htm

43 “PFA Family Involvement.” Illinois State Board of Education, 2012, p. 22.

http://www.isbe.net/earlychi/preschool/pfa-report/pfa-3-5/fam-involvement-fact-sheet.pdf 44

“Illinois Preschool for All (PFA) Program Evaluation.” Illinois State Board of Education, 2012, p. 26. http://www.isbe.net/earlychi/preschool/pfa-report/pfa-3-5/final-eval-rpt-0312.pdf

45 Ibid.



GEORGIA

In 2007, the Georgia Department of Education and Bright from the Start: Georgia Department of Early Care and Learning commissioned an evaluation of Georgia prekindergarten programs and licensed childcare centers. The purpose of the study was to examine “the quality of center-based care and pre-k programs” as well as the “types of services provided to infants, toddlers, and preschoolers served by these programs.”47 The study was conducted by the Frank Porter Graham Child Development Institute at the University of North Carolina.48 Researchers relied upon a random sample of 173 programs operated across the state. Childcare programs examined in the study included programs that do not currently operate state-funded prekindergarten programs to provide a comprehensive view of the opportunities generally available to support child development across the state. Within each program participating in the study, researchers selected one to three classrooms for observation. Researchers also reviewed program documentation and self-reported data from principals, program directors, and teachers.49 Research examined several key factors relating to family involvement, including:

Resource coordination: presence of a resource coordinator to promote “access to services that help enable the child to be ready for school”;

Family participation: availability of opportunities that permit families to become members of program advisory boards and “to read to children in classrooms, eat with their child’s class or help at meals, help out in the classroom, and participate in program activities for the whole family”;

Information provided to families: written information regarding “early literacy, overall child development, general safety issues, general health and well-being of children, and dental health”;

Services and supports provided to families: presence of program initiatives designed to “help families find community activities, school-age care, social services, mental health services; coordinate community services for families; provide a lending library for families; and send home reading activity packs”;

Communicating with families: types of communication that programs used to engage parents, including phone calls, newsletters, and parent conferences.50

46

“Preschool for All.” Illinois State Board of Education, 2011, p. 201. http://www.isbe.state.il.us/earlychi/pdf/ec_implementation.pdf

47 “Georgia Study of Early Care and Education: Findings from Georgia’s Pre-K Program.” Frank Porter Graham Child

Development Institute, 2009, p. 2. http://www.fpg.unc.edu/sites/fpg.unc.edu/files/resources/reports-and-policy-briefs/FPG_GeorgiaStudyofEarlyCareandEducation_%20FindingsfromGeorgia%27sPre-KProgram_2009.pdf

48 Ibid.

49 Ibid., pp. 2-4.

50 Ibid., pp. 21-23.



More recently, the Georgia Department of Education and Bright from the Start: Georgia Department of Early Care and Learning conducted an examination of the statewide family engagement initiative, 360 Degrees of Family Engagement. 360 Degrees of Family Engagement is a Pre-K-12 family engagement initiative that permits local districts to conduct independent evaluations with guidance from state agencies. Annual reporting procedures require program leaders complete a year-end report that describes parent attendance at specified events, activities implemented to fulfill specific program goals, quantitative and qualitative outcomes of individual activities, and plans for improvement.51 To describe outcomes of individual goals program leaders address the following questions:

How did the targeted goal results compare to the actual goal results?

What factors contributed to the success or lack of success in meeting the targeted goal results?

What vital family behaviors were learned?

Did the team see a connection between measures of effort and measures of effect in the activity? If so, how?

What activity efforts were the most successful in communicating with the target family population?

Did collaborative teaming efforts help in implementing the activity? If so, what would you do again? If not, what would you do differently?

Have you seen increased vital behaviors in families’ abilities to support student learning? If so, what from the activity do you think contributed to this?

Have you seen an increase in parent capacity due to contributing factors for the activity?

Does the goal result provide enough data evidence to show a logical correlation to the success or lack of success in meeting the overall student measurable goal?

Based on your data results, would you implement the same activity next year? If not, what might you revise?52

KANSAS

In a 2013 report, the National Center for Children in Poverty identified the Kansas state-funded prekindergarten program for exemplary practices in parent engagement.53 State evaluations of parent engagement activities are conducted using two separate annual surveys: a general family survey and a protective factors survey.54 Each survey is distributed

51

“Wrapping School, Home, and Community Engagement Activities Around Student Achievement Outcomes.” Georgia Department of Education, 2012, pp. 42-47. http://www.gadoe.org/School-Improvement/Federal-Programs/Documents/360_Curriculum_FINAL%20(%20Lakeisha).pdf

52 Bulleted items taken verbatim from: Ibid., pp. 39-40.

53 Smith, S., T. Robbins, S. Stagman, D. Mathur. “Parent Engagement from Preschool through Grade 3: A Guide for

Policymakers.” National Center for Children in Poverty, 2013, p. 11. http://www.nccp.org/publications/pdf/text_1084.pdf

54 “State Pre-Kindergarten.” Kansas State Department of Education.

http://www.ksde.org/Agency/DivisionofLearningServices/EarlyChildhoodSpecialEducationandTitleServices/EarlyChildhood/StatePreKindergarten.aspx



and collected by program teachers, who are also responsible for submitting parent responses using an online data collection system.55 The family survey primarily gathers information pertaining to family demographics, including:

Language spoken at home

Family income

Size of household

Family members in the armed forces

Access to state-funded services and financial relief, such as food stamps, unemployment benefits, public housing, and mental health services

Use of community services, such as libraries, medical services, and parks56

In addition to family demographics, the family survey also asks respondents to rate the frequency of the following items on a five-point scale. Respondents may indicate whether each activity occurs on a scale from “never” to “daily.”

Someone reads to my child in our home.

My child and I talk about things that happened during the day.

Someone is active and involved in my child’s play.

Someone takes my child to a museum, library, learning or activity center, or the zoo.

Someone talks with the teacher or educator of any early childhood programs that my child attends.57

The protective factors survey provides greater detail pertaining to family engagement at home and at school. The protective factors survey includes four sections, each described in Figure 2.3.

55

Ibid. 56

“Family Survey.” Kansas State Department of Education, July 2013, pp. 1-2. http://www.ksde.org/Portals/0/Early%20Childhood/State%20PreKindergarten/EC-SIS%20Family%20Survey.pdf

57 Taken verbatim from: Ibid., p. 2.



Figure 2.3: Protective Factors Survey Items

*Repondents indicate frequency on a scale from “Never” to “Always.” **Respondents indicate level of agreement on a scale from “Strongly disagree” to “Strongly agree.” Source: Kansas State Department of Education

58

MARYLAND

The Maryland State Department of Education funds dozens of organizations responsible for promoting school readiness in early care or educational settings.59 Currently, the Maryland Family Engagement Coalition is in the process of drafting goals and strategies for state-funded family engagement initiatives operating in these settings.60 The working draft of the

58

Survey items taken verbatim from: “Protective Factors Survey.” Kansas State Department of Education, August 2012, pp. 1-3. http://www.ksde.org/Portals/0/Early%20Childhood/State%20PreKindergarten/EC-SIS-ProtectiveFactorsSurvey.pdf

59 “The Early Childhood Family Engagement Framework: Maryland’s Vision for Engaging Families with Young

Children,” Op. cit., p. 1. 60

Ibid.

•In my family, we talk about problems.

•When we argue, my family listens to "both sides of the story."

•In my family, we take time to listen to each other.

•My family pulls together when things are stressful.

•My family is able to solve our problems.

Part I: Family relationships*

•I have others who will listen when I need to talk about my problems.

•When I am lonely, there are several people I can talk to.

•I would have no idea where to turn if my family needed food or housing.

•I wouldn't know where to go for help if I had trouble making ends meet.

•If there is a crisis, I have others I can talk to.

•If I needed help finding a job, I wouldn't know where to go for help.

Part II: Sources of support**

•There are many times when I don't know what to do as a parent.

•I know how to help my child learn.

•My child misbehaves just to upset me.

Part III: Parent-child relationship*

•I praise my child when he/she behaves well.

•When I discipline my child, I lose control.

•I am happy being with my child.

•My child and I are very close to each other.

•I am able to soothe my child when he/she is upset.

•I spend time with my child doing what he/she likes to do.

Part IV: Parenting**



Early Childhood Family Engagement Framework, which has been adapted from the Head Start Parent, Family, and Community Engagement Framework, establishes “common goals for family engagement across the early childhood system and within individual early care and education providers.”61 Guidance provide by the Maryland framework is also intended to support family engagement initiatives operated in other educational settings, including “museums, libraries, medical offices, hospitals, community colleges, and other community organizations” with similar family engagement goals.62 Figure 2.4 describes the family engagement goals and general strategies that form the Maryland framework.

Figure 2.4: Maryland Early Childhood Family Engagement Framework

Source: Maryland Family Engagement Coalition

63

The Maryland Family Engagement Initiative, which has spearheaded the development of the Maryland framework, notes that the strategies suggested in the framework are not necessarily “applicable for every early care and education provider” and should be considered “guidelines for helpful practices” rather than program requirements.64 Individual programs funded by specific grants have established evaluation requirements specific to those individual programs. Preschool services funded through the state Judith P. Hoyer Early Care and Education Enhancement grant must survey parents to assess satisfaction with services.65 Maryland Learning Links, which provides guidance to families and educators that serve students enrolled in special education programs and early intervention initiatives, requires programs to conduct family assessments using a locally

61

Ibid., p. 2. 62

Ibid. 63

Ibid., p. 4. 64

Ibid., p. 2. 65

“The Judith P. Hoyer Early Care and Education Enhancement Grant for Preschool Services,” Op. cit., p. 20.



developed family intervention tool and routines-based interviews, which provide educators with a better understanding of how families function in the home.66

66

[1] “Maryland’s Early Childhood Intervention and Special Education Evaluation and Assessment System.” Maryland State Department of Education, June 2011. http://olms.cte.jhu.edu/olms2/data/ck/sites/121/files/MD%20ECIE%20Evaluation%20and%20Assessment%20System%20Protocol%20with%20WSS%20implementation%20change%20June%202011(1).pdf

[2] “Routines-Based Interview.” Siskin Children’s Institute. http://www.siskin.org/www/docs/112.190



PROJECT EVALUATION FORM Hanover Research is committed to providing a work product that meets or exceeds partner expectations. In keeping with that goal, we would like to hear your opinions regarding our reports. Feedback is critically important and serves as the strongest mechanism by which we tailor our research to your organization. When you have had a chance to evaluate this report, please take a moment to fill out the following questionnaire. http://www.hanoverresearch.com/evaluation/index.php

CAVEAT The publisher and authors have used their best efforts in preparing this brief. The publisher and authors make no representations or warranties with respect to the accuracy or completeness of the contents of this brief and specifically disclaim any implied warranties of fitness for a particular purpose. There are no warranties which extend beyond the descriptions contained in this paragraph. No warranty may be created or extended by representatives of Hanover Research or its marketing materials. The accuracy and completeness of the information provided herein and the opinions stated herein are not guaranteed or warranted to produce any particular results, and the advice and strategies contained herein may not be suitable for every partner. Neither the publisher nor the authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Moreover, Hanover Research is not engaged in rendering legal, accounting, or other professional services. Partners requiring such services are advised to consult an appropriate professional.

http://www.hanoverresearch.com/evaluation/index.php



1700 K Street, NW, 8th Floor

Washington, DC 20006

P 202.559.0500 F 866.808.6585

www.hanoverresearch.com