Children and Youth Services Review 32 (2010) 1388–1395

Contents lists available at ScienceDirect

Children and Youth Services Review

j ourna l homepage: www.e lsev ie r.com/ locate /ch i ldyouth

Improving turnover in public child welfare: Outcomes from anorganizational intervention

Jessica Strolin-GoltzmanWurzweiler School of Social Work, Yeshiva University, USA

E-mail address: strolin@yu.edu.

0190-7409/$ – see front matter © 2010 Elsevier Ltd. Aldoi:10.1016/j.childyouth.2010.06.007

a b s t r a c t

a r t i c l e i n f o

Article history:Received 1 April 2010Received in revised form 25 May 2010Accepted 1 June 2010Available online 4 June 2010

Keywords:Child welfareStructural equation modelingOrganizational changeIntervention

This study examines the effects of an organizational intervention on intention to leave child welfare. Using anon-equivalent comparison group design, twelve child welfare agencies participated in either the DesignTeam intervention condition or a comparison condition. Organizational factors and intention to leave wereassessed pre and post intervention. Findings from GLM Repeated Measures indicate significant group bywave interactions for three of the six organizational variables (professional resources, commitment, andburnout) and intention to leave. All of these interactions showed a greater positive improvement for the DTgroup than the comparison group. Structural equation modeling demonstrates good model fit withsignificant pathways leading from the intervention through intervening organizational variables to intentionto leave. Intervening at the organizational level can help child welfare agencies improve organizationalshortcomings, while also decreasing intention to leave. Evidence suggests that by improving organizationalfactors affecting the workforce, service quality will improve.

l rights reserved.

© 2010 Elsevier Ltd. All rights reserved.

1. Introduction

Workforce turnover in child welfare causes multiple social andeconomic consequences for child welfare organizations and thechildren and families in the system. The National Council on Crimeand Delinquency (2006) detected significant relationships betweenstaff turnover, child welfare system functioning and recurrent childabuse. Other research shows a similar pattern of negative outcomesfrom turnover. Specifically, Flower, McDonald, and Sumski (2005)found that children with more than one worker are almost 60% lesslikely to be placed in a permanent situation within Adoption and SafeFamilies Act (ASFA) timeframes as compared to those with only oneworker. In another study, foster care youth discussed their experi-ences of the negative effects of workforce turnover such as a lack ofstability and loss of trusting relationships (Strolin-Goltzman, Kollar, &Youth in Progress, 2010). Further, Glisson and Hemmelgarn (1998)found that workers' perceptions of organizational climate was aprimary predictor of positive service outcomes and increased servicequality for clients in child welfare systems. In other words, if workersperceived their organizations in a positive manner they were morelikely to provide quality service and obtain positive client outcomes.In response to the consequences related to workforce turnover inchild welfare, there is a need for the development and evaluation ofinterventions aimed at improving retention.

In the past decade there has been an abundance of literaturedescribing the causes of turnover in child welfare and other humanservice systems (Landsman, 2001; Dickinson & Perry, 2002; US GeneralAccounting Office, 2003; Smith, 2005; Mor Barak, Levin, Nissly, & Lane,2006; Ellett, Ellis, Westbrook, & Dews, 2007; Scannapieco & Connell-Carrick, 2007; Cahalane & Sites, 2008; Strolin-Goltzman et al., 2008;Yankeelov, Barbee, Sullivan, & Antle, 2009; McGowan, Auerbach, &Strolin-Goltzman, 2009). In fact, in the early tomid 2000s, in an attemptto compile a coherent set offindings related to the causes and correlatesof workforce recruitment and retention, the Institute for the Advance-ment of Social Work Research, the University of Maryland's Center forFamilies & Institute for Human Services Policy, and the Annie E. CaseyFoundation (2005) collaborated in a project to undertake a systematicreview of the research related to recruitment and retention in childwelfare. The findings provided insight into various organizational andpersonal factors affecting intention to leave and turnover in public childwelfare (DePanfilis & Zlotnik, 2008). Personal factorswere identified as:professional commitment to children and families, previous workexperience, education, job satisfaction, efficacy, emotional exhaustionand role overload. The six organizational factors that were identifiedincluded: (1) Better salary; (2) Supervisory support; (3) Reasonableworkload; (4) Coworker support; (5) Opportunities for advancement;and (6) Organizational commitment and valuing employees. These sixfactors have provided researchers, administrators and other childwelfare experts guidelines for where to begin to intervene.

Since the publication of new guidelines, there have been few studieson interventions attempting to address the workforce epidemic at theorganizational level are meager. In fact the few studies that have been

1389J. Strolin-Goltzman / Children and Youth Services Review 32 (2010) 1388–1395

published were published prior to the release of the DePanfilis andZlotnik study. Further, approaches to reducing turnover and increasingquality practice in child welfare have focused primarily on individuallevel recruitment of Title IV-E, specially trained caseworkers with socialwork degrees (Gansle & Ellett, 2002; Gomez, Travis, Ayers-Lopez, &Schwab, 2010; Jones, 2002; Okamura & Jones, 1998). Throughout thechild welfare literature, there have been only two published studies onthe outcomes of organizational interventions aimed at improving childwelfare agency climate or workforce turnover: the ARC intervention(Glisson, Dukes, & Green, 2006) and the Design Team intervention(Strolin-Goltzman, Kollar, & Youth in Progress, 2010).

Glisson et al. (2006) report results from the Availability, Responsive-ness and Continuity (ARC) organizational intervention aimed atreducing caseworker turnover in children's service systems, along withimproving climate and culture. Their findings suggest that the ARCintervention is effective at reducing the probability of caseworkerturnover by two-thirds while also improving role conflict, role overload,emotional exhaustion, and depersonalization.

Strolin-Goltzman, Lawrence, et al. (2010) report evidence that theimplementation of Design Teams in public child welfare agencies iscorrelated with a reduction in intention to leave and actual turnoverrates at the organizational level. The findings demonstrated thatagencies with Design Teams had a 12% decrease in turnover rate frompre to post test,while the comparison agencies turnover rate increase by2%. However because the data were analyzed at the aggregate agencylevel, it is unclear through exactly what mechanisms these improve-ments occurred or whether the intervention actually created change inindividual perceptions of organizational factors over time. This paperfocuses on individual level changes as a result of the DT intervention.This purpose of this study is twofold: (1) to test the hypothesis thatparticipants inDTagencieswill have greater positive change frompre topost test on organizational factors than the comparison group and(2) using structural equation modeling, test themeditational pathwaysfrom the DT intervention to intention to leave (see Fig. 1).

2. Design Team intervention

2.1. Theoretical framework

Design and Improvement Teams (DT) are mechanisms for organi-zational learning and improvement founded on the principles of actiontheory and organizational learning theory. Organizational learningtheory (OL) was derived from the works of Argyris and Schön (1978)and furthered by Peter Senge in 1990. One of the main aims of theintervention is to move organizations from Model I organizations toModel II learning organizations.

Fig. 1. Theoretical model of DT influence on intention to leave.

Argyris and Schön (1978) introduced concepts of organizationalbehavior such as theories of action and model I and model II structures.Model I inhibits double loop learning,whileModel II enhances it (Smith,2005). Anypractice that reduces the chance that errorswill be identifiedand prevents workers from learning from their errors is a Model Iprocess. Argyris and Schön (1978) call this the “anti-learning process.”InModel I, any intervention, or action strategy is likely to fail because theproblem lies in the governingvariables and in the lack ofmonitoring andtesting.

Model II utilizes double loop learning, which encourages openquestioning and minimal defensiveness (see Fig. 2). Gaps betweenespoused theories (what one says they do) and theories in use (whatthey actually do) are attempted to beminimized through the collectionand utilization of quality data to make inferences and questionprocesses. This type of learning process values team learning. InModel II, the vision and goals are shared rather than unilateral. Whena vision emerges from all of the workers in an agency through dialogueand listening, it becomes a shared vision (Senge, 1990). Organizationspracticing Model II resolve difficult problems by immediately workingtoward the identification and treatment of the problem. The datacollected and utilized is “actionable”, or realistic and usable. It isproposed that Model II learning processes are achieved that organiza-tional factors will improve.

DTs attempt to build on action and organizational learning theoriesby using specific solution-focused activities to move participating childwelfare agencies toward Model II organizations.

2.2. Description of intervention

Design Teams are groups of employees who work together to solveorganizational issues leading to intention to leave and turnover in theirorganization. The teams begin by identifying the problems that employ-ees perceive to be the causes of turnover within their agency. The teamsidentify the causes and correlates of turnover through informal focusgroups and an agencywide survey called theWorkforce Retention Survey.TheDT then compile the results andprioritize the issues by feasibility andimportance. Each of the teams follow a specific solution-focused logicmodel that guides them toward developing solutions to the identifiedcauses of turnover in their organization. The key components of the DTsare described below. For amore in depth descriptions of the interventionsee Lawson et al. (2006) and Caringi et al. (2008).

2.2.1. Representative of the organizationIntervention Participants are selected from all levels (caseworker,

supervisor, management) and units (CPS, foster care, prevention,adoption, etc.) of the agency. The representativeness is essential as asimilar intervention study in a public for profit agency was completedwith specific units, rather than agency wide (Landsbergeris & Vivona-Vaughan, 1995). The Landsbergis study found a negligible or negativeimpact on job satisfaction among the entire agency suggesting thatorganizational interventions should be implemented with a crosssection of the entire agency.

2.2.2. SupportiveDT sessions began with a brief debriefing (approximately10 mi-

nutes) of the events since the last meeting. During this time team

Fig. 2. Single and double loop learning.

1390 J. Strolin-Goltzman / Children and Youth Services Review 32 (2010) 1388–1395

members and the facilitator are able to clean the slate of any work orpersonal challenges. “Process is key in the development of relation-ships and open communication between design team members, andthe construction of a learning system (Lawson, Anderson-Butcher,Petersen, & Barkdull, 2003, p. 168).” The process time allows teammembers to move forward toward developing and implementingsolutions.

2.2.3. Data drivenThe DTs are data driven in that they utilize the findings from the

Workforce Retention Survey. The survey is administered to all of theagency's employees as a means for identifying key factors affectingintention to leave in their agency. The DT compiles the results andprioritizes the issues by feasibility and importance.

2.2.4. Solution-focused and action orientedA logic model based on action and organizational learning theories

provides the scaffolding for organizational changes and helps tofacilitate the solution composition and implementation by followingseven structured steps:

(1) Clearly identifying the problem and/or need(2) Assessing causes of problem(3) Evaluating its effects on retention and workforce stability(4) Pondering the ideal situation(5) Discussing solutions already in place(6) Developing new feasible solutions(7) Identifying specific action steps that team members had to

complete prior to the next meeting.

2.2.5. External facilitation and intervention fidelityTwo External facilitators employed by a local university lead the

teams in a solution-focused process to identify and develop actionstrategies to solve organizational issues leading to turnover. All of thefacilitators are MSW educated group workers who completed a twoday initial training on DT facilitation. During the facilitation, Lawsonet al. (2003) important skills for facilitation were emphasized.

• “Facilitators make or break the process. They must know how andwhen to recognize key learning and development processes andevents” (p. 177).

• “Facilitators must know how to structure the process in a way thatencourages team members to collaborate among themselves anddevelop ownership of the team” (p.168).

• “Skills in group process are essential to provide a balance betweencontent, or outcome” (p. 168).

• “Content is also important for the group to feel successful andaccomplished” (p. 169).

To ensure intervention fidelity, facilitators participated in ongoingmeetings with project director to debrief DT progress and challenges.There were three phases to the external facilitation:

• Beginning: Meet with DT bi-weekly• Middle:Meet with DTmonthly however the team continued to meetweekly on their own. Team identifies one primary internalfacilitator. Some teams rotate facilitation among the DT members.

• End/Transition: Meet with team quarterly as needed• Self-sustaining DT: No external facilitation — fully self-sustainingteams institutionalized into their agencies. Teams made it to thisphase by two years.

3. Methods

3.1. Data and sample

The data in this study was collected in two waves in 2002 and again28–32 months later from public child welfare agencies in twelve

counties with chronic turnover of 25% or more for two consecutiveyears prior to the intervention. All of the twelve counties were in ruraland suburban regions. In 2002, the twelve agencies self selected toparticipate in a workforce retention study. The child welfare agencymanagers collaborated with university faculty to develop a survey thatwould result in a better understanding of the rationale for workerturnover in their agencies. In 2003, shortly after the completion of theinitial wave of the Workforce Retention Survey research study, theuniversity was awarded a Children's Bureau grant to address turnoverand other workforce concerns in the state. As a result, an organizationalintervention was implemented in 2003 in 5 of the 12 child welfareagencies. Of the 5 counties that initiated the intervention, 3 completedthe intervention and have sustainedDesign Teams institutionalized intotheir agencies. Previous papers have reported on the findings from theturnover study and on county level findings from the intervention.

Five hundred and twenty six frontline caseworkers and super-visors completed the surveys over the two waves. Two hundred andseventy five in 2002 and 251 at wave 2. Of the 275 that completed thesurveys at wave one (pre test) only 82 participants completed thesurvey again at wave 2 (post test) resulting in a response rate ofapproximately 30%. Although this is a low response rate, it is to beexpected due to the chronic high turnover (more than 25% in all 12counties at wave 1) in the study population. The current study utilizesthe individual level matched data to analyze effects of the interven-tion on the organizational factors and intention to leave amongparticipants in the three agencies that completed the intervention. Byutilizing only the 82 individuals we can assess individual levelperceptions of change rather than only providing a snapshot of theagency over time. Of the 82 participants 19 were from the 3 agenciescompleting the intervention. There were 63 participants from the no-intervention comparison agencies that completed both the pre andpost test.

3.2. Measures

The Workforce Retention Survey consisted of organizational andsupervisory scales, and questions related to demographics such asrace, age, job tenure, and salary. Organizational items were adaptedfrom the Emotional Exhaustion scale of the Maslach BurnoutInventory (Maslach & Jackson, 1986) as well as scales from otherWorkforce studies nationwide (Dickinson & Perry, 2002). Contentvalidity of the instrument was assessed via a review of the literatureon workforce issues in child welfare.

Organizational questions were subjected to Principal ComponentAnalysis (PCA) and four factors were extracted. Prior to performingthe PCA the correlation matrix was inspected in order to assesswhether there were coefficients above .3. The data revealed manycoefficients meeting the criteria suggesting suitability for factoranalysis. Examination of the Scree plot indicated a break betweencomponents four and five with diminishing return for the remainingcomponents, therefore four components were retained. Eigenvalueswere above 1.5 and the four components and explained 55% of thevariance. A combination of the results from the PCA along with theempirical and theoretical literature was utilized to create the fourfactors: (1) rewards and recognition, (2) professional resources, (3)agency commitment and job satisfaction, and (4) role clarity andburnout. Cronbach's alpha coefficients suggested adequate reliabilityfor all of the factors.

3.3. Intervening variables

The rewards variable consists of the mean score of 7 items thatassess how well the agency provides recognition and rewards to theparticipants. It includes items such as “I have support and recognitionfrom coworkers” and “There are clear incentives and rewards for a jobwell done” (α=.7).

1391J. Strolin-Goltzman / Children and Youth Services Review 32 (2010) 1388–1395

Professional resources consists of items that assess how well theagency provides workers with access to knowledge and skillsnecessary to complete their jobs. The variable is the mean score of11 items such as “I have training and support to make work relateddecisions”, “I have support from supervisors”, “The training providedby the agency is helpful to my work” and “computer technologiesmakemy work easier” (α=.7). The inclusion of supervisor support inthis factor is supported by the literature. In many studies, supervisorsupport is defined narrowly as emotional support, however theliterature points to a broader use of support to include the requisitionof resources (i.e. training and professional development) that can beutilized for increasing professionalization of the workforce. Specifi-cally Poulin (1994) found that supervision coupled with professionaldevelopment opportunities influence social workers' job satisfactionover time.

Organizational commitment and job satisfaction are combined toinclude items that assess whether the participant is committed to theagency and satisfied with the job. Throughout the literature jobsatisfactionhas beenhighly correlatedwithorganizational commitment(Bednar, 2003; Camp, 1994; Hellman, 1997; Seifert & Jayaratne, 1991;Tett & Meyer, 1993; Vinoker-Kaplan, 1991). According to Giffords(2003), organizational commitment is present when a worker expendsenergy on behalf of an organization, recommends the organization, andinternalizes the success or failure of the organization. The variableconsists of the mean score of 4 items such as “I recommend working atthis agency” and “All in all I am satisfied with my job” (α=.7).

The role clarity and (lack of) burnout variable consists of 7 items thatassess the clarity of job roles and expectations within the agency aswell as the level of emotional exhaustion and burnout a participantmay feel. Examples of these items include “I have sufficient emotionalenergy for the job”, “There are clear job expectations” and “I can domyjob and not burnout” (α=.7).

Workload and salary variables were added based on the strongevidence of their impact on turnover and intention to leave found inthe literature, however they were not a result of the PCA. Thesevariables measure satisfaction on a 5 point Likert scale.

Salary was found to be significant predictor of turnover in manyinstances (Gleason-Wynn, 1999; Koeske & Koeske, 2000; Reagh, 1994;Scannapieco&Connell, 2003). Still some studies did notfind a significantrelationship between salary and turnover (Jayaratne, Himle, & Chess,1991; Manlove & Guzell, 1997; Smith, 2005), suggesting otherorganizational factors may be influencing the relationship. It is includedin this analysis as a way to better understand its potential mediatingqualities in the relationship between the intervention and intention toleave. Salary and benefits is measured by asking “how satisfied are youwith your salary and how satisfied are you with your benefits?”

In the literature workload is defined as the amount of time it takesto complete all tasks related to job function (Jayaratne et al., 1991).Workload in this study is measured by 4 items such as “myworkload isreasonable”, “paperwork is reasonable” and “on call demands arereasonable”.

3.4. Dependent variables

Intention to leave is measured by one item asking “Have you lookedfor another job in the past year?” Participants who responded “yes”were coded 1 while those responding “no” were coded 0.

3.4.1. Independent variableParticipants whowere in a county that received theDT intervention

were coded 1 while those in comparison counties were coded 0.

3.5. Statistical analyses

All statistical analyses were conducted using SPSS for Windows,Version 17 and MPLUS version 4.1 (Muthén, & Muthén, 1998–2004).

Bivariate analyses, GLM Repeated Measures and structural equationmodeling (SEM) were completed. Bivariate analyses included Chi-square analyses, Pearsonproduct-moment correlation, and T-tests. GLMRepeatedMeasures is a procedure thatusesANOVA tomodeldependentvariables measured at multiple times (Tabachnick & Fidell, 2001). GLMRepeated Measures was performed to assess whether there weresignificant difference in the amount of DV change betweenwaves 1 and2 between the participants from the intervention agencies and theparticipants from the comparison agencies. In the GLM RepeatedMeasures the intervening variables were assessed as DVs. SEM wasconducted tomodel the pathways from theDT intervention through theintervening organizational variables to change in intention to leave.

3.6. SEM

The weighted least squares mean variance-adjusted estimator wasused to accommodate the modeling of ordered categorical andmultinomial variables of intention to leave. Model fit was assessedusing the chi-square test of model fit; the root mean square error ofapproximation index (RMSEA) (Brown & Cudek, 1993); the TuckerLewis Index (Tucker & Lewis, 1973) and the comparative fit index(CFI) (Bentler, 1990). CFI values above .95 indicate a good fit to thedata (Byrne 1994); RMSEA values below .08 indicate adequate modelfit (Ullman, 1996). TLI values greater than .9 are a good indication ofmodel fit. The statistical significance of the estimated parameters isexamined with z statistics and a .05 level of probability.

Change scores were computed for each of the organizationalvariables and used in the SEM. Change scores are often controversialdue to the potential for low reliability. However MacKinnon (2008)suggests that if the correlation is less than .5 on a measure at twopoints in time and the measure is generally reliable (α≥ .07), then thechange score will have acceptable reliability. In this data, thereliability of the measures is greater than or equal to .07, while allcorrelations between the two waves were less than .5, ranging from.285 to .474. Therefore, the researcher assumes reliability of thechange scores and uses them in the SEM.

Because intention to leave was a dichotomous variable, the changescore was calculated with the following formula (Y2−Y1) and thusincludes three ordinal categories: (1) participant identified intention toleave at post test but not pre test (decline), (0) participant did notchange intention to leave, and (−1) participant identified no intentionto leave at post test, but an intention to leave at pre test (improvement).

4. Results

4.1. Descriptive statistics

T-tests and Chi-square analyses were conducted to compare thedifferencebetween the intervention group and the comparison groupondemographics variables. Group means and percentages are reported inTable 1. No significant differenceswere foundon anyof the demographicvariables suggesting evidence of group equivalence pre-intervention. Adescription of the sample in its entirety is reported below.

4.2. Participant demographics

The average age of participants in the sample was 42. Eighty twopercentof the samplewas female and18%wasmale. Approximately98%of the sample was white, with 1% identifying themselves as AfricanAmerican and 1% as Hispanic. Front line supervisors represented 29% ofthe sample.

4.3. Career information

Five percent of the sample stated that casework was their firstcareer choice. The current job was the first full time job for 12% of the

Table 1Background characteristics (N=82).

Demographic characteristic Intervention (N=19) Comparison (N=63) Chi-square or t-statistic (df)

Female 79.0% 89.5% 1.05 (1)Male 21.0% 10.5%Race/ethnicity

Non-Hispanic White 100% 97.6% .63 (2)Black or African American 0% 1.2%Hispanic 0% 1.2%

SalaryUnder $30,000 36.8% 30.6% 2.64 (4)$30,001–$35,000 42.1% 29.0%$35,001–$45,000 15.8% 25.8%$45,001 or more 5.3% 12.9%

Work more than one job 21% 10% 1.8 (1)Supervisor (yes or no) 32% 29% t=.06Age 42.6 yrs 42.1 yrs t=.03

Career characteristicsCasework was first career choice 0% 6% t=1.3First full time job 5% 14% t=1.1“Step up” from last job 61% 64% t=.05Would make same decision again 65% 75% t=.76Years in current agency 9.2 yrs 10.1 yrs t=.24Years in current position 4.4 yrs 5.8 yrs t=1.2Years in the child welfare profession 10.2 yrs 11.0 yrs t=.19

No significant differences at or below pb .05.

Table 2GLM Repeated Measures illustrating means, standard deviations and group × timeinteractions.

Organizationalvariables

Design Team Comparison Wilk's Lambda F value

Mean SD Mean SD (group × wave)

Burnout and role clarityWave 1 2.9 .83 3.1 .57 4.59*Wave 2 3.2 .57 3.0 .57

Professional resources and trainingWave 1 2.8 .45 3.1 .50 13.81***Wave 2 3.2 .43 3.0 .52

Agency commitment and job satisfactionWave 1 3.1 .81 3.2 .67 6.62*Wave 2 3.6 .63 3.1 .63

WorkloadWave 1 2.4 .81 2.7 .76 3.41+Wave 2 3.0 .73 2.8 .67

Salary and benefitsWave 1 2.2 .95 2.4 .83 3.72+Wave 2 2.8 1.1 2.5 .91

RewardsWave 1 2.9 .52 2.9 .46 .054Wave 2 3.0 .43 3.0 .57

Intention to leaveWave 1 .58 .51 .62 .48 4.23*Wave 2 .32 .48 .68 .47

+pb .1; *pb .05; **pb .01; ***pb .001.

1392 J. Strolin-Goltzman / Children and Youth Services Review 32 (2010) 1388–1395

participants. Sixty three percent stated that this job was a step upfrom their last one, while 73% said that if asked again they wouldmake the same decision to take the job. The average time in thecurrent agency and in the child welfare profession were both reportedto be approximately 10 years. Participants reported the mean numberof years in their current position to be 5.5, while the median wasreported as 4.0 years.

4.4. Salary

Thirty two percent of the participants had a salary of less than$30,000; 32% reported a salary between $30–35,000; 24% had a salaryranging from $35–45,000; and 12% had a salary of more than $45,000.Twelve percent reported working more than one job.

4.5. Wave one organizational variables

No significant differences on any of the organizational variablesexcept for professional resources (t=5.02, (df=1); p=.028). Theintervention participants had a wave 1 mean score for professionalresources of 2.8, while the comparison group rated it significantlyhigher at 3.1 on a five point Likert scale of satisfaction. Actual meansfor both groups at each wave will be reported in the MANOVA table.

4.6. GLM Repeated Measures

Seven GLM Repeated Measures analyses were conducted todetermine the difference between the two groups over the 2 waveson the six intervening organizational variables (rewards, commit-ment, burnout, professional resources, salary and benefits, andworkload), and intention to leave. Group by wave interaction effectsare reported in Table 2.

As illustrated in the table, there were significant group by waveinteractions for three of the six organizational variables (Professionalresources, commitment, and burnout), while two of the organizationalvariables approached significancewith p values less than .10 (salary andbenefits and workload). There was no significant difference in thechange of perceptions of rewards from wave 1 to wave 2 between thegroups. A significant interaction was found on the dependent variableintention to leave. Only the findings with significant group by waveinteractions are described in the following sections.

4.7. Burnout and role clarity

Themean score for burnout and role clarity atwave one pre testwas3.1 for the comparison group and 2.9 for the DT group. At wave 2 postintervention, the mean scores increased to 3.2 for the DT group, butdecreased to 3.0 in the comparison group. A significant interactionbetweenwave and treatment conditionwas found in regards toburnoutand role clarity (F=4.59(1); p=.035). In sum, there was a significantlygreater improvement in burnout and role clarity fromwave 1 to wave 2for the treatment group than for the comparison group.

4.8. Professional resources and training

At wave 1 themean scores for the DT group and Comparison groupwere 2.8 and 3.1 respectively. At post test, the DT group improved

1393J. Strolin-Goltzman / Children and Youth Services Review 32 (2010) 1388–1395

their scores by almost 1/2 of a point while the comparison group'sscores declined by .1. The difference in the change between the groupswas significant (F=13.81(1); pb .001).

4.9. Agency commitment and job satisfaction

Themean score for agency commitment and job satisfaction atwaveonewas 3.2 for the comparison group and 3.1 for the DT group. At wave2, the mean scores increased by a half of a point to 3.6 for the DT group,but decreased to 3.1 for the comparison group. A significant interactionbetween wave and treatment condition was found (F=6.62(1);p=.012) suggesting that therewas a significantly greater improvementin agency commitment and job satisfaction from wave 1 to wave 2 forthe treatment group than for the comparison group.

4.10. Intention to leave

At wave 1, the comparison group had an intention to leave score of.62 while the DT group's score was .58. However by wave two theseresults were significantly different. At wave two, the comparisongroup's score had increased to .68while theDT's decreased by26% to .32(F=4.23 (1); p=.04).

4.11. Summary

Individually each of the findings from the Repeated MeasuresMANOVA informs us about what changes were made between thegroups over time, however it does not provide childwelfare experts andresearchers with information about the pathway from the interventionto intention to leave. Therefore a structural equation model wasconducted to better understand the mechanism by which theintervention affected intention to leave.

4.12. SEM model

The fit of the model in Fig. 3 was evaluated using the samplecovariance matrix as input and a weighted least squares mean variancewith a diagonal weight matrix (WLSMV) solution. This type of solutionis suggested with categorical and multinomial dependent variables(Muthén & Muthén, 1998–2004). The model is statistically over-identified and therefore the number of data points is less than the

Fig. 3. Pathways from design team to intention to leav

number of observed variables in the model (Ullman, 1996). A variety ofindices of model fit were evaluated. The overall chi-square was non-significant (X2=5.34 (df=4); p=.25). The Root Mean Square Error ofApproximation (RMSEA) was .06. Both the Comparative Fit Index (CFI)and the Tucker–Lewis Index (TLI)were greater than or equal to .95 (.97;.95). The Weighted Root Mean Square Residual (WRMR) was .51. Theindices uniformly point toward a good model fit. Inspection of themodification indices (using AMOS 17) revealed a relationship betweenprofessional resources and salary and benefits. This adjustment wasmade and rerun in MPLUS resulting in a better fit. Fig. 1 presents theparameter estimates for the model.

The model was able to account for 32% of the variance in intentionto leave, while the residual variance was .71. The variables in themodel accounted for 41% of the variance in commitment and jobsatisfaction, 21% of the variance in professional resources, 13% of lackof burnout and role clarity, and 8% of satisfaction with salary andbenefits. Standardized coefficients ranging from .54 to .21 showstrong to moderate relationships between the variables in the model.The strongest pathway in the model leads from the DT intervention tochange in professional resources.

In sum, the structural equation model illustrates how changeoccurred within the model. The design team first effected professionalresources. By increasing the amount of professional resources such asimmediate access to training, positive change occurred in perceptionsof supervisory support and recognition, agency commitment, salaryand benefits and role clarity and burnout. Further there was a directnegative relationship from professional resources to intention to leavewhich is also the strongest pathway to improving intention to leavechild welfare.

5. Discussion

Despite a small sample, the findings of this study are important forchild welfare researchers, supervisors and administrators for threereasons. First, it is one of only two intervention studies describing theeffects of an organizational intervention on intention to leave publicchild welfare. Second, the findings suggest that intervening at theorganizational levelwithDTs canpositively affectperceptionsof burnoutand role clarity, job satisfaction, agency commitment and decreaseintention to leave. Third, outcomes of the SEM suggest that professionalresources is a mediating variable between the DT intervention and

e (standardized coefficients are in parentheses).

1394 J. Strolin-Goltzman / Children and Youth Services Review 32 (2010) 1388–1395

intention to leave. Therefore agency administrators and supervisorsmayconsider implementing the DT intervention and focusing attention ondevelopingways forworkers to have access to professional developmenttraining, supervisor support and adequate technological resources.

There are several limitations to this study. First, this studyrepresents a small non-random sample (30%) of employees whoremained at the agencies from wave 1 to wave 2, and therefore it isdifficult to know how the intervention might have affected those wholeft the agency differently than those who remained. The small ruraland suburban sample and lack of diversity in the sample challengesthe generalizability of the results to urban child welfare organizations.Further, the participants worked at the agency an average of 9 yearswhich is greater than the general population of child welfare workers.However there were no significant differences between the twogroups on any of the demographic variables at wave one thereforesuggesting group equivalence.

Second, turnover was not measured at the individual level in thisstudy, although agency level turnover rates show a similar trend asthat depicted in this study' proxy measure of intent to leave (Strolin-Goltzman, Lawrence, et al., 2010), actual turnover would be a moreaccurate measure. Further, the study does not measure effects onactual outcomes for families and children. Future research shouldfocus on actual turnover and child and family outcomes.

The length of time from pre to post test was 3 years, during whichthe state and the consortium of child welfare agencies were activelyaddressing workforce issues. There were several co-occurring inter-ventions. However these interventions were delivered by the state toall counties participating in this study, and therefore should not resultin any threats to the validity of the intervention. Despite theselimitations, the study adds a new perspective to the organizationalliterature in child welfare by providing insight into how organiza-tional interventions such as the DT affect intention to leave. It also, forthe first time, identifies professional resources as a key variable in thechallenge to reduce workforce turnover.

Jensen, Dieterich, Brisson, Bender, and Powell (2010) report on thepaucity of intervention research among social work investigators andthe scarcity of peer reviewed publications describing outcomes fromintervention studies. This paper adds to the social work literature bydescribing outcomes of an organizational intervention in rural andsuburban public child welfare systems. The outcomes provideevidence that the DT intervention improves burnout and role clarity,job satisfaction and agency commitment, perceptions of salary andbenefits, and intention to leave public child welfare in rural andsuburban regions through improving access to professional resources.As Glisson and Hemmelgarn (1998) found, improvement of organi-zational factors affecting turnover is essential in obtaining qualityservice outcomes for children and families in the child welfare system.

References

Argyris, C., & Schön, D. (1978). Organizational learning: A theory if action perspective.Reading, MA: Addison Wesley.

Bednar, S. (2003). Elements of satisfying organizational climates in child welfareagencies. Families in Society, 84(1), 7−12.

Bentler, P. M. (1990). Comparative Fit Indexes in structural models. PsychologicalBulletin, 107, 238−246.

Brown, M. W., & Cudek, R. (1993). Alternative ways of assessing model fit. In K. A.Bollen, & J. S. Long (Eds.), Testing structural equation models (pp. 136−162).Newbury Park, CA: Sage Publications.

Byrne, B. M. (1994). Structural Equation Modeling With EQS And EQS/Windows, NewburyPark, Sage.

Cahalane, H., & Sites, E. (2008). The climate of child welfare employee retention. ChildWelfare, 87(1), 91−114.

Camp, S. D. (1994). Assessing the effects of organizational commitment and job satisfactionon turnover: An event history approach. The Prison Journal, 74(3), 279−305.

Caringi, J., Strolin-Goltzman, J., Lawson, H., McCarthy, M., Briar-Lawson, K., & Claiborne, N.(2008). child welfare design teams: An intervention to improve workforce retentionand facilitate organizational development. Research on Social Work Practice, 18,565−574.

DePanfilis, D., & Zlotnik, J. (2008). Retention of front-line staff in child welfare: Asystematic review of research. Children and Youth Services Review, 30(9),995−1008.

Dickinson, N., & Perry, R. (2002). Factors influencing the retention of specially educatedpublic child welfare workers. In K. Briar-Lawson, & J. Levy-Zlotnik (Eds.), EvaluationResearch in child Welfare: Improving outcomes through university–public agencypartnerships (pp. 89−103). New York: The Haworth Press.

Ellett, A., Ellis, J., Westbrook, T., & Dews, D. (2007). A qualitative study of 369 childwelfare professionals' perspectives about factors contributing to employeeretention and turnover. Children and Youth Services Review, 29(2), 264−281.

Flower, C., McDonald, J., & Sumski, M. (2005). Review of turnover in milwaukee countyprivate agency child welfare ongoing case management staff. Milwaukee, WI:Bureau of Milwaukee Child Welfare.

Gansle, K., & Ellett, A. (2002). Child welfare knowledge transmission, practitionerretention, and university–community impact: A study of title iv-e child welfaretraining. Journal of health and social policy, 15(3/4), 69−88.

Giffords, E. (2003). An examination of organizational and professional commitmentamong public, not-for-profit, and proprietary social service employees. Adminis-tration in Social Work, 27, 5−23.

Gleason-Wynn, C. H. (1999). A proposedmodel for predicting job satisfaction. Journal ofGerontological Social Work, 32(3), 65−79.

Glisson, C., Dukes, D., & Green, P. (2006). The effects of ARC organizational interventionon caseworker turnover, climate and culture in children's service systems. Childabuse and neglect, 30, 855−880.

Glisson, C., & Hemmelgarn, A. (1998). The effects of organizational climate andinterorganizational coordination on the quality and outcomes of children's servicesystems. Child Abuse & Neglect, 22(5), 401−421.

Gomez, R. J., Travis, D. J., Ayers-Lopez, S., & Schwab, J. (2010). In search of innovation:Child welfare recruitment and retention. Children and Youth Services Review, 32(5):664–671.

Hellman, C. M. (1997). Job satisfaction and intent to leave. The Journal of SocialPsychology, 137, 677−689.

Jayaratne, S., Himle, D., & Chess, W. (1991). Job satisfaction and burnout: Is there adifference? Journal of Applied Social Sciences, 15(2), 245−262.

Jensen, J., Dieterich, W., Brisson, D., Bender, K., & Powell, A. (2010). PreventingChildhood Bullying: Findings and Lessons From the Denver Public Schools Trial.Research on Social Work Practice, 32(5), 509−517.

Jones, L. (2002). A follow-up of a title iv-e program's retention rates in a public childwelfare agency. Journal of Health & Social Policy, 15(3/4), 39−51.

Koeske, G., & Koeske, R. (2000). The individual in the organization: The impact ofhuman service workers attributes on job response. In R. J. Patti (Ed.), The handbookof social welfare management (pp. 219−239). Thousand Oaks: Sage Publications.

Landsbergeris, P., & Vivona-Vaughan, E. (1995). Assessment of occupational stressintervention in a public agency. Journal of Organizational Behavior, 16(29–48).

Landsman, M. (2001, September). Commitment in public child welfare. Social ServiceReview, 387−419.

Lawson, H., Anderson-Butcher, D., Petersen, N., & Barkdull, C. (2003). Design teams aslearning systems for complex systems change: Evaluation data and implications forhigher education. Journal of Human Behavior in the Social Environment, 7(1/2),159−179.

Lawson, H., McCarthy, M., Briar-Lawson, K., Miraglia, P., Strolin, J., & Caringi, J. (2006). Acomplex partnership to optimize and stabilize the public child welfare workforce.Professional Development, 9(2/3), 122−139.

MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. NewYork:Taylor and Francis Group.

Manlove, E., & Guzell, J. (1997). Intention to leave, anticipated reasons for leaving and12 month turnover of child care staff. Early Childhood Research Quarterly, 12(145–167).

Maslach, C., & Jackson, S. E. (1986). TheMaslach Burnout Inventory, (2nd ed.). Palo Alto,CA: Consulting Psychologists Press.

McGowan, B., Auerbach, C., & Strolin-Goltzman, J. (2009). Turnover in the child welfareworkforce: A different perspective. Journal of Social Service Research, 35(3), 228−235.

Mor Barak, M., Levin, A., Nissly, C., & Lane, J. (2006). Why do they leave? Modeling childwelfare workers' turnover intentions. Children and Youth Services Review, 28(5),548−577.

Muthén, L. K., & Muthén, B. O. (1998–2004). Mplus user's guide, 3rd ed. . Los Angeles:Muthén & Muthén.

National Council on Crime and Delinquency (2006). Relationship between staffturnover, child welfare system functioning and recurrent child abuse. RetrievedMay 16, 2006 from http://www.cornerstones4kids.org/images/nccd_relation-ships_306.pdf.

Okamura, A., & Jones, L. P. (1998). Re-professionalizing child welfare services: Anevaluation of a title iv-e training program. Paper presented at the Annual ProgramMeeting of the Council on social Work Education, Orlando, Florida.

Poulin, J. (1994). Job task and organizational predictors of social worker job satisfactionchange: A panel study. Administration in Social Work, 18(1), 21−39.

Reagh, R. (1994). Public child welfare professionals — Those who stay. Journal ofSociology and Social Welfare, 21(3), 69−78.

Scannapieco, M., & Connell, K. (2003). Do collaborations with schools of social workmake a difference for the field of child welfare? Practice, retention and curriculum.Journal of Human Behavior in the Social Environment, 7(1/2), 35−51.

Scannapieco, M., & Connell-Carrick, K. (2007). Child welfare workplace: The state of theworkforce and strategies to improve retention. Child Welfare, 86(6), 31−52.

Seifert, K., & Jayaratne, S. (1991). Job satisfaction, burnout and turnover in health caresocial workers. Health & Social Work, 16(3), 193−203.

Senge, P. (1990). The fifth discipline: Art and practice of learning organizations. NewYork: Doubleday/Currency.

1395J. Strolin-Goltzman / Children and Youth Services Review 32 (2010) 1388–1395

Smith, M. K. (2001). Chris Argyris: theories of action, double-loop learning andorganizational learning. The encyclopedia of informal education, www.infed.org/thinkers/argyris.htm. Last update: September 7, 2009.

Smith, B. (2005). Job retention in child welfare: Effects of perceived organizationalsupport, supervisor support and intrinsic job value. Children and Youth ServicesReview, 27(2), 153−169.

Strolin-Goltzman, J. S., Kollar, S., & Youth in Progress (2010). Consumer opinions ofworkforce retention, turnover and recruitment: Voices of youth in foster care.Social Work, 55(1), 447−453.

Strolin-Goltzman, J., Lawrence, C., Auerbach, C., Caringi, J., Claiborne, N., Lawson, H.,et al. (2010). Design Teams: A promising organizational intervention for improvingturnover rates in the child welfare workforce. Child Welfare, 88(5), 149−168.

Strolin-Goltzman, J. S., McCarthy, M., Smith, B., Lawson, H., Bronstein, L., & Caringi, J.(2008). A comparison study of intention to leave among public child welfaresystems with high and low turnover rates. Child Welfare, 87(4), 125−143.

Tabachnick, B., & Fidell, L. (2001). Using multivariate statistics. Needham Heights, MA:Allyn and Bacon.

Tett, R., & Meyer, J. (1993). Job satisfaction, organizational commitment, turnoverintention, and turnover: Path analysis based on meta analysis findings. PersonnelPsychology, 46, 259−293.

Tucker, L. R., & Lewis, C. (1973). The reliability coefficient for maximim likelihood factoranalysis. Psychometrika, 38, 1−10.

Ullman, J. B. (1996). Structural equation modeling. In B. G. Tabachnick, & L. S. Fidell(Eds.), Using multivariate statistics (pp. 709−819)., Third Edition. New York, NY:HarperCollins College Publishers.

US General Accounting Office (2003). Child welfare: HHS could play a greater role inhelping child welfare agencies to recruit and retain staff. Washington, DC: USGeneral Accounting Office.

Vinoker-Kaplan, D. (1991). Job satisfaction among social workers in public andvoluntary child welfare agencies. Child Welfare, 70(1), 81−91.

Yankeelov, P., Barbee, A., Sullivan, D., & Antle, B. (2009). Individual and organizationalfactors in job retention in Kentucky's child welfare agency. Children and YouthServices Review, 31(5), 547−554.