Citizen Review Panel - PCA Arizona · 1912- Federal government established Children’s Bureau 1960- Child maltreatment received National attention Professional interest by physicians

Citizen Review Panel

Citizen Review Panel’sImpact and Influence on Child Welfare Policy, Practice and System

Citizen Review Panel’s

Presented By: Sandra Lescoe, MSWLois Sayers, PhD Janet Cornell, Cyleer Love

College of Public Service & Community Solutions

Our Discussion Today Background and Origin of Child Welfare SystemOrigin of Citizen Review Panels (CRP)s, Purpose of

CRP’s and Federal requirements of CRP’SImportance of the relationship between child

welfare agencies and CRPs Structure of Arizona CRP’s and how they are

working to impact the child welfare systemPreliminary results from an ongoing evaluation of

the three Arizona CRPs.

college of public service & community solutions

Questions

When you think about child welfare, who should be involved with these children and families?

What is reasonable to expect from the child welfare agency regarding the safety, care and well-being of vulnerable children in your community?

Do you think citizen involvement in government is useful or makes a difference?


What is the Child Welfare System?


Background and Evolution of Child Welfare System 1800- Form of child welfare begins 1912- Federal government established Children’s

Bureau 1960- Child maltreatment received National

attentionProfessional interest by physiciansMedia interestsPublic interest increasesCongress becomes more engaged in what was

historically a local or state level issue


The Federal Government Assumes a Leadership Role in Public Child Welfare1974 The Child Abuse Prevention and Treatment Act (CAPTA) enacted First national legislation addressing child maltreatment.

States were to way to identify and address child maltreatment emergence of State child welfare agencies private foundations advocacy and citizen review

Federal funding in support of prevention, assessment, investigation, prosecution, and treatment.

Grants to public agencies and non profit organizations for programs and projects.

Identified Federal role in supporting research, evaluation, technical assistance, and data collection activities; and

Established a minimum standard definition of child abuse and neglect


CAPTA also established a structure which expanded citizen participation In 1974 CAPTA Court Appointed Special Advocate programs were established

as an independent, citizen voice in court for specific children.

In 1980 federal legislation was passed: Foster Care Review Boards look at individual cases for

children in out-of-home every 6 months


The Federal and State role in the Child Welfare SystemThe federal and state government continued to assume more responsibility for administration and funding

Child abuse reporting laws and interventions created other public concerns Increase of children in long term foster care Family rights and child removals Children left or returned to unsafe environments


The Origin of Citizen Involvement in Public Child WelfareFederal and state government continued to implement new laws, policies and practice Driven by political, social and national climate

Greater demand for accountability and transparency of the child welfare system

Congress responded with an intent to replace extensive federal regulations and fines to states


The Origin of Citizen Involvement in Public Child Welfare and Congressional MotivesAlthough there was some citizen oversight establishedHowever, Congress responded by increasing citizen involvement with the following premise

“By allowing the Panels to have complete access to child protection cases, by requiring Panels to publicize their findings, and by requiring states to respond to criticisms and recommendations of the Panels, the Committee intends to subject states to public criticism and political repercussion if they fail to protect children”

~House report 104-081


Creation of Citizen Review Panels1996 CAPTA amended by Congress States required to establish minimum of 3 CRPs by 1999 if they continued

to receive CAPTA grants

Primary responsibility and purpose of the panels was to evaluate the extent to which the state and local child protection service agencies are discharging their child protection responsibilities and compliance with Child Abuse Prevention and Treatment Act (CAPTA) requirements Other criteria the panel considers important such as coordination with

foster care and adoption programs review of child fatalities and near fatalities; and

State CAPTA plan (basically any child protective services)


Federal Requirements of CRP’sThe 1996 CAPTA mandate required CRP’s to Be composed of volunteer members who are broadly

representative of the community in which the panel is established, and Include members with expertise in the prevention and

treatment of child abuse and may include adult former victims of child abuse or neglect.

Meet not less than once every 3 months. Maintain confidentiality


Amendments to CAPTA createdAdditional Requirements2003 CAPTA Reauthorization requires Panels to review and evaluate PRACTICE as well as policy and procedure

Develop a means for public comment and prepare and present an annual report which details their activities and recommendations

Child welfare agency is to respond in writing to the annual report within six months

2010 CAPTA Reauthorization of Required a study and report to Congress on the effectiveness of CRPs Panels may include adults who are former victims of child abuse and

neglect


CRP’s can Review, Evaluate and Examine any of the Following Parts of the Child Welfare System

Intake and initial screening Investigation or assessment Case determination Service planning,

implementation, and monitoring

Utilization of technology to determine outcomes

Case closure Crisis intervention;

Emergency placement; Family stabilization

Coordination of services Staff qualifications, training

and workload Review hard copies of case

information


CRP Recommendations Can address elements of

policy or practice that was not followed or unclear

Suggest modifying a policy which was followed but was identified as a concern or produced a bad outcome

Address issues not addressed in policy

Systemic issues

Should be focused and specific

Should deal with something that is within the agency’s control

Be factual to address policy, procedure or practice (avoid changes to Federal policy)

Be derived from the panel’s work during the year.



In an era of citizen participation, what is the role of the state child welfare agency and other stakeholders in the child

welfare system?

Collaboration

What impacts good and bad collaboration? Competing priorities Distrust Mindset Agreeing on what constitutes safety,

permanency, and well-being


Contentious relationship of distrust between the public and child protection agency does not improve outcomes for children and families


Child Welfare Mindset The public does not understand

our job or our workload Public is out to get us “ Those people (Citizen Review

Panel members) need to get a clue. They don’t know the first thing about what we do, but they want to judge us. They should just mind their own business!” ~ Child Welfare Administrator


Public Mindset often focuses on blame rather than problem-solving


Citizens have trouble understanding the complexities and bureaucracy of state agencies.

There is civic apathy

Public is under informed or misinformed

Their perspective is shaped by what they read in the paper or what they think is the reality

The government is constantly asking for more money

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&docid=H3jLnDLPeNOf-M&tbnid=04-PRd6GPcK-jM:&ved=0CAUQjRw&url=http://article.wn.com/view/2014/04/22/Some_reforms_of_Child_Protective_Services_could_put_children/&ei=tjiSU8mFENOGyAT1mYKQAg&bvm=bv.68445247,d.aWw&psig=AFQjCNFj_IQAyl0SjGuWUHJe3bH5FPf7jw&ust=1402177885827577

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&docid=H3jLnDLPeNOf-M&tbnid=04-PRd6GPcK-jM:&ved=0CAUQjRw&url=http://article.wn.com/view/2014/04/22/Some_reforms_of_Child_Protective_Services_could_put_children/&ei=tjiSU8mFENOGyAT1mYKQAg&bvm=bv.68445247,d.aWw&psig=AFQjCNFj_IQAyl0SjGuWUHJe3bH5FPf7jw&ust=1402177885827577

Do you think citizen involvement in government is useful or makes a difference?

Myths One person can’t make a difference You have to be a real expert and policy guru Only full-time lobbyists have an impact Somebody else will do it You have to be rich or well-known


New Era and Evolution of Citizen Involvement

Building trust is important to promoting good outcomes

Different perspectives are valuable Spend less energy being reactive and

more time being proactive System can work more efficiently and

effectively


Why is Citizen Involvement Important It prevents the child welfare agency from becoming

a “system unto itself” It moves us toward “community based” protection of

children versus the child protection system having sole responsibility

Citizens can be advocates for the agency Citizens are educated about what is really

happening with child abuse and neglect It’s democracy in action


Specific advantages of a new era of citizen involvement From watch dog to partnership Greater impact achieved by bringing many

groups/constituencies together around a coordinated message

Power in numbers Reduce competing messages delivered to policy

makers about a particular issue; demonstrates consensus

Unbiased citizen input


Arizona Citizen Review Panel Program

1999 3 CRP created in AZ Facilitated by DHS Funded by DCS

2008 ASU facilitates and partners with DCS Accomplishments to date Yearly reports including findings and recommendations Attended national conferences Provided trainings on citizens in child welfare

Current Activities Strategic planning Evaluation


Evaluation of Arizona CRP

• Two main evaluation questions1. Are CRP activities aligned with CAPTA mandates?

What is the vision and mission of the CRP members in Arizona?• Is the CRP aligned with core CAPTA values?

• Accountability• Transparency• Public participation and awareness

2. Are these mandated activities impactful?• Do panel members perceive activities as impactful?• What kind of activities do panel members see as impactful?• What are the issue areas where they have had the most impact? The least?• How impactful are CRP recommendations?


Evaluation of Arizona CRP

• Two main evaluation questions1. Are CRP activities aligned with CAPTA mandates?

What is the vision and mission of the CRP members in Arizona?• Is the CRP aligned with core CAPTA values?

• Accountability• Transparency• Public participation and awareness

2. Are these mandated activities impactful?• Do panel members perceive activities as impactful?• What kind of activities do panel members see as impactful?• What are the issue areas where they have had the most impact? The least?• How impactful are CRP recommendations?


Methodology• Survey of Panel members statewide

• Response rate above 50%(N=17)• Scaled items and Open-ended questions• Likert scale from 1-10

• Role relative to CAPTA mandates and values• Impact of panel activities • Impact of panel on child welfare issues

• Recommendations Analysis• Recommendation Ratings tool- 7 years of CRP recommendations

• Theme (policy, professional development, procedure, practice)• Implementation Complexity scale- Is collaboration required?

• Type-external/internal• Order-none, sequential, reciprocal• Amount-none, one, two or more

• SMART Criteria


Methodology• SMART Criteria for recommendations

• Specific• Measurable• Attainable• Realistic• Time Specific


What is the vision of CRP?

• Common Themes• Advocate• Case Reviews• Community Participation• Developing Recommendations• Improve DCS• Legislative Change• Support Children• Support DCS• Support families


What is the mission of CRP in AZ?

• Common Themes• Advocate• Case Reviews• Community Participation• Developing Recommendations• Improve DCS• Legislative Change• Support Children• Support DCS• Support families


Is the Arizona CRP aligned with CAPTA values?

• Accountability• Transparency• Public Participation• Public Awareness


Transparency• General consensus was that this CAPTA value was extremely

important. • Several individuals attributed transparency to being a state

mandated panel and being open to the community. • Panel members say:

“Any child welfare system should be transparent. It is after all publicly funded.”“We are a community advisory council and the community should know what

the group is doing.”“State panel that provides feedback on state child welfare agency shouldn't have

nothing to hide”“Transparency will enhance our ability to better serve the community that we

represent.”


How important is it for the CRP to be transparent, where it is easy for the public to see what CRP is doing?


This CAPTA value was rated extremely important with a mode of 10 and a standard deviation of 0.86.

Accountability• General consensus was that it is extremely important for CRP to be

accountable.• Participants associated this value with their federal mandate and

being publicly funded. • Panel members say:

“If we are a committee created by a federal mandate we need to be held accountable, the budget related to CAPTA.”“Any public funded entity needs to be accountable for its actions and spending.” “It is a mandated activity we are accountable to the public, the community needs to trust the work that is being done.”


How important is it for the CRP to be accountable, to answer for responsibilities assigned through mandates?

This CAPTA value was rated extremely important with a mode of 10 with some variation, showing a standard deviation of 1.41.


Public Awareness• Overall, this value was rated important but had significant

variation amongst respondents. • Some respondent’s felt it was important but raising public

awareness should not be a primary goal of CRP. • Panel members say:

“Public awareness, training, and education should be a component of core staff who work to ensure children’s safety. Perhaps more state funding is needed for this function.”“Don’t work in the field, don’t see CRP as a fundraiser or advocate, but see CRP as an educator”“There are a lot of other people that can raise public awareness (i.e., the Children’s Action Alliance) –Shouldn’t be primary goal.”“Public awareness is something to strive for but wasn’t indicated as a necessity for participation.”


Public Awareness continued• Others felt this CAPTA value was extremely important and was the

key to progress. • Panel members say:

“The public does not understand the child welfare system we need to be advocating and rising awareness on these issues.”

“Public awareness and education is the key to preventing child maltreatment.”

“If the public isn’t aware on how to assist children nothing will change because no one is held accountable.”


How important is it for the CRP to raise public awareness on child welfare issues?

This CAPTA value was overall rated extremely important with a mode of 10, but there was significant variation in responses, showing a standard deviation of 2.03.


Public Participation • Overall, this value was rated important but had some variation amongst

respondents.• Question was misunderstood by participants. • Respondents scores were based on how CRP was doing in promoting

public participation and not the importance of the value.• Limitation overlooked by evaluators• Panel members say:

“The public does not understand the child welfare system we need to be advocating and rising awareness on these issues.”

“Public awareness and education is the key to preventing child maltreatment.”

“If the public isn’t aware on how to assist children nothing will change because no one is held accountable.”


How important is it for the CRP to be an effective mechanism to promote/encourage public participation, in child welfare ?

This CAPTA value was overall rated extremely important with a mode of 10, but there was significant variation in responses, showing a standard deviation of 2.03.


Do panel members perceive activities as impactful in Arizona?• Survey measured both personal and panel impact• Rating scale from 1-10 (1=not impactful at all-10=extremely

impactful)

• Average score=6• Personal and panel perceptions were aligned • Comments suggested state politics and agency

reconfigurations may be potential sources of this perceptions


What kind of activities do panel members see as impactful?• Mandated and non-mandated activities • Activities either suggested or enacted by panel members• Likert scale from least impactful to most impactful 1-10


Impact on Activities

• Most Impactful– Attending panel meetings– Learning about child welfare

issues– Meeting with child welfare

system partners– Contributing to organizations

that advocate for children– Developing recommendations

for DCS– Advocating for child welfare

issues

• Least Impactful – Writing letters– Organizing around child

welfare issues– Contributing to the CRP

annual report

CRP Issue Areas

• What child welfare issues do panel members see as areas where they have been most impactful? Least impactful?• Issues derive from child welfare literature • Ten point Likert scale 1-not impactful 10-most impactful


Impact on Issues

• Most Impactful– Child Maltreatment– Children’s mental/behavioral

health needs– Congregate Care– Policy and Procedure– Substance Exposed Newborns

(SEN)– Staff training– Placement Stability– Adoption*– Foster care/Foster families*Variation among responses; median/mode

• Least Impactful – Children’s Educational Needs– Child Fatalities– Indian Child Welfare Act

(IWCA)– Kinship Care*– Permanency– Substance Abuse*– Medically Assisted

Treatment*– Case Planning

Key Mandated Activity: Developing Recommendations

• Recall panel members recommendations identified as impactful

• Empirical analysis of recommendations • 7 years of recommendations 2007-2014• 48 total recommendations

• Key questions• What are the central themes of the recommendations?• Are there barriers to implementation?


Analysis of past Recommendations

• Major themes in CRP recommendations overtime:• Create DCS database to improve access to civil and

criminal courts and communication with law enforcement agencies (2007-2008),

• Improve detailed documentation procedures (2008-2011),

• Collaborate with other agencies (2009-2014), • Improve training to DCS employees (2010-2014).


Analysis cont.

Classification of recommendations:Procedural (64.9%)Practices (39.6%)Professional development (12.5%)Policy (8.3%)


Analysis cont.• Informal review showed recommendations:

• Repeated within same year and from year to year

• Appeared complex and difficult to implement• Actual implementation varied

• Formal review of recommendations• Structured instrument• SMART Criteria


Complexity Rating ToolHow difficult is the recommendation to implement?

• Type of recommendation:• External actors must be engaged (41.7%)• Only actors internal to the agency must be engaged (58.3%)

• Amount:• No collaboration required (54.2%)• One other department/agency must be engaged (10.4%)• Two or more department/agencies must be engaged (35.4%)

• Level:• None: each actor can implement separately (79.2%)• Sequential: one actor must implement first (4.2%)• Reciprocal: actions are contingent on other actions (16.7%)


SMART CriteriaSpecific-Measurable-Attainable-Realistic-Time-Specific

• Of all 48 recommendations from 2007-2014, none met all five of the SMART criteria. However, 75% of the recommendations met three (33.3%) or four (41.7%) of the criteria.


What steps can panels take to improve the effectiveness of their recommendations?

Citizen Review Panels are most effective when they make recommendations that…• Are specific, measurable, actionable, realistic and contain a time-

frame (SMART).• Require single actor strategies rather than multi-actor strategies.• Are not contingent on other actions.• Have a balance of affecting Policy, Procedure, Practice, and

Professional Development


Summary and Preliminary Conclusions

Arizona Citizen Review Panels • Understand and value their CAPTA mandated and responsibilities• See many activities as critical; others no• See impact on some child welfare issue areas more than others• Promote Key CAPTA values• Tend to make procedural recommendations over policy• Tend to make recommendations that are specific, attainable and

realistic • May benefit from

• Balancing recommendations between policy and procedure• Utilizing SMART criteria to develop more measurable and time-

specific recommendations


Questions?


Documents

Citizen Review Panel - PCA Arizona · 1912- Federal government established Children’s Bureau 1960- Child maltreatment received National attention Professional interest by physicians