Richard P. Barth & Terry V. ShawOctober 23, 2014

CSWE APMTampa, Florida

Implications of CFSR 3 for IVE Programs

Purpose of the Presentation

• To stimulate thinking and discussion about:– What is the meaning of CFSR 3?– Does it create new opportunities for SSW

University-Agency Partnerships?– Does it suggest changes in SW Education?

CFSR 3 Guiding Principles• States and the Federal Government are unified in a

commitment to safety, permanency, and well-being for the children served by the public child welfare system.

• The CFSRs have led to the identification of opportunities for:– 1. program improvement – 2. improvement in outcomes for children and families.

• Further improvements in the CFSR and in the related CQI processes are critical to the continued CWs advances– Scientifically valid– Feasible– Understandable– Meet All Federal Requirements

National Standards (Issues)Proposal

InputAggregate

administrative data on safety and

permanency

Applied Methodology Multi-level Model

Risk Adjusted

OutputSet National Standard

at national averagefor all measures

Points of Agreement

Proposal is reasonable

Areas for Discussion

Recommendation to endorse CB proposal; or

Compute State-specific rates and calculate a national

average rate

Recommendation to endorse CB proposal; or

Endorse with additional factors; or; Do not risk adjust

National Standards

CFSR Round 2 CFSR Round 3 Differences

Of all children in foster care during the reporting period, what percent were not victims of substantiated or indicated maltreatment by a foster parent or facility staff member?

Of all children in foster care during a 12-month period, what is the rate of victimization per day of foster care? (most likely reported in a rate per 100,000 days)

Child maltreatment while in care is calculated by directly linking data from NCANDS (numerator) with the AFCARS file (denominator) using CHILDID.

All instances of maltreatment will be included regardless of type of perpetrator (including parents).

Safety Outcome 1: Children are, first and foremost, protected from abuse and neglect.

Safety Performance Measure 1: Maltreatment in Foster Care

Re-Report of Maltreatment (Issues)

ProposalPoints of

AgreementAreas for Discussion

Of all the children who were victims of substantiated or indicated abuse or neglect during the first 6 months of the reporting period, what percent experienced another screened-in report with a 12-month period?

It is important to include a measure of repeat maltreatment. No support for the measure as proposed “…..getting the measure right should trump concerns over current data capacities.”

Recommendations for appropriate measure(s)? Recommendations around data availability, quality and testing proposed measures?

National Standards

CFSR Round 2 CFSR Round 3 Differences

Of all children who were victims of substantiated or indicated maltreatment allegation during the first 6 months of the reporting period, what percent were not victims of another substantiated or indicated maltreatment allegation within a 6-month period?  

Of all children with a screened-in report of alleged maltreatment in a 12-month period, what percent had another screened-in report within 12 months of their initial report?

Lengthens the timeframes being considered (from 6 months to 12 months)

increases the scope to include all screened-in reports and not just Substantiated or Indicated.

Safety Outcome 1: Children are, first and foremost, protected from abuse and neglect.

Safety Performance Measure 2: Re-report of Maltreatment

National Standards

CFSR Round 2 CFSR Round 3 Differences

Composite 1.3: Of all children entering foster care for the first time in a 6-month period, what percent discharged to reunification (or live with relative) within 12 months of entering foster care or by the time they reached 18?

Of all children who enter foster care in a 12 month period, what percent discharged to permanency within 12 months of entering foster care?

Expanded the exits from reunification alone, to permanency (reunification, live with relative, adoption or guardianship). Expands base time from 6 months to 12 months and includes all entries, not solely first entries.

Permanency Outcome 1: Children have permanency and stability in their living situations.

Permanency Performance Measure 1: Permanency in 12 months for children entering foster care

National Standards

CFSR Round 2 CFSR Round 3 Differences

Composite 3.1: Of all children in foster care on the first day of a 12-month period who had been in foster care (in that episode) for 2 or more years, what percent discharged to permanency within 12 months of the first day or by the time they reached 18?

Of all children in foster care on the first day of a 12-month period who had been in foster care (in that episode) for 2 or more years, what percent discharged to permanency within 12 months of the first day?

No differences.

Permanency Outcome 1: Children have permanency and stability in their living situations.

Permanency Performance Measure 2: Permanency in 12 months for children in foster care for 2 years or more

National Standards

CFSR Round 2 CFSR Round 3 Differences

Composite 1.4: Of all children discharged from foster care to reunification or live with a relative in a 12- month period, what percent re-entered foster care in less than 12 months from the date of discharge?

Of all children who enter foster care in a 12- month period and discharged within 12 months to reunification, live with relative, or guardianship, what percent re-entered foster care within 12 months of their date of discharge?

The measure is now based on an entry cohort instead of an exit cohort.

The denominator has been expanded and now includes Guardianship as well as reunification, and live with relative.

Permanency Outcome 1: Children have permanency and stability in their living situations.

Permanency Performance Measure 3: Re-entry in 12 months

National Standards

CFSR Round 2 CFSR Round 3 Differences

Composite 4.1: Of all children served in foster care during the 12- month period, what percent had two or fewer placement settings?

Of all children who enter foster care in a 12- month period, what is the rate of placement moves per day of foster care? (most likely expressed as a rate per 1,000 days)

The measure is now based on an entry cohort instead of all children served and in care for a period.

All placement moves will be counted during a period rather than just identification of stable vs. not stable.

Permanency Outcome 1: Children have permanency and stability in their living situations.

Permanency Performance Measure 4: Placement stability

Risk Adjusted Analyses

Performance Goals and Thresholds (issues)

Proposal Points of Agreement

Areas for Discussion

InputState specific data on the most recent 3 years of performance

Applied MethodBootstrapping to set baselineEstimate grand mean and standard deviation

Out PutImprovement Factor Grand Mean/Grand Mean + 2SD Goal CalculationBaseline * Improvement FactorThreshold CalculationBaseline – 2SD

Applied MethodMin. & Max. Goals & ThresholdsMin. = Top of bottom 5th

Max. = Bottom of top 5thth Use Companion Measures

OutputMeasurement at 3 & 5 years for some measures and 4 & 5 years for one measurement

Bootstrapping is reasonable

Recommendation to incorporate trend consideration

Recommendation to adjust calculations to 1 SD

Recommendation to set at 5th and 45th State performance levelCompanion

Measures are reasonable

Measurement timeframes are

reasonable

Case Review Standards (issues)

Proposal Points of Agreement

Areas for Discussion

Case Review determines substantial conformity for 24 items that assess case-specific practices in both in-home and foster care cases.

A single standard is applied to all Case Review items.

Case Review is an important component of a CQI system. A single standard should be applied.

The Panel suggested that Case Review should not be used to determine Substantial Conformity. How would Case Review results be used to meet legislative objectives to measure State performance and improvement? What should the standard be?

Case Review Standards• In prior rounds onsite case reviews were conducted during

a one-week visit by external reviewers.• In round 3, qualifying states may conduct their own reviews

using a revised CFSR onsite review instrument. States have to meet three criteria in order to use their own case review process and be able to meet the following time lines.– Must use a sample period of 4/1 to 9/30 of year prior to review

year.– Conduct reviews from 4/1 to 9/30 of the year of review and

provide information to the Children’s Bureau– Report results no later than 11/15 of the year of review to the

Children’s Bureau

Case Review Standards

• Criterion 1 - The states operates an internal case review process at least annually that assesses statewide practice performance for the key child welfare areas using a uniform sampling process and methodology

• Criterion 2 - The state has a process in place for ensuring accurate and consistent case review ratings.

• Criterion 3 - The state uses the federal onsite review instrument and its instructions using the sample and method established above to collect data to be used for the initial determination of conformity.

Sampling (issues)

Proposal Points of Agreement

Areas for Discussion

Sample to allow for statistical inferences

Combined foster care and in-home case sample

Simple random sampling design

Statewide with options for geographic representation

Option to reduce sample size for ongoing program improvement

No support for a sampling approach that could eliminate children in rural areas No support for reducing sample size for ongoing program improvement

Should sample size be increased to allow for statistical inferences for foster care and in-home cases?

Is the proposed sample size feasible?

Is the proposed random sampling design feasible?

What are alternative sampling strategies?

Panel suggestion that targeted reviews could have a role in ongoing improvement monitoring.

Sampling

• Random selection; or a stratified selection representative of the states demographics and includes the largest metropolitan area.

• Minimum of 65 cases served during the sample period (40 foster care cases; 25 in-home cases, including Alternative Response);

• Out of home cases must be consistent with the listings reported to AFCARS during the period.

• In home cases (including AR) must have been opened for at least 45 consecutive days during the sampling period .

• The state must be able to consistently address document case elimination.

Standards for Systemic Measures (issues)

Proposal Points of Agreement

Areas for Discussion

The State’s CQI system would identify sources for quantifiable data for items TR1 – TR4 and FAP 2.

A single standard is applied to all systemic capacity items.

The prior method of assessing systemic measures was subjective.

The evidence connecting the systemic factors to family outcomes is limited.

What is the recommended standard and why?

90%75%

Should the standard be different for systemic capacity items assessed through case review?

Implications

1. The measures are simpler which will help in explaining what they mean to all the various audiences - this should make it clearer to develop strategies to influence outcomes.

2. Opportunities for SSWs to increase their role regarding development and implementation of CQI processes

3. Opportunities to work with state partners on the importance of understanding the trajectory of children into and through the service systems

4. Greater focus on CQI in social work education5. Critiquing CFSR 3 or Designing CFSR 4 could become a useful

activity for a research or CWS course