33
FRANKLIN COUNTY ADULT PROBATION Quality Assurance/Continuous Quality Improvement

Quality Assurance/Continuous Quality Improvement

Embed Size (px)

Citation preview

Page 1: Quality Assurance/Continuous Quality Improvement

FRANKLIN COUNTY ADULT PROBATION

Quality Assurance/Continuous Quality Improvement

Page 2: Quality Assurance/Continuous Quality Improvement

We supervise approximately 6,000 offenders a year

We have 60+ supervision officers, a PSI unit, a pre-trial unit and programming staff

We have 7 managers and 12 seniors

About Our Department

Page 3: Quality Assurance/Continuous Quality Improvement

We had some checks and balances in place, but no cohesive system

Officers and managers were attempting to adhere to evidence based practices, but again there was no systematic way to evaluate how well we were doing

In 2013, we received funding to add two Quality Assurance positions in our department

Where We Started

Page 4: Quality Assurance/Continuous Quality Improvement

Caseload Audits

Observation of Office Visits

Offender Surveys

Criminal Thinking Surveys

Changes We Have Made

Page 5: Quality Assurance/Continuous Quality Improvement

Each manager was responsible for doing audits on members of their team

Most audit questions were related to documentation in case files

Audit results were reviewed with the individual officer

Managers review audits on a random basis and can be assigned to any officer in the department

Audit questions were updated to be evidence based and examine how officers are interacting with offenders

Audit results tracked for department

Caseload Audits

Page 6: Quality Assurance/Continuous Quality Improvement

Audit procedures were developed for the department (how often audits would be completed, what would happen with audit results, what happens if corrections need made, etc)

Audit guidelines were also distributed to officers explaining what managers would be looking for in each section of the audit

Caseload Audits

Page 7: Quality Assurance/Continuous Quality Improvement

Audit results were collected and shared with staff

Caseload Audits

Updated ORAS

Correct Tool Timely man-ner

Accurate As-sess

Sup Approp Supervision Plan

October-Decem-ber 2013

0.924528301886792

0.905660377358492

0.760869565217391

0.966666666666667

0.865384615384616

0.461538461538462

January-March 2014

1 0.966666666666667

0.924528301886792

0.909090909090909

0.96296296296296

0.600000000000001

April-June 2014

0.938271604938273

0.950000000000001

0.840579710144928

0.957446808510638

0.941176470588236

0.692307692307693

July-Sep-tember 2014

1 0.9672 0.859600000000002

0.9286 0.918 0.861100000000001

10.00%30.00%50.00%70.00%90.00%

110.00%

Section IIAssessment Information

Page 8: Quality Assurance/Continuous Quality Improvement

Allows us to identify department strengths

Helps us identify areas where officers are struggling

Staff can see the results of their work and get an idea of how they compare to the rest of the department

Benefits of Audit Data

Page 9: Quality Assurance/Continuous Quality Improvement

Prior to implementing EPICS, office visits were never routinely observed and officers got very little feedback about how they were interacting with offenders

Officers now turn in audio tapes of office visits and get very detailed feedback from EPICS coaches

Results from tapes are tracked and progress documented

Observing Office Visits

Page 10: Quality Assurance/Continuous Quality Improvement

Barriers to Implementing EPICS

Department Size Officer Resistance

Caseload Size

Consistency among coaches

Page 11: Quality Assurance/Continuous Quality Improvement

Department Size Trained officers in three different waves

Needed to recruit a large number of coaches

Officers turn in a tape every month, but get feedback on one tape over a two month period

Page 12: Quality Assurance/Continuous Quality Improvement

Officer Resistance EPICS supported by chief and management

team

Coaches emphasize positive steps and find even “small victories” to encourage officers

Officers are given choices about which booster session to attend

Officers that demonstrate proficiency can be moved to a different schedule of observation

Page 13: Quality Assurance/Continuous Quality Improvement

EMVP AWARD

Officers must get 85% satisfactory on overall score for a minimum of 3 tapes.

Officers must have an answer of yes on at least 75% of their caseload audits for question #III b.

Once officers are deemed an EMVP, they will need to submit a tape once every 4 months. They will also be required to attend one booster session every 4 months.

Officers can be moved back to the regular recording schedule if their manager has any issues/concerns or if the scoring on their tapes significantly declines.

Page 14: Quality Assurance/Continuous Quality Improvement

Booster Sessions Coaches identify areas that need

improvement to focus on during booster sessions

Officers are required to attend one session in a two month period

Booster sessions are limited to 12 participants to encourage participation and allow environment for practicing skills

Page 15: Quality Assurance/Continuous Quality Improvement

Booster Sessions

 

Do your offenders ever come into your office visit with problems? This session will focus on deciding which intervention/tool would be most appropriate to use. Different scenarios will be played and officers will break into groups and discuss how they would handle the scenarios.

Do you help your offenders set goals during office visits? Goal setting is an important, but often missed step in the EPICS session. This training will focus on setting goals and include a review of what should happen during the check-in and review sections.

Page 16: Quality Assurance/Continuous Quality Improvement

Caseload Size Officers added additional report days, allowing

more time to meet with offenders

Caseload management based on risk level and need areas instead of conditions

Low risk offenders moved to quarterly reporting

Efforts were made to reduce paperwork, allowing officers more time with offenders

Page 17: Quality Assurance/Continuous Quality Improvement

Consistency Among Coaches Due to department size, 12 coaches were

trained

With help from UC, developed coaching checklist that all coaches fill out with feedback form

Meetings held once every two months to discuss coaching progress or concerns

Page 18: Quality Assurance/Continuous Quality Improvement

Coaching Checklist

Prior to giving feedback:1. I listened to the entire tape. Yes No2. I have filled out all applicable sections on the feedback form. Yes No3. I have included positive feedback for the officer on the form. Yes No4. I have included specific suggestions/recommendations on how the officer can improve on the form. Yes No

During feedback:1. I asked the officer several open ended questions. Yes No2 I asked permission to give feedback. Yes No3. I applied the sandwich approach (strengths/areas for improvement/strengths) when giving feedback. Yes No5. I was able to answer all of the officer’s questions during feedback. Yes No If no, what question(s) were you not able to answer? After feedback:1. What do you think went well during the feedback? 2. What could you have done better during the feedback?

Page 19: Quality Assurance/Continuous Quality Improvement

Allows us to determine how effectively officers are using skills

Allows us to provide positive feedback to officers and constructive criticism

Allows us to reward officers that are effectively applying techniques they have learned

Benefits of Observing Office Visits

Page 20: Quality Assurance/Continuous Quality Improvement

In July of 2013 we started to give offenders a survey to complete about their experience with probation and their P.O.

Prior to this date, offenders had never been asked for any formal feedback

Administered privately and can be anonymous

Offender Surveys

Page 21: Quality Assurance/Continuous Quality Improvement

My P.O. is firm but fair. My P.O. helped to arrange services or

programs for me. My P.O. listened to me. I felt heard and

understood. My P.O. helped me to learn how to solve my

problems. What was the most difficult part of being on

supervision for you? Please feel free to include any additional

comments.

Sample of Survey Questions

Page 22: Quality Assurance/Continuous Quality Improvement

Survey Results

My P.O. listened to meMy P.O. helped me to learn how to solve my problems.

Strongly Agree

Agree Disagree Strongly Disagree

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Group 1Group 2Group 3Group 4

Stro

ngly

Agr

ee

Agree

Disagr

ee

Stro

ngly

Disa

gree N/A

0%

10%

20%

30%

40%

50%

60%

70%

80%

Group 1Group 2Group 3Group 4

Page 23: Quality Assurance/Continuous Quality Improvement

Identifying Barriers

Transportation Nothing Reporting too often Missing Work Getting/Staying Clean Color Drops0%

5%

10%

15%

20%

25%

30%

Group 1Group 2Group 3Group 4

Page 24: Quality Assurance/Continuous Quality Improvement

My P.O. has saved my life and I am a different person because of her!

My P.O. is a positive role model and always took time to listen to my life problems and to help solve them.

I would really like to thank my P.O. for treating me like a person and not a felon.

My P.O. always encouraged me and told me that I was doing good. Once he said he was proud of me!

Positive Feedback

Page 25: Quality Assurance/Continuous Quality Improvement

Allows us to get the offender’s perspective of his/her experience on supervision

Helps us identify barriers for offenders and enables us to reduce some of those barriers

Gives positive feedback to P.O.s and validates the work they are doing with offenders

Benefits of Surveys

Page 26: Quality Assurance/Continuous Quality Improvement

We piloted two criminal thinking surveys with our intensive officers to see if we were impacting criminal thinking among our officers

We used the CSS-M (Criminal Sentiment Scale- Modified) and TCU’S CTS (Criminal Thinking Scales)

Criminal Thinking Surveys

Page 27: Quality Assurance/Continuous Quality Improvement

Collected data for a year

Had offenders fill out a survey at intake and at the 6 month mark

The CTS gave us better results than the CSS-M

Criminal Thinking Surveys

Page 28: Quality Assurance/Continuous Quality Improvement

Consists of 36 questions and uses Likert scale

Answers are divided into 6 domains1. Entitlement (feelings of privilege)2. Justification (minimizing the seriousness of

antisocial acts)3. Power Orientation (the need for power)4. Cold Heartedness (callousness)5. Criminal Rationalization (negative attitudes

toward law and those in authority)6. Personal Irresponsibility (lack of ownership for

one’s actions)

Criminal Thinking Scales

Page 29: Quality Assurance/Continuous Quality Improvement

  Intake Post Test T-Test Results

  Average Pre-Test Score for 63 probationers who completed 

post-test (All cases)

Average Post-test score for 63 probationers

p-value

Overall Scores 20.0 19.3 0.06*

Entitlement 15.3 14.6 0.15

Justification 16.1 15.9 0.40

Power Orientation 20.3 19.3 0.07*

Cold Heartedness 23.8 23.2 0.27

Criminal Rationalization 24.1 22.3 0.02*

Personal Irresponsibility 21.1 19.8 0.04*

Results with CTS

Page 30: Quality Assurance/Continuous Quality Improvement

  Intake Post Test T-test Results

  Average Pre-Test Score for 56 probationers who

completed post-test (Non Judicial Release cases)

Average Post-test score for 56 probationers

p-value

Overall Scores 20.3 19.4 0.04*

Entitlement 15.5 14.7 0.10*

Justification 16.4 16.2 0.30

Power Orientation 20.5 19.4 0.08*

Cold Heartedness 24.2 23.3 0.19

Criminal Rationalization 24.5 22.3 0.01*

Personal Irresponsibility 21.6 20.3 0.05*

Results with CTS after Removing Judicial Release Cases

Page 31: Quality Assurance/Continuous Quality Improvement

Allows us to have an objective measure to determine what impact we are having on reducing criminal thinking

Allows us to determine areas that need more focus

Will continue to collect data, but will look at 9 month intervals

Benefits of CTS

Page 32: Quality Assurance/Continuous Quality Improvement

Officer resistance

Increase in responsibilities for management staff

Department size

Training

Barriers to the QA/CQI Process

Page 33: Quality Assurance/Continuous Quality Improvement

Sara Shields Quality Assurance ManagerFranklin County Adult Probation Department

[email protected]

(614) 525-4687

Contact Information