Feedback Informed Treatment– an introduction

Dr. Ryan Melton Research Faculty/EASA Clinical Director

Portland State University

rymelton@pdx.edu

Therapists Rock!

• The effect size of therapy is .80! (the average treated individual is better off than 80% of untreated individuals)

• Couples/Adolescents = .75-.80 • Family Psychotherapy = .58-.70 • This equates to the point that therapy is cost

effective when compared to psychological and medical interventions

• Reference: (Minami, et al., 2008. Journal of Consulting and Clinical Psychology).

Therapists Suck!

• Make claims of effectiveness but only 3% of therapists count outcomes (Akins & Christensen, 2001).

• Despite all innovations (400 tx models) no improvement in 30 years!

• 47-50% of individuals drop out.

• Despite individuals feeling less stigmatized a lack of confidence in therapists exists.

• Continued emphasis on medical model despite better outcomes for most MH conditions.

• Ongoing claims of superiority amongst models without evidence.

– Model v. model= ES of .20

– Disorders

– Tx ingredients

– Individual demographics & dx (<1%)

– Therapist’s degree and demographics (0%)

– From The great psychotherapy debate by Wampold, B.E. (2001)

ARE ALL THERAPISTS FROM LAKE WOBEGON?

• COMPARED TO OTHER MENTAL HEALTH PROFESSIONALS WITHIN YOUR FIELD

(WITH SIMILAR CREDENTIALS), HOW WOULD YOU RATE YOUR OVERALL CLINICAL SKILLS AND EFFECTIVENESS IN TERMS OF A PERCENTILE? (PLEASE ESTIMATE FROM 0-100%. FOR EXAMPLE, 25% = BELOW AVERAGE, 50% = AVERAGE, 75% = ABOVE AVERAGE).

• WHAT PERCENTAGE (0-100%) OF YOUR CLIENTS GET BETTER (I.E., EXPERIENCE SIGNIFICANT SYMPTOM REDUCTION/RELIEF) DURING TREATMENT? WHAT PERCENTAGE STAY THE SAME? WHAT PERCENTAGE GET WORSE?

Walfish, S., McAllister, B., Lambert, M.J. (in press). Are all therapists from Lake Wobegon? An investigation of self-assessment bias in mental health providers.

LAKE WOBEGON

SURVEYED A REPRESENTATIVE SAMPLE PSYCHOLOGISTS, PSYCHIATRISTS, SOCIAL WORKERS, MARRIAGE AND FAMILY THERAPISTS FROM ALL 50 US STATES:

• NO DIFFERENCES IN HOW CLINICIANS RATED THEIR OVERALL SKILL LEVEL AND

EFFECTIVENESS LEVELS BETWEEN DISCIPLINES.

•ON AVERAGE, CLINICIANS RATED THEMSELVES AT THE 80TH %TILE

•NONE RATED THEMSELVES BELOW AVERAGE;

•LESS THAN 4% CONSIDERED THEMSELVES AVERAGE;

•ONLY 8% RATED THEMSELVES LOWER THAN THE 75TH %TILE;

•25% RATED THEIR PERFORMANCE AT THE 90TH% OR HIGHER

• BELIEVED: 80%, 17%, 3%. ---- DATA: 50% (50-70% FOR SS), 10% DETERIORATION

• Walfish, S., McAllister, B., Lambert, M.J. (in press). Are all therapists from Lake Wobegon? An

investigation of self-assessment bias in mental health providers.

Feedback Informed Treatment

• “ FIT is a meta-theoretical approach. Where traditionally counselors are guided by a particular treatment model or theoretical orientation, FIT is

guided by outcome and alliance feedback provided by clients. As such, FIT may be applied across

therapeutic modalities, disciplines, and service settings.”

FIT

• The client’s experience of the alliance and outcome are the best predictors of retention and progress in treatment;

• Because of the low correlation between client and clinician ratings of outcome and alliance, therapists must routinely seek client feedback via valid and reliable measures of the alliance and outcome;

• No one model, method, or clinician is sufficient for treating all problems; (Same outcomes for last 30 years despite 400+ models)

• Feedback is crucial to addressing the diverse problems and people seeking behavioral health services.

Implementation

ORS

SRS

• I REALLY WANT TO MAKE SURE OUR WORK TOGETHER IS HELPFUL TO YOU AND YOUR REASONS FOR COMING IN…..I HAVE A SIMPLE FORM THAT I USE

• THIS ALLOWS US TO TRACK HOW YOU THINK THINGS ARE CHANGING OR NOT CHANGING.

• IF THINGS AREN’T CHANGING OVER TIME THE WAY YOU HOPE THEN THIS WILL HELP US KNOW THAT AND ADJUST AS NEEDED

• IT JUST TAKES A FEW MINUTES TO COMPLETE. THANKS!

• Whether or not the sessions are going the way you hope and if I am providing what you want is really important to me

• Want to make sure that you are getting what you need;

• I really want any feedback you have, especially negative feedback. I can take it.

• Feedback is critical to success.

ORS/SRS www.scottdmiller.com

Outcome Rating Scale (ORS)

• 40 pt measure with 4 subscales

• Adult and child versions

• Higher scores=lower level of distress. Lower scores=higher level of distress

• Clinical cutoffs: 25 (>18), 28 (13-18), 32 (<12)

• 5 pt change is considerable reliable change.

• Complete at start of session. It takes 1 min.

Session Rating Scale (SRS)

• 40 pt measure with 4 subscales

• Adult and child version

• Scores below 36 should be discussed with client or any subscale below 9

• Lower scores early could mean anything-discuss. Low scores later 4x likely to drop out.

• Done at end of session, takes 1 min.

• Can plot ORS & SRS on Excel.

• All materials free at www.scottdmiller.com

EMPLOYING MEASURES WITH CLIENTS: BENEFITS

• SETTING A NORM FOR DATA COLLECTION

• PROGRAM EVALUATION FOR DATA-BASED DECISION MAKING (CCOs)

• EMPHASIS ON THE CLIENT PERSPECTIVE AND POTENTIAL FOR CLIENT ADVOCACY

PREPARING FEEDBACK

• MINDSET

• ADDRESSING OWN SELF-EFFICACY & OPENNESS TO FEEDBACK

• MEANINGFUL DATA BEYOND THE NUMBERS (FORMATIVE)

• INFORMED CONSENT: WHAT EVALUATION TOOLS WILL BE USED TO EVALUATE MY PERFORMANCE?

CLIENT FEEDBACK WITH TRAINEES: INFLUENCE ON SUPERVISION & COUNSELOR SELF EFFICACY(REESE ET AL.,

2009)

28 TRAINEES ASSIGNED TO FEEDBACK OR NO FEEDBACK

FINDINGS:

• CLIENTS IN FEEDBACK GROUP EXPERIENCED TWICE AS MUCH IMPROVEMENT IN OUTCOME

• SELF-EFFICACY SCORES INCREASED FOR BOTH GROUPS

• NO CORRELATION BETWEEN SELF-EFFICACY AND OUTCOME FOR NO-FEEDBACK GROUP

• SELF-EFFICACY IN THE FEEDBACK GROUP WAS STRONGLY CORRELATED WITH OUTCOME

Putting Feedback to work in practice: Three Steps

1. Create a “Culture of feedback” by integrating alliance and outcome feedback into practice.

2. Engage in deliberate practice.

3. Learn to “fail successfully.”

STEP 1: CREATE A CULTURE OF FEEDBACK

• WORK TOWARDS ADDRESSING BARRIERS TO GETTING OPEN CLIENT FEEDBACK (E.G., ARE WE OPEN TO FEEDBACK, WHAT DO WE COMMUNICATE NON-VERBALLY, WHAT ARE OUR RESISTANCES, SYSTEMIC BARRIERS)

• USE A SYSTEMATIC METHOD FOR ACQUIRING FEEDBACK THAT IS RELIABLE, VALID AND FEASIBLE

• FOLLOW A STANDARD CASE PRESENTATION MODEL

STEP 2: ENGAGE IN DELIBERATE PRACTICE

• LISTENING & RESPONDING TO CLIENT FEEDBACK

• WORKING HARD AT OVERCOMING “AUTOMATICITY”

• PLANNING, STRATEGIZING, TRACKING, REVIEWING, AND ADJUSTING

• VIEWING MISTAKES OR LOW RATINGS FROM CLIENTS AS AN OPPORTUNITY

• INCLUDE THE CLIENT’S VOICE IN SUPERVISION VIA THEIR FEEDBACK (USE THE GRAPH)

Step 3: Learning to Fail Successfully

• Know when the work is failing

• Things to change when improvement is not occurring: what, where, who????

• Gracefully get the client to another place to accomplish their goals.

CLIENT PREFRENCES

GOALS/MEANING/PURPOSE INTERVENTION

CLIENT VIEW OF THE RELATIONSHIP

Wrapping Up

• What have you learned?

• Did we meet our goals today?

• Any feedback?

Questions?