© Duncan Law

Making the CYP-IAPT Feedback and

Outcomes Monitoring Clinically Useful:

A Workshop

Duncan Law

5th Nov 2013

CYP-IAPT National Conference

Hertfordshire PartnershipUniversity NHS Foundation Trust

© Duncan Law

Feelings about routine feedback & outcomes monitoring?

Hate them

0

Love them

10

Mixed feelings

5

© Duncan Law

Values behind outcomes and feedback

1. Demonstrate/celebrate/prove

effectiveness of interventions and service

1. Enhance collaborative practice:

– Better service user voice in therapy

– Facilitates good clinical practice

© Duncan Law

Feedback & Outcomes

Monitoring

Demonstrating success

© Duncan Law

Demonstrating effectiveness

• 90% target

– at least two time points with repeated normed

measure

– Plus, at lest one report of Education

Employment and Training (EET)

– http://www.iapt.nhs.uk/silo/files/90.pdf

© Duncan Law

It’s not (just) about measurement,

it’s about a culture of collaboration

©Tom Frost

© Duncan Law

Feedback & Outcomes

Monitoring

Young peoples views

© Duncan Law

Video

• http://www.youtube.co

m/watch?v=SOayZgia

M7U

First, a word from the

people who matter

© Duncan Law

Young peoples views:

“Knowing when and how to use

the outcome measures is

important. They must become an

integrated part of the session - not

something that is kept as an

abstract form-filling exercise.”

“Outcomes tools can be really

useful, not only for the therapists

but for the young people. They

allow you to evaluate where

things are going well and where

they could be improved.

Feedback is important for us to

learn and grow. However, I do

think you must use outcomes

tools with the correct intention and

in the right way.”

© Duncan Law

Young peoples views:

“Whenever I was doing a

questionnaire, it felt like a tick box

exercise. I didn’t know why I was

doing it. In fact no one seemed to

understand the point of the

questionnaire, it was just a

standard process. Eventually I

stopped thinking what I wrote on it

as it just seemed pointless.”

“Young people never get any

feedback about what the

information they have given has

been used for or how it has been

implemented in services. This

makes it seem like the

questionnaires are just so that

they can tick a box to say they

have used outcome tools without

actually using the information

given.”

© Duncan Law

Feedback & Outcomes

Monitoring

The evidence base

© Duncan Law

Mike Lambert

• Feedback to clinicians on outcomes trajectories

• Reduced drop-out

• Better outcomes

• No advice given to clinicians on how to use the feedback

• Lambert, M. J. (2007). PRESIDENTIAL ADDRESS: A program of research aimed at improving

psychotherapy outcome in routine care: What we have learned from a decade of research.

Psychotherapy Research, 17, 1-14.

© Duncan Law

Len Bickman

• Big RCT (28 sites in 10 states) ‘real world CAMHS’

• Feedback weekly or every three months

• Faster improvement with weekly feedback

• Even better if clinicians looked at the feedback! (paraphrased)

• Bickman, L., Kelley, S., Breda, C., De Andrade, A, & Riemer, M. (2011): Effects of routine

feedback to clinicians on youth mental health outcomes: A randomized cluster design, Psychiatric

Services, 62(12), p.1423-1429

© Duncan Law

Feedback & Outcomes

Monitoring

Making them clinically useful

Assessment/Choice

• “What’s the problem?”

• “What do you want to change?”

Partnership/on-going work

• “How are we getting on together?”

• “How are things going?”

Review & Close

• “Have we done as much as we need to?”

• “How has this been generally?”

6 useful questions that forms can help with

© Duncan Law

Feedback & Outcomes

Monitoring

Clinicians views

© Duncan Law

clinicians talking about their experience

http://www.youtube.com/watch?v=5srUEXW

wMKU

Apprehension

Feeling ClunkyIntegrated in

practice

How it can feel to try out new ideas and approaches

Learning taken from the Closing the Gap ProjectNeus AbrinesJasmine Hoffman

© Duncan Law

Let’s give it a go!

© Duncan Law

In pairs, one clinician, one young person

Clinicians: try explain to the young person:

– the importance of completing a measure

– to complete it as openly and honestly as possible

Young people: during the role play, consider what the

clinician said or did that made you:

– Appreciate the importance of completing the measure

– want to fill the measure in openly and honestly

– DO NOT share this detail with your partner yet

© Duncan Law

How did that feel?

Young people:

• What did your partner do that made you feel that

completing the measure was:

– Important

– Worthwhile

– Useful

– Helpful

• What did your partner do to convince you to complete

the measure as honestly as possible?

© Duncan Law

How did that feel?

Clinicians:

• Did you feel confident in what you were saying to the

young person?

• What techniques did you try to use to convince them?

• What would have helped you to feel more confident?

© Duncan Law

Feedback & Outcomes

Monitoring

Making them clinically useful

© Duncan Law

Forms, forms and more forms!

Let’s brainstorm: what are all these forms for?

Barriers to using measures(with thanks to UPROMISE)

Balancing the seesaw of service and therapeutic need

(with thanks to UPROMISE)

SERVICE EVALUATION CLINICAL MEANINGFULNESS

SDQ RCADS

CGAS Goals

Symptom rating scales Session rating scales

CHI-ESQ

Assessment/Choice

• “What’s the problem?”

• “What do you want to change?”

Partnership/on-going work

• “How are we getting on together?”

• “How are things going?”

Review & Close

• “Have we done as much as we need to?”

• “How has this been generally?”

6 useful questions that forms can help with

© Duncan Law

At Assessment/ Choice

“What’s the problem?”

• SDQ

and/or

• RCADS

“What do you want to change?”

• Goals (GBO)

and

• ORS/CORS

© Duncan Law

Session-by-session

“How are things going?”• Symptom Tracker (in collaboration with child/family)

or

• Impact Tracker “how are you doing” or ORS/CORS

and

• Goals Based Outcomes (GBOs), or ORS/CORS

Hertfordshire PartnershipNHS Foundation Trust

© Duncan Law 2012

An Example of

a Symptom

Tracker

Worries

(GAD)

Hertfordshire PartnershipNHS Foundation Trust

Goals Based

Outcome Tool:

What does it look

like?

© Duncan Law

Session-by-Session

“How are we getting on together?”

• SRS

or

• Session Feedback Questionnaire

Introducing the RCADS: 2 ways

• In the following video, a clinician will introduce the RCADS to a young person and ask her to complete it in two different ways: http://vimeo.com/57925678

While watching, consider

• What are the pros and cons of asking a young person to complete measures in these ways?

• What might you do differently?

• What questions might the young person have?

Interpreting RCADS Scores

• What do you make of this?

• What might you want to discuss in supervision?

• What do you want to know?

• What questions would you ask?

Feeding back scores to clients

• The following video clip shows a therapist discussing scores on a measure with a young person: http://vimeo.com/57925681

• Do you think this was done in a collaborative manner?

• How could it have been more collaborative?

© Duncan Law

What’s going on here?

© Duncan Law

0

2

4

6

8

10

12

14

16

Session 1 Session 2 Session 3 Session 4 Session 5 Session 6

Su

bscale

Sco

re

Session

© Duncan Law

Clinical Cutoffs

A score below (or

sometimes above) which a

person is unlikely to have a

problem measured by a tool

Reliable Change

A change in scores is reliable

if it is larger than that

expected by measurement

error

With thanks to Andy Fugard

The stats bit!

© Duncan Law

What’s going on here?

© Duncan Law

Depression symptom tracker - 17 year old boy

Clinical cutoff = 12

Reliable Change = 6

© Duncan Law, Stephen Butler, & Alex Goforth

Service User Feedback &

Outcomes Monitoring

Resources and Further Information

© Duncan Law

CYP-IAPT Service user feedback and outcomes framework

© Duncan Law

CYP-IAPT Resources

http://www.iapt.nhs.uk/silo/files/a-

practical-guide-to-using-service-

user-feedback--outcome-tools-

.pdf

http://www.iapt.nhs.uk/cy

p-iapt/routine-outcome-

monitoring-as-part-of-

iapt/

© Duncan Law

Other Resources

• www.corc.uk.net

• www.myapt.org.uk

• www.youtube.com/channel/UCVr_XGnjA229P5b

UgfZ-y3w

• www.vimeo.com/tag:cyp+iapt

• Information on using feedback and outcomes

including training videos, information

© Duncan Law

Feelings about routine feedback & outcomes monitoring?

Hate them

0

Love them

10

Mixed feelings

5

© Duncan Law

© Duncan Law

Thanks for coming along

We hope you found this presentation useful

Feel free to use all are parts of it to help train other staff in your organisation & to

further implement CYP-IAPT

If you do use it please reference: ‘Duncan Law’

These slides must not be used in training for commercial gain