Patient monitoring and feedback in psychiatric care reduces depressive symptoms.

Elizabeth A. Newnham¹, Geoff R. Hooke¹², Andrew C. Page¹²

¹The University of Western Australia, Perth, Western Australia

²Perth Clinic, Perth, Western Australia

Short title: Feedback in psychiatric care

Key words: Outcome Assessment, Group Psychotherapy, Inpatient, WHO-Five,

Feedback

Corresponding author: Elizabeth A Newnham, School of Psychology, The University of

Western Australia, 35 Stirling Highway, Crawley, Western Australia. Phone: +61 8 6488

2479; Fax: +61 8 6488 1006; Email: Elizabeth.Newnham@graduate.uwa.edu.au

Abstract

Background: To date, the monitoring of patient progress using standardized

assessments has been neglected in hospital-based psychiatric care. Findings in outpatient

psychotherapy have demonstrated clinically significant benefits for providing feedback to

the sizeable minority of patients who were otherwise unlikely to experience positive

outcome [1]. However, a similar system for presenting feedback on patient progress has

not yet been assessed within psychiatric inpatient settings. The current study aimed to

develop and evaluate the effectiveness of a feedback system suitable for use in

psychiatric services. Methods: One thousand three hundred and eight consecutive

inpatients and day patients, whose diagnoses were primarily depressive and anxiety

disorders according to ICD-10 criteria, completed the World Health Organization’s

Wellbeing Index (WHO-5) routinely during a ten-day cognitive behavioral therapy

group. The first cohort (n=461) received treatment as usual. The second cohort (n=439)

completed monitoring measures without feedback, and for patients in the third cohort

(n=408), feedback on progress was provided to both clinicians and patients midway

through the treatment period. Results: Feedback was effective in reducing depressive

symptoms (F(1,649)=6.29, p<.05) for those patients at risk of poor outcome, but not

effective in improving wellbeing (F(1,569)=1.14, p>.05). Conclusions: Similar to

outpatient settings, feedback appears to be beneficial for improving symptom outcomes

but further time may be required for wellbeing to be affected. The current findings may

be generalized to patient samples that exhibit largely depressive disorders, and further

follow-up with a randomized trial design is warranted.

Psychiatric care, unlike physical healthcare, does not yet have the instruments

available to reliably monitor patient health and alert clinicians to a negative response to

treatment. Physical healthcare relies upon a suite of tools, such as the thermometer, to

guide treatment. Monitoring complements and informs the treatment process by

providing the clinician with quick, easy to interpret feedback on a patient’s response.

Unfortunately, psychiatric care is in the early phase of developing standard monitoring

instruments that may alert clinicians to a poor treatment response and signal a need to

evaluate and possibly alter therapy.

This absence is concerning because relatively high rates of negative outcome

occur in mental health care. Despite the widespread use of evidence-based treatments, a

large proportion of patients still fail to demonstrate reliable or clinically meaningful

improvement. Estimations of deterioration rates have been as high as 23%, and up to 40%

of patients show no change as a result of therapy, as illustrated in both clinical settings

[2-4] and randomized controlled trials [5], where a highly controlled treatment is assessed

under optimal conditions and with homogenous patient groups. Thus, despite the success

of efficacy and effectiveness studies in identifying valuable treatments; a large minority

of patients are not benefiting from psychological therapy. One response to these findings

has been to seek better and more refined treatments. Another, not mutually exclusive

response has been to highlight the need to complement the delivery of psychological

treatment with additional progress information to improve rates of clinical recovery. By

monitoring progress during the therapy period, clinicians have the opportunity to improve

outcomes in real time for the benefit of each particular individual [6-8].

Monitoring programs suitable for outpatient psychological therapy have been

developed in the United States and Europe, with notable success. In the US, Lambert and

colleagues have developed a program of feedback to alert therapists and patients to

deviations from expected response [1]. Administration of the 45-item Outcomes

Questionnaire (OQ-45) at each therapy session allows for the ongoing monitoring of

progress, which is mapped against the patient’s expected response trajectory. The

expected trajectory is a function based upon the patient’s severity at intake. Five studies

[4, 9-12], of which four were randomized controlled trials, have demonstrated that

providing feedback on patients’ treatment response improved rates of clinically

significant outcomes, but perhaps more importantly, reduced negative outcomes from

20% to 8% (when presenting feedback to therapists and patients, and providing clinical

support tools). Those studies that have demonstrated a significant improvement in

outcomes have also revealed cost-efficient benefits for the use of ongoing monitoring.

Whilst the minority of cases that were identified as ‘not on track’ were provided two to

three sessions extra, the majority were ‘on track’ and received one session less [1]. Thus

progress monitoring and feedback appears to improve treatment outcomes in an efficient

and cost-effective manner.

However, one issue not addressed by the studies to date relates to the quantity of

psychotherapy provided. Although participants were randomly assigned to conditions,

the number of sessions of therapy delivered varied across conditions. As mentioned, the

cost-effectiveness of the different numbers of sessions received could be highlighted, but

it does mean that the research has not revealed that the same benefits would be observed

if patients in the feedback and non-feedback groups received the same quantity of

therapy. Consequently, research is needed in which the amount of treatment delivered is

constant across feedback and non-feedback groups to address this issue.

At present, monitoring patient response has been illustrated as an important

addition to clinical practice and research within the scientific literature; however it has

been greatly underutilized in practice. There are a number of arenas in which it would be

beneficial to monitor treatment response more closely. The argument is particularly

compelling for fields in which early response to treatment is useful information, such as

drug trials; or the monitoring of patient health is particularly important, such as suicide

risk management or treatment for trauma. Time-intensive psychotherapy and inpatient

psychiatric care also require ongoing monitoring but as yet, an appropriate and efficient

monitoring system for short-term assessment has not been identified. This is a surprising

reality, given that Lambert rightly heralds the ongoing monitoring of patient progress a

clinician’s ethical responsibility [1].

Despite the temptation to extend the methods to all areas of mental health care, of

the models that have been developed for the ongoing monitoring of treatment response

[1, 13] none operate over a brief time-frame. For instance, treatment offered in inpatient

settings is often intensive and brief, being measured in days, as opposed to outpatient

psychotherapy that takes place over weeks. For patient groups that may be expected to

demonstrate changes within a number of days or when the duration of treatment is brief,

the OQ-45 and benchmarks provided by Lambert and colleagues would not be the most

appropriate, as they measure change over a one week period. Although a number of

health status measures are available for monitoring treatment response, all measure

progress within a timeframe of 1 week to 3 months, and most fail to demonstrate

appropriate psychometric features for individual monitoring [14, 15]. Accordingly, a

system of monitoring appropriate for settings in which changes may occur in days, not

weeks, is required.

In addition, current programs have been developed within outpatient samples, and

therefore the outcomes may not necessarily generalize to an inpatient group setting [16].

Inpatient care differs in that it can be more intensive, the population may experience a

greater level of disturbance, psychotherapy is delivered in the context of a variety of

other interventions (e.g., pharmacotherapy, ongoing nursing care) and patients are often

discharged with the expectation that further improvement will occur with ongoing

community care. Furthermore, while the average length of stay in psychiatric facilities

differs between nations, Australia, like the United States, tends to have relatively short

admissions [17]. It is therefore important to develop a monitoring program that has the

capacity to provide feedback within a shorter time-frame.

The current study comprised the development and evaluation of a monitoring

system suitable for use in an acute psychiatric setting. The World Health Organization’s

Wellbeing Index [WHO-5; 18] is a five-item self-report measure of positive wellbeing

that has performed reliably and sensitively in psychiatric samples [19]. The monitoring

program involved the routine administration of the WHO-5 as a measure of patient

progress, evaluated within a Cognitive Behavioral Therapy group at a private psychiatric

hospital. The aim of the study was to assess the effectiveness of monitoring patient

progress, using that information as feedback for clinicians and patients. It was anticipated

that patients receiving feedback on their progress during the group would exhibit

significantly improved outcomes on measures of wellbeing and symptom distress at

completion of the group.

Method

Research setting and participants

Participants were recruited from a 98 bed, private psychiatric hospital in Western

Australia. Eligible participants were English speaking inpatients or day patients who were

participating in the hospital’s two week cognitive behavioral therapy program. The

sample comprised of 1308 patients who participated in the trial; 408 who received

feedback on their WHO-5 scores, 439 completed the WHO-5 routinely but did not

receive feedback on scores until completion of the group, and a control group of 461 who

were not administered the WHO-5 while participating in the CBT treatment program.

Participants were diagnosed by their treating psychiatrist according to ICD-10-

AM criteria [20] and primary diagnoses consisted mostly of mood (67.7%), anxiety

(25.9%), and substance use (3.0%) disorders. Ages ranged from 16 years to 76 years with

a mean of 39.8, and 63.1% were female. There were no significant differences between

groups in diagnoses (F (2,1302) = 1.77, p > .05), sex (F (2,1305) = .476, p > .05) or age

(F (2,1305) = .295, p > .05). The University of Western Australia Human Research

Ethics Committee approved the study protocol prior to commencement, and patients

provided informed consent as part of the routine admission procedure at the hospital.

Monitoring and Outcome Measures

The World Health Organization’s Wellbeing Index [WHO-5; 18] is a 5-item scale

of positive wellbeing. Originally designed to screen for depression in diabetic samples

[18], the WHO-5 has also demonstrated reliability and validity in screening for

depression in primary care [21] and elderly samples [22]. Further to this, the

psychometric properties and clinical utility of the WHO-5 has warranted its use as a

measure for monitoring patient progress and treatment outcomes in psychiatric care [19].

The Index is rated on a five point scale, and for the purpose of the current study,

wellbeing was measured over the past day rather than previous two weeks [19]. Scores

thus range from 0 to 25 with higher scores indicating more positive wellbeing.

The Mental Health subscales of the Medical Outcomes Questionnaire Short Form

[SF-36; 23], are a reliable self-report measure of patient mental health status. They

comprise of Vitality, Social Function, Role Emotion and Mental Health scales, which

have previously demonstrated validity as measures of patient outcomes in psychiatric

care [2]. The scales have sound internal consistency, exceeding 0.8 for each of the scales

[24], together with strong content and construct validity [2, 23, 24].

The Depression Anxiety Stress Scale [DASS-21; 25] is a 21-item self-report

measure of psychopathology. A short form of the of the 42 item scale, the DASS-21 has

strong internal consistency [26] sound construct validity [27], and a cleaner factor

structure [28]. It is rated on a 5-point scale and high scores indicate more severe

psychopathology.

The Health of the Nation Outcome Scale [HoNOS; 29] is a 12-item measure of

patient mental health that covers a heterogeneous range of presenting issues, including

aggression and agitated behavior, hallucinations, depressed mood, difficulties with

activities of daily living, social relationships and housing issues. The HoNOS is

completed by therapy staff, psychiatric nurses or the treating psychiatrist, all of whom

have completed a training program for its administration. The Scale measures health as

reported for the previous two weeks when rated at admission and over the preceding 72

hours when rated at discharge. All items on the scale are rated from 0 (no problem) to 4

(severe problem); thus low scores indicate healthier functioning.

The SF-36, DASS-21 and HoNOS were administered as routine clinical practice

at admission and discharge for each patient.

Trial Design

The trial comprised a historical cohort design so that between January 2005 and

March 2006, patients were not administered the WHO-5; from April 2006 to July 2007

the WHO-5 was routinely administered during treatment, but feedback was not presented

to the patient or clinician until the final day of therapy; and between August 2007 and

January 2009, participants completed the WHO-5 routinely during therapy and were

provided with feedback on their progress at Day 5 and Day 10. The cohorts that were

administered the WHO-5 were matched according to the severity of their WHO-5 score

at Day 1 of treatment.

Group Treatment

The Cognitive Behavioral Treatment Program is a closed group of 6-8 members

that runs from 9am to 5pm over 10 working days. Each group is run by two therapists,

and covers depression and anxiety management, cognitive disputation, behavioral

activation and experiments, identification and modification of negative core beliefs, self

esteem, communication skills, stress management, and dealing with setbacks. Each

group member sets personal therapy goals and participates in homework tasks. The

treatment program has demonstrated effectiveness comparable to randomized controlled

trials [30].

Monitoring Intervention

Prior to the commencement of the group therapy, on days 1 (Monday), 3

(Wednesday), 5 (Friday), 7 (Tuesday), and 9 (Thursday) of the program, patients

completed the WHO-5. Occasionally, if a day was missed, the questionnaire would be

completed on the following day. For those in the No Feedback cohort, scores were

graphed and provided to patients on Day 10 (Friday), where they were given the

opportunity to discuss their scores with the therapist.

Using the data from the No Feedback group’s wellbeing scores, expected

treatment response cures were generated. The sample was divided into five groups

according to severity of wellbeing scores at Day 1, so that each group consisted of 20%

of the sample. For each group, means and standard deviations were calculated for each

measurement point, which illustrated a dose-response curve across the ten days of

therapy. A log linear curve was generated, one standard deviation around the mean for

each day, for each group (see Table 1). This curve became the trajectory of expected

response against which each patient’s actual scores were mapped (see Figure 1).

Consistent with the procedure of administration for No Feedback, those in the

Feedback condition completed the WHO-5 on days 1 (Monday), 3 (Wednesday), 5

(Friday), 7 (Tuesday), and 9 (Thursday) of the program. According to the patient’s score

at Day 1, an expected treatment response curve was generated, and their actual scores

mapped against it. This graph and an accompanying explanation was provided as

feedback to the clinician and patient at Day 5 (Friday of the first week of therapy) and

Day 10 (Friday of the final week of therapy). Feedback graphs were distributed during

group, and clinicians spent time discussing what the graphs meant, exploring the meaning

of fluctuations in scores, and discussing the opportunity that feedback provides to

examine one’s progress and re-assess the treatment goals. Clinicians were not given

specific directions on the use of feedback but clinical management meetings were held

during the trial to assess adherence to protocol and discuss the clinical management of

cases.

Day 5 was considered the most appropriate point for feedback for clinical and

evidence-based reasons. Regression analyses [see 19] deemed Day 5 to be the point at

which Day 9 scores could be best predicted and it provided sufficient time to modify the

treatment plan. Further to this, it was suggested by the clinical management team that

providing feedback on a Friday morning presented the optimal opportunity to review the

week’s work, set homework exercises for the weekend ahead, and evaluate and devise

treatment goals for the second week of therapy.

Results

Equivalence of Groups

To compare the effectiveness of providing feedback, groups were matched

according to severity at Day 1 on the WHO-5. This was calculated by comparing each

cohort according to their expected trajectory of response groupings. Accordingly, for the

analyses, each condition had a sample size of 379, with no significant differences in

severity between groups at Day 1.

To reduce the skew of the data, a square root of WHO-5 scores at each day was

taken. This score was used for all further WHO-5 analyses. Quality control charts

revealed three therapy groups with outlying scores (i.e. three standard deviations beyond

the mean) which were removed from the sample for further analyses.

Definition of Alarm

Consistent with Lambert’s research [4, 10-12], the data were analysed according

to whether a person was deemed ‘on track’ at the point of feedback. Following the

feedback processes used, those who were deemed to be ‘not on track’ at Day 5 (alarm

cases) were patients whose Day 1 scores were below 12 (thus within the unwell range at

admission to the CBT Program), and whose scores fell below the expected trajectory of

improvement at Day 5.

Intervention Outcomes

The distribution of raw scores was inspected and means and standard deviations

were calculated for each cohort across all outcome measures (see Table 2). The data

suggests that on average, patients in all cohorts move from the ‘unwell’ range to the

‘well’ range on the WHO-5 [19] and SF-36 [2] as a result of therapy.

Patient self-report outcomes

To assess the effectiveness of feedback in reducing patients’ symptoms and

improving wellbeing, a series of repeated-measures ANOVAs were conducted with the

outcome measure as the dependent variable, and condition and alarm status as between

group factors. A significant improvement over time was evident for both conditions so

that on average, patients improved in wellbeing as a result of therapy (F (1,569) = 237.1,

p<.05). However, there was no significant difference in wellbeing scores at Day 9

between feedback conditions for those patients on track or not on track (F (1,569) = 1.14,

p>.05). Thus feedback to staff and patients about scores on the WHO-5 did not

significantly improve patients’ wellbeing by Day 9.

In contrast, feedback was of benefit on some measures of symptom distress.

Again, significant improvement over time was evident for all conditions on the

Depression (F (1,649) = 438.5, p<.05), Anxiety (F (1,649) = 305.2, p<.05), and Stress (F

(1,649)=421.2, p<.05) scales of the DASS-21. A significant interaction was evident,

arising because those patients who were not on track and received feedback exhibited

relatively greater improvements in Depression scores on the DASS-21 (F (1,649) = 6.29,

p<.05). However no significant difference resulting from feedback was evident for any

patients on the Anxiety (F (1,649) = .496, p>.05) or Stress (F (1,649) = .628, p>.05)

subscales of the DASS-21.

Those patients not on track who received feedback exhibited a significant

improvement on the Vitality subscale (F (1,639) = 5.53, p<.05), and Role Emotion

subscale (F (1,635) = 4.11, p<.05) of the SF-36. Yet no significant interaction was

evident for Mental Health (F (1,639) = 2.28, p>.05) or Social Function subscales (F

(1,643) = 1.94, p>.05). The results suggest that measures of depressive symptoms, such

as vitality and emotion, demonstrate a significant improvement for those patients who

received feedback when at risk of poor outcome.

The clinical validity of the results was investigated using the criteria for clinical

significance outlined by Jacobson and Truax [31]. Consistent with the finding that

feedback does not affect outcomes for those patients on track during therapy, positive

outcomes on the Vitality subscale of the SF-36 remained constant across feedback

conditions (40.8% achieved reliable improvement without feedback and 40.2% with

feedback). However, the provision of feedback reduced deterioration rates from 5.3% in

the No Feedback cohort to 3.3% in the Feedback cohort.

In contrast, no significant change in rates of clinically significant improvement or

deterioration was evident for the DASS-21 Depression subscale. Positive outcomes

remained constant across cohorts (51.3% achieved reliable improvement without

feedback and 51.1% with feedback). Similarly, 48.7% of participants exhibited a negative

outcome (no change or deterioration) without receiving feedback, compared to 48.9%

with feedback. Deterioration rates showed no significant change across cohorts.

The calculation of clinical significance was not appropriate for the Role Emotion

subscale due to the limited variability in potential scores [2].

Clinician-rated outcomes

When examining the clinician-rated scores, again a significant effect of

improvement over time was evident so that patients are regarded as much improved

following therapy (F (1,615) = 639.7, p<.05). Yet, no effect of feedback was evident on

HoNOS total scores regardless of predicted outcome (F (1,615) = 3.20, p>.05).

Interestingly, the data depict an overall elevation of HoNOS scores for those patients not

on track regardless of whether feedback is provided, but no significant effect is evident

(F (1,615) = 3.15, p=.08).

To investigate the hypothesis that feedback improves the convergence of

clinician-rated scores with patient self-report, correlations were conducted between

HoNOS scores and the other outcome measures (see Table 3). There is some indication

of convergence between clinician-ratings and self-report as the correlations reliably

increase over time, however there is no significant change in correlations across cohorts.

Discussion

The current study aimed to assess the effectiveness of monitoring patient progress

and providing feedback about wellbeing in an inpatient and day patient psychiatric

setting. It was proposed that those patients who received feedback on their response to

treatment would exhibit significant improvements in symptom relief and wellbeing, when

compared to those who did not receive feedback. The study found, unexpectedly, that

providing feedback on wellbeing did not significantly improve patients’ wellbeing scores

at conclusion of the group. This result represents a divergence from previous findings in

patient-focused research that have illustrated large treatment effects for feedback when

measured against the same questionnaire at outcome [4, 9-11] and therefore warrants

further consideration.

There are a number of plausible reasons for this discrepancy. It may be the case

that wellbeing is a construct less susceptible to the effects of feedback, or that it requires

a longer period of time to realise significant change. The stages of change outlined in the

Phase Model [32] suggest that an improvement in general positive wellbeing and function

occur in a later stage of therapy, following periods of remoralization and remediation of

symptoms. Thus, the time-intensive program of measurement required for this study, as

opposed to the weekly points of measurement depicted in previous studies, may not

capture changes in wellbeing required to show a significant improvement resulting from

feedback.

Another reason may be that the WHO-5 scores illustrate a ceiling effect which

may indicate that a large proportion of the sample move into the ‘well’ range early in

therapy and thus their difficulties are not captured by the measure. To test this hypothesis,

the effects of feedback were assessed on two measures of symptom distress.

In an extension of previous studies in the field of patient-focused research, the

effectiveness of feedback in reducing patients’ presenting symptoms was assessed on two

convergent measures: the DASS-21 and the mental health subscales of the SF-36.

Traditionally, the effects of feedback have been investigated using a single measure for

monitoring and outcome assessment [4, 10-12]. Analyses revealed that those patients

who were not on track for improvement in therapy, but received feedback on their

progress demonstrated a significant improvement in depression, vitality and role emotion

scores. For a predominantly depressive disordered group, these treatment gains are of

great consequence. The gains suggest that providing depressed patients with feedback

that they are not improving as expected presents an opportunity for a crucial change in

direction. Importantly, these findings demonstrated clinical impact in that deterioration

rates in therapy reduced from 5.3% to 3.3% when feedback was provided. This finding

complements and extends Lambert and colleagues’ work in patient-focused research that

has illustrated the significant benefits of providing feedback to clinicians and patients

during therapy [12]. Thus an improvement in depressive symptoms is evident for those

patients demonstrating poor progress when the feedback and non-feedback groups

received the same quantity of therapy.

In contrast, feedback did not significantly improve outcomes as measured on the

broad mental health and social function subscales of the SF-36, or the anxiety and stress

subscales of the DASS-21. Thus, the benefits of feedback appear limited to the symptoms

which were the principal reasons for treatment. Furthermore, the improved outcomes

associated with feedback were not evident when measured on the clinician-rated HoNOS.

Interestingly, a trend in the data suggested that clinicians rated ‘alarm’ patients worse off,

regardless of whether they received feedback, which suggests that they are attuned to

their patients’ symptoms, despite the low correlations between clinician ratings and

patient self-report outcomes.

The current study aimed to extend the field of patient-focused research by

assessing the use of feedback in an inpatient and day patient setting, using a group

therapy format. Accordingly, a number of discrepancies exist between the current

research design and previous work that may account for the alternative findings. First, for

the current study, feedback was provided within a group format. Group treatment differs

from individual treatment in that the individual is able to process their own response to

treatment in the context of others’ progress. Group therapy offers a dynamic environment

in which a patient’s lack of change may be highlighted or normalized, and achievements

encouraged, depending on the progress of the group in which they are treated. Thus the

process of feedback within group settings may be a qualitatively difference experience

for patients and clinicians, and this requires further examination.

Second, inpatient and day patient therapy differs to outpatient therapy in that it is

time-limited, the therapy provided is intensive, and the severity of the sample is likely to

be greater, which may limit the utility of feedback in this setting. Lambert and

colleagues’ work suggests that treatment for those patients deemed on track during

therapy concludes sooner than those not on track when feedback is provided [1]. Thus

when the dose of therapy is fixed, it appears that feedback presents the opportunity to

attend more closely to those patients not responding.

Third, the current study employed the WHO-5 as a measure of patient progress,

which is a freely-available, quick, reliable and valid measure of individual progress and

outcome in psychiatric settings [19]; but has a ceiling effect. The low cut-off for

movement into the healthy range means that a large proportion of patients’ distress is not

captured by the measure, and similarly, the range of movement is restricted. It is

recommended that future patient-focused research includes the measurement of

symptoms which may be more sensitive to acute changes. In addition to this point,

outcome on the WHO-5 was measured at the beginning of Day 9, which does not

accurately reflect the patient’s health status at conclusion of the group (end of Day 10).

Thus, the symptom measures portray outcome more precisely.

The present study was conducted with a historical cohort design, and thus patients

were not randomized to conditions. The reliable finding that progress monitoring and

feedback improves outcomes has been demonstrated in a series of randomized controlled

trials [9-12], and thus the current study moved to extend the patient-focused methodology

to a novel setting. Whilst the study design reflects the real-world nature of the setting,

conclusions cannot be drawn on the reliability of improvements in symptom reduction

among inpatients and day patients without rigorous follow-up.

Patient-focused research has the potential to bridge the scientist-practitioner gap

and improve patient outcomes [8, 33]. The current findings provide limited support for

this proposition. Patient monitoring and feedback presents an opportunity to improve

safety issues, increase the reliability of outcome measurement, and foster a more

collaborative relationship between clinician and patient. In this regard, the patient is

informed about their progress in relation to their expected progress, and accordingly,

empowered to make decisions about their treatment management; an objective that

should be highlighted more frequently in psychotherapy research.

Acknowledgments:

The authors would like to thank Mrs Moira Munro, Perth Clinic, for ongoing support and

assistance. This study was supported by grants from the Medibank Private Safety and

Clinical Improvement Incentive Pool, and a PhD Completion Scholarship awarded by the

University of Western Australia.

References

1 Lambert MJ: Presidential address: What we have learned from a decade of

research aimed at improving psychotherapy outcome in routine care. Psychotherapy

Research 2007;17:1-14.

2 Newnham EA, Harwood KE, Page AC: Evaluating the clinical significance of

responses by psychiatric inpatients to the mental health subscales of the SF-36. Journal of

Affective Disorders 2007;98:91-97.

3 Parabiaghi A, Barbato A, D'Avanzo B, Erlicher A, Lora A: Assessing reliable and

clinically significant change on HoNOS: A method for displaying longitudinal data.

Australian and New Zealand Journal of Psychiatry 2005;39

4 Lambert MJ, Whipple JL, Vermeersch DA, Smart DW, Hawkins EJ, Nielsen SL,

al. e: Enhancing psychotherapy outcomes via providing feedback on patient progress: A

replication. Clinical Psychology and Psychotherapy 2002;9:91-103.

5 Hansen NB, Lambert MJ, Forman EM: The psychotherapy dose-response effect

and its implications for treatment delivery services. Clinical Psychology: Science and

Practice 2002;9:329-343.

6 Page AC, Stritzke WGK: Clinical Psychology for Trainees: Foundations of

Science-Informed Practice. Cambridge, UK, Cambridge University Press, 2006.

7 Lambert MJ, Harmon C, Slade K, Whipple JL, Hawkins EJ: Providing feedback

to psychotherapists on their patients' progress: Clinical results and practice suggestions.

Journal of Clinical Psychology 2005;In Session 61:165-174.

8 Lutz W: Efficacy, effectiveness, and expected treatment response in

psychotherapy. Journal of Clinical Psychology 2003;59:745-750.

9 Hawkins EJ, Lambert MJ, Vermeersch DA, Slade K, Tuttle K: The effects of

providing patient progress information to therapists and patients. Psychotherapy Research

2004;14:308-327.

10 Lambert MJ, Whipple JL, Smart DW, Vermeersch DA, Nielsen SL, Hawkins EJ:

The effects of providing therapists with feedback on patient progress during

psychotherapy: Are outcomes enhanced? Psychotherapy Research 2001;11:49-68.

11 Whipple JL, Lambert MJ, Vermeersch DA, Smart DW, Nielsen SL, Hawkins EJ:

Improving the effects of psychotherapy: The use of early identification of treatment

failure and problem solving strategies in routine practice. Journal of Counseling

Psychology 2003;58:59-68.

12 Harmon C, Lambert MJ, Smart DW, Hawkins EJ, Nielsen SL, Slade K, Lutz W:

Enhancing outcome for potential treatment failures: Therapist/client feedback and clinical

support tools. Psychotherapy Research 2007;17:379-392.

13 Lutz W, Leach C, Barkham M, Lucock M, Stiles WB, Evans C, Noble R, Iveson

S: Predicting change for individual psychotherapy clients on the basis of their nearest

neighbors. Journal of Consulting and Clinical Psychology 2005;73:904-913.

14 McHorney CA, Tarlov AR: Individual-patient monitoring in clinical practice: Are

available health surveys adequate? Quality of Life Research 1995;4:293-307.

15 Lambert MJ, Hawkins EJ: Measuring outcome in professional practice:

Considerations in selecting and using brief outcome instruments. Professional

Psychology: Research and Practice 2004;35:492-499.

16 Newnham EA, Page AC: Client-focused research: New directions in outcome

assessment. Behaviour Change 2007;24:1-6.

17 Page AC, Hooke GR, Rampono J: A methodology for timing reviews of inpatient

hospital stay. Australian and New Zealand Journal of Psychiatry 2005;39:198-201.

18 Bech P, Gudex C, Johansen KS: The WHO (Ten) Well-being Index: Validation in

Diabetes. Psychotherapy and Psychosomatics 1996;65:183-190.

19 Newnham EA, Hooke GR, Page AC: Monitoring treatment response and

outcomes using the World Health Organization's Wellbeing Index in psychiatric care.

Journal of Affective Disorders in press

20 National Centre for Classification in Health Publications: The International

Statistical Classification of Diseases and Related Health Problems, Tenth Revision,

Australian Modification (ICD-10-AM). Geneva, National Centre for Classification in

Health Publications, 2002.

21 Löwe B, Spitzer RL, Grafe K, Kroenke K, Quenter A, Zipfel S, et al.:

Comparative validity of three screening questionnaires for DSM-IV depressive disorders

and physician's diagnoses. Journal of Affective Disorders 2004;78:131-140.

22 Bonsignore M, Barkow K, Jessen F, Heun R: Validity of the five-item WHO

Wellbeing Index (WHO-5) in an elderly population. European Archives of Psychiatry

and Clinical Neuroscience 2001;251:II27-II31.

23 Ware JE, Snow KK, Kosinski M, Gandek B: SF-36 Health Survey: Manual and

Interpretation Guide. Boston, The Health Institute, New England Medical Centre, 1993.

24 Scott KM, Tobias MI, Sarfati D, Haslett S: SF-36 health survey reliability,

validity and norms for New Zealand. Australian and New Zealand Journal of Public

Health 1991;23:401-406.

25 Lovibond SH, Lovibond PF: Manual for the Depression Anxiety Stress Scales ed

2nd. Sydney, Psychology Foundation, 1995.

26 Crawford J, Henry JD: The Depression Anxiety Stress Scales (DASS): Normative

data and latent structure in a large non-clinical sample. British Journal of Clinical

Psychology 2003;42:111-131.

27 Lovibond SH, Lovibond PF: The structure of negative emotional states:

Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression

and Anxiety Inventories. Behavior Research and Therapy 1995;33:335-343.

28 Antony MM, Bieling PJ, Cox BJ, Enns MW, Swinson RP: Psychometric

properties of the 42-item and 21-item versions of the Depression Anxiety and Stress

Scales (DASS) in clinical groups and a community sample. Psychological Assessment

1998;10:176-181.

29 Wing JK, Beevor AS, Curtis RH, Park SBG, Hadden S, Burns A: Health of the

Nation Outcome Scales (HoNOS): Research and development. British Journal of

Psychiatry 1998;172:11-18.

30 Page AC, Hooke GR: Outcomes for depressed and anxious inpatients discharged

before or after group cognitive behavior therapy: A naturalistic comparison. Journal of

Nervous and Mental Disease 2003;191:653-659.

31 Jacobson NS, Truax P: Clinical significance: A statistical approach to defining

meaningful change in psychotherapy research. Journal of Consulting and Clinical

Psychology 1991;59:12-19.

32 Howard KI, Lueger RJ, Maling MS, Martinovich Z: A phase model of

psychotherapy: Causal mediation of outcome. Journal of Consulting and Clinical

Psychology 1993;61:678-685.

33 Howard KI, Moras K, Brill P, Martinovich Z, Lutz W: Evaluation of

psychotherapy: Efficacy, effectiveness, and patient progress. American Psychologist

1996;51:1059-1064.

Figure 1.

Example feedback graph of WHO-5 scores for a patient responding to feedback at day 5.

Table 1

Log linear upper and lower scores for the expected treatment response trajectories for

feedback.

WHO-5 Score at Admission Day 1 Day 3 Day 5 Day 7 Day 9

0-3 Upper 3.43 7.66 10.13 11.89 13.25

Lower 1.64 3.97 5.33 6.30 7.05

4-6 Upper 6.65 10.54 12.81 14.43 15.68

Lower 4.95 6.97 8.15 8.99 9.64

7-9 Upper 8.79 12.22 14.23 15.66 16.76

Lower 7.37 8.62 9.35 9.87 10.27

10-13 Upper 12.20 14.45 15.77 16.71 17.43

Lower 10.27 11.11 11.61 11.96 12.23

14-25 Upper 17.54 18.85 19.62 20.17 20.59

Lower 14.73 14.65 14.60 14.57 14.55

Table 2

Means and standard deviations (in parentheses) for each cohort across all outcome

measures.

Measure Control No Feedback Feedback

Day 1 8.10 (5.15) 8.12 (5.31) WHO-5

Day 9 13.21 (5.93) 12.98 (6.07)

Depression (Pre) 26.04 (12.92) 25.28 (13.17) 24.13 (12.69)

Depression (Post) 13.08 (11.29) 13.86 (11.07) 12.40 (10.79)

Anxiety (Pre) 20.00 (12.00) 19.24 (11.36) 19.57 (11.94)

Anxiety (Post) 11.41 (9.77) 11.41 (9.37) 10.92 (9.37)

Stress (Pre) 26.93 (11.38) 26.29 (11.23) 26.03 (11.19)

DASS-21

Stress (Post) 15.95 (10.47) 16.37 (10.67) 15.21 (10.07)

Mental Health (Pre) 42.00 (20.68) 43.46 (20.65) 43.69 (20.98)

Mental Health (Post) 62.21 (19.92) 64.27 (19.90) 65.70 (19.49)

Vitality (Pre) 32.00 (21.81) 33.39 (21.68) 33.94 (22.69)

Vitality (Post) 48.88 (22.33) 51.01 (22.24) 52.77 (22.10)

Role Emotion (Pre) 28.25 (36.77) 26.55 (35.89) 26.54 (35.41)

Role Emotion (Post) 57.05 (41.23) 58.40(41.47) 60.40 (40.16)

Social Function (Pre) 38.71 (27.00) 40.33 (26.12) 40.47 (26.81)

SF-36

Social Function (Post) 61.73 (27.39) 61.82 (27.69) 65.58 (25.86)

Total (Pre) 11.17 (5.50) 10.32 (4.53) 12.15 (3.89) HoNOS

Total (Post) 5.44 (4.06) 5.13 (3.39) 6.65 (4.05)

Table 3

Correlations between HoNOS post-treatment scores and patient self-report outcome measures across cohorts.

WHO-5

Day 1

WHO-5

Day 9

Depression

Anxiety

Stress

Role

Emotion

Vitality

Mental

Health

Social

Function

Control

- -

.385**

.350**

.326**

-.276**

-.297**

-.390**

-.334**

No Feedback

-.218**

-.397**

.467**

.366**

.444**

-.366**

-.373**

-.441**

-.482**

HoNOS

Score

Feedback

-.275**

-.476**

.518**

.485**

.477**

-.434**

-.417**

-.549**

-.498**

** p<.01