Publication Bias in Meta-Analyses of the Efficacy of PsychotherapeuticInterventions for Depression

Helen Niemeyer, Jochen Musch, and Reinhard PietrowskyHeinrich Heine University

Objective: The aim of this study was to assess whether systematic reviews investigating psychothera-peutic interventions for depression are affected by publication bias. Only homogeneous data sets wereincluded, as heterogeneous data sets can distort statistical tests of publication bias. Method: We appliedBegg and Mazumdar’s adjusted rank correlation test, Egger’s regression analysis, and the trim and fillprocedure to assess the presence and magnitude of publication bias in all homogeneous data sets ofsystematic reviews published up to September 2010. Results: Thirty-one data sets reported in 19meta-analyses fulfilled our inclusion criteria. Significant bias was detected in 5 (16.13%; rank correlationtest) and 6 (19.35%; Egger’s regression analysis) of these data sets. Applying the trim and fill procedureto amend presumably missing studies rarely changed the assessment of the efficacy of therapeuticinterventions, with 2 exceptions. In 1 data set psychotherapy was no longer found to be significantly moreefficacious than pharmacotherapy in reducing dropout at posttreatment when publication bias was takeninto account. In the 2nd data set, after correcting for publication bias, there was no longer evidence thatdepressed patients without comorbid personality disorder profited more from psychotherapy and phar-macotherapy than patients with comorbid personality disorder. Conclusions: The results suggest thattaken together, psychotherapy research for depression is only marginally affected by the selectivereporting of positive outcomes. With 2 notable exceptions, correcting for publication bias did not changethe evaluation of the efficacy of psychotherapeutic interventions.

Keywords: depression, publication bias, meta-analysis, psychotherapy research

Epidemiological studies report lifetime prevalences for depres-sion of 11.1%–14.6% (Bromet et al., 2011) and worldwide 12-month prevalences between 5.5% and 5.9% (Kessler et al., 2010).Twelve-month prevalence estimates are as high as 2.7% for majordepression even among U.S. children (Merikangas, He, Brody, etal., 2010), and a lifetime prevalence rate in U.S. adolescents of11.7% for major depressive disorder or dysthymia has been re-ported (Merikangas, He, Burstein, et al., 2010). As depressionimpairs function and is linked to an increased risk of suicide(Holma et al., 2010), efficacious treatments are essential to reducethe suffering. Several evidence-based treatments with high effi-cacy do exist, such as cognitive behavioral therapy (CBT) orinterpersonal therapy, and their efficacy for the treatment of de-pression has been investigated in a large number of empiricalstudies. Similarly, prevention programs have also been evaluatedthoroughly. To obtain an overview over the existing wealth ofstudies and to achieve results with increased inferential power and

validity, meta-analyses have been used to synthesize the results ofthese primary studies (Rustenbach, 2003). However, meta-analyses also suffer from several limitations, and one of the mostsevere threats to the validity of meta-analyses is publication bias(Rothstein, Sutton, & Borenstein, 2005). The present study aims ataddressing this bias in the evidence for the efficacy of psychother-apeutic and preventive interventions for depression.

Publication bias is a systematic bias characterized by the selec-tive publication of studies with positive results, as opposed tostudies with null or negative results (Hopewell, Clarke, & Mallett,2005). The state-of-the-art procedures used in evidence-based psy-chotherapy derive in the main from published rather than unpub-lished research (Gilbody & Song, 2000). Published results aremost likely to be retrieved in literature searches, and thereforemore often included in meta-analyses (Jackson, 2006). To obtainmore valid results, it is important to address the risk of publicationbias. If the sample of included studies is flawed, the validity of theresults of a meta-analysis may be seriously compromised, even ifthe analysis is of otherwise high quality. Publication bias may leadto an overestimation of the efficacy of therapeutic interventions,and the additional consideration of unpublished results mightreduce the overall effect size estimate and have severe practicalimplications if it changes the assessment of the efficacy of atherapeutic intervention (Rustenbach, 2003).

Therapeutic interventions in clinical practice should be evidencebased; that is, clinical decision making should be based on “bestpractice” (Slade & Priebe, 2001). If publication bias is not takeninto account, psychotherapists may be led to using interventionsseemingly based on apparent empirical evidence that, however, are

This article was published Online First December 17, 2012.Helen Niemeyer, Jochen Musch, and Reinhard Pietrowsky, Depart-

ment of Experimental Psychology, Heinrich Heine University, Düssel-dorf, Germany.

We would like to thank Helen-Rose Cleveland and Annett Schmitz fortheir helpful technical assistance.

Correspondence concerning this article should be addressed to HelenNiemeyer, now at Department of Health Sciences, Institute for HealthSociology, University of Potsdam, Am Neuen Palais 10, 14469 Potsdam,Germany. E-mail: helen.niemeyer@hhu.de

Journal of Consulting and Clinical Psychology © 2012 American Psychological Association2013, Vol. 81, No. 1, 58–74 0022-006X/13/$12.00 DOI: 10.1037/a0031152

58

only erroneously supported by the results of meta-analyses (Berlin& Ghersi, 2005). Treatments that are less efficacious than assumedalso result in unnecessarily high costs for society and the healthcare system.

Sterling (1959) conducted the first analyses of publication biasand found that in psychology, most of the published studies re-ported significant results. This finding was subsequently widelyconfirmed for the fields of social science (Glass, McGaw, &Smith, 1981; Smith, 1980) and medicine (Moscati, Jehle, Ellis,Fiorello, & Landi, 1994; Vickers, Gloyal, Harland, & Rees, 1998).As publication bias can occur in all fields of research, approachesto detect and reduce publication bias should always be applied.They encompass the search for unpublished studies when conduct-ing a meta-analysis, the registration of all studies at the time oftheir inception, and a search for publication bias in meta-analyseswith formal statistical methods (Berlin & Ghersi, 2005; Moshagen& Musch, 2008; Rustenbach, 2003).

Quantitative and hence objective methods to detect publicationbias are available and include Begg and Mazumdar’s rank corre-lation method (Begg & Mazumdar, 1994), Egger’s regressionanalysis (Egger, Davey Smith, Schneider, & Minder, 1997), andthe trim and fill procedure (Duval & Tweedie, 2000; Taylor &Tweedie, 2000). These methods, all based on the funnel plot, testfor asymmetry in the dispersion of the primary studies’ effect sizesaround the mean effect estimate. If bias is detected, the trim andfill procedure additionally computes the number of presumablymissing studies and a revised effect size estimate that is correctedfor bias (Duval, 2005).

The key assumption of all methods based on the funnel plot isthat observed asymmetries are due to publication bias. It is impor-tant to note, however, that the cause of an asymmetry cannot bedetermined from the asymmetry itself (Ioannidis & Trikalinos,2007; Sterne, Gavaghan, & Egger, 2000; Terrin, Schmid, Lau, &Olkin, 2003). Rather, asymmetry can result from many causes,including heterogeneous effects, undetected covariates, method-ological inadequacies, and chance (Sterne, Becker, & Egger,2005). If there is significant between-study heterogeneity in a dataset, researchers usually try to find the reasons for this heterogene-ity by applying techniques such as subgroup analysis or meta-regression (Borenstein, Hedges, Higgins, & Rothstein, 2009). Sta-tistical methods based on a homogeneity assumption areinappropriate and lead to false alarms if evidence for heterogeneityis found (Ioannidis & Trikalinos, 2007; Sterne et al., 2000; Terrinet al., 2003). In a reanalysis of existing data sets, Ioannidis andTrikalinos (2007) found that in most meta-analyses the homoge-neity requirement was in fact not met. Moreover, they found thatif publication bias was detected, heterogeneity frequently offeredan alternative explanation for the observed asymmetry.

To detect publication bias in studies investigating the efficacy ofpsychotherapeutic interventions for depression, Cuijpers, Smit,Bohlmeijer, Hollon, and Andersson (2010) recently conducted ameta-analytic study on publication bias. They selected all random-ized controlled studies in which the efficacy of different psycho-therapeutic treatments for depression was investigated from adatabase of published studies (http://www.evidencebasedpsycho-therapies.org; Department of Clinical Psychology, Vrije Univer-siteit Amsterdam, 2011). Overall, they included 117 primary stud-ies and calculated pooled mean effect size estimates for CBT, aswell as for a sample of other psychotherapeutic interventions taken

together, and applied Begg and Mazumdar’s rank correlationmethod (Begg & Mazumdar, 1994), Egger’s regression analysis(Egger et al., 1997), and the trim and fill procedure (Duval &Tweedie, 2000; Taylor & Tweedie, 2000) for a statistical assess-ment of publication bias. A substantial degree of publication biaswas detected, and Cuijpers, Smit, et al. therefore concluded thatthe efficacy of both CBT and other types of psychotherapy wasmost likely overestimated. However, heterogeneity was highlysignificant in 40 out of the 43 data sets investigated by Cuijpers,Smit, et al., offering an alternative explanation for their findings.

The present assessment of publication bias was therefore con-ducted to avoid the false positive identification of publication biasin meta-analyses in which in fact heterogeneous studies led toasymmetry in the funnel plot by choosing homogeneity of the dataas inclusion criterion (Ioannidis, 2005; Ioannidis & Trikalinos,2007; Terrin et al., 2003). Moreover, there is a large number ofstudies investigating the efficacy of depression treatments that wasnot included in the meta-analysis of Cuijpers, Smit, et al. (2010).We aimed at providing a more comprehensive assessment ofpublication bias by reassessing all existing meta-analyses of ther-apeutic interventions for the potential presence of publication bias.

A literature search up to September 2010, aimed at identifyingall meta-analyses investigating the efficacy of psychotherapeuticinterventions for depression (not combined with other disorders)and the prevention of depression, resulted in 85 meta-analyses. Inthese meta-analyses the following types of interventions wereinvestigated: CBT, behavioral activation, problem-solving ther-apy, the “coping with depression” course, interpersonal therapy,relaxation, psychoeducation, a combination of several types ofpsychotherapy (including some of the aforementioned as well asnondirective therapy, for example), a combination of psychother-apy and pharmacotherapy, couple therapy, group therapy, Internet-based therapy, short-term psychodynamic therapy, positive psy-chology, reminiscence therapy, case management in primaryhealth care, and preventive interventions against depression. Someof these interventions were investigated in particular samples, suchas adolescents, the elderly, in- and outpatient groups, women afterbirth, and patients suffering from somatic diseases and comorbiddepression (all of these meta-analyses are marked with an asteriskin the list of references). Most of these meta-analyses providedevidence that the therapies investigated are of moderate to largeefficacy. However, the validity of meta-analyses that are limited tothe inclusion of published studies is severely threatened by asystematic bias (Rustenbach, 2003), and only in 16 of the 85meta-analyses (18.82%), unpublished studies were included(Bohlmeijer, Smit, & Cuijpers, 2003; Bortolotti, Menchetti,Bellini, Montaguti, & Berardi, 2008; Cuijpers, 1998a; Cuijpers, Li,Hofmann, & Andersson, 2010; Cuijpers, van Straten, & Smit,2006; Driessen et al., 2010; Ekers, Richards, & Gilbody, 2008;Gloaguen, Cottraux, Cucherat, & Blackburn, 1998; Gregory,Schwer Canning, Lee, & Wise, 2004; Kühner, 2003; Mazzucchelli,Kane, & Rees, 2009; Michael & Crowley, 2002; Neumeyer-Gromen, Lampert, Stark, & Kallischnigg, 2004; Sin & Lyubomir-sky, 2009; Spek et al., 2007; Weisz, McCarty, & Valeri, 2006).Without the use of formal statistical methods to detect the presenceof publication bias, it is impossible to tell whether the selectivepublication of positive results has distorted the effect size esti-mates reported in these meta-analyses. Unfortunately, most of themeta-analyses did not employ formal statistical methods, or em-

59PSYCHOTHERAPEUTIC INTERVENTIONS FOR DEPRESSION

ployed only one of several statistical methods that are useful forthis purpose. In particular, the trim and fill procedure (Duval &Tweedie, 2000; Taylor & Tweedie, 2000) had been conductedin only 12 (14.12%) of these 85 meta-analyses (Andersson &Cuijpers, 2009; Cuijpers, Brannmark, & van Straten, 2008; Cuijpers,Li, et al., 2010; Cuijpers, Smit, et al., 2010; Cuijpers, Smit, & vanStraten, 2007; Cuijpers, van Straten, Andersson, & van Oppen,2008; Cuijpers, van Straten, Schuurmans, et al., 2010; Cuijpers,van Straten, Hollon, & Andersson, 2010; Cuijpers, van Straten,Smit, Mihalopoulos, & Beekman, 2008; Driessen et al., 2010;Mazzucchelli et al., 2009; Watanabe, Hunot, Omori, Churchill, &Furukawa, 2007). Moreover, in only three of the 85 meta-analyses(3.53%), Begg and Mazumdar’s (1994) rank correlation test hadbeen conducted (Cuijpers, Smit, et al., 2010; Ekers et al., 2008;Himelhoch, Medoff, & Oyeniyi, 2007), and only six meta-analyses(7.06%) applied Egger et al.’s (1997) regression analysis(Cuijpers, Smit, et al., 2010; Ekers et al., 2008; Gensichen et al.,2006; Himelhoch et al., 2007; Mazzucchelli et al., 2009; Watanabeet al., 2007). These results indicate a widespread lack of statisticalcontrol for publication bias; existing publication bias may there-fore well have gone undetected in previous meta-analyses, and theefficacy of the interventions therefore may have been overesti-mated. For this reason, we conducted a comprehensive analysis ofpublication bias in meta-analyses of the efficacy of therapeuticinterventions by employing the three most frequently recom-mended formal procedures to detect publication bias.

The present study differs from the meta-analysis of Cuijpers,Smit, et al. (2010) in several ways. First, we reviewed all meta-analyses that have been conducted regarding the prevention andtreatment of depression, as we intended to reassess all of themregarding the possible presence of publication bias. Second, Cui-jpers, Smit, et al. calculated the effect metric d based on therandom-effects and mixed-effects model, whereas we included allavailable effect metrics, also those based on a fixed-effects model.Third, Cuijpers, Smit, et al. focused exclusively on randomizedcontrolled trials, whereas we included all meta-analyses that havebeen conducted, including those that were not restricted to ran-domized controlled trials. Fourth, Cuijpers, Smit, et al. explicitlyexcluded unpublished studies, in contrast to the present analysis.Unlike the present study, Cuijpers, Smit, et al. limited their anal-ysis to depression in adults and excluded studies concerning chil-dren and adolescents. Cuijpers, Smit, et al. also excluded studiesabout disease management programs and the prevention of depres-sion as well as relapse prevention, and studies with inpatientsamples, all of which were included in our analysis with the aim ofa more comprehensive assessment of publication bias. Thus, thepresent analysis is characterized by a broader focus. However,whereas Cuijpers, Smit, et al. applied no language restrictions, welimited our analysis to meta-analyses reported in English andGerman. Moreover, although Cuijpers, Smit, et al. included ahigher number of primary studies in most of their data sets, almostall of their data sets are heterogeneous, and their assessment ofpublication bias may therefore have led to an unknown numberof false positives, which are unavoidably the result of a violationof the homogeneity assumption (Ioannidis & Trikalinos, 2007).We therefore limited our analysis to homogeneous data sets, inwhich this important precondition for an unbiased assessment ofthe presence of publication bias was met.

Method

Data Sources

To comprehensively identify all relevant meta-analyses on psy-chotherapeutic interventions and prevention programs for depres-sion, we conducted a literature search in the databases PsycINFOand PSYNDEX, as well as in reference lists of articles and bookchapters, following the search strategies recommended by Lipseyand Wilson (2001). Articles were retrieved for further assessmentif the title or abstract suggested that a meta-analysis concerningpsychotherapy or prevention methods for depression had beenconducted. All available meta-analyses up to September 2010 wereincluded. The keywords meta-analysis and systematic review com-bined with depression, major depression, depressive disorder, anddysthymia were used in all variations. The search was restricted toreviews and meta-analyses reported in either English or German.

Study Selection and Data Extraction

All articles, including those for which the abstract providedinsufficient information, were thoroughly examined in order not tomiss any relevant meta-analyses. Eligible meta-analyses had toinvolve major depression or dysthymic disorder according to thediagnostic criteria of either the Diagnostic and Statistical Manualof Mental Disorders (3rd ed., American Psychiatric Association[APA], 1980; 3rd ed., rev., APA, 1987; 4th ed., APA, 1994; 4thed., rev., APA, 2000) or the International Classification of Dis-eases (9th ed., Degkwitz, Helmchen, Kockott, & Mombour, 1980;10th ed., World Health Organization, 1992). Dysphoric sampleswith high scores on self-rating scales for depressive symptomatol-ogy were also included. Eligible meta-analyses for the preventionof depression or relapses did not have to include samples with acurrent episode of major depression, but had to measure depressivesymptoms before and after the application of the intervention. Theintervention also had to aim at preventing the development of adepressive disorder or a relapse. At least one psychotherapeutic orpreventive intervention had to be involved to make a meta-analysiseligible for inclusion. Interventions were not restricted to a specificschool of psychotherapy. Reviews in which primary studies withvarious disorders were pooled for the mean effect size calculations,and that thus were not restricted to the treatment of depression,were excluded. Finally, all meta-analyses were excluded that ex-clusively targeted the effectiveness of pharmacological treatmentor electroconvulsive treatment, without including any additionalcomparisons between pharmacotherapy and psychotherapy or datasets that merely dealt with the efficacy of psychotherapy.

A pooled effect size estimate, the primary studies’ effect sizes,and the measures of their precision are necessary to assess publi-cation bias. All arms of studies that fulfilled the following inclu-sion criteria were therefore included: (a) all effect sizes of theprimary studies, or raw data for their calculation, were given; (b)a pooled effect size estimate was reported and could be confirmedwith the original primary studies’ effect sizes; and (c) a measure ofprecision (confidence interval, standard deviation, standard error,or variance) was available for the raw data or primary studies’effect sizes. If the necessary data were not reported, an attempt wasmade to obtain them from the authors. If the original study re-ported a correlation coefficient but no measure of precision, the

60 NIEMEYER, MUSCH, AND PIETROWSKY

sample size was used to calculate this measure. An additionalinclusion criterion was that (d) data sets were homogeneous, toavoid a violation of the respective assumption underlying allstatistical methods to assess funnel plot asymmetry. Homogeneitywas assessed by means of the Q statistic. This statistic tests the nullhypothesis that effect sizes from each of the studies are similarenough that a common population effect size can be estimated, anda significant Q value rejects the null hypothesis of data homoge-neity (Borenstein et al., 2009; Lipsey & Wilson, 2001). Further-more, (e) at least six studies had to be included, because thedetection of publication bias is unreliable and of low power (Sterneet al., 2005) if less than six studies are analyzed (Egger et al.,1997). Finally, (f) we only included studies for which an effect sizeestimate corrected for publication bias had not yet been calculatedwith the trim and fill procedure.

Assessment of Publication Bias

As mentioned above, several procedures have been proposed forthe assessment of publication bias. The most widely used methodsare based on the funnel plot (Light & Pillemer, 1984). Funnel plotschart the primary studies’ effect sizes (the treatment estimates)against a measure of their precision. If no bias is present, thefunnel plot is symmetric about the mean effect size, with equaldispersion around the mean at any level of precision. The effectsize estimates at the bottom of the graph from smaller and thus lessprecise studies will vary more than those from larger studies, andwill therefore scatter more widely at the base of the plot. Thespread narrows toward the top of the funnel, due to the higherprecision of larger studies. Under the premise that smaller studieswith null or negative effects are nonrandomly excluded frompublication, the funnel plot becomes asymmetrical at its base(Gilbody & Song, 2000; Sterne et al., 2005). Since a visualassessment of asymmetry in the dispersion is subjective and thusunreliable (Terrin et al., 2003), three objective methods have beendeveloped, all of which were applied in the present study: Beggand Mazumdar’s rank correlation method (Begg & Mazumdar,1994), Egger’s regression analysis (Egger et al., 1997), and thetrim and fill procedure (Duval & Tweedie, 2000; Taylor &Tweedie, 2000).

Begg and Mazumdar’s rank correlation method and Egger’sregression analysis both investigate whether a correlation betweenthe variances and the effect sizes of the primary studies is present.In case of publication bias small studies with higher variances aremore likely to be published if they show large effect sizes. Thisleads to a positive association between variance and effect size,which is also called small study bias. The nonparametric adjustedrank correlation method detects small study effects by assigningranks to the standardized effect sizes and their variances andexamining the rank correlation between them. There is no rela-tionship between the effect sizes and their variances in the absenceof bias. The significance of correlation between the ranks is testedusing Kendall’s � (Sterne & Egger, 2005). In the parametric linearregression method the standard normal deviate—effect size di-vided by its standard error—is regressed against its precision,given by the inverse of its standard error. Magnitude and directionof the effect are indicated by the slope, while the intercept providesa measure of the degree of asymmetry, in which case the interceptdoes not run through the origin (Sterne & Egger, 2005). We

conducted both one- and two-sided significance tests with theType I error level set at .05. This decision was made becausewhereas under the typical scenario of publication bias, a one-tailedtest is usually conducted (Cuijpers, Smit, et al., 2010), in a scenariowhere larger effects in larger studies may also be observed, atwo-tailed test is more appropriate (Sterne & Egger, 2005).

Neither Begg and Mazumdar’s nor Egger’s method adjusts forpublication bias. Beyond the mere existence of bias, the nonpara-metric trim and fill procedure (Duval & Tweedie, 2000; Taylor &Tweedie, 2000) provides a corrected effect size estimate taking thebias into account. Rather than investigate the correlation betweenthe effect size and its variance, the mean effect estimate is used asa fixed point to test which studies with positive effect sizes have nomirror image counterparts with negative effect sizes. In an algo-rithm the rightmost studies in the funnel plot with the largest effectsizes, which do not have counterparts on the left side of the meaneffect size estimate, are trimmed off, and the mean effect isreestimated. Then, the studies and their missing counterparts areimputed. After a few iterations the trimmed and filled funnel plothas usually become symmetric and the procedure stops, outputtinga new bias-corrected effect estimate and the number of putativemissing studies. The corrected effect estimate is usually lower thanthe original estimate. The 95% confidence interval is recomputed,taking the observed and the inserted studies into account. Wetested the difference between the original and the corrected effectsize estimate provided by trim and fill for significance by inves-tigating whether the original still fell between the confidence limitsof the bias-corrected effect estimate. Importantly, the number ofimputed studies should not be regarded as the definite number ofmissing studies. The procedure should rather be seen as a sensi-tivity analysis of how robust the original effect size estimate isagainst a possible publication bias (Duval, 2005).

All data sets were analyzed with the Comprehensive Meta-Analysis software (Version 2.2; Borenstein, Hedges, Higgins, &Rothstein, 2005). The various methods for the detection of publi-cation bias can be applied after entering either the primary studies’effect size statistics or the raw data. The methods can be applied toeach effect size statistic (mean differences, correlations, or dichot-omous metrics).

Results

Data Selection

Of 85 meta-analyses, 19 provided data sets that fulfilled allinclusion criteria and qualified for reanalysis in the present study.Our reanalysis included 31 data sets from these meta-analyses,which were homogeneous and contained more than six primarystudies. We decided not to include the posttreatment and follow-upmean effects that were additionally reported in these 19 meta-analyses and assessed the efficacy of therapeutic interventions orprevention programs for depression, because we found that thesedata sets did not fulfill the inclusion criteria of (a) providing theeffect sizes of the included primary studies or raw data for theircalculation, (b) reporting a pooled effect size estimate that could beconfirmed with the primary studies’ effect sizes, (c) providing ameasure of precision for the raw data or primary studies’ effectsizes, (d) reporting homogeneous effects, (e) providing a data baseof at least six studies, and (f) lacking a previous correction for

61PSYCHOTHERAPEUTIC INTERVENTIONS FOR DEPRESSION

publication bias with the trim and fill procedure. The completedata set selection process including all 85 meta-analyses is shownin Figure 1.

Description of Meta-Analyses Investigated

We drew one data set from each of the following meta-analyses:Barbato and D’Avanzo (2008); Beltman, Voshaar, and Speckens(2010); Bortolotti et al. (2008); Cuijpers, van Straten, and Smit(2006); Cuijpers, van Straten, and Warmerdam (2007a); Cuijpers,van Straten, and Warmerdam, (2008); Dennis (2005); Ekers et al.(2008); Haby, Donnelly, Corry, and Vos (2006); and Himelhoch etal. (2007). We were able to include two data sets each fromCuijpers, van Straten, Smits, and Smit (2006); Cuijpers, Muñoz,Clarke, and Lewinsohn (2009); Lynch, Laws, and McKenna

(2010); Neumeyer-Gromen et al. (2004); Newton-Howes, Tyrer,and Johnson (2006); Pampallona, Bollini, Tibaldi, Kupelnick, andMunizza (2004); and Reinecke, Ryan, and DuBois (1998). Threedata sets were provided by de Maat, Dekker, Schoevers, and deJonghe (2007) and four by de Maat, Dekker, Schoevers, and deJonghe (2006). A complete list of all 31 data sets included inthe present study is given in Table 1.

Seven meta-analyses dealt with the efficacy of CBT. In three ofthem, the efficacy of CBT for depression was compared to controlgroups (Haby et al., 2006; Lynch et al., 2010; Reinecke et al.,1998). Two further meta-analyses dealt with the comparison ofbehavioral therapy to CBT (Cuijpers, van Straten, & Warmerdam,2007a; Ekers et al., 2008). In one meta-analysis, the efficacy ofCBT was compared for individual versus group therapy (Cuijpers,

Exclusion criteria Assessed data sets n = 1379*

Excluded n = 376 Measure of precision for primary studies´ effect sizes or raw data not given

Excluded n = 19 Pooled effect size estimate not reported or not confirmed

Excluded n = 98 Heterogeneity of data sets

Excluded n = 61 Less than six studies included

Excluded n = 15 Effect size estimate already corrected for publication bias (using Trim and Fill)

Effect sizes of primary studies or raw data for calculation not given

Excluded n = 810

Included data sets n = 31

Figure 1. Flowchart of the data set selection process for data sets regarding psychotherapy or prevention ofdepression. � Data sets from unpublished meta-analyses are not included because their number is unknown.

62 NIEMEYER, MUSCH, AND PIETROWSKY

Tab

le1

Res

ults

ofth

eT

rim

and

Fil

lP

roce

dure

,B

egg

and

Maz

umda

r’s

Ran

k-C

orre

lati

onT

est,

and

Egg

er’s

Reg

ress

ion

Ana

lysi

s

Dat

ase

tSt

udy

Inte

rven

tion/

depe

nden

tm

easu

re/ti

me

ofm

easu

rem

ent

Ori

gina

lef

fect

size

(95%

CI)

Rep

licat

edef

fect

size

(95%

CI)

Cor

rect

edef

fect

size

a

(95%

CI)

Num

ber

ofst

udie

sin

clud

ed

Num

ber

ofst

udie

sm

issi

ngb

Beg

gan

dM

azum

dar

(�)

and

Egg

er(�

0)

1H

aby

etal

.(2

006)

CB

Tvs

.co

ntro

l/sym

ptom

,fu

nctio

ning

,an

dqu

ality

oflif

eco

mbi

ned/

post

g�

0.54

c([

0.29

,0.7

9])

(RE

)0.

54a

([0.

29,0

.79]

)11

0�

�.2

,�0

��

0.4

2L

ynch

etal

.(2

010)

CB

Tvs

.co

ntro

l/de

pres

sive

sym

ptom

s(H

amilt

onD

epre

ssio

nR

atin

gSc

ale)

/pos

t

g�

�0.

28c

([�

0.45

,�0.

12])

(FE

)�

0.28

a([

�0.

44,�

0.12

])9

0�

�.1

7,�

0�

1.12

3L

ynch

etal

.(2

010)

CB

Tvs

.co

ntro

l/rel

apse

/fo

llow

-up

OR

�0.

53c

([0.

4,0.

72])

(FE

)0.

53a

([0.

4,0.

71])

90

��

�.9

3,�

0�

�0.

17

4R

eine

cke

etal

.(1

998)

CB

Tvs

.co

ntro

l/de

pres

sive

sym

ptom

s,ad

oles

cent

s/po

st

g�

�1.

02c

([�

1.23

,�0.

81])

d�

1.02

a([

�1.

22,�

0.82

])(F

E)

60

��

�.1

9,�

0�

�0.

77

5R

eine

cke

etal

.(1

998)

CB

Tvs

.co

ntro

l/de

pres

sive

sym

ptom

s,ad

oles

cent

s/fo

llow

-up

g�

�0.

61c

([�

0.88

,�0.

35])

d�

0.61

a([

�0.

86,�

0.35

])(F

E)

60

��

�.1

8,�

0�

�0.

04

6C

uijp

ers,

van

Stra

ten,

&W

arm

erda

m(2

008)

Indi

vidu

alvs

.gr

oup

ther

apy/

depr

essi

vesy

mpt

oms/

post

d�

0.20

c([

0.05

,0.3

5])

(RE

)0.

20a

([0.

06,0

.35]

)0.

17a

([0.

01,0

.33]

)15

2�

�.2

3,�

0�

0.83

7C

uijp

ers

etal

.(2

007a

)A

ctiv

itysc

hedu

ling

vs.

cont

rol

grou

p/de

pres

sive

sym

ptom

s/po

st

d�

0.87

c([

0.6,

1.15

])(F

E)

0.87

a([

0.6,

1.2]

)10

0�

�.4

7�,�

0�

3.21

�

8E

kers

etal

.(2

008)

BT

vs.

CT

orC

BT

/dep

ress

ive

sym

ptom

s/po

st

d�

0.08

([�

0.14

,0.3

])(R

E)

0.07

([�

0.11

,0.2

4])

�0.

02([

�0.

19,0

.15]

)12

3�

�.3

4�,�

0�

1.86

�

9B

eltm

anet

al.

(201

0)C

BT

vs.

cont

rol/s

omat

icdi

seas

ean

dde

pres

sion

,de

pres

sive

sym

ptom

s/po

st

d�

�0.

16c

([�

0.27

,�0.

06])

(FE

)�

0.18

a([

�0.

28,�

0.08

])16

0�

��

.43�

†,�

0�

�2.

12�†

10Pa

mpa

llona

etal

.(2

004)

PTan

dan

tidep

ress

ant

vs.

antid

epre

ssan

t/re

spon

ders

/no

nsig

nifi

cant

OR

�1.

86c

([1.

38,2

.52]

)(R

E)

1.86

a([

1.38

,2.5

1])

160

��

0.00

,�0

�0.

71

11Pa

mpa

llona

etal

.(2

004)

PTan

dan

tidep

ress

ant

vs.

antid

epre

ssan

t/dro

pout

/no

nsig

nifi

cant

OR

�0.

86([

0.60

,1.2

4])

(RE

)0.

86([

0.60

,1.2

4])

0.72

([0.

49,1

.06]

)16

4�

�.1

7,�

0�

0.78

12de

Maa

tet

al.

(200

6)PT

vs.

phar

mac

othe

rapy

/dr

opou

t/pos

tR

R�

1.29

c([

1.07

,1.5

7])

(FE

)1.

27a

([1.

05,1

.54]

)1.

19([

0.99

,1.4

2])

103

��

.38,

�0

�1.

07�†

13de

Maa

tet

al.

(200

6)PT

vs.

phar

mac

othe

rapy

/re

mis

sion

/pos

tR

R�

0.91

([0.

79,1

.06]

)(F

E)

0.93

([0.

8,1.

08])

100

��

�.6

�†,�

0�

�1.

51

14de

Maa

tet

al.

(200

6)PT

vs.

phar

mac

othe

rapy

/re

mis

sion

/pos

tO

R�

0.87

([0.

68,1

.10]

)(F

E)

0.9

([0.

72,1

.18]

)10

0�

��

.22,

�0

��

0.23

(tab

leco

ntin

ues)

63PSYCHOTHERAPEUTIC INTERVENTIONS FOR DEPRESSION

Tab

le1

(con

tinu

ed)

Dat

ase

tSt

udy

Inte

rven

tion/

depe

nden

tm

easu

re/ti

me

ofm

easu

rem

ent

Ori

gina

lef

fect

size

(95%

CI)

Rep

licat

edef

fect

size

(95%

CI)

Cor

rect

edef

fect

size

a

(95%

CI)

Num

ber

ofst

udie

sin

clud

ed

Num

ber

ofst

udie

sm

issi

ngb

Beg

gan

dM

azum

dar

(�)

and

Egg

er(�

0)

15de

Maa

tet

al.

(200

6)PT

vs.

phar

mac

othe

rapy

/re

laps

e/fo

llow

-up

RR

�0.

46c

([0.

33,0

.65]

)(F

E)

0.49

a([

0.35

,0.6

9])

60

��

�.3

3,�

0�

�0.

68

16de

Maa

tet

al.

(200

7)PT

vs.

PTan

dph

arm

acot

hera

py/

drop

out/p

ost

RR

�1.

03([

0.82

,1.3

])(F

E)

1.04

([0.

83,1

.31]

)7

0�

��

.05,

�0

�0.

27

17de

Maa

tet

al.

(200

7)PT

vs.

PTan

dph

arm

acot

hera

py/

rem

issi

on/p

ost

RR

�1.

32c

([1.

12,1

.56]

)(F

E)

1.33

a([

1.13

,1.5

6])

70

��

�.0

5,�

0�

�1.

48

18de

Maa

tet

al.

(200

7)PT

vs.

PTan

dph

arm

acot

hera

py/

rem

issi

on/p

ost

OR

�1.

59c

([1.

22,2

.09]

)(F

E)

1.6a

([1.

22,2

.1])

70

��

�.3

3,�

0�

�1.

07

19C

uijp

ers,

van

Stra

ten,

Smits

,&

Smit

(200

6)

Scre

enin

gan

dPT

/dep

ress

ive

sym

ptom

s/po

st

d�

0.58

c([

0.37

,0.7

8])

(FE

)0.

59a

([0.

39,0

.79]

)0.

41a

([0.

23,0

.59]

)8

5�

�.7

3�†,�

0�

2.27

�†

20C

uijp

ers,

van

Stra

ten,

Smits

,&

Smit

(200

6)

Scre

enin

gan

dPT

/dep

ress

ive

sym

ptom

s,on

est

udy

less

/pos

t

d�

0.72

c([

0.45

,0.9

9])

(FE

)0.

76a

([0.

51,1

.02]

)0.

56a

([0.

33,0

.79]

)7

4�

�.6

7�†,�

0�

3.54

�†

21C

uijp

ers,

van

Stra

ten,

&Sm

it(2

006)

PTvs

.co

ntro

l/lat

e-lif

ede

pres

sion

,de

pres

sive

sym

ptom

s/po

st

d�

0.72

c([

0.57

,0.8

7])

(FE

)0.

72a

([0.

57,0

.87]

)0.

69a

([0.

54,0

.84]

)14

2�

�.2

5,�

0�

0.74

22D

enni

s(2

005)

PTvs

.co

ntro

l/pre

vent

ion

ofpo

stna

tal

depr

essi

on,

depr

essi

vesy

mpt

oms/

post

g�

�0.

06([

�0.

37,0

.26]

)(F

E)

�0.

06([

�0.

37,0

.26]

)7

0�

��

.14,

�0

��

0.34

23N

ewto

n-H

owes

etal

.(2

006)

PTan

dph

arm

acot

hera

py,

com

orbi

dPD

vs.

noPD

/RC

Ts,

depr

essi

vesy

mpt

oms/

post

OR

�1.

6c([

1.25

,2.0

6])

(RE

)1.

61a

([1.

25,2

.06]

)1.

29([

0.98

,1.6

9])

145

��

.36�

,�0

�1.

67�†

24N

ewto

n-H

owes

etal

.(2

006)

PT,

com

orbi

dPD

vs.

noPD

/dep

ress

ive

sym

ptom

s/po

st

OR

�1.

74c

([1.

25,2

.42]

)(R

E)

1.74

a([

1.25

,2.4

1])

1.71

a([

1.23

,2.3

7])

101

��

.11,

�0

�0.

72

25C

uijp

ers

etal

.(2

009)

Cop

ing

with

depr

essi

on,

prev

entio

n/de

pres

sive

sym

ptom

s/po

st

RR

�0.

62c

([0.

43,0

.91]

)(R

E)

0.62

a([

0.43

,0.9

1])

60

��

�.2

,�0

��

1.21

26C

uijp

ers

etal

.(2

009)

Cop

ing

with

depr

essi

on,

ther

apy/

depr

essi

vesy

mpt

oms/

post

d�

0.28

c([

0.18

,0.3

8])

(RE

)0.

28a

([0.

19,0

.38]

)0.

19a

([0.

08,0

.3])

188

��

.29�

†,�

0�

1.79

�†

27B

arba

to&

D’A

vanz

o(2

008)

Cou

ple

vs.

indi

vidu

alth

erap

y/dr

opou

t/pos

tR

R�

1.16

([0.

67,2

.01]

)(R

E)

1.16

([0.

67,2

.02]

)6

0�

�.0

7,�

0�

�1.

44

28B

orto

lotti

etal

.(2

008)

PTvs

.G

P/de

pres

sive

sym

ptom

s/po

std

��

0.42

c([

�0.

59,�

0.26

])(F

Ean

dR

E)

�0.

43a

([�

0.59

,�0.

26])

�0.

46a

([�

0.62

,�0.

3])

61

��

.2,�

0�

0.99

64 NIEMEYER, MUSCH, AND PIETROWSKY

van Straten, & Warmerdam, 2008). Finally, the efficacy of CBT inreducing comorbid depression in individuals with somatic diseasescompared to control groups was investigated by Beltman et al.(2010).

The efficacy of psychotherapy combined with pharmacotherapywas compared to pharmacotherapy alone by Pampallona et al.(2004), or psychotherapy alone by de Maat et al. (2007). In ameta-analysis by de Maat et al. (2006), psychotherapy was com-pared to pharmacotherapy. We will refer to combinations of dif-ferent approaches in the same data set as psychotherapy (PT).

Six meta-analyses dealt with the efficacy of assorted psycho-therapeutic approaches and for different samples. Cuijpers, vanStraten, Smits, and Smit (2006) investigated the efficacy of PT fordepressed adolescents, applied after a screening in school. Thesecond meta-analysis investigated PT for older adults with late-lifedepression (Cuijpers, van Straten, & Smit, 2006). The third meta-analysis dealt with the prevention and therapy of postnatal andpostpartum depression (Dennis, 2005). Newton-Howes et al.(2006) compared PT for depressed participants with comorbidpersonality disorders to those without personality disorder. Theefficacy of the coping with depression course as compared tocontrol groups was assessed by Cuijpers et al. (2009). In a lastmeta-analysis, the efficacy of couple therapy was compared toindividual therapy (Barbato & D’Avanzo, 2008).

PT can also be delivered to depressive patients in medicalcontexts, and to patients with somatic diseases and comorbiddepression. Bortolotti et al. (2008) investigated the efficacy of PTdelivered by medical practitioners in primary care, and Neumeyer-Gromen et al. (2004) investigated the efficacy of disease manage-ment programs, including psychoeducation, in reducing depres-sion. The efficacy of PT in individuals with AIDS who suffer fromcomorbid depression was assessed by Himelhoch et al. (2007).

Unpublished studies were included in five of the 19 meta-analyses (Beltman et al., 2010; Cuijpers, van Straten, & Smit,2006; Cuijpers, van Straten, & Warmerdam, 2008; Ekers et al.,2008; Neumeyer-Gromen et al., 2004). A statistical assessment ofpublication bias had been performed in five of the meta-analyses:A fail-safe N was conducted in two (Cuijpers, van Straten, &Warmerdam, 2008; Reinecke et al., 1998), Egger’s regressionmethod was applied in three (Ekers et al., 2008; Himelhoch et al.,2007; Pampallona et al., 2004), and Begg and Mazumdar’s rankcorrelation test was performed in one meta-analysis (Himelhoch etal., 2007). All results of Begg and Mazumdar’s and Egger’s testswere nonsignificant. In five meta-analyses funnel plots were usedto detect asymmetry, albeit not always for the data sets that weincluded (Beltman et al., 2010; Ekers et al., 2008; Himelhoch etal., 2007; Neumeyer-Gromen et al., 2004; Newton-Howes et al.,2006). Some results of these visual examinations of the funnelplots suggested asymmetries that were discussed in the originalanalyses, but only Ekers et al. (2008) and Himelhoch et al. (2007)also used statistical tests, with nonsignificant results.

Results of the Trim and Fill Calculations

Our recalculations of effect sizes with the raw data matched theoriginal ones in most of the 31 data sets included with minordeviations in only a few cases (by .01 for three of the meandifferences d, by .02 and by .04 for one of the mean differences d,respectively; by .01 for three relative risk, by .02 for two relativeT

able

1(c

onti

nued

)

Dat

ase

tSt

udy

Inte

rven

tion/

depe

nden

tm

easu

re/ti

me

ofm

easu

rem

ent

Ori

gina

lef

fect

size

(95%

CI)

Rep

licat

edef

fect

size

(95%

CI)

Cor

rect

edef

fect

size

a

(95%

CI)

Num

ber

ofst

udie

sin

clud

ed

Num

ber

ofst

udie

sm

issi

ngb

Beg

gan

dM

azum

dar

(�)

and

Egg

er(�

0)

29N

eum

eyer

-G

rom

enet

al.

(200

4)

DM

Pvs

.us

ual

prim

ary

care

/dep

ress

ive

sym

ptom

s/fo

llow

-up

RR

�0.

75c

([0.

7,0.

81])

(FE

)0.

75a

([0.

7,0.

8])

100

��

�.0

9,�

0�

�0.

3

30N

eum

eyer

-G

rom

enet

al.

(200

4)

DM

Pvs

.us

ual

prim

ary

care

/dep

ress

ive

sym

ptom

s,no

min

orde

pres

sion

/fol

low

-up

RR

�0.

74c

([0.

69,0

.8])

(FE

)0.

74a

([0.

7,0.

8])

100

��

�.2

,�0

��

1.07

31H

imel

hoch

etal

.(2

007)

Gro

upPT

vs.

cont

rol/d

epre

ssio

nan

dH

IV,

depr

essi

vesy

mpt

oms/

post

d�

0.38

c([

0.23

,0.5

3])

(RE

)0.

38a

([0.

23,0

.53]

)0.

33a

([0.

17,0

.48]

)8

3�

�.4

4�,�

0�

1.47

�

Not

e.C

I�

conf

iden

cein

terv

al;

CB

T�

cogn

itive

beha

vior

alth

erap

y;R

E�

rand

om-e

ffec

tsm

odel

;FE

�fi

xed-

effe

cts

mod

el;

OR

�od

dsra

tio;

BT

�be

havi

oral

ther

apy;

CT

�co

gniti

veth

erap

y;PT

�ps

ycho

ther

apy;

PT�

psyc

hoth

erap

y;R

R�

rela

tive

risk

;PD

�pe

rson

ality

diso

rder

;RC

T�

rand

omiz

edco

ntro

lled

tria

l;G

P�

gene

ralp

ract

ition

erca

re;D

MP

�di

seas

em

anag

emen

tpro

gram

.a

For

publ

icat

ion

bias

.b

Acc

ordi

ngto

trim

and

fill.

cSi

gnif

ican

tef

fect

size

with

a95

%C

Ino

tin

clud

ing

0(f

orR

Ran

dO

R:

CI

not

incl

udin

g1)

.d

The

inte

grat

ion

mod

elw

asno

tsp

ecif

ied

inth

ear

ticle

.†

p�

.05,

two-

side

d.�

p�

.05,

one-

side

d.

65PSYCHOTHERAPEUTIC INTERVENTIONS FOR DEPRESSION

risk, and by .03 for one relative risk; and lastly by .01 and by.03 for one of the odds ratios, respectively). These small dis-crepancies were probably the result of different rounding. Therecalculated effect sizes were used for the statistical assessmentof publication bias. The original and recalculated effect sizesare shown in Table 1.

According to the trim and fill calculations, at least one missingstudy was identified in each of 12 data sets, reported by Bortolottiet al. (2008); Cuijpers et al. (2009); Cuijpers, van Straten, and Smit(2006); Cuijpers, van Straten, Smits, and Smit (2006); Cuijpers,van Straten, and Warmerdam (2008); de Maat et al. (2006); Ekerset al. (2008); Himelhoch et al. (2007); Newton-Howes et al.(2006); and Pampallona et al. (2004). We calculated correctedmean effect size estimates for all asymmetric data sets with theresult that 11 of the 31 effect sizes were somewhat reduced. Oneof the corrected effect size estimates was actually enlarged. How-ever, none of these changes in the effect sizes was significant, andin all but four cases all therapeutic approaches were assessed assignificant both before and after correction. However, two effectsize estimates that were nonsignificant before and after the cor-rection indicated an equal efficacy of two therapeutic approachesinvestigated, and two effect sizes changed from significance tononsignificance, also indicating equal efficacy of the involvedtreatments after the correction for publication bias. Detailed infor-mation about all asymmetric data sets is presented below. Table 1displays results of the trim and fill calculations.

Table 1 shows the estimates of the efficacy of CBT for depres-sion and reports nine data sets (1–9), provided in seven meta-analyses. Two of these data sets (6, 8) showed evidence of publi-cation bias, and additional studies were therefore imputed by thetrim and fill procedure. In Data Set 6, two missing studies wereimputed, but the difference between the original and the correctedeffect size estimates was nonsignificant, indicating that publicationbias did not have a strong impact on the result. Furthermore, boththe original and the corrected effect size estimates themselvesdiffered significantly from zero, proving higher efficacy of indi-vidual CBT as compared to group CBT. In Data Set 8, threestudies were found to be missing, with no significant differencebetween the original and the corrected effect size estimates. Bothpooled effect estimates were nonsignificant, indicating that CBTand BT are equally efficacious. No studies had to be imputed in theremaining data sets (1–5, 7, 9).

Missing studies were found in two of nine data sets estimatingthe efficacy of PT alone compared to pharmacotherapy or a com-bined treatment of PT and pharmacotherapy to pharmacotherapy.Four studies were found to be missing in Data Set 11, but thedifference between the original and the corrected effect size esti-mates was nonsignificant, and the original and corrected effect sizeestimates themselves did not differ significantly from 0. Threestudies were found to be missing in Data Set 12. The estimate ofthe relative risk originally differed significantly from 1, but thecorrected relative risk no longer significantly differed from 1.Thus, the efficacy of PT and pharmacotherapy no longer differedonce publication bias was taken into account. It is our assertionthat psychotherapy has only due to publication bias a significantlygreater effect on dropout than pharmacotherapy, but after correc-tion there is no difference. No missing studies had to be imputedin the other data sets (10, 13–18).

From six meta-analyses investigating the efficacy of variouspsychotherapeutic approaches for an assortment of subsampleswith depressive symptoms, nine data sets fulfilled the inclusioncriteria of the present study. Of these, six were found to beasymmetric. The trim and fill procedure imputed five missingstudies in Data Set 19 and four in Data Set 20. In both data sets thedifference between the original and the corrected effect size esti-mates was nonsignificant. Furthermore, both corrected effect sizesremained significant, indicating that PT is efficacious in reducingdepressive symptoms. In Data Set 21, two missing studies wereimputed by the trim and fill procedure, but again there was nosignificant change between the original and the corrected effectsize estimates. Both the original and the corrected effect sizeestimates themselves differed significantly from 0, and thus PT isefficacious in reducing late-life depression. Data Set 22 needed nocorrection.

Odds ratios larger than 1 indicated that treatment was moreefficacious for depressed samples without comorbid personalitydisorder, than with personality disorder, in the two data sets 23 and24. Effect size estimates in both data sets were reduced by the trimand fill procedure, indicating that six (five in Data Set 23 and onein Data Set 24) primary studies with nonsignificant effects weremissing. In both data sets no significant differences between theoriginal and the corrected effect size estimates occurred. However,in Data Set 23 the original effect size was significant, and thecorrected effect size turned nonsignificant. This indicates thattreatment for patients with and without comorbidity is equallyefficacious. In Data Set 24 both the original and the reduced effectestimates themselves differed significantly from 1, and thus PT fordepressed patients without comorbidity is more efficacious. Nomissing studies were found in Data Set 25. In contrast, the pooledeffect estimate of Data Set 26 was corrected by the trim and fillprocedure. But even though eight missing studies had to be im-puted, the resulting reduction of mean effect size was not signif-icant. The original and corrected effect size estimates themselveswere both significant, indicating efficacy of the coping with de-pression course. Lastly, no missing studies had to be imputed inData Set 27 (see Table 1).

Two data sets from the three meta-analyses investigating theefficacy of therapeutic interventions in medical contexts wereaffected by missing studies. The corrected effect size estimate ofData Set 28 was actually larger after adding one study in favor ofPT, but the difference between the original and the corrected effectsize estimates was nonsignificant. This correction did not changethe assessment of the efficacy of PT, as both the original and thecorrected effect size estimates themselves were significant. Nostudies were missing in Data Sets 29 and 30, indicating that theyare not affected by publication bias. In Data Set 31, three studieswere found to be missing, but with no significant differencebetween the original and the corrected effect size estimates. Fur-thermore, both the original and the corrected effect size estimatesthemselves differed significantly from 0, indicating efficacy ofgroup psychotherapy for reducing depressive symptoms in patientswith HIV.

Figure 2 shows the funnel plot for the data set with the largestnumber of studies estimated to be missing (N � 8; Data Set 26),investigating the reduction of depressive symptoms due to therapywith the coping with depression course (Cuijpers et al., 2009).

66 NIEMEYER, MUSCH, AND PIETROWSKY

Results of the Rank Correlation and RegressionMethod

With the rank correlation method (Begg & Mazumdar, 1994),one-sided testing indicated significant asymmetry in nine data sets,and in five data sets with two-sided testing. The nine asymmetricdata sets found under one-sided testing (7, 8, 9, 13, 19, 20, 23, 26,and 31) refer to the comparison of activity scheduling versuscontrol conditions regarding the reduction of depressive symptoms(7), a comparison of the efficacy of BT versus cognitive therapyand CBT in reducing depressive symptoms (8), the comparison ofCBT versus control groups for the reduction of depressive symp-toms in participants with somatic diseases and comorbid depres-sion (9), the efficacy of PT compared to pharmacotherapy inreducing remission rates (13), two data sets investigating theefficacy of psychotherapeutic treatment for adolescents afterscreening in schools (19 and 20), a comparison of PT and phar-macotherapy for depressed samples with and without comorbidpersonality disorder (23), therapy with the coping with depressioncourse (26), and the efficacy of group psychotherapy compared tocontrol conditions in reducing depressive symptoms in sampleswith a HIV infection and comorbid depression (31). The fiveasymmetric data sets significant under two-sided testing (9, 13, 19,20, and 26) were also significant under one-sided testing.

The linear regression method (Egger et al., 1997) identified asignificant relationship between effect sizes and their variances innine data sets (one-sided testing): in Data Sets 7, 8,1 9, 19, 20, 23,26 and 31, as was the case with Begg and Mazumdar’s method;and for Data Set 12, investigating the efficacy of PT comparedto pharmacotherapy in reducing dropout rates. In six data sets,publication bias was found in two-sided testing (9, 19, 20, and26), as was the case in the rank correlation test, and in 12 and23, as was the case for one-sided testing of the regressionanalysis (see Table 1).

Concordance Between the Three Methods

In the majority of the 31 assessed data sets (21 cases; Data Sets1–6, 10, 11, 14–18, 21, 22, 24, 25, 27–30), none of the methodssignificantly indicated the presence of publication bias. Begg andMazumdar’s and Egger’s tests were not significant in any of these21 data sets, and the occasional imputation of missing studiesaccording to the trim and fill procedure never resulted in a signif-icant difference between the original and the corrected mean effectsize estimates. Thus, there was high concordance between thethree methods for the majority of cases. However, in 10 data setseither Begg and Mazumdar’s or Egger’s method indicated publi-cation bias. When tested for significance one-sided, both methodsled to the same conclusions for eight data sets, for four sets whentested two-sided. Of the 10 data sets in which either Begg andMazumdar’s or Egger’s method indicated bias, the trim and fillprocedure found missing studies in seven cases, but none of thereductions was significant. However, in two of these data sets,the effect size changed from significance to nonsignificance afterthe correction. Thus, the efficacy of the respective interventionswas changed by the publication bias, but the corrected effect sizesindicated an equal efficacy of two treatments in one of these datasets and equal efficacy of PT and pharmacotherapy for depressedpatients with and without comorbid personality disorder in theother data set.

1 In the meta-analysis Data Set 8 was drawn from (Ekers et al., 2008), anonsignificant intercept for Egger’s regression analysis was reported. Eventhough we used the same data, we arrived at a different intercept byapplying the software Comprehensive Meta-Analysis. We did not find anexplanation for this discrepancy.

-2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0

0.0

0.1

0.2

0.3

0.4

Sta

ndar

d E

rror

Standard difference in means

Figure 2. An illustration of the trim and fill method showing the funnel plot for the reduction of depressivesymptoms due to therapy with the coping with depression course (Data Set 26; Cuijpers et al., 2009). Darkcircles: the eight studies identified as missing.

67PSYCHOTHERAPEUTIC INTERVENTIONS FOR DEPRESSION

Discussion

The present study aimed at investigating publication bias inmeta-analyses of the efficacy of psychotherapeutic and preventiveinterventions for depression. An inappropriate application of themethods to detect publication bias was avoided by restricting theanalysis to homogeneous data sets including at least six primarystudies (Egger et al., 1997; Ioannidis & Trikalinos, 2007). Wefound 31 data sets that fulfilled these inclusion criteria. Our resultsindicated only slight bias in most of these data sets. Begg andMazumdar’s rank correlation method (Begg & Mazumdar, 1994)and Egger’s regression analysis (Egger et al., 1997) both indicatedpublication bias in nine data sets (29.03%) for one-sided testing,and in five (16.13%) versus six (19.35%) for two-sided testing.Both methods find essentially the same evidence for selectivepublication of positive results. In 12 data sets, including seven ofthe data sets for which either of these methods indicated bias, thetrim and fill procedure (Duval & Tweedie, 2000; Taylor &Tweedie, 2000) imputed missing studies, but the reduction of thepooled effect size estimates never reached significance. Moreover,the imputation of presumably missing studies only resulted in asignificant change of the effect estimates in two cases. The changeof the respective effect sizes was not large, but as their significancechanged, in the first data set psychotherapy can no longer beregarded as more efficacious than pharmacotherapy after takingpublication bias into account. Thus, the efficacy of both treatmentsneeds to be regarded as equal. However, as this finding is based onmerely one data set, it should not be generalized, and needsconsiderable replication. For pharmacotherapy compared to psy-chotherapy, or combined treatment, other empirical evidences donot consistently prove equal efficacy of the approaches, and formoderate to severe depression combined treatment is regarded asmost efficacious (National Institute for Health and Clinical Excel-lence, 2009). However, most of these evidences have not beentested for publication bias and could possibly be distorted. Fur-thermore, with Data Set 23, psychotherapy and pharmacotherapyhave to be considered to be as effective for depressed patients withcomorbid personality disorder as they are for those without per-sonality disorder. However, this finding again is based on merelyone data set and awaits further replication. A more comprehensiveassessment for publication bias is necessary to generalize thisfinding.

In summary, it is a very important result that despite a minortendency toward a selective publication of positive results, theefficacy of all reviewed interventions remains substantial evenafter correction with the trim and fill procedure. These resultsdemonstrate that publication bias alone cannot explain their con-siderable efficacy. It can be concluded that despite some evidencefor publication bias, CBT and other psychotherapeutic interven-tions can still be considered efficacious and recommended for thetreatment of depression.

However, even though we limited our analyses to data sets witha minimum of six studies, 22 of the 31 data sets still included only10 or fewer studies, and thus we have to make the reservation thatpower for these tests was low to moderate. As power is also afunction of the size of the effect under investigation, for the nullresults we found in the data sets consisting of 10 or fewer studies,we can therefore at least exclude the possibility of a very largepublication bias. In addition, we even found bias in some data sets

despite low power. We can therefore conclude that in most of thedata sets where some evidence for publication bias was found, theefficacy of psychotherapy for depression and of preventive inter-ventions was unchanged by the selective publication of positiveresults, and even in those data sets where it was changed, theefficacy of the interventions remained quite substantial.

The scientist-practitioner model calls for clinical psychologiststo let empirical results inform their work. With the aim of movingaway from opinion and experience in therapeutic decision makingtoward the use of research results in practice, clinical practiceshould be evidence based (Shapiro, 2002). The strength of recom-mendations found in practice guidelines for depression reflects thestrength of the evidence upon which each is based (Chambless &Ollendick, 2001), and it is important that the evidence of meta-analyses on which some of the strongest recommendations arebased be unbiased. Therefore, the topic of the present study is notjust a methodological problem. The most important practical im-plication of our results is that clinicians who treat depressedpatients can rely on the efficacy of the therapeutic interventionsthat have been tested for publication bias and found not to beoverestimated. Thus, our study helps to provide more valid rec-ommendations for these interventions and informs clinical practi-tioners about the unbiased effect sizes that might be expected. Thisalso helps to assure quality in clinical practice (Shapiro, 2002).

Furthermore, if clinical practitioners read in research articlesthat it is not clear whether the effects of psychotherapy are over-estimated, this is confusing and unsettling for them. If they are notassured that the techniques they apply in practice are unbiased andeffective, this uncertainty might well impair the quality of theirapplication of the interventions. Without a doubt the evidence basewill always remain incomplete and its application to many clinicalsituations uncertain, but we are confident that our findings con-tribute to making the evidence we have tested more reliable andhelp strengthen clinicians’ assurance in the application of inter-ventions due to greater confidence in their efficacy.

Finally, with regard to the scientist-practitioner model, our studyalso warns practitioners against blind faith in the results of meta-analyses. They are well advised to inform themselves about valid-ity criteria and to assess evidence for the inclusion of unpublishedstudies to provide unbiased results as well as for the application ofmethods of testing for publication bias in meta-analyses.

Our results, taking heterogeneity into account, are considerablydifferent from those of Cuijpers, Smit, et al. (2010), who found alarge amount of publication bias in almost all of their data sets witha strong impact on mean effect size estimates. As we have pointedout, homogeneity is a necessary prerequisite of the application ofstatistical methods to detect publication bias. Applying methodsbased on the assumption of homogeneity leads to false positives ifthere is heterogeneity in the data (Ioannidis & Trikalinos, 2007).We therefore limited our analysis to homogeneous data sets, tomake sure that heterogeneity is not mistakenly interpreted asevidence for publication bias. The causes for heterogeneity shouldalways be explored and resolved, for example, by moderatoranalyses (Lipsey & Wilson, 2001), but before the reason for anunobserved heterogeneity is identified, no strong meta-analyticconclusions should be drawn on the basis of a heterogeneous set ofdata (Borenstein et al., 2009). The results of Cuijpers, Smit, et al.might be attributable to false positive hits, as the alternativeexplanation of heterogeneity cannot be ruled out for their results.

68 NIEMEYER, MUSCH, AND PIETROWSKY

The large differences between the original and the corrected effectsizes in the analysis of Cuijpers, Smit, et al.—for example, thesignificant reduction of the overall mean effect size estimate fromg � 0.67 to g � 0.42, with 51 imputed studies—is in strongcontrast to the much smaller amounts of missing studies and effectreductions we have found. The difference between the results ofthe two analyses of publication bias is likely due to the emphasison homogeneity in the present study.

Importantly, our analysis also differed from that of Cuijpers,Smit, et al. (2010), as described in the introduction. Cuijpers, Smit,et al. did not recalculate published meta-analyses, but conducted anew meta-analysis. Therefore, the sample of primary studies in-cluded in our reanalysis differs from that in Cuijpers, Smit, et al.The overlap varies from zero to 12 studies between the 19 meta-analyses we included in the present study and Cuijpers, Smit, etal.’s analysis. The small amount of overlap is not surprising, sincewe aimed at including all meta-analyses for psychotherapeutic andpreventive interventions for depression. A possible reason for thediscrepancy between the analysis of Cuijpers, Smit, et al. and thepresent analysis is that a different set of studies was being used.

Moreover, Cuijpers, Smit, et al. (2010) included only publishedstudies in their analyses, whereas some unpublished studies wereincluded in the data sets we assessed, albeit only few. Merely threeof the included data sets comprehended unpublished studies. DataSet 6 included one unpublished study (Cuijpers, van Straten, &Warmerdam, 2008), Data Set 8 two unpublished studies (Ekers etal., 2008), and Data Set 21 one unpublished study (Cuijpers, vanStraten, & Smit, 2006). The inclusion of these four unpublishedstudies may have contributed only little to the relatively smalltendency toward publication bias we found. Publication bias mightbe higher in samples with only published studies, but it is unlikelythat this explains the difference in the results of the two analyses.In summary, however, even if the primary studies differ betweenthe two analyses and the sample in which high publication biaswas found included exclusively published studies (Cuijpers, Smit,et al., 2010), it is very unlikely that this can explain the largepublication bias they found, in contrast to the present results.

Lastly, the significant results of Begg and Mazumdar’s andEgger’s methods in our analysis were dispersed over all ap-proaches. Also the occasional imputation of missing studies by thetrim and fill procedure occurred in different approaches. A smalltendency for selective reporting of positive outcomes seems toaffect the different approaches quite equally, and no intervention isespecially prone to the selective publication of positive results.

Limitations of the Present Study

Publication bias could only be tested for with small to moderatepower for the 22 of the 31 data sets for which 10 or fewer studieswere available. Therefore, the null findings of no publication biasin the smallest data sets should be interpreted with caution due tothe low statistical power. However, Sterne et al. (2005) reportedthat in medical research the number of available studies is oftenless than 10, and Sterne et al. (2000) nevertheless recommendedthat “tests for small study effects should routinely be performed inmeta-analysis” (p. 1119). Moreover, it is also important to notethat in spite of the reduced power, we did in fact find evidence forpublication bias in two of the data sets.

When using methods based on an investigation of funnel plotasymmetry to detect the presence of publication bias, it is impor-tant to note that the asymmetry itself does not disclose the cause ofasymmetry. No statistical test can provide direct evidence ofpublication bias. Other sources of asymmetry need to be consid-ered when interpreting the significant result of a rank correlation orregression analysis (Niemeyer, Musch, & Pietrowsky, 2012). Inparticular, study quality (e.g., implementation of the psychother-apeutic interventions) may be a function of the sample size(Borenstein et al., 2009; Sterne et al., 2005). Thus, althoughmethods to assess a possible publication bias are looking for arelationship between precision and effect size, it is important tokeep in mind that publication bias is only one of several possiblereasons for such a relationship. If the efficacy of an intervention isa function of the size of the study, it remains ambiguous how anintervention will work in routine care (Sterne et al., 2005). Fur-thermore, Begg and Mazumdar’s and Egger’s methods rely on theeffect size index and might result in different findings due to thisfact, whereas the algorithm used in the trim and fill method can beaffected by a few aberrant studies (Borenstein et al., 2009).

Another limiting factor in our study is that only a few meta-analyses could be included in our assessment, as the necessary rawdata needed for a post hoc assessment of publication bias was notavailable for most of the meta-analyses. The reporting of theprimary studies’ effect statistics and measures of their precisionwould be helpful for future investigations. Statistical guidelinessuch as the Meta-Analysis Reporting Standards (MARS) includedin the sixth edition of the Publication Manual of the AmericanPsychological Association (American Psychological Association,2010) and the Preferred Reporting Items for Systematic Reviewsand Meta-Analyses (PRISMA; Moher, Liberati, Tetzlaff, &Altmann, 2009) are available good standards that will help anyresearcher conducting a meta-analysis. Both recommend providingeffect size statistics.

Eighty-two percent of the meta-analyses of psychotherapy andprevention for depression did not consider publication bias in theiranalyses, and 81% did not include unpublished studies. Research-ers conducting meta-analyses should also take pains to includeunpublished studies. However, since unpublished studies are dif-ficult to obtain, this is often not feasible. Moreover, it may often beimpossible to ensure that all unpublished studies in a given fieldhave been found (Hopewell et al., 2005). The solution that mightbe best suited to prevent publication bias is to implement registers,in which psychotherapy research studies are collected system-atically and comprehensively already at the time of their incep-tion. Such preferably mandatory registers could provide unbi-ased samples (Berlin & Ghersi, 2005; Dickersin, 2005;Rustenbach, 2003). The use of statistical post hoc methods forassessing publication bias may even become obsolete in re-search domains in which such registers have been implemented(Niemeyer et al., 2012). However, no such registers have yetbeen implemented in the field of depression research, and todetect a possible publication bias, researchers interested indepression therefore still have to resort to methods such as thetrim and fill procedure, Begg and Mazumdar’s rank correlationmethod, and Egger’s regression analysis (Borenstein et al.,2009; Sterne & Egger, 2005; Sterne et al., 2000).

Therefore, it is necessary to emphasize the importance of as-sessing publication bias as a sensitivity analysis routinely, as is

69PSYCHOTHERAPEUTIC INTERVENTIONS FOR DEPRESSION

also recommended by the MARS and PRISMA guidelines. Thiswould solve the problem of missing data for follow-up analyses byother researchers. The MARS and PRISMA guidelines should bemandatory in psychotherapeutic research. In addition, when meth-ods for the detection of publication bias based on symmetryassumptions are applied, necessary statistical conditions for theiremployment must be met, in order to avoid erroneous interpreta-tions of their results (Ioannidis & Trikalinos, 2007).

In summary, the positive conclusion that can be drawn from thepresent analysis is that publication bias does not seem to haveinvalidated the results of most of the meta-analyses of the efficacyof therapeutic approaches to treat depression that were tested. Forthe most part, publication bias was either not present or did notchange the conclusions. Thus, efficacious preventive and psycho-therapeutic interventions for depression are available, and theirefficacy does not appear to be based on a reporting bias. However,as the correction for publication bias resulted in nonsignificance ofeffect sizes in two cases, further research is necessary to generalizethe findings and to be able to draw firm conclusions about possiblepublication bias in the whole field of research on the use ofpsychotherapy in depression.

References

References marked with an asterisk indicate studies included in themeta-analysis.

American Psychiatric Association. (1980). Diagnostic and statistical man-ual of mental disorders (3rd ed.). Washington, DC: Author.

American Psychiatric Association. (1987). Diagnostic and statistical man-ual of mental disorders (3rd ed., rev.). Washington, DC: Author.

American Psychiatric Association. (1994). Diagnostic and statistical man-ual of mental disorders (4th ed.). Washington, DC: Author.

American Psychiatric Association. (2000). Diagnostic and statistical man-ual of mental disorders (4th ed., rev.). Washington, DC: Author.

American Psychological Association. (2010). Publication manual of theAmerican Psychological Association (6th ed.). Washington, DC: Author.

�Andersson, G., & Cuijpers, P. (2009). Internet-based and other comput-erized psychological treatments for adult depression: A meta-analysis.Cognitive Behaviour Therapy, 38, 196 –205. doi:10.1080/16506070903318960

�Barbato, A., & D’Avanzo, B. (2008). Efficacy of couple therapy as atreatment for depression: A meta-analysis. Psychiatric Quarterly, 79,121–132. doi:10.1007/s11126-008-9068-0

Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rankcorrelation test for publication bias. Biometrics, 50, 1088–1101. doi:10.2307/2533446

�Bell, A. C., & D’Zurilla, T. J. (2009). Problem-solving therapy fordepression: A meta-analysis. Clinical Psychology Review, 29, 348–353.doi:10.1016/j.cpr.2009.02.003

�Beltman, M. W., Voshaar, R. C. O., & Speckens, A. E. (2010). Cognitivebehavioural therapy for depression in people with a somatic disease:Meta-analysis of randomised controlled trials. British Journal of Psy-chiatry, 197, 11–19. doi:10.1192/bjp.bp.109.064675

Berlin, J. A., & Ghersi, D. (2005). Preventing publication bias: Registriesand prospective meta-analysis. In H. R. Rothstein, A. J. Sutton, & M.Borenstein (Eds.), Publication bias in meta-analysis: Prevention, as-sessment and adjustments (pp. 35–48). Chichester, England: Wiley.doi:10.1002/0470870168.ch3

�Bledsoe, S. E., & Grote, N. (2006). Treating depression during pregnancyand the postpartum: A preliminary meta-analysis. Research on SocialWork Practice, 16, 109–120. doi:10.1177/1049731505282202

�Bohlmeijer, E., Smit, F., & Cuijpers, P. (2003). Effects of reminiscenceand life review on late-life depression: A meta-analysis. InternationalJournal of Geriatric Psychiatry, 18, 1088–1094. doi:10.1002/gps.1018

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2005).Comprehensive Meta-Analysis (Version 2.2) [Computer software].Englewood, NJ: Biostat.

Borenstein, M., Hedges, L. V.,Higgins, J. P. T., & Rothstein, H. R. (2009).Introduction to meta-analysis. Chichester, England: Wiley. doi:10.1002/9780470743386

�Bortolotti, B., Menchetti, M., Bellini, F., Montaguti, M. B., & Berardi, D.(2008). Psychological interventions for major depression in primarycare: A meta-analytic review of randomized controlled trials. GeneralHospital Psychiatry, 30, 293–302. doi:10.1016/j.genhosppsych.2008.04.001

Bromet, E., Andrade, L. H., Hwang, I., Sampson, N. A., Alonso, J., deGirolamo, G., . . . Kessler, R. C. (2011). Cross-national epidemiology ofDSM–IV major depressive episode. BMC Medicine, 9, 90. doi:10.1186/1741-7015-9-90

Chambless, D. L., & Ollendick, T. H. (2001). Empirically supportedpsychological interventions: Controversies and evidence. Annual Reviewof Psychology, 52, 685–716. doi:10.1146/annurev.psych.52.1.685

�Chan, E. K.-H. (2006). Efficacy of cognitive-behavioral, pharmacological,and combined treatments of depression: A meta-analysis. DissertationAbstracts International: Section B. Sciences and Engineering, 67(4),2218.

�Cuijpers, P. (1997). Bibliotherapy in unipolar depression: A meta-analysis. Journal of Behavior Therapy and Experimental Psychiatry, 28,139–147. doi:10.1016/S0005-7916(97)00005-0

�Cuijpers, P. (1998a). A psychoeducational approach to the treatment ofdepression: A meta-analysis of Lewinsohn’s “coping with depression”course. Behavior Therapy, 29, 521–533. doi:10.1016/S0005-7894(98)80047-6

�Cuijpers, P. (1998b). Psychological outreach programs for the depressedelderly: A meta-analysis of effects and dropout. International Journal ofGeriatric Psychiatry, 13, 41– 48. doi:10.1002/(SICI)1099-1166(199801)13:1�41::AID-GPS729�3.0.CO;2-B

�Cuijpers, P., Brannmark, J. G., & van Straten, A. (2008). Psychologicaltreatment of postpartum depression: A meta-analysis. Journal of Clini-cal Psychology, 64, 103–118. doi:10.1002/jclp.20432

�Cuijpers, P., Li, J., Hofmann, S. G., & Andersson, G. (2010). Self-reported versus clinician-rated symptoms of depression as outcomemeasures in psychotherapy research on depression: A meta-analysis.Clinical Psychology Review, 30, 768–778. doi:10.1016/j.cpr.2010.06.001

�Cuijpers, P., Muñoz, R. F., Clarke, G. N., & Lewinsohn, P. M. (2009).Psychoeducational treatment and prevention of depression: The “copingwith depression” course thirty years later. Clinical Psychology Review,29, 449–458. doi:10.1016/j.cpr.2009.04.005

�Cuijpers, P., Smit, F., Bohlmeijer, E., Hollon, S. D., & Andersson, G.(2010). Efficacy of cognitive-behavioural therapy and other psycholog-ical treatments for adult depression: Meta-analytic study of publicationbias. British Journal of Psychiatry, 196, 173–178. doi:10.1192/bjp.bp.109.066001

�Cuijpers, P., Smit, F., & van Straten, A. (2007). Psychological treatmentsof subthreshold depression: A meta-analytic review. Acta PsychiatricaScandinavica, 115, 434–441. doi:10.1111/j.1600-0447.2007.00998.x

�Cuijpers, P., van Straten, A., Andersson, G., & van Oppen, P. (2008).Psychotherapy for depression in adults: A meta-analysis of comparativeoutcome studies. Journal of Consulting and Clinical Psychology, 76,909–922. doi:10.1037/a0013075

�Cuijpers, P., van Straten, A., Hollon, S. D., & Andersson, G. (2010). Thecontribution of active medication to combined treatments of psychother-apy and pharmacotherapy for adult depression: A meta-analysis. Acta

70 NIEMEYER, MUSCH, AND PIETROWSKY

Psychiatrica Scandinavica, 121, 415– 423. doi:10.1111/j.1600-0447.2009.01513.x

�Cuijpers, P., van Straten, A., Schuurmans, J., van Oppen, P., Hollon,S. D., & Andersson, G. (2010). Psychotherapy for chronic major de-pression and dysthymia: A meta-analysis. Clinical Psychology Review,30, 51–62. doi:10.1016/j.cpr.2009.09.003

�Cuijpers, P., van Straten, A., & Smit, F. (2006). Psychological treatmentof late-life depression: A meta-analysis of randomized controlled trials.International Journal of Geriatric Psychiatry, 21, 1139–1149. doi:10.1002/gps.1620

�Cuijpers, P., van Straten, A., Smit, F., Mihalopoulos, C., & Beekman, A.(2008). Preventing the onset of depressive disorders: A meta-analyticreview of psychological interventions. American Journal of Psychiatry,165, 1272–1280. doi:10.1176/appi.ajp.2008.07091422

�Cuijpers, P., van Straten, A., Smits, N., & Smit, F. (2006). Screening andearly psychological intervention for depression in schools. Systematicreview and meta-analysis. European Child & Adolescent Psychiatry, 15,300–307. doi:10.1007/s00787-006-0537-4

�Cuijpers, P., van Straten, A., & Warmerdam, L. (2007a). Behavioralactivation treatments of depression: A meta-analysis. Clinical Psychol-ogy Review, 27, 318–326. doi:10.1016/j.cpr.2006.11.001

�Cuijpers, P., van Straten, A., & Warmerdam, L. (2007b). Problem solvingtherapies for depression: A meta-analysis. European Psychiatry, 22,9–15. doi:10.1016/j.eurpsy.2006.11.001

�Cuijpers, P., van Straten, A., & Warmerdam, L. (2008). Are individualand group treatments equally effective in the treatment of depression inadults? A meta-analysis. European Journal of Psychiatry, 22, 38–51.doi:10.4321/S0213-61632008000100005

�Cuijpers, P., van Straten, A., Warmerdam, L., & Smits, N. (2008).Characteristics of effective psychological treatments of depression: Ametaregression analysis. Psychotherapy Research, 18, 225–236. doi:10.1080/10503300701442027

Degkwitz, R., Helmchen, H., Kockott, G., & Mombour, W. (Eds.). (1980).Diagnoseschlüssel und Glossar psychiatrischer Krankheiten: DeutscheAusgabe der internationalen Klassifikation der Krankheiten der WHO,ICD, 9. Revision [International Classification of Diseases, 9thReview ofGeneral PsychologyBerlin, Germany: Springer.

�de Maat, S. M., Dekker, J., Schoevers, R. A., & de Jonghe, F. (2006).Relative efficacy of psychotherapy and pharmacotherapy in the treat-ment of depression: A meta-analysis. Psychotherapy Research, 16, 562–578. doi:10.1080/10503300600756402

�de Maat, S. M., Dekker, J., Schoevers, R. A., & de Jonghe, F. (2007).Relative efficacy of psychotherapy and combined therapy in the treat-ment of depression: A meta-analysis. European Psychiatry, 22, 1–8.doi:10.1016/j.eurpsy.2006.10.008

�Dennis, C.-L. (2005). Psychosocial and psychological interventions forprevention of postnatal depression: Systematic review. British MedicalJournal, 331, 15. doi:10.1136/bmj.331.7507.15

Department of Clinical Psychology, Vrije Universiteit Amsterdam. (2011).Psychotherapy: Randomized controlled and comparative trials [Elec-tronic database]. Retrieved from http://www.evidencebasedpsychothera-pies.org/index.php?id�23

Dickersin, K. (2005). Publication bias: Recognizing the problem, under-standing its origins and scope, and preventing harm. In H. R. Rothstein,A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis:Prevention, assessment and adjustments (pp. 11–34). Chichester, Eng-land: Wiley. doi:10.1002/0470870168.ch2

�Dobson, K. S. (1989). A meta-analysis of the efficacy of cognitive therapyfor depression. Journal of Consulting and Clinical Psychology, 57,414–419. doi:10.1037/0022-006X.57.3.414

�Driessen, E., Cuijpers, P., de Maat, S. C. M., Abbass, A. A., de Jonghe,F., & Dekker, J. J. M. (2010). The efficacy of short-term psychodynamicpsychotherapy for depression: A meta-analysis. Clinical PsychologyReview, 30, 25–36. doi:10.1016/j.cpr.2009.08.010

Duval, S. (2005). The trim and fill method. In H. R. Rothstein, A. J. Sutton,& M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention,assessment and adjustments (pp. 127–144). Chichester, England: Wiley.doi:10.1002/0470870168.ch8

Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot basedmethod of testing and adjusting for publication bias in meta-analysis.Biometrics, 56, 455–463. doi:10.1111/j.0006-341X.2000.00455.x

Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias inmeta-analysis detected by a simple, graphical test. British MedicalJournal, 315, 629–634. doi:10.1136/bmj.315.7109.629

�Eifert, G. H., & Crail, L. (1989). The relationship between affect, behav-iour, and cognition in behavioural and cognitive treatments of depressionand phobic anxiety. Behaviour Change, 6, 96–103.

�Ekers, D., Richards, D., & Gilbody, S. (2008). A meta-analysis of ran-domized trials of behavioural treatment of depression. PsychologicalMedicine, 38, 611–623. doi:10.1017/S0033291707001614

�Engels, G. I., & Vermey, M. (1997). Efficacy of nonmedical treatments ofdepression in elders: A quantitative analysis. Journal of Clinical Gero-psychology, 3, 17–35.

�Friedman, M. A., Detweiler-Bedell, J. B., Leventhal, H. E., Horne, R.,Keitner, G. I., & Miller, I. W. (2004). Combined psychotherapy andpharmacotherapy for the treatment of major depressive disorder. ClinicalPsychology: Science and Practice, 11, 47– 68. doi:10.1093/clipsy.bph052

�Gaffan, E. A., Tsaousis, I., & Kemp-Wheeler, S. M. (1995). Researcherallegiance and meta-analysis: The case of cognitive therapy for depres-sion. Journal of Consulting and Clinical Psychology, 63, 966–980.doi:10.1037/0022-006X.63.6.966

�Gellatly, J., Bower, P., Hennessy, S., Richards, D., Gilbody, S., & Lovell,K. (2007). What makes self-help interventions effective in the manage-ment of depressive symptoms? Meta-analysis and meta-regression. Psy-chological Medicine, 37, 1217–1228. doi:10.1017/S0033291707000062

�Gensichen, J., Beyer, M., Muth, C., Gerlach, F.-M., von Korff, M., &Ormel, J. (2006). Case management to improve major depression inprimary health care: A systematic review. Psychological Medicine, 36,7–14. doi:10.1017/S0033291705005568

�Gerson, S., Berlin, T. R., Kaufman, A., Mintz, J., & Jarvik, L. (1999).Pharmacological and psychological treatments for depressed older pa-tients: A meta-analysis and overview of recent findings. Harvard Reviewof Psychiatry, 7, 1–28.

Gilbody, S. M., & Song, F. (2000). Publication bias and the integrity ofpsychiatry research. Psychological Medicine, 30, 253–258. doi:10.1017/S0033291700001732

Glass, G. V.,McGaw, B., & Smith, M. L. (1981). Meta-analysis in socialresearch. Beverly Hills, CA: Sage.

�Gloaguen, V., Cottraux, J., Cucherat, M., & Blackburn, I. M. (1998). Ameta-analysis of the effects of cognitive therapy in depressed patients.Journal of Affective Disorders, 49, 59 –72. doi:10.1016/S0165-0327(97)00199-7

�Gregory, R. J., Schwer Canning, S., Lee, T. W., & Wise, J. C. (2004).Cognitive bibliotherapy for depression: A meta-analysis. ProfessionalPsychology: Research and Practice, 35, 275–280. doi:10.1037/0735-7028.35.3.275

�Haby, M. M., Donnelly, M., Corry, J., & Vos, T. (2006). Cognitivebehavioural therapy for depression, panic disorder and generalized anx-iety disorder: A meta-regression of factors that may predict outcome.Australian and New Zealand Journal of Psychiatry, 40, 9–19. doi:10.1080/j.1440-1614.2006.01736.x

�Haby, M. M., Tonge, B., Littlefield, L., Carter, R., & Vos, T. (2004).Cost-effectiveness of cognitive behavioural therapy and selective sero-tonin reuptake inhibitors for major depression in children and adoles-cents. Australian and New Zealand Journal of Psychiatry, 38, 579–591.doi:10.1080/j.1440-1614.2004.01421.x

71PSYCHOTHERAPEUTIC INTERVENTIONS FOR DEPRESSION

�Himelhoch, S., Medoff, D., & Oyeniyi, G. (2007). Efficacy of grouppsychotherapy to reduce depressive symptoms among HIV-infectedindividuals: A systematic review and meta-analysis. AIDS Patient Careand STDs, 21, 732–739. doi:10.1089/apc.2007.0012

Holma, K. M., Melartin, T. K., Haukka, J., Holma, I. A. K., Sokero, T. P.,& Isometsae, I. T. (2010). Incidence and predictors of suicide attemptsin DSM–IV major depressive disorder: A five-year prospective study.American Journal of Psychiatry, 167, 801–808. doi:10.1176/appi.ajp.2010.09050627

Hopewell, S., Clarke, M., & Mallett, S. (2005). Grey literature and sys-tematic reviews. In H. R. Rothstein, A. J. Sutton, & M. Borenstein(Eds.), Publication bias in meta-analysis: Prevention, assessment andadjustments (pp. 49–72). Chichester, England: Wiley. doi:10.1002/0470870168.ch4

�Imel, Z. E., Malterer, M. B., McKay, K. M., & Wampold, B. E. (2008).A meta-analysis of psychotherapy and medication in unipolar depressionand dysthymia. Journal of Affective Disorders, 110, 197–206. doi:10.1016/j.jad.2008.03.018

Ioannidis, J. P. A. (2005). Differentiating biases from genuine heteroge-neity: Distinguishing artifactual from substantive effects. In H. R. Roth-stein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 287–302).Chichester, England: Wiley. doi:10.1002/0470870168.ch15

Ioannidis, J. P. A., & Trikalinos, T. A. (2007). The appropriateness ofasymmetry tests for publication bias in meta-analyses: A large survey.Canadian Medical Association Journal, 176, 1091–1096. doi:10.1503/cmaj.060410

Jackson, D. (2006). The implications of publication bias for meta-analysis’other parameter. Statistics in Medicine, 25, 2911–2921. doi:10.1002/sim.2293

�Jané-Llopis, E., Hosman, C., Jenkins, R., & Anderson, P. (2003). Predic-tors of efficacy in depression prevention programs. British Journal ofPsychiatry, 183, 384–397. doi:10.1192/bjp.183.5.384

�Kang-Yi, C. D., & Gellis, Z. D. (2010). A systematic review ofcommunity-based health interventions on depression for older adultswith heart disease. Aging & Mental Health, 14, 1–19. doi:10.1080/13607860903421003

Kessler, R. C., Birnbaum, H., Shahly, V., Bromet, E., Hwang, I.,McLaughlin, K. A.,. . . Stein, D. J. (2010). Age differences in theprevalence and comorbidity of DSM–IV major depressive episodes:Results from the WHO World Mental Health Survey Initiative. Depres-sion and Anxiety, 27, 351–364. doi:10.1002/da.20634

�Klawansky, S. (1994). Meta-analysis on the treatment of depression in latelife. In L. S. Schneider, C. F. Reynolds, B. D. Lebowitz, & A. J.Friedhoff (Eds.), Diagnosis and treatment of depression in late life:Results of the NIH Consensus Development Conference (pp. 333–352).Washington, DC: American Psychiatric Association.

�Koder, D. A., Brodaty, H., & Anstey, K. J. (1996). Cognitive therapy fordepression in the elderly. International Journal of Geriatric Psychiatry,11, 97–107. doi:10.1002/(SICI)1099-1166(199602)11:2�97::AID-GPS310�3.0.CO;2-W

�Kühner, C. (2003). Das Gruppenprogramm “Depression bewältigen” undseine Varianten—Eine aktualisierte Metaanalyse [The “coping with de-pression” course and its variants—An updated meta-analysis]. Ver-haltenstherapie, 13, 254–262. doi:10.1159/000075841

�Leichsenring, F. (2001). Comparative effects of short-term psychody-namic psychotherapy and cognitive-behavioral therapy in depression: Ameta-analytic approach. Clinical Psychology Review, 21, 401–419. doi:10.1016/S0272-7358(99)00057-4

�Lewinsohn, P. M., & Clarke, G. N. (1999). Psychosocial treatments foradolescent depression. Clinical Psychology Review, 19, 329–342. doi:10.1016/S0272-7358(98)00055-5

Light, R. J., & Pillemer, D. B. (1984). Summing up: The science ofreviewing research. Cambridge, MA: Harvard University Press.

Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. ThousandOaks, CA: Sage.

�Lynch, D., Laws, K. R., & McKenna, J. (2010). Cognitive behaviouraltherapy for major psychiatric disorder: Does it really work? A meta-analytical review of well-controlled trials. Psychological Medicine, 40,9–24. doi:10.1017/S003329170900590X

�Mazzucchelli, T., Kane, R., & Rees, C. (2009). Behavioral activationtreatments for depression in adults: A meta-analysis and review. ClinicalPsychology: Science and Practice, 16, 383–411. doi:10.1111/j.1468-2850.2009.01178.x

�McCusker, J., Cole, M., Keller, E., Bellavance, F., & Berard, A. (1998).Effectiveness of treatments of depression in older ambulatory patients.Archives of Internal Medicine, 158, 705–712. doi:10.1001/archinte.158.7.705

�McDermut, W., Miller, I. W., & Brown, R. A. (2001). The efficacy ofgroup psychotherapy for depression: A meta-analysis and review of theempirical research. Clinical Psychology: Science and Practice, 8, 98–116. doi:10.1093/clipsy.8.1.98

Merikangas, K. R., He, J.-P., Brody, D., Fisher, P. W., Bourdon, K., &Koretz, D. S. (2010). Prevalence and treatment of mental disordersamong US children in the 2001–2004 NHANES. Pediatrics, 125, 75–81. doi:10.1542/peds.2008-2598

Merikangas, K. R., He, J.-P., Burstein, M., Swanson, S. A., Avenevoli, S.,Cui, L., . . . Swendsen, J. (2010). Lifetime prevalence of mental disor-ders in U.S. children: Results from the National Comorbidity SurveyReplication–Adolescent Supplement (NCS–A). Journal of the AmericanAcademy of Child & Adolescent Psychiatry, 49, 980–989. doi:10.1016/j.jaac.2010.05.017

�Michael, K. D. (2000). The efficacy of treatments for childhood depres-sion: An integrative review. Dissertation Abstracts International: Sec-tion B. Sciences and Engineering, 60(11), 5784.

�Michael, K. D., & Crowley, S. (2002). How effective are treatments forchild and adolescent depression? A meta-analytic review. Clinical Psy-chology Review, 22, 247–269. doi:10.1016/S0272-7358(01)00089-7

Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferredreporting items for systematic reviews and meta-analysis: The PRISMAstatement. Annals of Internal Medicine, 151, 264–269.

�Mohr, D. C., & Goodkin, D. E. (1999). Treatment of depression inmultiple sclerosis: Review and meta-analysis. Clinical Psychology: Sci-ence and Practice, 6, 1–9. doi:10.1093/clipsy.6.1.1

Moscati, R., Jehle, D., Ellis, D., Fiorello, A., & Landi, M. (1994). Positive-outcome bias: Comparison of emergency medicine and general medicineliteratures. Academic Emergency Medicine, 1, 267–271. doi:10.1111/j.1553-2712.1994.tb02443.x

Moshagen, M., & Musch, J. (2008). Publication bias in studies on theefficacy of hypnosis as a therapeutic tool. Contemporary Hypnosis, 25,94–99. doi:10.1002/ch.355

National Institute for Health and Clinical Excellence. (2009). Depressionin adults (update) (NICE Clinical Guideline 90). London, England:Author.

�Nelson, M. K. (2002). Meta-analysis: Hypnotherapy/cognitive-behavioraltherapy and its efficacy on depression compared to pharmacotherapy.Dissertation Abstracts International: Section B. Sciences and Engineer-ing, 63(2), 1040.

�Neumeyer-Gromen, A., Lampert, T., & Kallischnigg, G. (2003). DiseaseManagement Programme für Depression. Eine systematische Ubersicht-sarbeit und Meta-Analyse randomisierter kontrollierter Studien [Dis-ease management programs for depression: A systematic review andmeta-analysis of randomized controlled trials]. Berlin, Germany:Zentrum Public Health.

�Neumeyer-Gromen, A., Lampert, T., Stark, K., & Kallischnigg, G. (2004).Disease management programs for depression: A systematic review andmeta-analysis of randomized controlled trials. Medical Care, 42, 1211–1221. doi:10.1097/00005650-200412000-00008

72 NIEMEYER, MUSCH, AND PIETROWSKY

�Newton-Howes, G., Tyrer, P., & Johnson, T. (2006). Personality dis-order and the outcome of depression: Meta-analysis of publishedstudies. British Journal of Psychiatry, 188, 13–20. doi:10.1192/bjp.188.1.13

Niemeyer, H., Musch, J., & Pietrowsky, R. (2012). Publication bias inmeta-analyses of the efficacy of psychotherapeutic interventions forschizophrenia. Schizophrenia Research, 138, 103–112. doi:10.1016/j.schres.2012.03.023

�Nietzel, M. T., Russell, R. L., Hemmings, K. A., & Gretter, M. L. (1987).Clinical significance of therapy for unipolar depression: A meta-analyticapproach to social comparison. Journal of Consulting and ClinicalPsychology, 55, 156–161. doi:10.1037/0022-006X.55.2.156

�Oei, T. P. S., & Free, M. L. (1995). Do cognitive behaviour therapiesvalidate cognitive models of mood disorders? A review of the empiricalevidence. International Journal of Psychology, 30, 145–180. doi:10.1080/00207599508246564

�Pampallona, S., Bollini, P., Tibaldi, G., Kupelnick, B., & Munizza, C.(2004). Combined pharmacotherapy and psychological treatment fordepression. Archives of General Psychiatry, 61, 714–719. doi:10.1001/archpsyc.61.7.714

�Pinquart, M., Duberstein, P. R., & Lyness, J. M. (2006). Treatments forlater-life depressive conditions: A meta-analytic comparison of pharma-cotherapy and psychotherapy. American Journal of Psychiatry, 163,1493–1501. doi:10.1176/appi.ajp.163.9.1493

�Pinquart, M., Duberstein, P. R., & Lyness, J. M. (2007). Effects ofpsychotherapy and other behavioral interventions on clinically depressedolder adults: A meta-analysis. Aging & Mental Health, 11, 645–657.doi:10.1080/13607860701529635

�Pinquart, M., & Sörensen, S. (2001). How effective are psychotherapeuticand other interventions with older adults? A meta-analysis. Journal ofMental Health and Aging, 7, 207–243.

�Reinecke, M. A., Ryan, N. E., & DuBois, D. L. (1998). Cognitive-behavioral therapy of depression and depressive symptoms during ado-lescence: A review and meta-analysis. Journal of the American Academyof Child & Adolescent Psychiatry, 37, 26–34. doi:10.1097/00004583-199801000-00013

�Robinson, L. A., Berman, J. S., & Neimeyer, R. A. (1990). Psychotherapyfor the treatment of depression: A comprehensive review of controlledoutcome research. Psychological Bulletin, 108, 30–49. doi:10.1037/0033-2909.108.1.30

Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). Publication biasin meta-analysis. In H. R. Rothstein, A. J. Sutton, & M. Borenstein(Eds.), Publication bias in meta-analysis: Prevention, assessment andadjustments (pp. 1– 8). Chichester, England: Wiley. doi:10.1002/0470870168.ch1

Rustenbach, S. J. (2003) Metaanalyse: Eine anwendungsorientierte Ein-führung [Meta-analysis: An application-oriented introduction]. Bern,Switzerland: Huber.

�Schening, L. J. (2004). Refractory depression treatment options: A meta-analysis. Dissertation Abstracts International: Section B. Sciences andEngineering, 64(10), 5233.

�Scogin, F., & McElreath, L. (1994). Efficacy of psychosocial treatmentsfor geriatric depression: A quantitative review. Journal of Consultingand Clinical Psychology, 62, 69–74. doi:10.1037/0022-006X.62.1.69

Shapiro, D. (2002). Renewing the scientist-practitioner model. The Psy-chologist, 15, 232–234.

�Sin, N. L., & Lyubomirsky, S. (2009). Enhancing well-being and allevi-ating depressive symptoms with positive psychology interventions: Apractice-friendly meta-analysis. Journal of Clinical Psychology, 65,467–487. doi:10.1002/jclp.20593

Slade, M., & Priebe, S. (2001). Are randomised controlled trials the onlygold that glitters? British Journal of Psychiatry, 179, 286–287. doi:10.1192/bjp.179.4.286

Smith, M. L. (1980). Publication bias and meta-analysis. Evaluation inEducation, 4, 22–24.

�Souter, M. A., & Miller, M. D. (2007). Do animal-assisted activitieseffectively treat depression? A meta-analysis. Anthrozoös, 20, 167–180.doi:10.2752/175303707X207954

�Spek, V., Cuijpers, P., Nyklicek, I., Riper, H., Keyzer, J., & Pop, V.(2007). Internet-based cognitive behaviour therapy for symptoms ofdepression and anxiety: A meta-analysis. Psychological Medicine, 37,319–328. doi:10.1017/S0033291706008944

�Steinbrueck, S. M., Maxwell, S. E., & Howard, G. S. (1983). A meta-analysis of psychotherapy and drug therapy in the treatment of unipolardepression with adults. Journal of Consulting and Clinical Psychology,51, 856–863. doi:10.1037/0022-006X.51.6.856

Sterling, T. D. (1959). Publication decisions and their possible effects oninferences drawn from tests of significance—or vice versa. Journal ofthe American Statistical Association, 54, 30–34.

Sterne, J. A. C., Becker, B. J., & Egger, M. (2005). The funnel plot. InH. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication biasin meta-analysis: Prevention, assessment and adjustments (pp. 75–98).Chichester, England: Wiley. doi:10.1002/0470870168.ch5

Sterne, J. A. C., & Egger, M. (2005). Regression methods to detectpublication and other bias in meta-analysis. In H. R. Rothstein, A. J.Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis:Prevention, assessment and adjustments (pp. 99–110). Chichester, Eng-land: Wiley. doi:10.1002/0470870168.ch6

Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000). Publication and relatedbias in meta-analysis: Power of statistical tests and prevalence in theliterature. Journal of Clinical Epidemiology, 53, 1119–1129. doi:10.1016/S0895-4356(00)00242-0

�Stuart, S., & Bowers, W. A. (1995). Cognitive therapy with inpatients:Review and meta-analysis. Journal of Cognitive Psychotherapy, 9,85–92.

Taylor, S., & Tweedie, R. (2000). A nonparametric “trim and fill” methodof accounting for publication bias in meta-analysis. Journal of theAmerican Statistical Association, 95, 89–98.

�Taylor, T. L., & Montgomery, P. (2007). Can cognitive-behavioral ther-apy increase self-esteem among depressed adolescents? A systematicreview. Children and Youth Services Review, 29, 823–839. doi:10.1016/j.childyouth.2007.01.010

Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003). Adjusting forpublication bias in the presence of heterogeneity. Statistics in Medicine,22, 2113–2126. doi:10.1002/sim.1461

Vickers, A., Gloyal, N., Harland, R., & Rees, R. (1998). Do certaincountries produce only positive results? A systematic review of con-trolled trials. Controlled Clinical Trials, 19, 159–166. doi:10.1016/S0197-2456(97)00150-5

�Vittengl, J. R., Clark, L. A., Dunn, T. W., & Jarrett, R. B. (2007).Reducing relapse and recurrence in unipolar depression: A comparativemeta-analysis of cognitive-behavioral therapy’s effects. Journal of Con-sulting and Clinical Psychology, 75, 475–488. doi:10.1037/0022-006X.75.3.475

�Wagner, J. J. (2005). A meta-analysis/literature review comparing theeffectiveness of SSRI antidepressants, cognitive behavioral therapy, andplacebo for the treatment of depression. Dissertation Abstracts Interna-tional: Section B. Sciences and Engineering, 66(4), 2319.

�Wampold, B. E., Minami, T., Baskin, T. W., & Tierney, S. C. (2002). Ameta-(re)analysis of the effects of cognitive therapy versus “other ther-apies” for depression. Journal of Affective Disorders, 68, 159–165.doi:10.1016/S0165-0327(00)00287-1

�Watanabe, N., Hunot, V., Omori, I. M., Churchill, R., & Furukawa, T. A.(2007). Psychotherapy for depression among children and adolescents:A systematic review. Acta Psychiatrica Scandinavica, 116, 84–95.doi:10.1111/j.1600-0447.2007.01018.x

73PSYCHOTHERAPEUTIC INTERVENTIONS FOR DEPRESSION

�Weisz, J. R., McCarty, C. A., & Valeri, S. M. (2006). Effects of psychother-apy for depression in children and adolescents: A meta-analysis. Psycho-logical Bulletin, 132, 132–149. doi:10.1037/0033-2909.132.1.132

�Westen, D., & Morrison, K. (2001). A multidimensional meta-analysis oftreatments for depression, panic, and generalized anxiety disorder: Anempirical examination of the status of empirically supported therapies.Journal of Consulting and Clinical Psychology, 69, 875–899. doi:10.1037/0022-006X.69.6.875

�Wexler, B. E., & Cicchetti, D. V. (1992). The outpatient treatment ofdepression. Implications of outcome research for clinical practice. Jour-

nal of Nervous and Mental Disease, 180, 277–286. doi:10.1097/00005053-199205000-00001

World Health Organization. (1992). The ICD–10 classification of mentaland behavioral disorders: Clinical descriptions and diagnostic guide-lines. Geneva, Switzerland: Author.

Received December 5, 2011Revision received October 5, 2012

Accepted November 7, 2012 �

Correction to Amir and Taylor (2012)

In the article “Interpretation Training in Individuals With Generalized Social Anxiety Disorder: ARandomized Controlled Trial” by Nader Amir and Charles T. Taylor (Journal of Consulting andClinical Psychology, 2012, Vol. 80, No. 3, pp. 497–511), a disclosure should have been noted thatNader Amir is the co-founder of a company that markets anxiety relief products.

DOI: 10.1037/a0031156

74 NIEMEYER, MUSCH, AND PIETROWSKY