Received: 5 May 2016 Revised: 7 February 2017 Accepted: 13 February 2017

DO

I: 10.1002/cpp.2080

R E S E A R CH AR T I C L E

Information order effects in clinical psychological diagnoses

Jan Christopher Cwik | Jürgen Margraf

Mental Health Research and Treatment

Center, Department of Psychology, Ruhr‐Universität Bochum, Bochum, Germany

Correspondence

Jan C. Cwik, Department of Clinical

Psychology and Psychotherapy, Ruhr‐Universität Bochum, Massenbergstraße 9‐13,44787 Bochum, Germany.

Email: jan.cwik@rub.de

Funding information

Alexander von Humboldt‐Stiftung

1142 Copyright © 2017 John Wiley & Sons, L

Despite the wide application and long history of diagnostic systems, several sources of diagnostic

errors remain in the criterion‐based diagnosing of mental disorders. The aim of this study was to

investigate whether the presentational order of diagnosis‐relevant information and pretreatment

reports predict diagnostic errors. One hundred twenty psychotherapists participated in the pres-

ent online study. The study employed a 2 (symptom presentation: core symptoms at vignette's

beginning vs. core symptoms at the end of the case vignette) × 2 (pretreatment report: receiving

a pretreatment report with an incongruent diagnosis to the case vignette vs. receiving no pre-

treatment report) between‐subjects experimental design, with random assignment. Participants

were asked to make diagnoses after reading three case vignettes describing patients with differ-

ent disorder constellations. Additionally, participants rated their confidence in the diagnoses and

their estimation of the severity of each diagnosed condition. Results indicated that order of symp-

tom descriptions predicted the correctness of diagnostic decisions, with a recency effect causing

more fully correct diagnostic decisions in cases where diagnostic information was presented last.

Receiving incongruent pretreatment reports was predictive for diagnostic errors. In conclusion,

the results of this study indicate that diagnoses of mental disorders can depend on the way symp-

toms are presented or reported.

Key Practitioner Message:

• Therapists' diagnostic decisions are not influenced by pretreatment reports.

• Diagnostic decisions are affected by information order effects.

• Diagnostic accuracy of psychotherapists is debatable.

• High rate of misdiagnoses in case vignette with comorbid disorders.

KEYWORDS

Decoy Effect, Diagnostic Accuracy, Diagnostic Errors, Pretreatment Report, Recency Effect

1 | INTRODUCTION

An accurate diagnosis is critical in choosing an appropriate course of

therapy for mental health patients. Diagnostic classification systems

for mental disorders, including the 10th edition of the International

Classification of Diseases and Related Health Problems (ICD‐10;

World Health Organization [WHO], 1992) and the fifth edition of the

Diagnostic and Statistical Manual of Mental Disorders (DSM‐5; Amer-

ican Psychiatric Association [APA], 2013), provide a clear structure for

making diagnoses. Diagnosticians assess a core symptom (e.g., the

exposure to a traumatic event in posttraumatic stress disorder [PTSD])

and a number of other symptoms as well as estimate the degree of suf-

fering and exclude criteria associated with other disorders. Diagnosti-

cians can and should refrain from assessing further symptoms of a

td. wileyonlinelibrar

disorder when core symptoms are not present. Comorbid diagnoses

can complicate the process. While structured interviews can provide

the diagnostician with a reliable and valid tool for systematically mak-

ing and ruling out diagnoses (In‐Albon et al., 2008; Suppiger et al.,

2008), they are still rarely used in everyday clinical practice; indeed

only in about 15% of clinical interviews (Bruchmüller, Margraf,

Suppiger, & Schneider, 2011). Psychotherapists tend to believe their

open clinical judgment—that is a diagnostic interview without using

a diagnostic tool—is more useful than judgments based on structured

diagnostic interviews in making correct diagnoses (Bruchmüller et al.,

2011). However, considering that therapists are often confronted

with multiple sources of information, as well as influenced by extra-

neous information (Bruchmüller & Meyer, 2009; Bruchmüller &

Schneider, 2012; Meyer & Meyer, 2009; Wolkenstein, Bruchmüller,

Clin Psychol Psychother. 2017;24:1142–1154.y.com/journal/cpp

CWIK AND MARGRAF 1143

Schmid, & Meyer, 2011), open clinical judgment is error‐prone and

can lead to diagnostic errors. For example, psychotherapists are more

likely to diagnose attention‐deficit/hyperactivity disorder (ADHD) in

a boy than in a girl after reading case vignettes that described several

ADHD symptoms but stated that other ADHD criteria were not

fulfilled (Bruchmüller, Margraf, & Schneider, 2012). The authors

interpret these results as an indication of therapists' diagnostic

decision‐making apart from following diagnostic manuals. Other

studies indicate that some diagnostic‐irrelevant information, such as

hallucinations (Meyer & Meyer, 2009) or reduced sleep (Bruchmüller

& Meyer, 2009), decreases the possibility that psychotherapists will

recognize bipolar disorder. Thus, especially prototypical symptoms

are a double‐edged sword in the field of diagnostic decision‐making

related to mental disorders. On the one hand, when more prototypi-

cal symptoms are present therapists are more likely to make correct

diagnostic decisions (Horowitz, Post, de Sales, Wallis, & Siegelman,

1981). On the other hand, a recent study illustrated that therapists

are also more likely to misclassify suicidal behavior as nonsuicidal

self‐directed violence when a patient with borderline personality dis-

order (BPD) was described in a case vignette compared to a patient

with major depressive disorder (Cwik & Teismann, 2016). Taking

these examples into account, structuring diagnostic reasoning by

using diagnostic interviews is recommended. In line with this, Mitch-

ell, Vaze, and Rao (2009) illustrated in their meta‐analysis that diag-

nostic concordance between general practitioners' diagnostic

decisions and diagnostic decisions based on structured diagnostic

interviews is relatively low with 47.3% of all cases. Our recent

meta‐analysis also indicated poor consensus between diagnoses

based on open clinical judgment versus structured interviews (Cwik

et al., 2017).

In order to obtain a better understanding of the reasons for the dis-

cordance between diagnostic decisions based on open clinical judg-

ment and structured diagnostic interviews, it could be useful to

investigate the processing of psychotherapists' diagnostic decision‐

making, such as the order of information collection in daily practice.

Therapists aim to form their own picture of symptoms patients

suffer from. Using an open clinical judgment approach, patients report

symptoms according to their sense of relevance or in the order that the

information comes to their mind. Thus, the order of reporting diagnos-

tically relevant information usually varies between patients and is com-

monly not in line with the structure of diagnostic manuals.

Consequently, psychotherapists using open clinical judgment must

mentally refer to and organize the diagnostic criteria in question. Study

results suggest an effect of the order of information on the integration

of diagnostic information (see for review Hogarth & Einhorn, 1992),

and that psychotherapists can be subject to confirmation bias (Margraf

& Schneider, 2009; Schulz‐Hardt, Frey, Luthgens, & Moscovici, 2000;

Shemberg & Doherty, 1999). It is conceivable that the first information

presented to the therapist may have a particularly strong influence on

the diagnosis, and thus which information is presented first during a

diagnostic assessment could have an impact to the final diagnostic out-

come. There is evidence that such primacy effects occur in diagnostic

reasoning (Cunnington, Trunbull, Regehr, Marriott, & Norman, 1997;

Rebitschek, Krems, & Jahn, 2015); for instance, the information that

there is the swelling of an eyelid has the greatest impact on diagnostic

judgments when reported first. Conceivably, this phenomenon could

also partly explain misdiagnoses of mental disorders. However, studies

also suggest that diagnostic reasoning stepwise in parts counteracts

the primacy effect (Rebitschek, Bocklisch, Scholz, Krems, & Jahn,

2015) and can even lead to a recency effect (Hogarth & Einhorn,

1992). Furthermore, research indicates that besides the first also the

last information has significant impact on diagnostic judgment (primacy

and recency effects; Rebitschek, Bocklisch, et al., 2015). Investigations

of Chapman, Bergus, and Elstein (1996) illustrated that a recency effect

can also occur in single end‐of‐sequence probability judgments. There-

fore, as both primacy and recency effect have been shown to be

related to diagnostic reasoning of mental disorders, further investiga-

tions are needed. Thus, an investigation of order effects could add rel-

evant knowledge about the reasons for misdiagnoses. Additionally, a

better understanding of potential order effects could help to optimize

diagnostic tools and approaches.

Based on our clinical experience in daily practice, patients some-

times seek psychotherapeutic help after reoccurrence of symptoms

and some of them provide a pretreatment report from their previous

therapist to communicate relevant information. Such information can

be diagnostically useful and provide additional information at the

beginning of therapy. However, the information could also be based

on inaccurate diagnostic decisions. Given that therapists make false

diagnostic assumptions near the mark (e.g., physical symptoms and

worries in relation to a generalized anxiety disorder [GAD] are

misinterpreted as panic attack symptoms and avoidance behavior

related to panic disorder [PD]), one could assume that such nearby

misleading information could lead to misdiagnoses causing decoy

effects—also known as attraction effect (see Connolly, Reb, & Kausel,

2013). As Kostopoulou, Russo, Keenan, Delaney, and Douiri (2012)

showed, physicians are more likely to maintain their diagnostic deci-

sions after having developed an initial diagnostic leaning, even though

they might receive additional conflicting information. Similar to decoy

effects, therapists trust pretreatment reports and develop an initial

diagnostic leaning based on the information of the report. Especially

in realistic choice situations, decoy effect has been shown as a relevant

explanation for diagnostic errors in decision‐making situations (Slaugh-

ter, Sinar, & Highhouse, 1999) and particularly in medical decision‐

making (see for a review Blumenthal‐Barby & Krieger, 2015). However,

in the case of first given information, a decoy effect appears to be

more likely when the judging person has less knowledge in the field

(Mantonakis, Rodero, Lesschaeve, & Hastie, 2009). However, to date

studies on decoy and order effects related to diagnosing of mental dis-

orders are rare.

Thus, this study aimed to investigate the association between the

order in which symptoms are presented and the correctness of diag-

nostic decisions. We expected that participants who receive a version

of case vignettes describing the core symptoms of the disorder at the

beginning of the vignette would be significantly more likely to make a

correct diagnostic decision (primacy effect). Based on the assumption

that a misleading information order (core symptoms at the end of the

vignette) give therapists the impression of a more complex case, we

expected a lower level of confidence in psychotherapists' diagnostic

decisions and a higher estimation of the severity of diagnosed

conditions.

1144 CWIK AND MARGRAF

Second, aim of this study was to investigate the effect of incon-

gruent pretreatment reports on diagnostic decisions by psychothera-

pists upon reading case vignettes of various cases with different

mental disorders. Based on the results that clinicians are more likely

to maintain their diagnostic decisions after having developed an initial

diagnostic leaning (Kostopoulou et al., 2012), we hypothesized that

therapists who receive an incongruent pretreatment report develop

an initial diagnostic leaning and are biased by it. We expected that

therapists with a pretreatment report are significantly more likely to

make false diagnostic decisions than participants who did not receive

such a report. Additionally, we expected that the receipt of an incon-

gruent pretreatment report creates the impression of a more complex

case and thus results in less confidence in psychotherapists' diagnostic

decisions and in a higher estimation of the severity of diagnosed

conditions.

2 | METHODS

2.1 | Sample

2.1.1 | Response rate

For the recruitment of our sample, we used the official email lists of

the psychotherapists' chambers of all federal states of Germany. The

email addresses of 1,490 psychotherapists were chosen randomly.

Two hundred sixty five participants responded to the invitation

(response rate: 17.23%). Participants included 11 psychiatrists as well

as 28 psychotherapy trainees and 226 licensed psychotherapists. Both

trainees and licensed psychotherapists were clinical psychologist grad-

uates (diploma or Master's degree). It is possible that several of the

contacted therapists did not receive our invitation because their email

addresses were no longer in use or they were on holiday during the

period of the study based on their notices of absence. For the follow-

ing analyses, we included n = 120 (45.28%) questionnaires where both

the demographic questionnaire and the diagnostic decision question-

naire related to the case vignettes had been completed.

A dropout analysis revealed that participants who dropped out

after reading case vignette 1 where significantly more likely to work

in a research facility (Χ2[1, n = 197] = 5.230, P = .037), whereas partic-

ipants who dropped out after reading case vignette 2 were more likely

to have another therapeutic orientations than a psychoanalytic orien-

tation (Χ2[1, n = 149] = 4.972, P = .033), and participants who dropped

out after reading case vignette 1 where significantly less confident

with DSM‐5 (t[127] = −1.194, P = .012). Related to all other demo-

graphic variables no significant differences were found (see Figure 1).

2.1.2 | Participants' characteristics

On average, the age of participants was 45.78 years (SD = 12.81), 94

(78.3%) were female, and the average number of years of work experi-

ence was 14.46 years (SD = 10.14). Highest academic level data were

as follows: 104 participants had a Master of Science or diploma

(94.2%) and 6 participants (5.0%) had another level (e.g., state examina-

tion). Of all participants, 14 (11.7%) had an academic title (PhD or MD).

In relation to therapeutic training, 101 (84.8%) were licensed thera-

pists, 17 (14.2%) were trainees, and 2 (1.7%) gave no answer.

Participants partly had multiple therapeutic orientations in cognitive

behavioral therapy (70.8%), followed by psychoanalytic therapy

(42.5%), Rogerian therapy (11.7%), and systemic therapy (8.3%). Some

participants were trained in 2 (21.7%) or 3 (5.8%) therapeutic

orientations. Several participants had even further therapeutic

trainings. Of all 120 participants, 41 (34.2%) had one, 14 (11.7%) had

two trainings, 2 (1.7%) had three, and 2 (1.7%) had four further thera-

peutic trainings (e.g., Trauma Focused Therapy, Schema Therapy, or

Hypnotherapy). The working environment of most therapists was their

own private practice (n = 103; 85.8%), followed by inpatient‐unit

(n = 13; 10.8%), research facility (n = 6; 5.0%), day unit (n = 6; 5.0%),

and others (n = 15; 12.5%). Ninety‐six participants (80.0%) engaged a

supervisor, whereof 35 (29.2%) rarely, 52 (43.3%) commonly, and 8

(6.7%) always engaged a supervisor (1 answer missing). In relation to

the question whether the therapists decline to treat patients with spe-

cialized mental disorders, 59 (49.2%) decline at least one disorder,

whereas 25 (20.8%) were specialized for at least one disorder. The

familiarity with diagnostic systems was assessed on a scale from

1 = “not familiar” to 4 = “very familiar”. Participants were most familiar

with the 10th edition of the International Classification of Diseases

and Related Health Problems (ICD‐10; WHO, 1992) (M = 3.40;

SD = 0.57), followed by the 4th edition of the Diagnostic and Statistical

Manual of Mental Disorders (DSM‐IV‐TR; APA, 2000) (M = 1.97;

SD = 0.96) and the 5th edition of DSM (DSM‐5; APA, 2013)

(M = 1.42; SD = 0.62). Finally, participants were asked for an estimation

of their diagnostic qualities in relation to other therapists, on a scale

ranging from 0 = “worse” to 100 = “superior” (M = 58.81; SD = 15.62).

2.2 | Design

Participants were randomized within each of two conditions (pretreat-

ment report condition and order‐of‐symptom‐description‐condition).

As can be seen inTable 1 and Figure 2, within the pretreatment report

condition, participants received either a pretreatment report prior to

the three case vignettes (n = 60) or only three case vignettes

(n = 60). Within the order‐of‐symptom‐description‐condition, partici-

pants received either three case vignettes with descriptions of core

symptoms at the beginning of the vignettes (n = 60) or at the end of

the vignettes (n = 60). Thus, the design resulted in a 2 × 2 between‐

subjects design (n = 120).

2.3 | Procedure

All participants received an email request to take part in the online sur-

vey with a link to the survey questionnaire. The survey questionnaire

was composed with SoSci Survey (Leiner, 2014) and made available

to participants on www.soscisurvey.com. After clicking on the link, par-

ticipants received a short description of the survey questionnaire and

the expected amount of time to complete it (35 min). Those who

wished to participate received assurance of confidentiality and were

asked to give their informed consent. Then the survey automatically

started. The procedure of the survey questionnaire can be seen in

Figure 2.

As can be seen in Figure 2, participants allocated to the pretreat-

ment report condition first received a pretreatment report. Participants

TABLE 1 Between‐subjects design (2 × 2) with information about sample size within each condition (N = 120)

2: Order‐of‐symptom‐description condition

2A Core symptomsat the beginning

2B Core symptomsat the end

Participants in the order‐of‐symptom‐description conditions (2A + 2B: N = 120)

1: Pretreatmentreport condition

1A Pretreatment report 1A‐2A (n = 30) 1A‐2B (n = 31) n = 611B No pretreatment report 1B‐2A (n = 30) 1B‐2B (n = 29) n = 59

Participants in the pre‐treatment reportconditions (1A + 1B: N = 120)

n = 60 n = 60

Note. The 120 participants in the two pretreatment report conditions were the same participants as in both order‐of‐symptom‐description conditions.

FIGURE 1 Flow diagram of enrollment anddropout of participants

CWIK AND MARGRAF 1145

who were not in this condition received the first case vignette immedi-

ately. Participants in the pretreatment report condition could proceed

with the survey after reading the report. According to Slaughter,

Kausel, and Quiñones (2011), a decoy effect occurs when additional

information (e.g., a pretreatment report) lead to changes in choices

while the decoy is added to only one of the two possible choice condi-

tions. This is particularly the case when the additional information is

highly attractive for the deciding person (Connolly et al., 2013). Thus,

adding decoys is an effective method to study decision‐making pro-

cesses (Choplin & Hummel, 2005). On average, participants spent

3.94min (SD = 13.87) on each pretreatment report whereupon the case

vignette was presented. Participants in the pretreatment report condi-

tion spent M = 6.69 min (SD = 10.28) on each case vignette, whereas

participants in the condition without receiving a pretreatment report

FIGURE 2 Procedure of the surveyquestionnaire as well as mean duration ofparticipants in the pretreatment‐reportcondition and the condition without receivingpretreatment‐reports (survey without greysheets). BPD = borderline personality disorder;GAD = generalized anxiety disorder;HYP = hyperactivity; PD = panic disorder;PTSD = posttraumatic stress disorder

1146 CWIK AND MARGRAF

spentM = 7.69 min (SD = 14.55) on each case vignette. After reading a

case vignette, participants stated whether they would give at least one

diagnosis for the case. They then automatically proceeded to the diag-

nostic questionnaire related to the particular case vignette, where they

were asked to make at least one diagnosis. Subsequently, the next pre-

treatment report and case vignette were automatically presented.

The Ethics Committee of Ruhr‐Universität Bochum approved the

study.

2.4 | Materials

2.4.1 | Background questionnaire

The demographic section of the assessment asked participants for

their gender and age. In the second section, job‐related information

was assessed, including highest academic degree obtained, academic

title, therapeutic approach, professional experience (in years), working

environment, familiarity with diagnostic manuals, use of supervision

(1 = never to 4 = always), exclusion of patients with specific mental dis-

orders, specialization in specific mental disorders, and number and ori-

entation of advanced therapeutic trainings.

2.4.2 | Pretreatment reports

For the design of the pretreatment reports, we used an original

template used at the Mental Health Research and Treatment Cen-

ter of the Ruhr‐Universität Bochum. Pre‐treatment reports included

information about patient's treatment, the length of therapy, time

frame, number of therapeutic sessions, explicit naming of

previously given diagnosis/diagnoses with the corresponding ICD‐

10 code, description of the complaints at the beginning of the

therapy, and description of the therapeutic process, including infor-

mation about therapy content and the used therapeutic rationale

(e.g., Dialectical behavior therapy (DBT); see Figure 3).

All reports were incongruent with each of the following case

vignettes. The assignment of pretreatment reports with corresponding

case vignettes was the same for each participant. Two pretreatment

reports provided information that matched the symptom described in

the vignettes, but reported discordant diagnoses. The first one was

the pretreatment report presented prior to the GAD case vignette.

Here, the somatic symptoms and worries of a patient with GAD were

affiliated to PD and hypochondriasis in the pretreatment report. The

second pre‐treatment report presented prior to the PD case vignette,

provided information that matched the symptom described in the

vignette, but were incongruent with the case vignette. In this report,

somatic symptoms of a patient with PD were affiliated to GAD. These

combinations were chosen because each of these diagnoses should be

differentially diagnostically clarified according to DSM (see APA, 2013,

pp. 213 and 314). Thus, the misdiagnoses named in the pretreatment

report seemed to be misleading. Nevertheless, it also does not seem

unlikely that in daily practice these disorders are incorrectly diagnosed

related to the symptoms reported in the case vignette. With respect to

the BPD/PTSD case, we presented a pretreatment report that only

reported BPD as given diagnosis. Doing so, we expected that such mis-

leading pretreatment reports affect therapists to neglect PTSD

symptomatology.

FIGURE 3 Example of a pretreatment report [Colour figure can be viewed at wileyonlinelibrary.com]

CWIK AND MARGRAF 1147

2.4.3 | Case vignettes

Contrary to other studies that use case vignettes for the investiga-

tion of diagnostic accuracy (e.g., Bruchmüller et al., 2012; Meyer &

Meyer, 2009), we decided to send three case vignettes with

descriptions of three different mental disorders. The main reason

for this procedure was that we aimed to investigate decoy and

information order effects more generally in relation to different dis-

orders and comorbidity. Additionally, to investigate whether these

effects occur consequently, the usage of more than one vignette

was necessary.

The first vignette described a patient fulfilling GAD, based on a

DSM‐III case description (Spitzer, Gibbon, Skodol, Williams, & First,

1991). For this case vignette we shortened the original description,

attuned to German culture and to DSM‐5 criteria. To ensure that par-

ticipating therapists who use ICD‐10 diagnostic criteria in their daily

routine would be able to diagnose GAD, we additionally included all

required symptoms based on ICD‐10. The second vignette described

a patient fulfilling all general criteria for a personality disorder accord-

ing to DSM‐5 as well as seven required criteria for BDP. The case

vignette was based on a BPD case description of Zaudig, Wittchen,

1148 CWIK AND MARGRAF

and Sass (2000). Additionally, all required symptoms of PTSD accord-

ing to DSM‐5 and ICD‐10 were added. We decided to include the

criteria of PTSD to investigate effects that cause misdiagnoses in

comorbidity considering that this could be of high relevance for daily

practice. These two disorders (BPD and PTSD) were combined due

to their high rates of comorbidity (Grant et al., 2008; Pabst et al.,

2012). Notably, an integration of the disorders has been discussed in

the field (Eichelman, 2010; Hodges, 2003). Studies have shown that

the especially complex symptomatology blur boundaries between the

two disorders (Cloitre, Garvet, Weiss, Calrson, & Bryant, 2014). The

third case vignette was a patient fulfilling all symptoms of a PD accord-

ing to DSM‐5 and ICD‐10. This case vignette was created for the cur-

rent study by the authors.

To validate the diagnostic criteria of the underlying disorders,

seven licensed psychotherapists of the Mental Health Research and

Treatment Center of the Ruhr‐Universität Bochum reviewed the

vignettes. All psychotherapists diagnosed the disorders in each

vignette correctly without an additional (comorbid) diagnosis.

As an example, we present the GAD case vignette (according to

Spitzer et al., 1991) in the following. The underlined part of the

vignette was either at the beginning of the case vignette (as in this

example) or at the end of the case vignette (after the last passage):

“Mrs. L., a 44‐year‐old married administrative assistant contacts you

to receive outpatient psychotherapeutic treatment. She

tells you that she has completed psychotherapy to

address this problem some time ago. She complains

about dizziness, sweaty hands, palpitations, and ringing

in her ears. At night Mrs. L. stares at the ceiling and

ruminates. She is always worried about the health of her

parents. Her father had a myocardial infarction 2 years

ago, but feels healthy today. She also worries about

being a “good mother,” whether her husband could leave

her (there was no indication that she is dissatisfied with

the marriage), and whether her colleagues accept her.

Despite her better judgment that there are no problems

concerning these worries, she is not able to suppress

these thoughts and the inner tension. Mrs. L. also

experienced that her mouth and her throat were dry,

and that she was periodically uncontrollably shaking as

well as having a persistent feeling of “being always on

the go”. This influenced her ability to concentrate. These

feelings have continuously occurred for the last 2 years

and were not restricted to limited periods of time.

Because of these problems, Mrs. L. contacted her family

doctor, a neurologist, a neurosurgeon, a chiropractor,

and an ENT specialist. She was put on a hypoglycemic

diet, received physical therapy because of a pinched

nerve, and she was informed that she has got a

“problem with her inner ear.” Mrs. L. has had only

limited social contacts in recent years because of her

nervous symptoms. Although she had to flee from work

several times because of her symptoms, she still works

at the road traffic office where she has been working

since the beginning of her career. She tends to conceal

her symptoms from her husband and the children. For

them she wants to appear “perfect.” She reports that

her nervousness problems are not related to her family.”

2.4.4 | Diagnostic survey

After reading a vignette, participants were asked to give a diagnostic

assessment of the vignette. Participants were instructed to use their

clinical judgment without reference to auxiliary materials (e.g., DSM

or ICD). For their diagnostic appraisal, they could choose up to three

diagnoses out of 19 listed diagnoses and were asked to indicate

whether each selected disorder was present with clinical or subclinical

intensity. The diagnoses used in the present study were derived from

another study testing a similar research question (see Cwik, Papen,

Lemke, & Margraf, 2016). As that previous study only used 12 diagno-

ses, seven additional diagnoses were added. Thus, there were a total of

19 diagnoses, and all of them represented high prevalence rates

according to the DSM‐5. Each participant received the list of disorders

in the same order after each vignette. They were also asked to indicate

their confidence in each diagnosis chosen, using a scale ranging from

0 = “absolutely no confidence” to 100 = “complete confidence”. This type

of confidence rating has already been used in former studies (e.g.,

Meyer & Meyer, 2009; Schmidt, Salas, Bernert, & Schatschneider,

2005). Finally, they were asked for their assessment of severity of

diagnosed conditions for each disorder with clinical intensity.

For the order‐of‐symptom‐description between‐subjects manipu-

lation, the core symptoms (criterion A in DSM‐IV) were described

either at the beginning or the end of the vignette. In the GAD case

vignette, participants either received the criterion “expressive anxiety

and worry” at the beginning or at the end of the vignette. The vignette

of the BPD and comorbid PTSD was varied so that participants

received a version with either the BPD criterion at the beginning of

the vignette, followed by the PTSD criterion naming the concrete trau-

matic event, or a version with the PTSD criterion at the beginning,

starting with the traumatic event and followed by the BPD description.

In the PD vignette, participants received either a version describing the

criterion “recurrent unexpected panic attacks” at the very beginning or

at the end of the vignette. At the end, participants were debriefed on

the study, receiving information about the aims of the study and the

experimental manipulation.

2.5 | Statistical analysis

First, we examined whether participants in each condition across the

two groups (pretreatment report vs. no pretreatment report; core

symptom at the beginning of the vignette vs. core symptom at the

end of the vignette) varied in demographic variables. To control for sig-

nificant differences and trends (P < .10) in demographic variables, pro-

pensity scores were calculated (Austin, 2011; Bartak et al., 2009;

Rosenbaum & Rubin, 1983). All variables with significant differences

and trends between groups were used for the calculation of propensity

scores for each condition, thus each participant had one propensity

score for the information order condition and another propensity score

for the pretreatment condition. The propensity scores were calculated

using logistic regression with the respective conditions as dependent

variables and the demographic variables showing significant

CWIK AND MARGRAF 1149

differences between groups as independent variables (Austin, 2011;

Bartak et al., 2009; D'Agostino, 1998; Rosenbaum & Rubin, 1983;

Weinberger, Maciejewski, McKee, Reutenauer, & Mazure, 2009). The

propensity scores were then included in the corresponding regression

analyses for controlling significant group differences.

For the investigation of the diagnostic decisions of participants,

we first rated whether participants gave the so‐called “aimed diagno-

sis.” This was the case when participants made the GAD diagnosis in

the GAD vignette, the BPD and PTSD diagnoses in the BPD and PTSD

vignette, and the PD diagnosis in the PD vignette. Next, we investi-

gated the rates of participants who made the aimed and at least one

other “false comorbid diagnosis” (e.g., GAD diagnosis and a major

depressive disorder diagnosis in the GAD vignette). Last, we investi-

gated whether participants made a “fully correct diagnostic decision”

(e.g., GAD diagnosis and no other diagnosis in the GAD vignette).

In order to examine whether the order of information about symp-

toms and the presentation of a pretreatment report is associated with

diagnostic decisions, we conducted multiple binary logistic regression

analyses with only fully correct diagnostic decisions as dependent var-

iables (yes vs. no). To examine the association between conditions and

diagnostic confidence and severity of diagnosed conditions, linear

regression analyses were conducted. Subsequently, results of all

regression analyses were Bonferroni‐corrected for multiple compari-

sons. We chose logistic and linear regression analyses as analyzing

methods, which enabled us to control for group differences by

including propensity scores and each noninteresting condition. Data

analyses were conducted using SPSS version 22.0 for Mac (IBM

Corporation, 2013).

3 | RESULTS

First, we examined whether participants in each condition (across the

two groups) varied significantly with respect to differences in demo-

graphic variables. In the case of the pre‐treatment report condition,

work environment (research unit) varied between groups: There was

a trend toward a higher proportion of researchers in the group that

did not receive the pre‐treatment report (Χ2[1, N = 120] = 2.95,

P = .086). Regarding the information order condition, participants

who received all vignettes describing the core symptoms at the begin-

ning were significantly more often men (Χ2[1, N = 120] = 7.070,

P = .008) and were significantly less familiar with ICD‐10

(U[60.60] = 1285, P = .002). In addition, there was a trend that these

participants had earned a PhD (Χ2[1, N = 120] = 2.911, P = .088), were

working in a research unit (Χ2[1, N = 120] = 2.807, P = .094), had fewer

further therapeutic trainings (t[120] = −1.805, P = .074), and were less

likely to be specialized in treating trauma‐related disorders (Χ2[1,

N = 120] = 2.727, P = .099). Thus, we used these variables for the cal-

culation of propensity scores. Furthermore, regarding the “informa-

tion‐order condition,” an additional propensity score was calculated,

solely based on variables revealing significant group differences for

further analyses. In all other variables, no significant differences were

found between participants.

At least one false diagnosis without giving the aimed diagnosis

was given by 62 (51.7%) participants in the GAD case vignette, by 75

(62.5%) in the BPD and PTSD case vignette, and by 50 (41.7%) in the

PD case vignette. Across all conditions, participants made the aimed

diagnosis in each case vignette in fewer than half of the cases: GAD

was diagnosed by 58 (48.4%) participants in the GAD case vignette,

BPD and PTSD were both diagnosed by 45 (37.5%) participants in

the BPD and PTSD case vignette, and PD was diagnosed by 70

(58.3%) participants in the PD case vignette. The number of psycho-

therapists who made the aimed and at least one false comorbid diag-

nosis was relatively low: Of all participants, 23 (19.2%) made a false

comorbid diagnosis in the GAD case vignette, 45 (37.5%) in the BPD

and PTSD case vignette, and 28 (23.3%) in the PD case vignette.

Finally, 35 (29.2%) participants made a fully correct diagnostic decision

in the GAD case, none of the participants made a fully correct diagnos-

tic decision in the BPD and PTSD case vignette, and 42 (35.0%) partic-

ipants made a fully correct diagnostic decision in the PD case vignette.

Frequencies and percentages of diagnostic decisions for each of the

pretreatment and order‐of‐symptom‐description conditions are illus-

trated in Table 2. The fully correct diagnostic decisions were used as

dependent variable for the subsequently conducted logistic regression

analyses.

In relation to the confidence with given diagnoses, participants

were least confident in the GAD case vignette (n = 116; M = 60.60,

SD = 23.58), followed by the PD case vignette (n = 116; M = 72.09,

SD = 22.55), and the BPD and PTSD case vignette (n = 117;

M = 74.94, SD = 21.66). Analyses of the estimation of the severity of

each case indicated that participants rated the lowest severity in the

PD case vignette (n = 116; M = 67.30, SD = 15.51), followed by the

GAD case vignette (n = 113; M = 70.78, SD = 13.08), and the BPD

and PTSD case vignette (n = 117; M = 89.34, SD = 8.43).

Additionally, regarding the confidence with given diagnoses and

the estimation of severity of described diagnoses, there was no signif-

icant association between receiving a pretreatment report and a higher

expectation of the severity of the described disorders.

As expected, the groups did also not differ in the level of confi-

dence with the given diagnoses and the expectation of the severity

of described diagnoses (see Table 3).

As can be seen in Table 4, significant associations were observed

regarding the order of symptom descriptions. Psychotherapists who

received a GAD case vignette with the description of the GAD core

symptoms at the end of the vignette detected GAD significantly more

often than those receiving a vignette describing the core symptoms

at the beginning (GAD: odds ratio [OR] = 2.89, P = .017), and thus,

they made significantly more fully correct diagnostic decisions. In

the PD case, participants who received a vignette with the core

symptoms at the end also made significantly more fully correct diag-

nostic decisions (PD: OR = 2.65, P = .024). Both results would remain

significant, even if the calculated propensity score would have been

calculated solely based on variables that significantly differed

(P < .05) between groups (GAD: OR = 2.61, P = .025; PD: OR = 2.44,

P = .034). Admittedly, in that case, the results would not persist

Bonferroni‐corrections for multiple comparisons. In the BPD and

PTSD case, all participants made a false diagnosis. Thus, it was not

possible to investigate whether there was an association between

the order of symptom descriptions and false diagnoses or the correct-

ness of diagnostic decisions.

TABLE 3 Results of the linear regression models associating the independent variables (pretreatment‐report and information order conditions)with the dependent variable (confidence ratings and severity ratings of diagnoses)

95%‐CI for B

Lower B (SE) Upper β P

GAD vignette

Confidence ratings (0–100)

Pretreatment report 43.554 80.943 (18.869) 118.332 ‐0.114 0.238

Information order ‐9.396 0.289 (4.888) 9.975 0.006 0.953

Severity ratings of diagnosis (1–8)

Pretreatment report ‐1.680 3.316 (2.520) 8.132 0.127 0.191

Information order ‐4.277 1.148 (2.737) 6.572 0.044 0.676

BPD/PTSD vignette

Confidence ratings (0–100)

Pretreatment report ‐11.158 ‐3.060 (4.087) 5.038 ‐0.071 0.456

Information order ‐8.814 0.012 (4.454) 8.838 0.000 0.998

Severity ratings of diagnosis (1–8)

Pretreatment report ‐4.011 ‐0.870 (1.585) 2.271 ‐0.052 0.584

Information order ‐2.346 1.076 (1.727) 4.499 0.064 0.534

PD vignette

Confidence ratings (0–100)

Pretreatment report ‐8.973 ‐0.448 (4.302) 8.077 ‐0.010 0.917

Information order ‐5.551 3.783 (4.710) 13.117 0.084 0.424

Severity ratings of diagnosis (1–8)

Pretreatment report ‐2.711 3.128 (2.946) 8.966 0.101 0.291

Information order ‐4.917 1.146 (3.219) 7.841 0.047 0.651

Note. BPD = borderline personality disorder; GAD = generalized anxiety disorder; PD = panic disorder; PTSD = posttraumatic stress disorder.

TABLE 2 Frequency of diagnostic decisions for the pretreatment report and order‐of‐symptom‐description conditions (N = 120)

Pretreatment reportcondition N (%)

Order‐of‐symptom‐descriptioncondition N (%)

With report No reportCore symptomsat the beginning

Core symptomsat the end Σ N (%)

Aimed diagnosis/diagnoses GAD 33 (54.1) 25 (42.4) 25 (41.7) 33 (55.0) 58 (48.4)BPD/PTSD 24 (39.3) 21 (35.6) 19 (31.7) 26 (43.3) 45 (37.5)PD 37 (60.7) 33 (55.9) 32 (53.3) 38 (63.3) 70 (58.3)

False comorbid diagnosis/diagnoses GAD 13 (21.3) 10 (16.9) 11 (18.3) 12 (20.0) 23 (19.2)BPD/PTSD 24 (39.3) 21 (35.6) 19 (31.7) 26 (43.3) 45 (37.5)PD 15 (24.6) 13(22.0) 17 (28.3) 11 (18.3) 28 (23.3)

Fully correct diagnostic decision GAD 20 (32.8) 15 (25.4) 14 (23.3) 21 (35.0) 35 (29.2)BPD/PTSD 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0)PD 22 (36.1) 20 (33.9) 15 (25.0) 27 (45.0) 42 (35.0)

Note. BPD = borderline personality disorder; GAD = generalized anxiety disorder; PD = panic disorder; PTSD = posttraumatic stress disorder.

TABLE 4 Results of the logistic regression analyses associating the independent variables (pre‐treatment‐report and information order conditions)with the dependent variable (fully correct diagnostic decision).

95%‐CI for OR

B (SE) Wald Lower OR Upper P

GAD vignette

Pretreatment report ‐0.102 (0.386) 0.070 0.289 0.634 1.391 0.256

Information order 1.060 (0.446) 5.659 1.205 2.886 6.912 0.017 *

PD vignette

Pretreatment report ‐0.102 (0.386) 0.707 0.424 0.903 1.925 0.792

Information order 0.975 (0.431) 5.124 1.140 2.652 6.170 0.024 *

Note. GAD = generalized anxiety disorder; PD = panic disorder.

1150 CWIK AND MARGRAF

CWIK AND MARGRAF 1151

In relation to the pretreatment report condition, there was no sig-

nificant association between receiving a report and fully correct diag-

nostic decisions (GAD: OR = 0.634, P = .256; PD: OR = 0.903,

P = .792; see Table 4). Psychotherapists who received a pretreatment

report were not significantly more likely to make a fully correct diag-

nostic decision. In relation to the third case vignette (BPD and PTSD),

we were not able to investigate the associations between the condi-

tion and the diagnostic decisions, because none of the 120 participants

identified BPD and PTSD correctly.

4 | DISCUSSION

Considering the structure of diagnostic classification systems—

assessing core criteria first and subsequently proceeding to verify or

falsify a diagnosis by assessing the other criteria required—diagnosti-

cians might also follow this structure when making diagnoses. There-

fore, we expected that the correctness of diagnostic decisions would

be associated with the presentation of the core symptoms of mental

disorders at the beginning of the case vignettes. If so, results could

have been seen as an indication for a primacy effect in clinical psycho-

logical diagnosing. According to the results of our study, this hypothe-

sis has to be rejected. Rather, the data showed significant associations

in the opposite direction in two case vignettes, indicating a recency

effect. Additionally, as we expected, the order of symptom description

was also related to psychotherapists' confidence with diagnostic deci-

sions or their estimation of disorder severity. This hypothesis has also

to be rejected. However, studies in diagnostic reasoning indicate that—

depending on the presentation rate—both a primacy effect and a

recency effect occur when the order of symptom presentation is var-

ied (Lange, Thomas, Buttaccio, Illingworth, & Davelaar, 2012; Lange,

Thomas, & Davelaar, 2012; Rebitschek, Bocklisch, et al., 2015). Never-

theless, there is also evidence for a recency effect in medical decision‐

making (Bergus, Chapman, Levy, Ely, & Oppliger, 1998; Croskerry,

2003; Lawson & Daniel, 2011), consistent with our findings.

As mentioned previously, a stepwise diagnostic process may

reduce the primacy effect in diagnostic reasoning. Further, Hogarth

and Einhorn (1992) asserted that diagnosticians are more likely to be

subject to the recency effect when they have to process and decide

stepwise a sequence of information. Several other studies lend support

to this assertion. For example, Kerstholt and Jackson (1998) showed

that judicial decisions are more likely to be associated with a recency

effect when the defendant's probability of guilt is considered in a step-

wise manner, and multiple studies demonstrate a recency effect in

diagnostic reasoning when it occurs stepwise (Bergus et al., 1998;

Rebitschek, Bocklisch, et al., 2015).

In addition, it appears that the likelihood of a recency effect

increases in relation to the complexity and length of the information

source (Hogarth & Einhorn, 1992). This assumption has been well‐

evidenced. For example, Canic and Pachur (2014) demonstrated that

longer sequences of information increase the probability of a recency

effect. Furthermore, Bergus et al. (1998) reported an increased likeli-

hood of considering clinical data when it was presented last. Com-

pared to other studies in medical decision‐making, the

diagnosticians in this study engaged in more complex decision‐

making than participants in studies that reported, for example, a pri-

macy effect in medical decision‐making. Higher complexity and lon-

ger length of sequences could be the reason for the recency effect

observed in this study. All three case vignettes depicted several

symptoms and additional information about the patients. Aside from

the description level, the diagnosticians evaluated whether each

symptom is part of a hypothesized mental disorder and/or whether

it is an indication for an exclusion of another disorder. Handling mul-

tiple levels of information makes the diagnostic process in relation to

mental disorders complex and may increase the likelihood of false

decisions. The nonsignificant association between the information

order and the estimation of disorder severity and confidence with

given diagnoses was as expected, as merely the order of information

was manipulated, not the content of the vignettes.

Also contrary to our hypothesis, we did not observe significant

associations between the receipt of incongruent pretreatment

reports and misdiagnoses in any of the three case vignettes. Further,

pretreatment reports did not affect the therapists' confidence with

diagnostic decisions. One could speculate that participating therapists

were not distracted by the decoy diagnoses named in the pretreat-

ment reports, and therefore did not include the information from

these reports in their diagnostic judgment. Support for this assump-

tion comes from Mantonakis et al. (2009), putting forward that infor-

mation given first is more likely to be included when the judging

person has less knowledge in the field, and that judging persons with

less knowledge are less likely to compare initially given information

and information given later. The participants in this study had a high

level of knowledge of mental disorders and diagnostic criteria, as

indicated by their average 14.46 (SD = 10.14) years of work experi-

ence and high familiarity with the diagnostic systems ICD‐10, DSM‐

IV, and DSM‐5.

Furthermore, Rebitschek, Bocklisch, et al. (2015) showed how

stepwise diagnostic reasoning partially counteracts the primacy effect

and can even lead to a recency effect (see experiments 1B, 2B, and 3).

Given this procedural influence, one could speculate that stepwise

diagnostic reasoning may also counteract decoy effects. In contrast,

Hogarth and Einhorn (1992) showed that it can even lead to a recency

effect. Additionally, a study on sequential diagnostic reasoning demon-

strated that increased activation of the initially formed hypothesis

when reasoning about multiple candidate hypotheses leads to a pri-

macy effect (Rebitschek, Krems, et al., 2015). Given these results,

one could speculate that stepwise diagnostic reasoning also counter-

acts decoy effects. The diagnostic process in relation to mental disor-

ders is characterized by a continual reconciliation of symptoms with

criteria of several disorders. Hence, each symptom listed in a case

vignette could create a new diagnostic hypothesis or be used to reject

previous hypotheses. Such a stepwise nature of clinical psychological

diagnosing could be one possible explanation why we did not observe

the expected decoy effect. Thus, it is possible that such multiple

hypotheses testing could lead therapists to desist from hypothesis

caused by the decoy. Alternatively, it is also possible that a memory‐

based reasoning‐effect could explain these results: Complex vignettes

are associated with substantial workload, which could have either

interfered with memorizing the pre‐treatment report information or

could have removed initial hypotheses that were built on the

1152 CWIK AND MARGRAF

information from the pretreatment report from memory. However,

given our design, we are unable to fully explain why we did not find

a decoy effect.

Additionally, recent social psychological findings illustrate that

new information (e.g., counterevidence) can result in fast and long‐last-

ing reinterpretations of previous assumptions (Cone & Ferguson,

2015), which may be related to diagnostic processes (Mann, Cone, &

Ferguson, 2015). This is in line with a study on clinical psychologists,

which illustrated that this specific professional group is likely to incor-

porate new information in their diagnostic reasoning (De

Kwaadsteniet, Kim, & Yopchick, 2013). In our study, the information

in the case vignettes could be seen as new information that induced

a reinterpretation of information and consequently might have led to

a rejection of diagnostic hypotheses.

Conceivably, psychotherapists who received a pretreatment

report in the present study integrated the information of the case

vignette or consecutively compared the diagnoses named in the pre-

treatment report with their hypotheses. They then would proceed to

evaluate the symptom description in the case vignette as incongruent

with the pretreatment report and to reject counteracting hypotheses

based on information named in the incongruent pretreatment reports.

Accordingly, there was no significant association between diagnosti-

cians' confidence with diagnostic decisions and the receipt of the

report. This result suggests an independent evaluation of cases by psy-

chotherapists, while taking into account that such diagnostic process-

ing is not necessarily associated with the correctness of diagnostic

decisions, as the data also indicates. Finally, it is possible that psycho-

therapists did not trust the pretreatment reports and thus were not

distracted by the misleading diagnoses named in the reports. This

would be in line with a study indicating that psychotherapists consider

their own experience indispensable (Bruchmüller et al., 2011).

Simultaneously, the present study also shows that the receipt of a

pretreatment report as an additional information source does not

impact the diagnostic evaluation of psychotherapists concerning

severity ratings. This was surprising to us, but could be explained by

the fact that psychotherapists generally seem to conduct diagnostics

independent of pretreatment reports. Future studies should addition-

ally assess to what extent psychotherapists trust the given pretreat-

ment reports.

More broadly, the results of this study illuminate again the prob-

lem of misdiagnoses in daily practice. In all three case vignettes, psy-

chotherapists made aimed diagnoses in only almost a half of the

cases, and false comorbid diagnoses in 65.0–100% of all cases. Indeed,

fully correct diagnostic decisions were made at the most in 35.0% of

the cases, with false diagnostic decisions as high as 100%, in the case

of comorbid BPD and PTSD.

The high rate of misdiagnoses illustrates how susceptible psy-

chotherapists are to diagnostic errors. Together with the results of

the recency effect in relation to the order of information, our results

suggest that an unstructured process in diagnosing mental disorders

is highly prone to error. Diagnostic tools such as structured inter-

views may help to reduce diagnostic errors and give the user struc-

ture and decision support (Merten, Cwik, Margraf, & Schneider,

2017; Mitchell et al., 2009; Rettew, Lynch, Achenbach, Dumenci, &

Ivanova, 2009).

5 | LIMITATIONS

The present study has some limitations. First, we used case vignettes

to investigate diagnostic processes, which has inherent limitations.

The case description in a vignette is only roughly comparable to

obtaining information in a real therapeutic setting (Garb, 1998). The

diagnostic information in vignettes is somewhat filtered. Patients'

descriptions are usually not as structured as descriptions in vignettes.

Additionally, diagnosticians are not able to pose additional inquiries

when they are not confident with given information. Moreover, the

present case vignettes presented participants with more or less simple

cases that are not necessarily comparable to real cases, as Wolkenstein

et al. (2011) have pointed out. However, case vignettes are a feasible

tool for standardized investigations of diagnostic accuracy and are fre-

quently used in such studies (e.g., Dalgleish, 2004; Flanagan &

Blashfield, 2005; Meyer & Meyer, 2009; Wolkenstein et al., 2011).

Second, we only used cases with fulfilled diagnoses. As a consequence,

it is not clear whether the observed associations are assignable to sub-

clinical diagnoses (e.g., missing one criterion) that can take place in

daily practice. Third, the exclusion rate of 54.72% of questionnaires

was high. This could be the result of the relatively complex nature of

the survey or the length of the questionnaire (35 min). Additionally,

there were significant associations between dropouts and some demo-

graphic variables. Thus, it is unclear whether the results of this study

may be affected by a selection bias. Fourth, for the calculation of pro-

pensity scores, we also included demographic variables whereby the

groups only differed on the trend‐level. The rationale for this approach

was to control for all potential effects of group differences. However,

if only significant (P < .05) demographic variables would have been

used, all results would have remained significant but would not have

survived the threshold for multiple comparisons. Taking this into con-

sideration, even if our results regarding the recency effect are in line

with former studies, it is possible that these results are affected by

an accumulation of alpha errors. Thus, our results should be

interpreted with caution and need to be replicated. Fifth, only German

clinicians participated and, therefore, it is unclear whether these

results can be generalized to therapists in other countries. Sixth, it

should be considered that participants were instructed to judge with-

out auxiliary materials, which possibly contradicted their day‐by‐day‐

practice. Additionally, we did not assess participants' experience with

open clinical judgment and thus, are unable to test whether the groups

differed in this respect. However, participants did not differ signifi-

cantly regarding years of work experience. Finally, the vignettes

described constellations of symptoms based on the DSM‐5, whereas

German therapists are encouraged by the national health system to

use ICD‐10 criteria in providing treatment rationales for health insur-

ance billing, and are thus more familiar with ICD‐10 compared to

DSM‐IV and DSM‐5.

CONCLUSION

This study makes important contributions that further advance our

understanding of the influence of pre‐treatment reports and order of

information on diagnostic decisions. Surprisingly, the receipt of pre‐

CWIK AND MARGRAF 1153

treatment reports was not related to misdiagnoses. Replication in

other populations should be undertaken before generalizing this result.

It remains unclear whether pre‐treatment reports bias diagnostic deci-

sions when they are received subsequent to a diagnostic interview,

which may often occur in daily practice. In contrast, the results sup-

ported the hypothesis that information order has an effect on diagno-

sis. Considering the method of case vignettes and the limitations of

this study, reported results should be seen as preliminary and thus

should not be generalized for daily practice.

ACKNOWLEDGMENTS

We followed the TREND Statement (Des Jarlais, Lyles, Crepaz, & the

TREND Group, 2004) for the manuscript preparation. We would like

to thank Kristen Lavallee, Helen Copeland‐Vollrath, and Simon E.

Blackwell for critical reading and language editing of this manuscript.

The authors have no conflict of interest to report.

ROLE OF FUNDING

This research was funded by an Alexander von Humboldt professor-

ship, awarded to Jürgen Margraf.

REFERENCES

American Psychiatric Association. (2000). Diagnostic and statistical manualof mental disorders (4th–TR ed.). Washington, DC: American PsychiatricAssociation. 10.1176/appi.books.9780890423349

American Psychiatric Association. (2013). Diagnostic and statistical manualof mental disorders (5.). Washington, DC: American PsychiatricAssociation.

Austin, P. C. (2011). An introduction to propensity score methods forreducing the effects of confounding in observational studies. Multivari-ate Behavioral Research, 46(3), 399–424 http://doi.org/10.1080/00273171.2011.568786

Bartak, A., Spreeuwenberg, M. D., Andrea, H., Busschbach, J. J. V., Croon,M. A., Verheul, R., … Stijnen, T. (2009). The use of propensity scoremethods in psychotherapy research: A practical application. Psychother-apy and Psychosomatics, 78(1), 26–34. doi:10.1159/000162298

Bergus, G. R., Chapman, G. B., Levy, B. T., Ely, J. W., & Oppliger, R. A.(1998). Clinical diagnosis and the order of information. Medical DecisionMaking, 18(4), 412–417.

Blumenthal‐Barby, J. S., & Krieger, H. (2015). Cognitive biases and heuris-tics in medical decision making: A critical review using a systematicsearch strategy. Medical Decision Making, 35(4), 539–557.doi:10.1177/0272989X14547740

Bruchmüller, K., & Meyer, T. D. (2009). Diagnostically irrelevant informa-tion can affect the likelihood of a diagnosis of bipolar disorder. Journalof Affective Disorders, 116, 148–151. doi:10.1016/j.jad.2008.11.018

Bruchmüller, K., & Schneider, S. (2012). Fehldiagnose Aufmerksamkeitsdefizit‐und Hyperaktivitätssyndrom? Psychotherapeut, 57(1), 77–89. doi:10.1007/s00278‐011‐0883‐7

Bruchmüller, K., Margraf, J., Suppiger, A., & Schneider, S. (2011). Popular orunpopular? Therapists' use of structured interviews and their estima-tion of patient acceptance. Behavior Therapy, 42(4), 634–643.doi:10.1016/j.beth.2011.02.003

Bruchmüller, K., Margraf, J., & Schneider, S. (2012). Is ADHD diagnosed inaccord with diagnostic criteria? Overdiagnosis and influence of clientgender on diagnosis. Journal of Consulting and Clinical Psychology,80(1), 128–138. doi:10.1037/a0026582

Canic, E., & Pachur, T. (2014). Serial‐position effects in preference con-struction: A sensitivity analysis of the pairwise‐competition model.Frontiers in Psychology, 5, 902. doi:10.3389/fpsyg.2014.00902

Chapman, G. B., Bergus, G. R., & Elstein, A. S. (1996). Order of informa-tion affects clinical judgment. Journal of Behavioral Decision Making,9, 201–211.

Choplin, J. M., & Hummel, J. E. (2005). Comparison‐induced decoy effects.Memory and Cognition, 33(2), 332–343. doi:10.3758/BF03195321

Cloitre, M., Garvet, D., Weiss, B., Calrson, E. B., & Bryant, R. A. (2014).Distinguishing PTSD, complex PTSD, and borderline personality disor-der: A latent class analysis. European Journal of Psychotraumatology, 5,25097. doi:10.3402/ejpt.v5.25097

Cone, J., & Ferguson, M. J. (2015). He did what? The role of diagnosticity inrevising implicit evaluations. Journal of Personality and Social Psychology,108(1), 37–57. doi:10.1037/pspa0000014

Connolly, T., Reb, J., & Kausel, E. E. (2013). Regret salience and accountabil-ity in the decoy effect. Judgment and Decision making, 8(2), 136–149.

Croskerry, P. (2003). The importance of cognitive errors in diagnosis andstrategies to minimize them. Academic Medicine, 78(8), 775–780.

Cunnington, J. P. W., Trunbull, J. M., Regehr, G., Marriott, M., & Norman, G.R. (1997). The effect of presentation order in clinical decision making.Academic Medicine, 72(10), 40–42.

Cwik, J. C., & Teismann, T. (2016). Misclassification of self‐directed vio-lence. Clinical Psychology and Psychotherapy. doi:10.1002/cpp.2036

Cwik, J. C., Papen, F., Lemke, J.‐E., & Margraf, J. (2016). An investigation ofdiagnostic accuracy and confidence associated with diagnostic check-lists as well as gender biases in relation to mental disorders. Frontiersin Psychology, 7, 1813. doi:10.3389/fpsyg.2016.01813

Cwik, J. C., Merten, E. C., Woud, M. L., Shkreli, L., Schneider, S., & Margraf,J. (2017). Biased diagnoses? Meta‐analyses of the effects of gender,ethnicity and unstructured assessment. Manuscript under Revision.

D'Agostino, R. B. Jr. (1998). Tutorial in biostatistics: Propensity scoremethods for bias reduction in the comparison of a treatment to anon‐randomized control group. Statistics in Medicine, 17, 2265–2281.

Dalgleish, T. (2004). Cognitive approaches to posttraumatic stress disorder:The evolution of multirepresentational theorizing. Psychological Bulletin,130(2), 228–260. doi:10.1037/0033‐2909.130.2.228

De Kwaadsteniet, L., Kim, N. S., & Yopchick, J. E. (2013). How dopractising clinicians and students apply newly learned causalinformation about mental disorders? Journal of Evaluation in ClinicalPractice, 19(1), 112–117. doi:10.1111/j.1365‐2753.2011.01781.x

Des Jarlais, D. C., Lyles, C., Crepaz, N., & theTREND Group (2004). Improv-ing the reporting quality of nonrandomized evaluations of behavioraland public health interventions: The TREND statement. American Jour-nal of Public Health, 94(3), 361–366.

Eichelman, B. (2010). Borderline personality disorder, PTSD, and suicide.American Journal of Psychiatry, 167(10), 1152–1154. doi:10.1176/appi.ajp.2010.10060870

Flanagan, E. H., & Blashfield, R. K. (2005). Gender acts as a context forinterpreting diagnostic criteria. Journal of Clinical Psychology, 61(12),1485–1498. doi:10.1002/jclp.20202

Garb, H. N. (1998). Studying the clinician: judgment research and psychologi-cal assessment. Washington, DC: American Psychological Association.

Grant, B. F., Chou, P., Goldstein, R. B., Huang, B., Stinson, F. S., Saha, T. D.,… Ruan, W. J. (2008). Prevalence, correlates, disability, and comorbidityof DSM‐IV borderline personality disorder: Results from the wave 2national epidemiologic survey on alcohol and related conditions. Journalof Clinical Psychiatry, 69(4), 533–545.

Hodges, S. (2003). Borderline personality disorder and posttraumatic stressdisorder: Time for integration? Journal of Counseling & Development,81(4), 409–417.

Hogarth, R. M., & Einhorn, H. J. (1992). Order effects in belief updating: Thebelief‐adjustment model. Cognitive Psychology, 24(1), 1–55.doi:10.1016/0010‐0285(92)90002‐J

1154 CWIK AND MARGRAF

Horowitz, L. M., Post, D. L., de Sales, F. R., Wallis, K. D., & Siegelman, E. Y.(1981). The prototype as a construct in abnormal psychology: II. Clarify-ing disagreement in psychiatric judgements. Journal of AbnormalPsychology, 90, 575–585.

IBM Corporation. (2013). SPSS. Armonk, New York: IBM Corporation.

In‐Albon, T., Suppiger, A., Schlup, B., Wendler, S., Margraf, J., & Schneider,S. (2008). Validität des Diagnostischen Interviews bei psychischenStörungen (DIPS für DSM‐IV‐TR). Zeitschrift für Klinische Psychologieund Psychotherapie, 37(1), 33–42. doi:10.1026/1616‐3443.37.1.33

Kerstholt, J. H., & Jackson, J. L. (1998). Judicial decision making: Order ofevidence presentation and availability of background information.Applied Cognitive Psychology, 12, 445–454.

Kostopoulou, O., Russo, J. E., Keenan, G., Delaney, B. C., & Douiri, A.(2012). Information distortion in physicians' diagnostic judgments.Med-ical Decision Making, 32(6), 831–839. doi:10.1177/0272989X12447241

Lange, N. D., Thomas, R. P., Buttaccio, D. R., Illingworth, D. A., & Davelaar,E. J. (2012). Working memory dynamics bias the generation of beliefs:The influence of data presentation rate on hypothesis generation.Psychonomic Bulletin and Review, 20, 171–176. doi:10.3758/s13423‐012‐0316‐9

Lange, N. D., Thomas, R. P., & Davelaar, E. J. (2012). Temporal dynamics ofhypothesis generation: The influences of data serial order, data consis-tency, and elicitation timing. Frontiers in Psychology, 3, 215.doi:10.3389/fpsyg.2012.00215

Lawson, A. E., & Daniel, E. S. (2011). Inferences of clinical diagnostic rea-soning and diagnostic error. Journal of Biomedical Informatics, 44(3),402–412. doi:10.1016/j.jbi.2010.01.003

Leiner, D. J. (2014). SoSci survey (version 2.4.00‐i) [computer software].Retrieved from http://www.soscisurvey.com

Mann, T. C., Cone, J., & Ferguson, M. J. (2015). Social‐psychological evi-dence for the effective updating of implicit attitudes. Behavioral andBrain Sciences, 38, . doi:10.1017/S0140525X14000223.e15

Mantonakis, A., Rodero, P., Lesschaeve, I., & Hastie, R. (2009). Order inchoice: Effects of serial position on preferences. Psychological Science,20(11), 1309–1312. doi:10.1111/j.1467‐9280.2009.02453.x

Margraf, J., & Schneider, S. (2009). Diagnostik psychischer Störungen mitstrukturierten Interviews. In J. Margraf, & S. Schneider (Eds.), Lehrbuchder Verhaltenstherapie, Band 1: Grundlagen, Diagnostik, Verfahren,Rahmenbedingungen (3rd ed.). (pp. 344–345). Heidelberg: Springer.

Merten, E. C., Cwik, J. C., Margraf, J., & Schneider, S. (2017). Overdiagnosisof mental disorders in children and adolescents (in developed coun-tries). Child and Adolescent Psychiatry and Mental Health, 11, 5.doi:10.1186/s13034‐016‐0140‐5

Meyer, F., & Meyer, T. D. (2009). The misdiagnosis of bipolar disorder as apsychotic disorder: Some of its causes and their influence on therapy.Journal of Affective Disorders, 112, 174–183. doi:10.1016/j.jad.2008.04.022

Mitchell, A. J., Vaze, A., & Rao, S. (2009). Clinical diagnosis of depression inprimary care: A meta‐analysis. The Lancet, 374, 609–619. doi:10.1016/S0140‐6736(09)60879‐5

Pabst, A., Schauer, M., Bernhardt, K., Ruf, M., Goder, R., Rosentraeger, R., …Seeck‐Hirschner, M. (2012). Treatment of patients with borderline per-sonality disorder and comorbid posttraumatic stress disorder usingnarrative exposure therapy: A feasibility study. Psychotherapy and Psy-chosomatics, 81(1), 61–63. doi:10.1159/000329548

Rebitschek, F. G., Bocklisch, F., Scholz, A., Krems, J. F., & Jahn, G. (2015).Biased processing of ambiguous symptoms favors the initially leadinghypothesis in sequential diagnostic reasoning. Experimental Psychology,62(5), 287–305. doi:10.1027/1618‐3169/a000298

Rebitschek, F. G., Krems, J. F., & Jahn, G. (2015). Memory activation of mul-tiple hypotheses in sequential diagnostic reasoning. Journal of CognitivePsychology, 27(6), 780–796. doi:10.1080/20445911.2015.1026825

Rettew, D. C., Lynch, A. D., Achenbach, T. M., Dumenci, L., & Ivanova, M. Y.(2009). Meta‐analyses of agreement between diagnoses made fromclinical evaluations and standardized diagnostic interviews. InternationalJournal of Methods in Psychiatric Research, 18(3), 169–184.doi:10.1002/mpr.289

Rosenbaum, B. Y. P. R., & Rubin, D. B. (1983). The central role of the pro-pensity score in observational studies for causal effects. Biometrika,70(1), 41–55.

Schmidt, N. B., Salas, D., Bernert, R., & Schatschneider, C. (2005). Diagnos-ing agoraphobia in the context of panic disorder: Examining the effectof DSM‐IV criteria on diagnostic decision‐making. Behaviour Researchand Therapy, 43(9), 1219–1229. doi:10.1016/j.brat.2004.09.004

Schulz‐Hardt, S., Frey, D., Luthgens, C., & Moscovici, S. (2000). Biasedinformation search in group decision making. Journal of Personality andSocial Psychology, 78, 655–669.

Shemberg, K. M., & Doherty, M. E. (1999). Is diagnostic judgment influ-enced by a bias to see pathology? Journal of Clinical Psychology, 55(4),513–518.

Slaughter, J. E., Sinar, E. F., & Highhouse, S. (1999). Decoy effects and attri-bute‐level inferences. Journal of Applied Psychology, 84(5), 823–828.doi:10.1037/0021‐9010.84.5.823

Slaughter, J. E., Kausel, E. E., & Quiñones, M. A. (2011). The decoy effect asa covert influence tactic. Journal of Behavioral Decision Making, 24(3),249–266. doi:10.1002/bdm.687

Spitzer, R. L., Gibbon, M., Skodol, A. E., Williams, J. B., & First, M. B. (1991).Falldarstellungen Diagnostisches und statistisches Manual PsychischerStörungen (DSM‐III‐R). In R. L. Spitzer, M. Gibbon, A. E. Skodol, J. B.Williams, & M. B. First (Eds.), Falldarstellungen Diagnostisches undstatistisches Manual Psychischer Störungen (DSM‐III‐R). (pp. 257–258).Weinheim: Beltz Verlag.

Suppiger, A., In‐Albon, T., Herren, C., Bader, K., Schneider, S., & Margraf, J.(2008). Reliabilität des Diagnostischen Interviews bei PsychischenStörungen (DIPS für DSM‐IV‐TR) unter klinischen Routinebedingungen.Verhaltenstherapie, 18(4), 237–244. doi:10.1159/000169699

Weinberger, A. H., Maciejewski, P. K., McKee, S. A., Reutenauer, E. L., &Mazure, C. M. (2009). Gender differences in associations between life-time alcohol, depression, panic disorder, and posttraumatic stressdisorder and tobacco withdrawal. The American Journal on Addictions,18(2), 140–147. doi:10.1080/10550490802544888

Wolkenstein, L., Bruchmüller, K., Schmid, P., & Meyer, T. D. (2011).Misdiagnosing bipolar disorder ‐ do clinicians show heuristic biases?Journal of Affective Disorders, 130(3), 405–412. doi:10.1016/j.jad.2010.10.036

World Health Organization (1992). Tenth revision of the international classi-fication of diseases, Chapter V (F): Mental and behavioural disorders(including disorders of psychological development). Clinical descriptionsand diagnostic guidelines. Geneva: World Health Organization.

Zaudig, M., Wittchen, H.‐U., & Saß, H. (2000). DSM‐IV und ICD‐10 Fallbuch:Fallübungen zur Differentialdiagnose nach DSM‐IV und ICD‐10.Göttingen: Hogrefe.

How to cite this article: Cwik JC, Margraf J. Information order

effects in clinical psychological diagnoses. Clin Psychol

Psychother. 2017;24:1142–1154. https://doi.org/10.1002/cpp.2080