jpd282_Final Thesis Edited

M.D. CRITICAL THINKING AND DECISION MAKING 1

CRITICAL THINKING AND DECISION-MAKING IN MODERN DAY PHYSICIANS

by: James Patrick Dunlea

A thesis submitted to the faculty of Cornell University

In partial requirement of the requirements for the

2015-2016 UNDERGRADUATE HONORS PROGRAM

Human Development Cornell University

May, 2016

Advisors:

Professor Stephen J. Ceci, Department of Human Development

Professor Wendy M. Williams, Department of Human Development


© James Patrick Dunlea


ABSTRACT

The present study investigated how individuals from different occupations/educational levels

critically think and make decisions. Specifically, undergraduates, graduate students, and post-

graduates in both medical and legal fields were sampled to explore differences in critical

thinking ability and decision-making preferences within and between groups. Members of the

general population were also surveyed. Subjects completed a battery of six cognitive tests—

namely, the Critical Reflection Test (CRT), Abridged Numeracy Scale, Linda Problem, Wason

Cards Task, and two versions of the Beads Task—to gauge their critical thinking and decision

making preferences. Results indicated that individuals from different occupations differed in

terms of critical thinking abilities. Specifically, physicians were found to underperform on

certain critical-thinking measures when compared to other highly educated samples, such as

lawyers. Results from the present study warrant the re-examination of both the medical education

system and the cognitive strategies employed by physicians.


BIOGRAPHICAL SKETCH

James Patrick Dunlea was born and raised in Wayne, Illinois. He will be graduating with Honors

from Cornell University with a Bachelor of Science in Human Development in May 2016.

Starting in the fall of 2016, James will be matriculating as a full-time graduate student at

Northwestern University’s Pritzker School of Law to pursue a Masters of Science in Law (MSL).

He will be concentrating in Regulatory Analysis and Strategy. Upon completion of his MSL

degree, James hopes to establish a psycho-legal research career and matriculate as a J.D./Ph.D.

student studying the underpinnings of punishment, reward, and the psychological roots of the

bias that is feeding the current trend favoring mass-incarceration/retribution over

healing/education.


ACKNOWLEDGMENTS

I would like to thank Professor Stephen J. Ceci and Professor Wendy M. Williams for

their support, guidance, and hospitality throughout the duration of my time in the Child Witness

and Cognition Lab. Additionally, this project would not have been possible without doctoral

student Kayla A. Burd—words cannot express how thankful I am for your unconditional support,

time invested in me, and friendship throughout my undergraduate years. I would also like to

thank Jonathan G. Lash for his generous assistance with my project. Thank you as well to the

other graduate Child Witness and Cognition Lab members Rebecca K. Helm and Michael D.

Creim for their invaluable insights with this project.

Finally, I would also like to thank my mother, Helen Dunlea, and my father, Sean

Dunlea, and my siblings, Kayla and Patrick, for giving me the world and so much more.


TABLE OF CONTENTS

Page Number

Biographical Sketch………………………………………………………………….....................4

Acknowledgments…………………………………………………………………………………5

List of Tables……………………………………………………………………………………...8

List of Figures……………………………………………………………………………………10

Introduction………………………………………………………………………………………11

Comparing Medical Decision-Making to Non-Medical Groups………………………...15

The Present Study………………………………………………………………………..18

Methodology……………………………………………………………………………………..19

Participants……………………………………………………………………………….19

Design……………………………………………………………………………………20

Materials…………………………………………………………………………………20

Procedures………………………………………………………………………………..23

Results……………………………………………………………………………………………24

Cognitive Reflection Test (CRT) …………………………………………………...…...24

Abridged Numeracy Scale………………………………………………………...……..26

Linda Problem……………………………………………………………...…………….28

Wason Cards Task……………………………………………………………………….32

Beads Tasks……………………………………………………………………………...35

Discussion………………………………………………………………………………………..40

Implications……………………………………………………………………………....44

Limitations and Future Directions……………………………………………………….49


References………………………………………………………………………………………..52

Appendices……………………………………………………………………………………….66

Appendix A……………………………………………………………..………………..66

Appendix B……………………………………………………………..………………..67

Appendix C…………………………………………………………………..…………..68

Appendix D………………………………………………………………..……………..69

Appendix E……………………………………………………………………………....70

Appendix F……………………………………………………………………………….71


LIST OF TABLES

Table Page Number

1. Means and Standard Deviations for CRT Score—Between Groups Comparison………25

2. Means and Standard Deviations for CRT Scores—Within Legal Education

Comparison……………………………………………………………………………...26

3. Means and Standard Deviations for CRT Scores—Within Medical Education

Comparison……………………………………………………………………………...26

4. Means and Standard Deviations for Abridged Numeracy Scale Scores—Between Groups

Comparison……………………………………………………………………………...27

5. Means and Standard Deviations for Abridged Numeracy Scale Scores— Within Legal

Education Comparison…………………………………………………………………..28

6. Means and Standard Deviations for Abridged Numeracy Scale Scores— Within Medical

Education Comparison………………………………………………………………......29

7. Linda Problem Between Groups with J.D.s as Reference Point…………………………30

8. Linda Problem Between Groups with M.D.s as Reference Point………………………..31

9. Linda Problem Within Groups with J.D.s as Reference Point…………………………...32

10. Linda Problem Within Groups with M.D.s as Reference Point………………………….32

11. Wason Cards Task Between Groups with J.D.s as Reference Point…………………….33

12. Wason Cards Task Between Groups with M.D.s as Reference Point……………….….34

13. Wason Cards Task Within Groups with J.D.s as Reference Point……………………....34

14. Wason Cards Task Within Groups with M.D.s as Reference Point……………………..35

15. Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task…....36

16. Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task Within


Legal Education…………………………………………………………………………37

17. Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task Within

Medical Education………………………………………………………………………37

18. Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task—

Between Between Groups………………………………………..………………………38

19. Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task

Within Legal Education…………………………………………………………………39

20. Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task

Within Medical Education………………………………………………………………39


LIST OF FIGURES

Figure Page Number

1. Mean CRT Composite Scores by Educational Track…………………….……….......…60

2. Percent Correct on the Linda Problem by Educational Track…………………………...61

3. Percent Correct on the Wason Cards Task by Educational Track……………….………62

4. Beads Drawn on Easy Beads Task by Education Track………………………..………..63

5. Beads Drawn on Difficult Beads Task by Education Track……………………………..64

6. Abridged Numeracy Scale Mean Composite Scores by Educational Track……………..65


Critical Thinking and Decision-Making in Modern Day Physicians

In our increasingly growing age of technology, information is both literally and

figuratively at our fingertips. In response to the new, technology-savvy generation, the way

information is disseminated has evolved in tandem with the trend favoring convenience

(Motiwalla, 2007). Information has become routinely published and widely accessible on the

world-wide-web. Unlike in more primitive times, the curious, information-seeking individual is

no longer prompted to leave his or her own comfort to find the knowledge he or she seeks

(Bennett, Maton, & Kervin, 2008). Additionally, Bennett and colleagues (2008) propose that

citizens of the modern era have access to more information than one can quantitatively perceive;

simply put, we are living in a robustly information-saturated environment.

This increase in the amount of information available may be viewed as the result of

humans’ obsession for minimizing both risk and uncertainty (Tversky, Kahneman, & Thaler

1991). Furthermore, Tversky and colleagues (1991) argue that the drive to create this so called

“information-obsessed society” may be reduced to human innate desire to control the

surroundings in which we live. In short, it is the consensus among researchers that it is

normative human inclination to strive for empirical truth. There is undoubtedly no feasible way

to completely erase risk and uncertainty throughout decision-making processes. Understanding

this fact, researchers believe that the best way to minimize uncertainty is to ascertain factual

quanta of knowledge from outside sources that are reputable, clear, and accessible (Henrich &

Gil-White, 2000). The current information-saturated society attempts to do just this. Being

cognizant and proactive about self-education with the relevant and available information, a

decision-making individual is capable of mentally employing cognitive cost-benefit analyses to


decide which course of action will yield the most fruitful and least harmful experience to

themselves (Jaeger, Webler, Rosa, & Renn, 2013).

However, this cost-benefit analysis is not always correctly executed; human decision-

making is inherently imperfect due, in part, to constraints on our information processing (e.g.,

working memory limits). For example, when experiencing stress or cognitive dissonance while

making a decision, heuristics are often implemented to make quick plans of action (Neth &

Gigerenzer, 2015). Although heuristics offer quick ways to make judgments about one’s course

of action, this algorithm-type cognitive tool may draw upon extraneous information that may

lead to flawed evaluations. When using heuristics to make a decision or form an opinion,

tangential pieces of stored knowledge that have been encoded in memory may be introduced into

decision-making processes in the form of biases (Gigerenzer & Gaissmaier, 2011). Bringing in

bias introduces complications in the decision-making process by increasing the probability that a

prediction error—an incorrect assessment of the situation and subsequent action—will occur

(Neth & Gigerenzer, 2015). By increasing the prediction error, one is proportionally amplifying

the chance that he/she misjudges the situation being confronted, and acts in a potentially harmful

manner to oneself or others.

Interestingly, flawed decision-making may begin before one is actually confronted with a

stimulus that requires a decision. Researchers have found that memory—sometimes described as

a “metaphorical ball of malleable putty”—is continually undergoing shaping and re-coding

processes by the experiences one has throughout his or her lifetime (Ceci & Bruck, 1995). This

memory reconstruction effectively plays a significant role in cognitive processes and the way

one makes decisions (Reyna & Lloyd, 1997). Memory reconstruction is a process that is capable

of altering the architecture of previously formed memories. This process is therefore able to


deceive the brain into “thinking” that all of its stored memories are true. A commonly held lay,

albeit incorrect, perception of memory is that the brain works analogously to a video camera

(Ceci & Bruck, 1995): When one experiences a series of events, it seems as if the events are

seamlessly recorded by the brain. However, contemporary research supports the claim that there

is no such veridicality in information encoding and storage processes; it is safe to say that false

memories—whether they are purposefully or passively accumulated—appear real, and at play in

daily life (Ceci & Bruck, 1995).

To exemplify the relevance and subtleness of how memory manipulation may occur,

conceptualize the following scenario: if one was given a list of the words to memorize such as

‘winter’, ‘frosty’, ‘cold’, ‘ice’, ‘sledding’, ‘igloo’, ‘Eskimo’, ‘holiday’, ‘white’, ‘mittens’, and

‘flake’ and after an elapsed period of time, asked if he or she heard the word ‘snow’, it is quite

likely that this word will be falsely remembered being on the list of words (Roediger &

McDermott, 1995). In their study, Roediger and McDermott found that unsuspecting college

participants falsely remembered the non-represented word (in this case, ‘snow’) 40% of the time.

This is because the word ‘snow’ has been stored in the same mental scheme as the other words

presented. These words have been categorized in a similar manner within the individual’s neural

network. Therefore, it is difficult to discriminate between the actual words and semantic

associations that were unconsciously activated by our brain processes. Since people may draw

from their stored repertoire of knowledge to make decisions, it is plausible that their cognitive

sets may contain pockets of falsely remembered memories, potentially derailing decision-making

(Burke & Miller, 1999).

In addition to automatic memory alteration processes that may eventually taint decision-

making, individual cognitive predispositions may also lead to skewed information digestion. As


suggested by many researchers, the increase in information does not necessarily linearly

correlate with the general public’s access, consumption, and, most importantly, correct

internalization of this knowledge (Perlis, Purang, & Anderson, 1998; Blum-Kulka & Weizman,

1988). For example, Kahan, Jenkins-Smith, and Braman (2011) argue that the scientific

community has largely been unsuccessful at convincing the public about certain empirical

findings. Numerous examples of this can be found. Debates regarding the appropriateness of

prescribing and using medical marijuana (Adler & Colbert, 2013), the true effects of hydraulic

fracturing—commonly termed “fracking”—(Sovacool, 2014), and the importance (or lack there

of) of eating breakfast in the morning (Levitsky & Pacanowski, 2013) have drawn serious

attention to the notion that “facts”—seemingly deductive products of empirical research, testing,

and data collection—are able to be framed in different manners.

The skewed consumption and internalization of facts largely stems from the ideology that

the general population heeds because it is culturally predisposed to believe, while other ideas are

discredited for the same reason (Tomasello, Carpenter, Call, Behne & Moll, 2005). This

phenomenon is termed “cultural cognition” and can further be illustrated in Hmong culture,

where tonic-clonic seizures, commonly known as gran mal seizures, are viewed as being divinely

inspired (Fadiman, 1997). The Hmong people believe that when an epileptic individual seizes, a

wicked spirit, or a dab, infiltrates that person’s body; it is this physical manifestation that causes

the associated jerking and shaking movements. Although the Hmong do view this as being a

serious condition, they believe only a select few are chosen to be inflicted with this condition;

since affected individuals are rare, they are often elected to become shamans or sages. Therefore,

it is unsurprising that Hmong people living in the United States have been documented as

refusing medications to treat epileptic children. To many Westernized individuals, though, there


is confusion as to why a debate about administering medicine to an epileptic child exists in the

first place. The answer to the question as to why the debate exists can be grounded in cultural

cognition theory (Kahan et al., 2011). To the Hmong people, it is held true that epilepsy is a

revered celestial affliction, whereas to most Western individuals, it is true epilepsy is a serious,

threatening condition that must be treated with medication. These are two “facts”, yet they are

certainly contradictory to some degree. Therefore, if individuals from each respective culture

were asked which side of the debate was “correct”, different conclusions would be drawn. This

example is analogous to how people are biased towards drawing certain conclusions despite

being given the same information. The area of “framing effects” in judgment and decision-

making research is replete with similar examples.

Comparing Medical Decision-Making to Non-Medical Groups

In sum, there are empirical and theoretical bases for the conclusion that the common

person is susceptible to making a flawed decision—a nervous suitor may be mistaken for dull on

his first date, a female corporate executive’s desire to come off as authoritative may be perceived

by her co-workers as boorish, etc. The contexts in which a mistake can be made are endless.

Despite the large body of literature on the dangers of flawed cognition, few researchers have

examined what factors may influence certain groups more prone to make irrational decisions.

The current study will examine how formal education shapes the way one thinks and

subsequently makes decisions. Specifically, physicians’ cognitive styles and decision-making

processes will be examined and compared to both members of the general population and other

highly educated professionals, namely, lawyers.

Physicians, throughout history, have been held in a high regard (Maulitz, 1988). In

modern times, they surpass other highly educated, prestigious occupations such as physicists,


chemists, mathematicians, architects, politicians, military officers, engineers, lawyers, and

economists, an empirically-based ranking of occupational prestige (Ganzeboom, De Graaf, &

Treiman, 1992; Ganzeboom & Treiman, 1996; Nakao & Treas, 1992). As cited in Malmsheimer

(1988), the general public often views physicians as “essentially flawless individuals, dedicated

to healing both the physical and emotional ills of patients and often providing advice about the

solutions to nonmedical problems” (pp. 173). One of the characteristics most used to describe

physicians is altruism—the trait that entails an individual serving the needs of others with no

tangible benefit for himself/herself (McClelland, 2014).

Seeing the modern-day physician through a rose-colored lens perpetuates the “doctor

knows best” mentality; although this perception is beneficial in terms of establishing trust

between physician and patient, it also sows the seed that depicts physicians as immune to the

fallacious reasoning that afflicts the rest of us. Moreover, researchers have found that physicians

themselves may hold similar perceptions about themselves—Bornstein and Emler (2001) found

that a widespread consensus among physicians is that they were more or less immune to making

errors. Furthermore, Russo and Schoemaker (2002) purport that individuals who often engage in

fallacious decision-making tend to believe that they are exceptional decision-makers.

In order to examine whether the “infallible doctor” conception is accurate, the types of

fallacies in which a physician might make needs to be investigated: in the next chapter ethical,

perceptual, and reasoning fallacies will be discussed and put in the context of medicine. The

previous research raises a concern for medical professionals specifically, but also for those who

are in any decision-making role.

In 1998, British gastroenterologist Dr. Andrew Jeremy Wakefield published an

interesting article in the British Journal of Medicine. In this article, Dr. Wakefield cited data that


pointed to the conclusion that the vaccine against measles, mumps, and rubella (MMR) may lead

children to acquire developmental disorders, such as autism, and digestive tract inflammation,

called non-specific colitis. The article went viral. Thirteen years after Wakefield’s initial

publication, the British Journal of Medicine retracted the publication, ridiculing the physician for

fraudulent data collection, and stripped Wakefield of his license to practice (Godlee, 2011).

Although the article has been retracted by the journal, scientists must continue the fight to

convince the public that there is no causal connection between vaccinations and

developmental/gastrointestinal disorders (see news story by Haberman, 2015). This is an

example of an ethical fallacy in which a physician failed to abide by a code of conduct honoring

truthful data collection and reporting. Although ethical lapses are a very important flaw in the

medical field, my study will explore cognitive fallacies that lead to flawed decision-making.

Another type of commonly made mistake in decision-making is a perceptual fallacy. This

type of fallacy can more simply be described as a sort of “inattentional blindness” where an

observer fails to notice a seemingly blatant perceptual element in the larger environment (Drew,

Võ, Wolfe, 2013). In 2013, Drew and colleagues conducted an experiment that put the perceptual

keenness of seasoned radiologists to the test: 24 radiologists who routinely scanned for lung

cancer tumors were allowed 3 minutes to scroll through a chest computerized tomography (CAT)

scan to identify any abnormal nodules. To the unsuspecting physicians, a picture of a gorilla—

one that is 48 times the size of a typical nodule—was superimposed on one of the CAT scans

they saw. Drew and colleagues (2013) reported that 83% of seasoned radiologists failed to see

the gorilla’s picture in the CAT scans and reported that there was nothing wrong. This startling

statistic demonstrates that even highly trained professionals, such as physicians, are sometimes


flawed in their decision making abilities—in this case, in misperceiving their surroundings and

using information to draw incorrect conclusions.

To demonstrate reasoning fallacies in the medical profession, Hoffrage and Gigerenzer

(1998) tested 48 German physicians of varying subspecialties on their ability to calculate

positive predictive values on certain medically related diagnostic tests. In order to calculate the

positive predictive value for a patient, Hoffrage and Gigerenzer (1998) provided the physicians

with the base rate in the general population for a disease, the percentage of cases in which the

diagnostic test correctly predicts a positive test result, and the percentage in which the diagnostic

test incorrectly predicts a positive test results. The researchers reported that physicians were able

to correctly calculate the positive predictive value only 10% of the time (Hoffrage & Gigerenzer,

1998). In one variation of the study where the physicians were asked to predict the probability

that a patient had colorectal cancer based on the statistics provided, answers ranged between 1

percent and 99 percent! Therefore, Hoffrage and Gigerenzer (1998) were able to reveal the

deficits in critical thinking skills and reasoning that characterize physicians.

The Present Study

The current study focused on the reasoning type cognitive fallacy when making

decisions. In the study, the frequency of reasoning fallacies from a physician sample representing

the general population and a sample of lawyers was compared. In order to ascertain how the

curriculum of one’s formal education shapes his/her cognition, the frequency of reasoning

fallacies made by undergraduate students who have not yet matriculated into medical/law school

(“pre-medical/law students”) and current medical/law students was also compared. This

comparison was achieved by administering a battery of critical thinking measures to current


undergraduate pre-med/pre-law students, matriculated medical/law students, degree-bearing

physicians/lawyers and to members of the general population via an online (Qualtrics) survey.

As cited by Loftus, Levidow, and Duensing (1992), group differences in memory exist

among individuals from different occupations. In their study, Loftus and colleagues found that

artists and architects exhibited a superior memory when compared to law enforcement agents,

and homemakers, for example. As there is evidence for differences in cognitive mechanisms in

memory (Loftus et al., 1992), it was hypothesized that there would also be differences in the way

people of different occupations yield to reasoning fallacies that impact critical-thinking and

decision-making. More specifically, it was predicted that both physicians and lawyers, on

average, will score higher on critical thinking tests than members of the general population.

However, it was posited that physicians’ critical thinking abilities will fall short when pitted

against individuals holding a law degree due to the latter group’s training that emphasizes logic

and deductive reasoning (e.g., in contract law class). Additionally, it was hypothesized that

lawyers and physicians would have different decision-making preferences—specifically, lawyers

would be more motivated than physicians to make decisions that are clearly backed by

substantial and unambiguous evidence.

Methodology

Participants

A total of 738 participants completed all tasks included in the present study. 239

participants were recruited via Cornell University’s Psychology Experiment Sign-Up System

(SONA), while 203 participants were recruited from Amazon’s Mechanical Turk. Additionally,

294 individuals were recruited via the snowball sampling method.


Participants were offered different incentives based on their recruitment platform:

participants who were recruited via Cornell University’s Psychology Experiment Sign-Up

System (SONA) were awarded 1 SONA credit for their participation, while those who were

recruited from Amazon’s Mechanical Turk were given $0.50 for their participation. Individuals

who were recruited by the snowball sampling method (e.g., via email and social media outlets,

such as Facebook) were not compensated, but instead thanked for their participation. Of the

entire sample, 38.3% participants identified as male, 61.3% as female, and .4% did not identify

with either gender. Participant ages ranged from 18 to 78 (M = 29.76, SD = 13.61).

Design

A cross-sectional, correlational design was utilized that compared individuals’

performances on a battery of critical thinking and decision-making tasks. Specifically,

individuals were grouped according to self-report of his/her current professional field.

Regardless of professional field, all participants were administered an identical battery of

cognitive tasks. Participant scores on all cognitive measures were then used in subsequent

regression analyses.

Measures

Professional field. All participants completed an initial questionnaire regarding intended

professional track (e.g., “pre-med”, “pre-law”) or current professional field. If the participant

indicated that he/she was enrolled as a student in either medical school or law school, he/she was

asked if he/she intended to specialize in a specific sub-field of law or medicine, respectively.

Participants indicating that they were graduates of medical school or law school were asked to

indicate specialty (if any) and number of years since receiving his/her professional degree (See

Appendix A).


Cognitive Reflection Test (CRT). The Cognitive Reflection Test (Frederick, 2005) is a

short questionnaire that measures cognitive reflection—the measurement used to determine how

capable one is to appreciate or recognize the effects of delayed gratification. For example, an

individual scoring high on cognitive reflective abilities would be able to appreciate the benefits

of interest rates offered by the market over time. Additionally, CRT scores can be used to further

predict the degree to which one relies on mental heuristics in decision-making processes and

his/her time preferences in cognitive processes. Liberali, Reyna, Furlan, Stein, and Pardo (2012)

report a moderate to high reliability (α = 0.64 – 0.74) (See Appendix B).

Abridged numeracy scale (5-item). The abridged numeracy scale used is part of a

longer, 11-item measure developed by Lipkus, Samsa, and Rimer (2001). In this task,

participants were prompted to respond to questions such as “Imagine that we rolled a fair, six-

sided die 1,000 times. Out of 1,000 rolls, how many times do you think the die would come up

even (2, 4, or 6)?”. These questions were used to gauge participants’ mathematical and statistical

competencies (See Appendix C).

The Linda problem. Participants’ adeptness to make a rational probability judgment was

tested with the Linda Problem (Tversky & Kahneman, 1983). In the Linda problem, a fictional

character, Linda, is portrayed as a socially liberal woman invested in “issues of discrimination

and social justice”. Following this description, participants are prompted to chose whether Linda

would be more likely to be solely a “bank teller” or “a bank teller and active in the feminist

movement”. When answering the Linda problem, many participants may find themselves relying

upon irrelevant information (the statements portraying Linda through a liberal lens) and make a

conjunction fallacy—a misjudgment in logical cognition that occurs when two simultaneous

conditions are deemed more likely to occur than either one independently. Previous work


suggests that an individual’s performance on the Linda problem may be related to reliance on

possibly fallacious heuristic based decision making strategies (Tversky & Kahneman, 1983) (See

Appendix D).

Wason selection task. The Wason Selection Task (Wason, 1968) measures participants’

capabilities to deductively reason and subsequently make a decision after being given a set of

rules. Participants were asked to answer an “if–then” logic puzzle using four white-faced cards

labeled with either black-type letters or numbers (“A”, “K”, “4”, and “7”, respectively).

Specifically, participants were given a rule-based statement (“If a card has an even number on

one side, then it has a vowel on the other”) and subsequently asked to select a card dyad that

fulfilled the conditions set in the aforementioned rule (See Appendix E).

Beads task. The tendency for an individual to jump to conclusions was tested by a

common probabilistic reasoning measure, the Beads Task (Averbeck, Evans, Chouhan, Bristow,

& Shergill, 2011; Garety et al., 1991; Garety & Freeman, 1999; Huq, Garety, & Hemsley, 1988).

Jumping to conclusions, as defined by Garety and colleagues (1991), provides information

regarding the tendency of an individual to “request [a small amount of] information to reach a

decision” (p. 570). Although previous research has used the Beads Task to assess “reasoning

bias [encountered] across the continuum of delusional ideation” (p. 1255) in delusional and

delusion-prone individuals (Warman, Lysaker, Martin, Davis, & Haudenschield, 2006), new

research has used the Beads Task with healthy, neurotypical individuals (Evans et al., 2012).

In the present study, two versions (“easy” vs. “difficult”) of the Beads Task were

administered to participants. In the “easy” version of the Beads Task, participants were presented

with two jars (labeled “A” and “B”, respectively). In the “easy” version, Jar “A” was comprised

of 15% blue beads and 85% red beads; conversely, Jar “B” was filled with beads that were 15%


red and 85% blue. The participants were instructed that “The computer will now select one of the

jars and draw random beads from it”. Given this information, participants were instructed to

guess which jar the given bead was drawn from. The same set of instructions was given during

the “difficult” Bead Task. The sole difference between the “easy” and “difficult” Bead Task

versions was that the “difficult” Bead Task displayed jars with 60:40 bead color ratios to

participants as opposed to 85:15 bead color ratios. In both conditions, participants were not

limited to the number of beads they were able to draw (See Appendix F).

Demographic questionnaire. Lastly, participants responded to a number of demographic

questions, including questions regarding age; gender; race; highest level of education completed;

number of academic courses in science, technology, engineering, and math (STEM) fields; and

household income. If the participant was financially dependent on a caretaker/guardian, he/she

was asked to report the income of his/her caretaker/guardian.

Procedures

All participants completed the present study online using the Qualtrics platform.

Participants accessed the survey link on Cornell’s SONA system, Amazon’s Mechanical Turk, or

by clicking on the survey link received via email or a social networking website.

After clicking on the survey link, participants gave informed consent for their

participation. Next, participants indicated his/her intended or current professional track/field.

Then, participants completed a battery of twelve cognitive and decision-making tasks based on

random assignment. More specifically, participants answered three CRT questions, five

Abridged Numeracy Scale questions, the Linda problem, the Wason Selection Task problem, and

completed two trials of the Beads Task. Lastly, participants completed several demographic

questions.


Results

What follows are a series of analyses that examined both inter-professional differences in

cognitive performance (e.g., comparing physicians with lawyers), and intra-professional

differences in cognitive performance that focus on the effects of training within a profession

(e.g., comparing law school students with full-fledged law school graduates).

CRT

Between-group comparisons. In order to test the hypothesis that there would be

differences in critical-thinking abilities between participants from different professions, mean

composite scores on the CRT were compared across education/occupation groups. Results from

a one-way ANOVA revealed a significant effect of one’s education level/occupation on mean

CRT scores, F (7, 718) = 6.25, p < .001, ηp2 = .057. Pair-wise comparisons using the Bonferroni

correction revealed several group differences: “pre-medical” undergraduate students had

significantly lower average CRT scores than medical school students (p = .024), law school

students (p = .003), lawyers (p = .001), and those with careers as engineers (p = .005). On

average, participants whose careers were classified as “Other” had significantly lower CRT

scores compared to law school students (p = .01), lawyers, (p = .003), and engineers, (p = .017).

No other comparisons revealed significant differences (See Table 1).

Within-group comparisons.

Legal education. In order to explore whether differences in critical-thinking abilities

would emerge among participants with legal education, mean composite scores on the CRT were

compared among “pre-law” students, current law school students, and law school graduates to

examine whether attending law school influenced CRT performance or if those who became

lawyers were self-selecting into the profession. Results from a one-way ANOVA revealed a


significant effect of education on mean CRT score, F (2, 158) = 4.00, p = .02, ηp2 = .048. Pair-

wise comparisons using the Bonferroni correction revealed that “pre-law” undergraduate

students had significantly lower CRT scores compared to law school graduates, (p = .031). No

other comparisons revealed significant differences (see Table 2).

Table 1

Means and Standard Deviations for CRT Score—Between Groups Comparison

CRT score

Educational Track n M SD

Pre-Med 116 1.51 a*,b**, c**, d** 1.18

Medical School 43 2.16 a* .97

M.D. 53 1.79 .91

Pre-Law 54 1.74 1.09

Law School 60 2.18b**, e* .98

J.D. 45 2.30c** .90

Other (non-engineers) 282 1.63e*, f* 1.16

Other (engineers) 73 2.12d**, f* 1.07

Total 726 1.80 1.12

Notes. * p <.05, ** p <.01

Medical education. In order to explore whether differences in critical-thinking abilities

exist among participants with varying degrees of medical education, mean composite scores on

the CRT were compared among “pre-med” students, current medical school students, and

medical school graduates. Results from a one-way ANOVA revealed a significant effect of

education on mean CRT score, F (2, 211) = 6.34, p = .002, ηp2 = .057. Pair-wise comparisons


using the Bonferroni correction revealed that “pre-med” undergraduate students had significantly

lower CRT scores compared to current medical school students, (p = .002). No other

comparisons revealed significant differences (see Table 3).

Table 2

Means and Standard Deviations for CRT Scores—Within Legal Education Comparison

CRT score


Pre-Law 54 1.74 1.09

Law School 60 2.18a* .98

J.D. 45 2.30a* .90

Notes. * p <.05

Table 3

Means and Standard Deviations for CRT Scores—Within Medical Education Comparison

CRT score


Pre-Med 116 1.51 a** 1.18

Medical School 43 2.16 a** .97

M.D. 53 1.79 .91

Notes. ** p <.01

Abridged Numeracy Scale

Between-group comparisons. In order to explore whether numeracy skills—“the

understanding of basic probability and mathematical concepts” (Lipkus, Samsa, & Rimer, 2001,


pp. 37)—differed among participants, a one-way ANOVA was used to compare mean composite

scores on the Abridged Numeracy Scale. This analysis revealed a significant effect of one’s

education on mean numeracy scores, F (7, 712) = 3.12, p = .003, ηp2 = .03. Pair-wise

comparisons using the Bonferroni correction revealed that medical school graduates had

significantly higher numeracy scores than “pre-med” undergraduate students, (p = .021). No

other comparisons revealed significant differences amongst all other groups, (see Table 4).

Table 4

Means and Standard Deviations for Abridged Numeracy Scale Scores—Between Groups Comparison

Abridged Numeracy Scale score


Pre-Med 115 3.72a* 1.51

Medical School 41 4.51a* .93

M.D. 50 4.34 .96

Pre-Law 52 3.75 1.61

Law School 62 4.27 1.16

J.D. 50 4.16 1.48

Other (non-engineers) 278 3.93 1.21

Other (engineers) 71 4.06 1.30

Total 720 4.00 1.29

Notes. * p <.05


Legal education. In order to discover whether differences in numeracy among “pre-law”

students would emerge, current law school students, and law school graduates, mean Abridged


Numeracy Scale scores were compared within groups. Results from a one-way ANOVA

revealed no significant differences amongst these groups, with means ranging only between 3.76

and 4.27, F (2, 162) = 1.99, p = .139, ηp2 = .024, (see Table 5).

Table 5

Means and Standard Deviations for Abridged Numeracy Scale Scores —Within Legal Education Comparison



Pre-Law 52 3.75 1.61

Law School 62 4.27 1.16

J.D. 50 4.16 1.48

Notes. Pairwise comparisons revealed no significant differences among groups.

Medical education. Investigating whether there would be differences in numeracy among

“pre-med” students, current medical school students, and medical school graduates, mean

abridged numeracy scale scores were compared within these groups. Results from a one-way

ANOVA revealed a significant effect of education on mean numeracy scores, F (2, 204) = 7.23,

p = .001, ηp2 = .066. Subsequent pair-wise comparisons using the Bonferroni correction revealed

that “pre-med” undergraduate students had significantly lower numeracy scores compared to

current medical school students, (p = .004). Additionally, analyses indicated “pre-med”

undergraduate students also had significantly lower numeracy scores compared to medical

school graduates, (p = .019). No other comparisons revealed significant differences, (see Table

6).

The Linda Problem

Between-group comparisons.


Legal education. A binary logistic regression was utilized to explore the effects of one’s

formal education/career path on performance on the Linda problem. First, law school graduates

were used as the comparison group. The omnibus test was significant, (p < .001). Analyses

revealed that lawyers were significantly more accurate when answering the Linda Problem

compared to any other group. Specifically, lawyers boasted an accuracy rate 2.21 times that of

“pre-med” undergraduate students (p < .001), 1.56 times that of medical school students (p =

.008), 1.63 times that of physicians (p = .004), 2.34 times that of “pre-law” undergraduate

students (p < .001), 1.53 times that of law students (p = .007), 1.74 times that of individuals

whose careers were categorized as “Other” (p = .001), and 2.68 times that of engineers, (p <

.001). No other comparisons revealed significant differences, ps > .05 (see table 7).

Table 6

Means and Standard Deviations for Abridged Numeracy Scale Scores —Within Medical Education Comparison



Pre-Med 115 3.72a**, b* 1.51

Medical School 41 4.51a** .93

M.D. 50 4.34 b* .96

Notes.* p <.05, ** p <.01

Medical education. A binary logistic regression was utilized to explore the effects of

one’s formal education/career path on performance on the Linda problem. Here, medical school

graduates were used as the comparison group. The omnibus test of the model was significant, (p

< .001). Analyses revealed that lawyers were significantly more accurate when answering the

Linda Problem than physicians were: specifically, lawyers were 1.56 times more likely to


correctly answer the Linda Problem correctly than physicians, (p = .004). Conversely, physicians

were 1.64 times more likely to answer the Linda Problem correctly than individuals whose

occupations were categorized as “Other”, (p = .007). No other comparisons revealed significant

differences, ps > .05 (see table 8).

Table 7

Linda Problem Between Groups with J.D.s as Reference Group

Educational Track b SE Wald Accuracy Rate Pre-Med -1.72 0.37 21.13 0.34***

Medical School -1.16 0.44 7.06 0.48** M.D. -1.22 0.42 8.39 0.46**

Pre-Law -1.82 0.44 17.51 0.32*** Law School -1.10 0.41 7.31 0.49**

J.D. --- --- 45.74 0.75a Other (non-engineers) -1.35 0.4 11.42 0.43**

Other (engineers) -2.04 0.35 34.46 0.28*** Note. * p <.05, ** p <.01, *** p <.00

Within-group comparisons. In order to examine the effects of formal education/career

path on cognitive performance, a series of logistic regressions were run to contrast performance

by those still engaged in formal training (law and med students) with those who already

completed such training (full-fledged physicians and attorneys). Such contrasts inform the

question of primacy or causality—does training improve problem-solving, or are there

preexisting differences that are not further amplified by training?

Legal education. A binary logistic regression was utilized to explore the effects of legal

education on performance on the Linda problem. Here, law school graduates were used as the

comparison group. The omnibus test of the model was significant, (p < .001). Specifically,

analyses revealed that lawyers were 2.34 times more accurate when answering the Linda

Problem than “pre-law” undergraduate students, (p < .001). Additionally, law school graduates


were 1.53 times more accurate than law school students, (p = .01). Both of these last two

findings suggest that training within a field does impart cognitive advantages over and above any

self-selection mechanism. No other comparisons revealed significant differences, ps > .05 (see

table 9).

Table 8

Linda Problem Between Groups with M.D.s as Reference Group

Educational Track b SE Wald Accuracy Rate Pre-Med -0.50 0.33 2.22 0.34

Medical School 0.06 0.40 0.02 0.48 M.D. --- --- 45.76 0.46a

Pre-Law -0.60 0.40 2.25 0.32 Law School 0.12 0.37 0.10 0.49

J.D. 1.22 0.42 8.39 0.75** Other (non-engineers) -0.82 0.30 7.27 0.43**

Other (engineers) -0.12 0.36 0.12 0.28 Note. * p <.05, ** p <.01, *** p <.001

Medical education. A binary logistic regression was utilized to determine the effects of

medical education on performance on the Linda Problem. Here, medical school graduates were

used as the comparison group. The omnibus test of the model was not significant, (p = .185),

suggesting that the experience of medical training did not influence reasoning accuracy on this

task. Current medical school students were no more accurate when answering the Linda problem

than full-fledged physicians, (p = .959), and although physicians were 1.35 times more accurate

than “pre-medical” students, this advantage was not statistically reliable (p = .136). No other

comparisons revealed significant differences, ps > .05 (see table 10).


Table 9

Linda Problem Within Groups with J.D.s as Reference Group

Educational Track

b SE Wald Accuracy Rate

Pre-Law -1.803 0.428 17.773 0.32*** Law School -1.03 0.402 6.583 0.49*

J.D. --- --- 17.814 0.75a Note. * p <.05, ** p <.01, *** p <.001

Table 10

Linda Problem Within Groups with M.D.s as Reference Group

Educational Track


Pre-Med -0.50 0.33 2.22 0.34*** Medical School 0.02 0.40 0.003 0.48

M.D. --- --- 3.36 0.46a Note. * p <.05, ** p <.01, *** p <.001

The Wason Cards Task

Between-group comparisons.

Legal education. A binary logistic regression was utilized to determine the effects of

one’s formal education/career path on performance on the Wason Cards Task. First, law school

graduates were used as the comparison group. The omnibus test of the model was significant, (p

= .002). Analyses revealed that lawyers were approximately 4 times more accurate than

physicians, (p = .021). No other comparisons revealed significant differences, ps > .05 (see Table

11).


one’s formal education/career path on performance on the Wason Cards Task. Here, physicians


were used as the comparison group. The omnibus test of the model was significant, (p = .002).

Analyses revealed that medical school students were 3.8 times more accurate than physicians, (p

= .031). Additionally, pre-law” undergraduate students were 3.4 times more accurate than

physicians, (p = .045), law students were 5.8 times more accurate than physicians, (p = .001),

and lawyers were 4 times more accurate than physicians (p = .021). Engineers were also 4 times

more accurate than physicians, (p = .016). No other comparisons revealed significant differences,

ps > .05 (see Table 12).


Legal education. A binary logistic regression was utilized to determine the effects of

legal education level on performance on the Wason Cards problem. Here, law school graduates

were used as the comparison group. The ombinus test of the model was not significant, (p =

.224). There was no reliable trend for current law school students to outperform law school

graduates on the Wason Cards task, (p = .959). Similarly, there was no trend for “pre-law”

undergraduate students to perform worse than law school graduates, (p = .851). No other

comparisons revealed significant differences, ps > .05 (see Table 13).

Table 11

Wason Cards Task Between Groups with J.D.s as Reference Group

Educational Track b SE Wald Accuracy Rate Pre-Med -0.60 0.42 2.06 0.12

Medical School -0.08 0.48 0.03 0.19 M.D. -1.56 0.67 5.32 0.05*


J.D. --- --- 5.32 0.20a Other (non-engineers) 0.02 0.43 2.06 0.20

Other (engineers) -0.62 0.37 0.03 0.12 Note. * p <.05, ** p <.01, *** p <.001


Table 12

Wason Cards Task Between Groups with M.D.s as Reference Group

Educational Track b SE Wald Accuracy Rate Pre-Med 0.96 0.65 2.17 0.12

Medical School 1.48 0.69 4.63 0.19* M.D. --- --- 21.99 0.05a

Pre-Law 1.37 0.69 4.00 0.17* Law School 2.08 0.64 10.45 0.29*

J.D. 1.56 0.67 5.32 0.20* Other (non-engineers) 1.58 0.66 5.79 0.20*

Other (engineers) 0.94 0.62 2.30 0.12 Note. * p <.05, ** p <.01, *** p <.001; a = reference point

Table 13

Wason Cards Task Within Groups with J.D.s as Reference Group

Educational Track



J.D. --- --- 2.99 0.20a Note. * p <.05, ** p <.01, *** p <.001


medical education on performance on the Wason Cards problem. Here, physicians were used as

the comparison group. The omnibus test of the model was not significant, (p = .069). Trends

revealed that “pre-med” undergraduate students had accuracy rates that were 2.4 times higher

than the accuracy rate of medical school graduates, although this superiority failed to reach

significance (p = .141). In contrast, medical school students were 3.8 times more accurate than

physicians when answering the Wason Cards Task, (p = .034), suggesting that training itself did

not impart an advantage, assuming that the current cohort of medical school students are not


being exposed to dramatically different types of cognitive training than the cohort of physicians.

No other comparisons revealed significant differences, ps > .05 (see Table 14).

Table 14

Wason Cards Task Within Groups with M.D.s as Reference Group

Educational Track


Pre-Med 0.96 0.65 2.17 0.12 Medical School 1.46 0.69 4.50 0.19*

M.D. --- --- 4.59 0.05a Note. * p <.05, ** p <.01, *** p <.001

Beads Task

Easy Version


differences in the amount of information individuals of different educational/occupational

backgrounds prefer when making decisions, the mean number of drawn beads during the easy

version of the beads task was compared utilizing a one-way ANOVA. Analyses revealed a

significant effect of one’s education on the number of beads drawn, F (7, 629) = 2.11, p = .04,

ηp2 = .023. Despite the model’s overall significance, no reliable differences in the number of

beads pulled were found between any of the compared groups (see Table 15).


Legal education. Exploring whether there would be differences in the amount of

information individuals of varying legal educational backgrounds prefer when making decisions,

the mean number of drawn beads during the easy version of the beads task was compared

utilizing a one-way ANOVA. Analyses revealed no significant effect of one’s education on the

number of beads drawn although it appeared to be trending toward significance, F (2, 122) =


2.57, p = .08, ηp2 = .04, (see Table 16).

Medical education. In order to discover if there would be differences in the amount of

information individuals of varying medical educational backgrounds prefer when making

decisions, the mean number of drawn beads during the easy version of the beads task was

compared utilizing a one-way ANOVA. Analyses revealed no significant effect of one’s

education on the number of beads drawn, F (2, 122) = .052, p = .95, ηp2 = .001(see Table 17).

Table 15

Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task

Number beads drawn


Pre-Med 115 3.91 3.33

Medical School 28 4.04 3.68

M.D. 48 3.83 2.56

Pre-Law 49 3.57 3.23

Law School 31 5.03 5.87

J.D. 42 6.00 6.15

Other (non-engineers) 262 4.43 3.90


Total 637 4.41 4.03

Note. Pairwise comparisons revealed no significant differences among groups.


Table 16

Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task Within Legal Education

Number beads drawn


Pre-Law 49 3.57 3.23

Law School 31 5.03 5.87

J.D. 42 6.00 6.15


Table 17

Means and Standard Deviations for Number of Beads Drawn in Easy Beads Task Within Medical Education

Number beads drawn


Pre-Med 115 3.91 3.33


M.D. 48 3.83 2.56


Difficult Version


differences in the amount of information individuals of different educational/occupational

backgrounds prefer when making decisions, the mean number of drawn beads during the difficult

version of the beads task was compared utilizing a one-way ANOVA. Analyses revealed a

significant effect of one’s education on the number of beads drawn, F (7, 696) = 2.89, p = .005,

ηp2 = .028. Specifically, pair-wise comparisons using the Bonferroni correction revealed that law


school graduates drew significantly more beads than both “premed” undergraduate students, (p =

.011), and those whose occupations were classified as “Other”, (p = .046). No other comparisons

revealed significant differences, (see Table 18).


Legal education. In order to explore if there are differences in the amount of information

individuals with varying legal educational backgrounds prefer when making decisions, the mean

number of drawn beads during the difficult version of the beads task was compared utilizing a

one-way ANOVA. Analyses revealed no significant effect of one’s education on the number of

beads drawn, F (2, 154) = 2. 15, p = .12, ηp2 = .027 (see Table 19).

Table 18

Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task—Between

Between Groups

Number beads drawn


Pre-Med 113 6.67a* 5.57


M.D. 50 8.48 7.45

Pre-Law 50 7.22 6.06

Law School 59 8.37 a*, b* 6.95

J.D. 47 10.43 8.58

Other (non-engineers) 275 7.39b* 5.48


Total 704 7.87 6.14

Note. * p <.05


Medical education. In order to explore if there are differences in the amount of

information individuals with varying medical educational backgrounds prefer when making

decisions, the mean number of drawn beads during the difficult version of the beads task was

compared utilizing a one-way ANOVA. Once again, analyses revealed no significant effect of

one’s education on the number of beads drawn, F (2, 202) = 2.00, p = .138, ηp2 = .019 (see Table

20).

Table 19

Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task Within Legal Education

Number beads drawn


Pre-Law 50 7.22 6.06

Law School 59 8.37 6.95

J.D. 47 10.43 8.58


Table 20

Means and Standard Deviations for Number of Beads Drawn in Difficult Beads Task Within Medical Education

Number beads drawn


Pre-Med 113 6.67 5.57


M.D. 50 8.48 7.45



Discussion

This study examined the effect of one’s formal education on his/her critical thinking

abilities and decision-making preferences. More specifically, the present study explored both

inter-professional and intra-professional differences in cognitive performance. Rather

surprisingly, the results of the present study found that highly educated individuals who

completed either medical or law school did not have higher accuracy rates than individuals from

the general population on critical thinking measures. However, it should be noted that

individuals who have obtained a law degree had, on average, better accuracy on critical thinking

measures than medical school graduates. Finally, results indicated no differences in decision-

making preferences among medical and law school graduates. Exploratory analyses not only

suggested that individuals drawn from the general population may have similar critical thinking

profiles than medical school graduates, but also that they appear to have similar decision-making

preferences to both law school graduates and medical school graduates.

In conclusion, the hypothesis that medical and law school graduates would have better

critical thinking abilities than individuals from the general population was only partially

supported. The results offered more substantial support for the hypothesis that law school

graduates would have more adept critical thinking abilities than their medical school graduate

counterparts. Contrary to previous assumptions, the hypothesis that there would be differences in

decision-making preferences between law school graduates and medical school graduates was

not supported by the results.

Inter-Professional Critical Thinking Differences

Analyses revealed that medical school graduates outperformed members from the general

population on the Linda Problem. Additionally, law school graduates outperformed individuals


whose occupations were classified as “Other” on the CRT and the Linda Problem. Relatedly,

Fredrick (2005) reported that highly educated individuals studying at extremely rigorous

undergraduate universities had higher average CRT scores than individuals from the general

population. Additionally, individuals whose occupations were classified as “Other” had accuracy

rates that were indistinguishable from those of medical school graduates on the CRT. Lastly, it

was found that both medical and law school graduates performed tantamount to members of the

general population on the Wason Cards Task. In sum, the hypothesis that both medical and law

school graduates would perform better on measures of critical thinking than members of the

general population was only partially supported.

Additionally, analyses revealed trends indicating that law school graduates had higher

average scores than medical school graduates on all critical thinking tasks (see Figures 1 – 3).

However, the difference in physicians’ and lawyers’ mean composite CRT scores failed to reach

statistical significance. Taken together, the above results may call attention to core differences in

cognitive profiles among members of the legal and medical professions. The results may also

indicate that the curriculum implemented in medical education may not necessarily accentuate

the importance of critical thinking to the extent that legal education does. In conclusion, the

hypothesis that law school graduates would achieve higher accuracy rates on critical thinking

tasks compared to medical school graduates was offered partial support.

Intra-Professional Critical Thinking Differences

Exploratory analyses revealed notable trends concerning critical thinking trajectories

throughout medical and legal disciplines. Not surprisingly, results revealed that “pre-medical”

undergraduate students had significantly lower accuracy rates on the CRT than medical school

students did; however, somewhat shockingly, medical school graduates had statistically

indistinguishable accuracy rates with “pre-medical” undergraduates on the CRT. In short, critical


thinking gains made by medical school curriculum were found to be diminished in medical

school graduates. Furthermore, exploratory analyses also revealed that medical school graduates

had similar rates of accuracy to both “pre-medical” undergraduate students and current medical

school students when answering the Linda Problem. Possibly most surprisingly, it was found that

“pre-medical” undergraduate students performed slightly better than physicians on the Wason

Cards Task; similarly, current medical school students significantly outperformed the medical

school graduates on this task. Therefore, these results expose a noticeable degeneration in

physicians’ critical thinking skills after they complete their extensive medical education.

Alternatively, these results may indicate that the sample of full-fledged physicians was not as

cognitively elite as the students. Future research will be needed to determine which of these

explanations are true (e.g., controlling MCAT scores in analyses).

Analyses revealed that, unlike their medically-oriented peers, lawyers continue to reap

critical thinking gains after completing their formal legal education: Law school graduates

boasted higher accuracy rates than “pre-law” undergraduates on the CRT. Further, despite failing

to reach significance, law school graduates also had slightly higher accuracy rates than current

law school students on the CRT. Additionally, exploratory analyses revealed that lawyers

outperformed both “pre-law” undergraduates and current law students on the Linda Problem.

This trend offers support to the suggestion that law school graduates may continue to accrue

certain critical thinking skills throughout their professional career. The only instance where law

school graduates’ accuracy rates waned in compared to current law school students was on the

Wason Cards Task—it was found that law school students outperformed both “pre-law”

undergraduates and law school graduates on this task. It is possible that this particular cohort of


law students was advantaged by current changes to the curriculum that influenced reasoning on

the CRT, but this is sheer surmise and will require follow-up research to determine.

Inter-Professional Decision-Making Differences

It was hypothesized that law school graduates would prefer to make decisions based on

more evidence and certainty than medical school graduates. However, this hypothesis was not

supported: no significant differences in decision-making preferences were found between that

law or medical school graduates. However, analyses revealed trends indicating that, on average,

lawyers drew more beads than physicians on both the “Easy” and “Difficult” Beads Tasks (see

Figure 4). Although failing to reach significance, this trend suggests there might be differences in

nuanced decision-making preferences between medical and legal professionals. More

specifically, this trend suggests that lawyers may have a higher intrinsic need to make more well-

informed decisions compared to medical school graduates.

Intra-Professional Decision-Making Differences

Exploratory analyses revealed trends suggesting that one’s formal education may produce

slightly different decision-making preferences among between members of different fields. For

example, results from the “Easy” Beads Task reveal that both “pre-medical” and “pre-law”

undergraduate students rely on similar amounts of information before feeling comfortable to

make a well-informed decision. However, trends reveal that, unlike individuals pursuing a

medical career, individuals interested in law demonstrate an increased preference for additional

information before making a decision. In fact, analyses revealed that medical school students and

medical professionals failed to show any preference for increasing the amount of information

needed to make a decision from the time they were “pre-medical” undergraduate students. A

similar pattern emerged for the “Difficult” Beads Task: Analyses revealed a trend suggesting


legal education may place emphasis on gathering substantive amounts of concrete evidence

before making a decision (see Figure 5). However, analyses revealed a less influential effect of

one’s medical education on such decision-making preferences.

Inter-Professional Numeracy Differences

Results did not support the hypothesis that numeracy differences exist between medical

school and law school graduates. Despite failing to reach significance, trends revealed that both

graduate medical and legal curricula might lead to a slight increases in numeracy skills.

Similarly, there was a non-significant trend indicating a very slight decline in numeracy for both

individuals in medical and legal fields after completion of graduate schooling (see Figure 6).

Intra-Professional Numeracy Differences

Exploratory analyses revealed that one’s formal education did not impact numeracy for

individuals interested in pursuing medical or legal careers. Specifically, it was found that

medical school students and graduates both had significantly higher numeracy scores than “pre-

medical” undergraduate students. Based on these results, it is presumed that graduate medical

education may have an effect on numeracy ability. Therefore, results suggest that the medical

curriculum may emphasize numeracy more so than traditional legal education does. This makes

sense in that law school does not entail quantitative reasoning coursework where some medical

training does entail further exposure to physical science.

Implications

Broadly speaking, the results of present study suggest that the medical education fails to

instill and maintain critical thinking abilities, careful decision-making preferences, and numeracy

abilities. Traditionally, the Medical College Admission Test (MCAT) was composed of four

sections—verbal reasoning, writing, biological sciences, and physical sciences. However,


medical school curriculum developers, practicing physicians, and other healthcare practitioners

noted that the MCAT was testing progressively outdated concepts—the test needed to be

revamped as the categories tested were seen as less representative of the multifaceted and

complex topics confronting modern day physicians (Kaplan, Satterfield, & Kington, 2012).

Therefore, in the spring of 2015, the Association of American Medical Colleges (AAMC)

decided to implement two new sections—“Psychological, Social, and Biological foundations of

Behavior” and “Critical Analyses and Reading”—to the MCAT exam. According to Kaplan,

Satterfield, and Kington (2012), the new MCAT exam would “build a better physician” (pp.

1265). This substantive shift in the MCAT offers support to the notion that up until the current

year, the medical education system has not equipped contemporary physicians with the

educational tools needed to fully address the issues in modern progressive society. Recognizing

this need for reform in medical school curricula, the current study sheds light on the flaws of the

current medical education training.

Critical Thinking. As previously mentioned, physicians’ lackluster performance on all

critical thinking tasks prompts the following question—why do physicians fail to critically think

at the level of other highly-educated professionals, such as lawyers? Despite the fact that future

research must be conducted to address this question in greater depth, there are serious and

unnerving implications associated with this trend. Inadequate critical thinking skills may allow

physicians to fall prey to irrational and flawed cognition (Croskerry, 2003; Elstein & Schwarz,

2002). For example, physicians who fail to critically think may misinterpret patient symptoms,

laboratory results, or other patient-related information to make faulty diagnoses or prescriptions.

Such flawed cognition and misinterpretation could lead to negative health outcomes for

individuals coming in contact with modern day physicians (Croskerry, 2003; Elstein & Schwarz,


2002). Additionally, trends indicating a decline in critical thinking skills from medical school

students to post-graduate physicians should be of concern to medical professionals and educators

alike.

In order to address this decline in critical thinking skills, educators, physicians, and other

curriculum developers should consider requiring medical professionals to complete regular

“cognitive upkeep” training after they have received their graduate medical degrees (Crosskery,

2013; Weinberger, Pereira, Iobst, Mechaber, & Bronze, 2010). By doing so, the critical thinking

abilities could be equated with the critical thinking cognitive trajectories of law school graduates.

Additionally, the quality of medical education as a whole should be examined. Weinberger and

colleagues (2010) propose that modern-day medical education fails to train medical school

students and physicians to integrate and apply information outside of the explicit knowledge that

is learned from the textbook. Instead of critically thinking and adapting to the evolving needs and

problems presented by the current healthcare system, medical school students and physicians are

hindered by their reliance on specific quanta of facts (Weinberger et al., 2010). Additionally, the

rigor of the medical education system should be examined to help explain the trend suggesting a

decline in physicians’ critical thinking skills post-medical school—provided that this is not an

artifact of cohort differences rather than reflecting genuine post-medical school functioning. It

has been widely documented that the medical education system requires long working hours,

countless assessments, and intrinsic drive (Chang, Eddins-Folensbee, & Coverdale, 2012;

Mandal et al., 2012; Rosen, Gimotty, Shea, & Bellini, 2006). Some assert that such rigorous

standards and non-optimal working environments for medical professionals may be linked to a

“burnout”—a condition leading to a decrease in both cognitive performance and patient care

among those pursuing a medical profession (Montgomery, Todorova, Baban, & Panagopoulou,


2013; Shanafelt, Bradley, Wipf, & Back, 2002). In short, many factors associated with the

medical education system may be to blame for the decline in critical thinking skills in medical

professionals after graduating from medical school, assuming such decline is real and not a

cohort effect.

Decision-Making Preferences. As previously discussed, distinctive trends in decision-

making preferences were apparent among medical school and law school graduates. Specifically,

the results suggested that medical school graduates had a weaker preference to rely on large

amounts of evidence before making decisions compared to law school graduates. This trend may

have relevance to professionals in the medical field. As documented by many researchers,

individuals who rely upon little information when making decisions may be efficient in terms of

wasting little time pondering future actions. Although seemingly helpful, this speed benefit

comes at a potentially serious cost—these individuals may also be at risk for subconsciously

including extraneous information in his/her cognitive evaluation that may ultimately lead to

flawed decision-making (Gigerenzer & Gaissmaier, 2011; Jaeger, Webler, Rosa, & Renn, 2013;

Neth & Gigerenzer, 2015). Specifically, this trend has implications for medical professionals

because physicians are often required to make rapid decisions regarding a patient’s health. For

example, in her article “Missing a Cancer Diagnosis” (2014), columnist Susan Gubar recounted

her own physician misinterpreting multiple retrospectively blatant signs of ovarian cancer: Gubar

reports that physicians often attribute “[b]loating, satiety, fatigue and constipation” to

“menopause, aging, indigestion, irritable bowel syndrome, depression, laziness or whining.”

However, these symptoms are also associated with more severe illnesses such as ovarian cancer.

A report by the National Coalition on Health Care and Best Doctors, Inc. (2013) supports

Gubar’s anecdote: They estimate that upwards of 28% of oncologists reach faulty diagnoses due


to misinterpretation of patient symptoms. Together, Gubar’s experience and the report by the

National Coalition on Health Care and Best Doctors, Inc. reveal how modern day physicians may

rely on limited amounts of information before coming to faulty conclusions.

Furthermore, the ontogeny of these decision-making preferences should also be

examined. Block and colleagues (2013) report that modern day physicians are being forced to

spend an alarmingly small amount of time with patients compared to physicians of the past: The

average physician spends approximately eight minutes in total with each patient before making a

diagnosis or treatment plan—“[r]educed work hours in the setting of increasing complexity of

medical inpatients, growing volume of patient data, and increased supervision may limit the

amount of time interns spend with patients” (pp. 1042). Based upon the presented information,

future policy-based interventions should take aim at allowing physicians to spend more time with

each patient. By increasing patient-physician interaction times, physicians will be allowed to

holistically evaluate each patient and make a beneficial, carefully thought-out, and individualized

treatment plan (Dugdale, Epstein, & Pantilat, 1999). Dugdale and colleagues (1999) propose that

increased patient-physician interaction time may lead to “effective information gathering by

patients, more information provided by physicians, more conversation by patients relative to the

physician, and more expression of affect” (pp. 35). Therefore, increased patient-physician

interaction time may allow physicians’ to gather more information per patient. Ultimately, this

may lead to a shift in physician cognition that emphasizes the importance of drawing upon

adequate amounts of information when making important conclusions.

Numeracy. The results from the Abridged Numeracy Task also raise questions pertaining

to the efficacy of a rigorous STEM-heavy education for individuals interested in pursuing a

medical career. Intuitively, one may expect that an individual exposed to a STEM-intensive


curriculum would have higher numeracy skills compared to another individual without such

exposure; however, the results from the present study do not show significant differences in

numeracy among individuals pursuing less STEM-intensive career paths. To address this issue,

medical education pedagogy and curriculum development should be examined to identify

educational domains that can be tweaked so that numeracy ability may be increased. For

example, educators and curriculum-developers should aim to dismantle the current trend that

favors students relying heavily on problem-solving tools, such as calculators, to answer

numeracy-based problems (see Mead, 2014).

Limitations and Future Directions

The current study investigated the relationship between one’s educational track and

his/her critical thinking and decision making skills. In order for the present results to generalize

to a broader, more representative population, future research should aim to explore critical

thinking and decision-making differences among specific medical specialties (e.g.,

otolaryngologist, radiologists, emergency medicine physicians). By doing so, researchers can

make more accurate claims regarding the effect of more specific types of medical education on

cognitive patterns. In addition to exploring the effect of medical education by specialty on

medical school graduates’ cognitive profiles, future studies may aim to explore specific intervals

that may serve as “critical periods” for learning certain skills throughout one’s medical

education. For example, one may find that first year medical school students experience a

significant gain in critical-thinking skills. If this is true, educators may be able to determine what

salient feature of first year medical school curriculum is responsible for this increase in critical

thinking abilities; therefore, this salient component of medical school curriculum can be

implemented in subsequent years of schooling as well.


Additionally, future studies should aim to sample a more diverse array of individuals. In

the present study, snowball sampling was used to gather medical school students, law school

students, physicians, and lawyers. Therefore, the present study’s sample had relatively small

variance in academic, socioeconomic, and ethnic diversity. A homogenous participant cohort

may also help explain non-significant differences in some analyses. Increased diversity may

yield more distinctive differences in critical thinking and decision-making profiles may become

more evident. Scientific sampling is needed to distinguish between age/training effects and

cohort differences; as noted earlier, it could be that for some reason the full-fledged physicians in

the current study were trained differently or possessed lower cognitive ability than the med

school students, and it is one of these factors rather than medical training per se that explains

why they did poorer than students on some of the tasks. Furthermore, using different cognitive

tests may also help elicit clearer, more generalizable results. Many of the measures used in the

present study, such as the CRT and Abridged Numeracy Scale, were known to be internally

consistent but nevertheless comprised of only a few questions. That these measures were not

very sensitive in measuring the constructs they were meant to tap into may be due to their

uneven sampling of the cognitive domain. Additionally, the Abridged Numeracy Scale had

extremely high accuracy rates for all participant groups tested, thus restricting the range and

setting limits on the regression analyses. Future studies may aim to use more sensitive measures

to tap into numeracy ability among participant groups.

Finally, future studies may aim to utilize cognitive tasks that produce more generalizable

results for the decision-making tasks. Although the present study’s Beads Task may give

rudimentary insight into decision-making preferences among participants of different

occupations, the task can be modified so that professionals can make decisions with information


they are more familiar with dealing on a daily basis. For example, in future studies using

physicians in the participant sample, red and blue beads may be replaced with pieces of

medically relevant information needed to make a diagnosis. Similarly, for lawyers, the original

red and blue beads may be replaced with pieces of evidence needed to reach grounds for criminal

prosecution or acceptance of a plea offer.

Although physicians are perceived as among the most highly knowledgeable and

prestigious professionals, the present findings suggest that physicians, too, may at times fall prey

to cognitive reasoning errors. In addition to vast amounts of rudimentary medical knowledge,

modern-day and future physicians are now being held responsible for keeping up with the rapidly

changing medical system: They are asked to make and assess complex diagnoses, and make

quick, yet evidence-based treatment decisions in the patients’ best-interest. It is unsurprising that

physicians may have an occasional lapse in critical thinking or decision-making processes.

If physicians are to successfully deal with such complex cases, increased working

demands, and greater culpability, they must become increasingly skilled in more rigorous critical

thinking and decision-making skills. In order to adequately provide present and future physicians

with these fundamental cognitive skills, medical school curricula should consider placing a

greater emphasis on teaching the importance of critical thinking skills and appropriate decision-

making strategies.


REFERENCES

Adler, J.N., & Colbert, J.A. (2013). Medical use of marijuana – polling results. NEJM: The New

England Journal of Medicine, 368, 866 – 868. doi: 10.1056/NEJMclde1305159

Averbeck, B.B., Evans, S., Chouhan, V., Bristow, E., & Shergill, S.S. (2011). Probabilistic

learning and inference in schizophrenia. Schizophrenia Research, 127, 115 – 122. doi:

10.1016/j.schres.2010.08.009

Bennett, S., Maton, K. A., & Kervin, L. (2008). The ‘digital natives’ debate: a critical review of

the evidence. British Journal of Educational Technology, 39, 775 – 786. doi:

10.1111/j.1467-8535.2007.00793.x

Block, L., Habicht, R., Wu, A.W., Desai, S.V., Wang, K., Silva, K.N., Niessen, T., Oliver, N., &

Feldman, L. (2013). In the wake of the 2003 and 2011 duty hours regulations, how do

internal medicine interns spend their time? Journal of Internal Medicine, 28, 1042 –

1047. doi: 10.1007/s11606-013-2376-6

Blum-Kulka, S., & Weizman, E. (1988). The inevitability of misunderstandings: Discourse

ambiguities. Text, 8, 649 – 679. doi: 10.1515/text.1.1988.8.3.219

Bornstein, B.H., & Elmer, A.C. (2001). Rationality in medical decision making: a review of the

literature on doctors’ decision-making biases. Journal of Evaluation in Clinical Practice,

7, 97 – 107. doi: 10.1046/j.1365-2753.2001.00284.x

Burke, L.A., & Miller, M.K. (1999). Taking the mystery out of intuitive decision making. The

Academy of Management Executive, 13, 91 – 99. doi: 10.5465/AME.1999.2570557

Ceci, S.J., & Bruck, M. (1995). Jeopardy in the courtroom: A scientific analysis of children’s

testimony. Washington, D.C: American Psychological Association.


Chang, E., Eddins-Folensbee, F., & Coverdale, J. (2012). Survey of prevalence of burnout,

stress, depression, and the use of supports by medical students at one school. Academic

Psychiatry, 36, 177 – 182. doi: 10.1176/appi.ap.11040079

Crosskery, P. (2013). From mindless to mindful practice—cognitive bias and clinical decision

making. New England Journal of Medicine, 368, 2445 – 2448. doi:

10.1056/NEJMp1303712

Crosskery, P. (2002). The importance of cognitive errors in diagnosis and strategies to minimize

them. Academic Medicine, 78, 775 – 780. doi: 10.1097/00001888-200308000-00003

Drew, T., Võ, M.L.H., & Wolfe, J.M. (2013). The invisible gorilla strikes again: Sustained

inattentional blindness in expert observers. Psychological Science, 24, 1848 – 1853. doi:

0.1177/0956797613479386

Dugdale, D.C., Epstein, R., & Pantilat, S.Z. (1999). Time and the patient-physician relationship.

Journal of General Internal Medicine, 14, 34 – 40. doi: 10.1046/j.1525-

1497.1999.00263.x

Elstein, A. S., & Schwarz, A. (2002). Clinical problem solving and diagnostic decision making:

Selective review of the cognitive literature. British Medical Journal, 324, 729 – 732.

Evans, S., Almahdi, B., Sultan, P., Sohanpal, I., Brandner, B., Collier, T., Shergill, S.S., Cregg,

R., & Averbeck, B.B. (2012). Performance on a probabilistic inference task in healthy

subjects receiving ketamine compared with patients with schizophrenia. Journal of

Psychopharmacology, 26, 1211 – 1217. doi: 10.1177/0269881111435252

Fadiman, A. (1997). The spirit catches you and you fall down: A Hmong child, her American

doctors, and the collision of two cultures. New York, NY: Farrar, Straus & Giroux.


Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic

Perspectives, 19, 25 – 42. doi: 10.1257/089533005775196732

Ganzeboom, H.B.G., & De Graaf, P.M., & Treiman, D.J. (1992). A standard international socio-

economic index of occupational status. Social Science Research, 21, 1 – 56. doi:

10.1016/0049-089X(92)90017-B

Ganzeboom, H.B.G., & Treiman, D.J. (1996). Internationally comparable measures of

occupational status for the 1998 international standard classification of occupations.

Social Science Research, 25, 201 – 239. doi: 10.1006/ssre.1996.0010

Garety, P. A., & Freeman, D. (1999). Cognitive approaches to delusions: A critical review of

theories and evidence. British Journal of Clinical Psychology, 38, 113 – 154. doi:

10.1348/014466599162700

Garety, P.A., Hemsley, D.R., & Wessely, S.M.R.C. (1991). Reasoning in deluded schizophrenic

and paranoid patients: Biases in performance on a probabilistic inference task. The

Journal of Nervous and Mental Disease, 179, 194 – 201. doi:

http://dx.doi.org/10.1097/00005053-199104000- 00003

Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic decision making. Annual Review of

Psychology, 62, 451 – 482. doi: 10.1146/annurev-psych-120709-145346

Godlee, F., Smith, J., Marcovitch, H. (2011). Wakefield’s article linking MMR vaccine and

autism was fraudulent: Clear evidence of falsification of data should now close the door

on this damaging vaccine scare. BMJ: British Medical Journal, 342, 64 – 66.

Gubar, S. (2014, January 2). Missing a cancer diagnosis. The New York Times. Retrieved from

http://well.blogs.nytimes.com/2014/01/02/missing-a-cancer-diagnosis/


Haberman, C. (2015, February 1). A discredited vaccine study’s continuing impact on public

health. The New York Times. Retrieved from http://www.nytimes.com/2015/02/02/us/a-

discredited-vaccine-studys-continuing-impact-on-public-health.html?_r=0

Henrich, J., & Gil-White, F.J. (2001). The evolution of prestige: Freely conferred deference as a

mechanism for enhancing the benefits of cultural transmission. Evolution and Human

Behavior, 22, 165 – 196. doi: 10.1016/S1090-5138(00)00071-4

Hoffrage, U., & Gigerenzer, G. (1998). Using natural frequencies to improve diagnostic

inferences. Academic Medicine, 73, 538 – 540. doi:

http://dx.doi.org/10.3389/fpsyg.2015.00642

Huq, S.F., Garety, P.A., & Hemsley, D.R. (1988). Probabilistic judgments in deluded and non-

deluded subjects. The Quarterly Journal of Experimental Psychology, 40, 801 – 812. doi:

10.1080/14640748808402300

Jaeger, C.C., Webler, T., Rosa, E.A., & Renn, O. (2013) Risk, uncertainty, and rational action.

New York, NY: Earthscan.

Kahan, D.M., Jenkins-Smith, H., & Braman, D. (2011). Cultural cognition of scientific

consensus. Journal of Risk Research, 14, 147 – 174. doi: 10.1080/13669877.2010.511246

Kaplan, R.M., Satterfield, J.M., & Kington, R.S. (2012). Building a better physician—the case

for the new MCAT. New England Journal of Medicine, 366, 1265 – 1268. doi:

10.1056/NEJMp1113274

Mead, K. (2014). Exploring everyday math without using technology (Unpublished doctoral

dissertation). State University of New York (SUNY), Fredonia, New York.

Levitsky, D.A., & Pacanowski, C.R. (2013). Effect of skipping breakfast on subsequent energy

intake, Physiology & Behavior, 119, 9 – 16. doi: 10.1016/j.physbeh.2013.05.006


Liberali, J.M., Reyna, V.F., Furlan, S., Stein, L.M., & Pardo, S.T. (2012). Individual differences

in numeracy and cognitive reflection, with implications for biases and fallacies in

probability judgment. Journal of Behavioral Decision Making, 25, 361 – 381. doi:

10.1002/bdm.752.

Lipkus, I.M., Samsa, G., & Rimer, B.K. (2001). General performance on a numeracy sale among

highly educated samples. Medical Decision Making, 21, 37 – 44. doi:

10.1177/0272989X0102100105

Loftus, E.F., Levidow, B., & Duensing, S. (1992). Who remembers best? Individual differences

in memory for events that occurred in a science museum. Applied Cognitive Psychology,

6, 93 – 107. doi: 10.1002/acp.2350060202

Malmsheimer, R. (1988). Doctors only: the evolving image of the American physician.

Contributions in medical studies. New York, NY: Greenwood Press.

Mandal, A., Ghosh, A., Sengupta, G., Bera, T., Das, N., & Mukherjee, S. Factors affecting the

performance of undergraduate medical students: A perspective. Indian Journal of

Community Medicine: Official Publication of Indian Association of Preventive & Social

Medicine, 37, 126-129. doi: 10.4103/0970-0218.96104

Maulitz, R.C. (1988). The physician and authority: a historical appraisal. In King, N.M.P.,

Churchill, L.R., & Cross, A.W. (Eds.), The physician as captain of the ship (pp. 3 – 21).

Netherlands: Springer. doi: 10.1007/978-0-585-27589-5_1

Montgomery, A., Todorova, I., Baban, A., & Panagopoulou, E. (2013). Improving quality and

safety in the hospital: The link between organization culture, burnout, and quality of care.

British Journal of Health Psychology, 18, 656 – 662. doi: 10.1111/bjhp


Motiwalla, L.F. (2007). Mobile-learning: A framework and evaluation. Computers and

Education, 49, 581 – 596. doi: 10.1016/j.compedu. 2005.10.011

National Coalition on Health Care & Best Doctors, Inc. (2013). Exploring diagnostic accuracy in

cancer: A nationwide survey of 400 leading cancer specialists. Boston, MA.

Nakao, K., & Treas, J. (1992). The 1989 socioeconomic index of occupations: Construction from

the 1989 occupational prestige scores. Chicago: National Opinions Research Center.

Neth, H., & Gigerenzer, G. (2015). Heuristics: Tools for an uncertain world. In R. Scott & S.

Kosslyn (Eds.), Emerging trends in the social and behavioral sciences: An

interdisciplinary, searchable, and linkable resource (pp. 1 – 18). New York, NY: Wiley

Online Library. doi: 10.1002/9781118900772.etrds0394

Perlis, D., Purang, K., & Anderson, C. (1998). Conversational adequacy: Mistakes are the

essence. International Journal of Human-Computer Studies, 48, 553 – 575. doi:

http://dx.doi.org/10.1006/ijhc.1997.0181

Reyna, V.F., & Lloyd, F. (1997). Theories of false memory in children and adults. Learning and

Individual Differences, 9, 95 – 123. doi: 10.1016/S1041-6080(97)90002-9

Roediger, H.L., & McDermott, K.B. (1995). Creating false memories: Remembering words not

presented in lists. Journal of Experimental Psychology: Learning, Memory, and

Cognition, 21, 803 – 814. doi: http://dx.doi.org/10.1037/0278-7393.21.4.803

Rosen, I.M., Gimotty, P.A., Shea, J.A., & Bellini, L.M. (2006). Evolution of sleep quantity, sleep

deprivation, mood disturbances, empathy, and burnout among interns. Academic

Medicine, 81, 82 – 85. doi: 10.1097/00001888-200601000-00020

Russo, J.E., & Schoemaker, P.J. (2002). Winning decisions: Getting it right the first time. M.

Hittleman (Ed.). New York, NY: Random House, Inc.


Shanafelt, T.D., Bradley, K.A., Wipf, J.E., & Back, A.L. (2002). Burnout and self-reported

patient care in an internal medicine residency program. Annals of Internal Medicine, 136,

358 – 367. doi: 10.1001/archinternmed.2012.3199

Sovacool, B.K. (2014). Diversity: Energy studies need social science. Nature, 511, 529 – 30.

Tomasello, M., Carpenter, M., Call, J., Behne, T., & Moll, H. Understandings and sharing

intentions: The origins of cultural cognition. Behavioral and Brain Sciences, 28, 675 –

691. doi: http://dx.doi.org/10.1017/S0140525X05000129

Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction

fallacy and probability judgment. Psychological Review, 90, 293 – 315. doi:

http://dx.doi.org/10.1037/0033-295X. 90.4.293

Tversky, A., Kahneman, D., Thaler, R.H. (1991). The endowment effect, loss aversion, and

status quo bias. The Journal of Economic Perspectives, 5, 193 – 201. doi:

10.1257/jep.5.1.193

Wakefield, A.J., Murch, S.H., Anthony, A., Linnell, J., Casson, D.M., Malik, M., Berelowitz,

M., Dhillon, A.P., Thompon, M.A., Harvey, P., Valentine, A., Davies, S.E., Walker-

Smith, J.A. (1998). Illeal-lymphoid-nodular hyperplasia, non-specific colitis, and

pervasive developmental disorder in children.

Warman, D.M., Lysaker, P.H., Martin, J.M., Davis, L., & Haudenschield, S.L. (2007). Jumping

to conclusions and the continuum of delusional beliefs. Behaviour Research and

Therapy, 45, 1255– 1269. doi: 10.1016/j. brat.2006.09.002

Wason, P.C. (1968). Reasoning about a rule. The Quarterly Journal of Experimental Psychology,

20, 273 – 281. doi: 10.1080/14640746808400161


Weinberger, S.E., Pereira, A.G., Iobst, W.F., Mechaber, A.J., Bronze, M.S. (2010). Competency-

based education and training in internal medicine. Annals of Internal Medicine, 153, 751-

756. doi: 10.1059/0003-4819-153-11-201012070-00009


Figure 1. Mean CRT composite scores by educational track.

0

0.5

1

1.5

2

2.5

3

Undergraduate Graduate Post-Graduate

Mea

n C

ompo

site

CRT

Sco

re

Education Level

Medical Track Law Track


Figure 2. Percent correct on the Linda problem by educational track.

0102030405060708090

100


Perc

ent C

orre

ct

Educational Track



Figure 3. Percent correct on the Wason cards task by educational track.

05

101520253035


Perc

ent C

orre

ct

Educational Track



Figure 4. Beads drawn on easy beads task by education track.

012345678


Mea

n B

eads

Dra

wn

Education Level Medical Track Law Track


Figure 5. Beads drawn on difficult beads task by education track.

0

2

4

6

8

10

12


Num

ner B

eads

Dra

wn

Education Level



Figure 6. Abridged numeracy scale mean composite scores by educational track.


Appendix A


Appendix B


Appendix C


Appendix D


Appendix E


Appendix F

Easy Version


Difficult Version

Documents

jpd282_Final Thesis Edited