Personality Tests, Predictive Validity II,

and Failure

Personality Tests and Predictive Validity• 89 of the Fortune 100 Companies use the Myers-Briggs Test

• 2.5 million Americans take the Myers-Briggs each year

Why do employers so love personality tests so much?

What are the primary problems with personality tests?

(a.) binary categoriese.g., Walter Mischel’s studies of childhood aggressiveness

(b.) consistency/reliabilitye.g., circumstantial effects

(c.) indirect measuree.g., context, situation, contingency

(d.) “emergent trait” issuese.g., interaction effects (people we don’t find loyal, nice and interesting have friends too)

Personality Tests and Predictive ValiditySteps in doing any kind of research:

(1) theory/idea/observation: UofR students are very homogenous (similar)

(2) formulate hypothesis(es): The admissions process strongly favors only a few personality types OR only a few personality types lead to successful admissions

(3) conceptualize and measure variables: Myers-Briggs or Gladwell’s “Canine/Feline, More/Different, Insider/Outsider, Nibbler/Gobbler”

(4) design research: Have a fairly representative class of students take the test.

(5) collect data: e-mail, in person, phone, mail, etc.

(6) analyze data and draw inferences: next slide

(7) go back again to alter or come up with a new theory/idea

Personality Tests and Predictive ValidityResults: Myers-BriggsENTJ ENFJ INTP ESFJ ESTJ ESFJ ESFP ISTJ YEAR 4 3 2 2 2 2 1 1 2005 1 1 1 1 1 2 1 1 2007

Extrovert/Introvert; Intuition(N)/Sensing; Thinking/Feeling; Judging/Perceiving

Results: GladwellCMIN CMIG FDIN CDIG FMON FMIG FDOG CDIN YEAR 4 2 2 2 2 2 2 1 2005

1 2 2 1 1 1 2007

Canine/Feline; More/Different; Insider/Outsider; Nibbler/Gobbler

How homogenous would you say UR students are based on this data?

ENTJ’s: Bill Gates, Quentin Tarantino • Among top 4 types in college GPA.

• In national sample of "Leisure Activities," overrepresented in "Working out/exercising."

• Academic subjects preferred: English, science

• In national sample, highest of all types in liking work environments with "Variety of tasks," "People from a variety of backgrounds," and "International opportunities"; lowest of all types in liking work environments with "No expectation of extra hours" and "Toeing the line." Include "Independence and achievement" and "Clear structure" among top 3 desirable characteristics.

• Greatest work environment satisfiers are opportunities to use talents and contribute to society, job security, opportunities for learning and for accomplishment.

• In national sample, 1 of 2 types most satisfied with their work, where they work, and future work opportunities, unlikely to leave job, and among those with the highest income.

• With ESTJs, had the 2nd highest mean level of coping resources.

• Ranked 1st of all 16 types in using physical coping resources.

• In national sample, highest in coping with stress by "Trying to think of options."

• In national sample, ranked 2nd highest in "No" and 4th highest in "Not sure" re: "Belief in higher spiritual power."

ESTJ’s: Bill O’Reilly, Drew Carrey • Most frequent type among school administrators.

• In national sample "Leisure Activities," overrepresented in "Playing sports" and "Watching sporting events," underrepresented in "Listening to music."

• Academic subjects preferred: math, practical skills

• Highest of all types in national sample liking work environments with "Clear structures & responsibilities" and "Working as part of a team"; 1 of 3 highest types favoring "Toeing the line" and "Going by the book" and include "Variety of tasks" among top 3 desirable characteristics.

• Most important feature of an ideal job: "A stable and secure future."

• In national sample, were among those most satisfied with their work and working conditions.

• In national sample, dissatisfied with "Promotions" in their work.

• Overrepresented among both male and female small business owners compared with national sample.

• Ranked 1st of all 16 types in using cognitive coping resources and 2nd in using emotional and social coping resources.

• Overrepresented among those having Type A behavior.

• In national sample, ranked 3rd highest in satisfaction with "Marriage/intimate relationship."

Stress, Failure, and the Power of ContextPsychologists’ research on “explicit learning” and

“implicit learning”

Research has shown which system of learning takes over under conditions of stress? Why? Stress wipes out _____-____ ________.

Panic is the opposite of choking. Choking is about thinking too much.

Panic is about thinking too little.

Failure and the Power of Context: Choke & Panic Jana Novotna Chuck Knoblauch Romo JFK, Jr.

“Panicking is conventional failure, of the sort we sort of understand. But choking makes little intuitive sense (it’s paradoxical).” Why?

“Sometimes a poor performance reflects not the innate ability of the performer but the environment; sometimes a poor test score is the sign not of a ______ student but of a ______ one.”

“Stereotype Threat,” the SAT, and the Power of Context

Stanford social psychologist, Claude Steele:

stereotype threats are essentially self-fulfilling prophecies

GETncm/justsaycust-recrate-itemcommunittg/stores/dtg/stores/d-favorite-listrue

Intelligence Reform and Predictive ValiditySenator Shelby’s investigative report on 9/11 argues that the

fundamental failure was the Intelligence Community’s inability to what: ______ ___ _____?

“creeping determinism”psychologist Baruch Fischhoff’s study of Richard Nixon’s trip to China

The central challenge of intelligence gathering has always been the problem of ______. Signals exist in a context of ______.

Seldom does intelligence information touch on both ______ and ________.

Intelligence Reform and Predictive Validity- 1970s study by Stanford psychology professor David

Rosenhan: labeling and self-fulfilling prophecies

- lessons from Pearl Harbor, the CIA, and the Bay of Pigs

- “groupthink” and the tendency to oscillate between centralization (JFK, Bush II) and constructive rivalries (FDR, Clinton)

- FBI and CIA’s rivalry/relationship pre-9/11

- Humans have a craving for predictive validity and

a visceral aversion to ambiguity/uncertainty.

Puzzles,Mysteries, and National SecurityWhere is Osama bin Laden? (puzzle)

What will happen in Iraq after the toppling of Saddam? (mystery)

Mysteries require judgments and assessments of uncertainty, and the hard part is not that we have too little data/information but that we have too MUCH of it.

Puzzles stem from a LACK of data/information.

The distinction is crucial.

Question: Was 9/11 a puzzle or a mystery?

Puzzles,Mysteries, and National SecurityIf things go wrong with a puzzle, identifying the culprit is easy: it’s

the person who withheld information (presidential aides, corporate executives, informants, etc.).

Mysteries, though, are a lot murkier.

Puzzles tend to come to satisfying conclusions (where is Saddam?). They grow simpler with the addition of each new piece of information/data.

Mysteries rarely grow simpler; instead, they grow more complex and harder to solve with the addition of new information/data.

Classic PuzzleWatergate

August 9, 1974

“Deep Throat”Mark Felt

Prototypical MysteryDiagnosing prostate cancer (used to be a puzzle):

Doctors used to do a rectal exam and feel for a lumpy tumor on the surface of the patient’s prostrate.

Today doctors don’t wait for patients to develop the painful symptoms of prostate cancer. They regularly test—collect data/info—on middle-aged men (elevated PSA levels).

If the results look abnormal (2 standard deviations above the mean for middle-age men), they use ultrasound imagine to take a picture. If the picture looks abnormal—compared to middle-age men without prostate cancer—they take a biopsy (removing tiny slices of the gland) and examine the tissue under a microscope.

Rarely are the findings absolutely definitive (unless the cancer is very advanced, by which time the doctor would have already found it through the rectal exam).

Thus, the doctor is no longer confirming the presence of a malignancy.

Instead, he or she is predicting it, and the certainties of previous doctors have been replaced with outcomes than can only be said to be “highly probable” or “tentatively” estimated. Doctors disagree about the predictive validity of the PSA test & biopsy.

This form of medical progress has involved the switch from solving a puzzle to pursuing a mystery.

Puzzle or Mystery?

Enron: Puzzle or Mystery?Clue: company paid no income tax in 4 of its last 5 years (1997-2001)

All the information/data used to write the critical Wall Street Journal article on September 20, 2000 by Jonathan Weil came directly from the company’s annual reports and quarterly filings. No “Deep Throat” or insider info/data.

Similarly, all the information/data used to write the critical Fortune article, “Is Enron Overpriced?” in March 2001 came from personal meetings with Enron’s senior managers and from the same annual reports and quarterly filings.

SPE’s (Special Purpose Entities) where Enron hid its bad debt were required to publicly file their financial numbers (revenues, expenses, etc.) with the SEC (Securities and Exchange Commission).

They were available to anyone and everyone, including students at Cornell….

Enron: Puzzle or Mystery? Cornell 1998