ContentServer (10).pdf

287287

The Journal of General Psychology, 2008, 135(3), 287300Copyright 2008 Heldref Publications

Science Versus the Stars: A Double-Blind Test of the Validity of the NEO Five-Factor

Inventory and Computer-Generated Astrological Natal Charts

ALYSSA JAYNE WYMANSTUART VYSE

Connecticut College

ABSTRACT. The authors asked 52 college students (38 women, 14 men, M age = 19.3 years, SD = 1.3 years) to identify their personality summaries by using a computer-generated astrological natal chart when presented with 1 true summary and 1 bogus one. Similarly, the authors asked participants to identify their true personality profile from real and bogus summaries that the authors derived from the NEO Five-Factor Inventory (NEO-FFI; P. T. Costa Jr. & R. R. McCrae, 1985). Participants identified their real NEO-FFI profiles at a greater-than-chance level but were unable to identify their real astrological summaries. The authors observed a P. T. Barnum effect in the accuracy ratings of both psychological and astrological measures but did not find differences between the odd-numbered (i.e., favorable) signs and the even-numbered (i.e., unfavorable) signs.

Keywords: astrology, double-blind test, five-factor model, NEO-FFI, personal validation, personality

ASTROLOGERS NATAL CHARTS and psychologists personality profiles share a common purposeto provide a description of the respondents personal-ityand they are based on at least two common assumptions. First, astrologers and personality psychologists assume that people possess stable characteristics that, to varying degrees, determine their behavior. This assumption gives person-ality assessment its value. Second, both astrologers and psychologists assume that the instruments they have developed for this purpose can measure these traits. Despite these similarities, astrology and trait psychology represent very differ-ent theories about the causes of personality. Astrologers believe an individuals character is determined by the arrangement of the planets and stars in relation

Address correspondence to Stuart Vyse, Department of Psychology, Box 5621, Connecticut College, New London, CT 06320, USA; [email protected] (e-mail).

288 The Journal of General Psychology

to the moment of that persons birth. As a result, the natal charts that astrolo-gers construct are designed to identify the relative locations of various celestial objects at the precise time and place of birth. In contrast, many contemporary personality psychologists place the source of personality 9 months earlierat conceptionwhen the individuals genetic profile is determined (Brody, 1994). The personality inventories that psychologists use are designed to measure these traits by asking participants to report on their own cognitions and behavior. Groups of specific cognitions and behaviors are thought to be indicative of more general underlying traits.

The similarities between these psychological and astrological personality assessments have led to comparisons of their relative validity. Carlson (1985) tested participants ability to recognize their astrological charts and their Califor-nia Personality Inventory (CPI) profiles. Presented with personality descriptions based on their own astrological charts and two other personality descriptions randomly selected from those of others, Carlsons participants were unable to identify their astrological profiles at greater-than-chance levels. Moreover, using the same procedure, he found that participants were no more skilled at identify-ing their CPI profiles.

A number of factors have been implicated in peoples acceptance of personal-ity feedback, regardless of whether it is psychological or astrological. Taylor and Brown (1988) suggested that many people hold an unrealistically enhanced view of themselves, a characteristic that may make it difficult for them to accurately identify their own personality profiles. If, as Taylor and Brown suggested, this self-enhancing phenomenon were widespread, it would throw into question the usefulness of the personal validation method of Carlsons (1985) test. Doubts about the accuracy of self-knowledge are further supported by Sundbergs (1955) study showing that participants were unable to discriminate between their own profiles and bogus profiles derived from the Minnesota Multiphasic Personality Inventory. However, other tests with the CPI have shown that participants can identify their own personality profiles at greater-than-chance levels (Greene, Harris, & Macon, 1979; Word, 1996). Therefore, although personal validation has not yet been successfully demonstrated for astrological charts, a limited number of studies have shown that participants have sufficient self-understanding to discriminate true CPI from bogus CPI (e.g., Word) and 16 Personality Factors (16PF; Hampson, Gilmour, & Harris, 1978) reports.

Several studies have shown that the P. T. Barnum effect (Meehl, 1956), a general tendency for individuals to accept vague and ambiguous descriptions as typical of themselves, influences the acceptance of both astrological (Glick, Got-tesman, & Jolton, 1989) and psychological (Dickson & Kelly, 1985; Forer, 1949; Guastello & Rieke, 1990; ODell, 1972; Sundberg, 1955; Word, 1996) personal-ity feedback. Furthermore, there is evidence that, for both kinds of assessment, positive feedback is more readily accepted. For example, in the study by Greene et al. (1979), participants who misidentified their CPI profiles were most likely

to pick the more positive of the two profiles presented. Other researchers have found similar results (e.g., Collins, Dmitruk, & Ranney, 1977; Glick et al., 1989; Hamilton, 1995). In the case of astrology, the odd-numbered sun signs (i.e., Aries, Gemini, Leo, Libra, Sagittarius, Aquarius) are thought to be more favorable in content than the even-numbered ones (i.e., Taurus, Cancer, Virgo, Scorpio, Cap-ricorn, Aquarius), and some researchers have found a difference in the degree of acceptance of charts for odd sun signs versus even sun signs (Glick et al., 1989; Hamilton, 2001). However, a more recent large-scale study did not replicate this effect (Wunder, 2003). Last, researchers have shown that prior knowledge of an individuals astrological sun sign is associated with a bias toward acceptance of astrological readings (Hamilton, 1995; Van Rooij, 1994).

We designed the present study as a replication of Carlsons (1985) study with several substantive modifications. First, in the intervening years, both astrology and personality psychologies have changed. For astrology, recent decades have seen the development of sophisticated computer programs capable of converting birth data into detailed natal charts that include lengthy personality descriptions. In the field of personality, the years since Carlsons study have included the emer-gence of the Five-Factor Model (FFM), or Big Five, as the dominant trait theory of personality (Costa & McCrae, 1992b; Goldberg, 1993). One of our goals in the present study was to update Carlsons work by using computer-generated natal charts and the NEO Five-Factor Inventory (NEO-FFI; Costa & McCrae, 1985).

Carlsons (1985) participants were not able to identify either their astrologi-cal personality descriptions or their CPI profiles given a three-choice task. In other tests with the CPI, participants have successfully identified their own pro-files in two- (Greene et al., 1979) and three-choice (Word, 1996) tasks; however, no studies have reported successful identification of astrological profiles. To maximize the likelihood of correct identification, we used a simple two-choice task. We presented each participant with one real astrological personality descrip-tion and one bogus astrological personality description. Similarly, identifications of the NEO-FFI involved one real profile and one bogus profile. In addition, both to provide more detailed information about participants accuracy judgments and to assess the P. T. Barnum effect, sun-sign bias, and odd-even sun-sign effect, we asked participants in the present study to make detailed accuracy ratings of both the astrological charts and the personality test results.

On the basis of previous research indicating that, at least in some instances, participants were able to correctly identify their CPI profiles (Greene et al., 1979; Word, 1996) and the presumed influence of the simpler two-choice arrangement of the present study, we hypothesized that participants would be able to correctly identify their real NEO-FFI profiles. But because no previous study had used a two-choice test with astrological profiles, we were unable to make a hypothesis in this case. We also asked participants to rate the accuracy of all four personality reports (bogus and real astrological charts and NEO-FFI profiles) on a 9-point Likert-type scale ranging from 1 (This is a completely inaccurate description of me)

Wyman & Vyse 289


to 9 (This is a completely accurate description of me), and on the basis of previous research we hypothesized that participants would give the real NEO-FFI profile significantly higher accuracy ratings than they would the bogus NEO-FFI profile. We did not make a hypothesis regarding the accuracy ratings of the astrologi-cal summaries. In addition, we hypothesized that participants ratings of all the profiles, astrological and psychological as well as bogus and real, would be rated somewhat favorably, consistent with the P. T. Barnum effect, and that participants who knew their sun signs would judge their real astrological readings to be more accurate, consistent with the sun-sign bias. Last, due to the conflicting results of prior studies, we made no hypothesis about differences in the accuracy ratings of odd and even signs because of the positive and negative personality descriptions associated with them.

Method

Participants

Participants were 52 Connecticut College students (38 women, 14 men). The ages of the students were between 18 and 22 years (M age = 19.3 years, SD = 1.3 years). We recruited participants from an introductory psychology class and through fliers posted around campus inviting any student to participate in a study of astrology and psychological assessment. There were 14 psychology majors, 28 nonpsychology majors, and 10 participants who were undecided about their major. Students from the introductory psychology class received course credit, and all volunteers received their personalized astrological natal charts as a reward for participation.

Measures

NEO Five-Factor Inventory (NEO-FFI). The NEO-FFI is a 60-item scale derived from the revised NEO Personality Inventory (Costa & McCrae, 1985; 1992a) that measures five facets of personality: Neuroticism (N), Extraversion (E), Open-ness to experience (O), Agreeableness (A), and Conscientiousness (C). Example items include, I try to be courteous to everyone I meet (A) and I like to be where the action is (E), and all items are rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The NEO-FFI has shown stable Cronbachs alphas ranging from .68 to .83 for domains of N, E, and O and alphas from .63 to .79 for A and C domains.

Knowledge and beliefs questionnaire. The only other measure was a brief question-naire that asked for demographic information and assessed knowledge of astrology and belief in the validity of both astrology and psychological personality tests. Two questions asked participants to rate the accuracy of personality descriptions derived from astrological charts and psychological tests on a Likert-type scale ranging

from 1 (very accurate) to 7 (very inaccurate). Five questions assessed astrological knowledge. The first three asked if the participant knew his or her sun sign and, if so, to name it and list three adjectives associated with people who have the sign. The last two asked participants whether they knew and, if so, to name their ris-ing or ascendant sign and their moon sign. This questionnaire also asked for the participants date, location, and time of birth; gender; and major.

Materials

Participants made their assessments on NEO-FFI summary feedback sheets and astrological charts that were modified to allow for accuracy ratings. The NEO-FFI summaries use one of three statements to indicate whether the respon-dent is high, moderate, or low on each of the five traits. For example, the high-N statement is, sensitive, emotional, and prone to experience feelings that are upsetting and the low-O statement is, down to earth, practical, traditional, and pretty much set in your ways. We modified the feedback sheets to allow partici-pant ratings of the accuracy of each of the five traits, as well as a single global rating of the report as a whole. These ratings were made on a 9-point Likert scale ranging from 1 (This is a completely inaccurate description of me) to 9 (This is a completely accurate description of me).

We generated astrological personality profiles by using Estoric Technolo-gies Solar Fire Five v5.0.19 (Dawson & Johnson, 2000) program. Although no conventional reliability or validity data are available for this program, the National Council for Geocosmic Research recommended it for research (National Center for Geocosmic Research, 2004), and it was endorsed by Burk (2001), who said, Solar Fire continues to be the cutting-edge astrology program avail-able (p. 342). We edited the natal chart output so that all references to the signs, planets, or houses were gone, to avoid bias due to astrological knowledge. We also removed personality descriptions derived from planetary aspects because astrological charts contain different numbers of aspects and including them would have resulted in real and bogus charts of different lengths. The resulting charts included 29 one- to four-sentence personality descriptions (e.g., You take pride in your home and family. You like being the center of attention in your home envi-ronment; You love spontaneity and new games. You also love new and creative projects, often initiating them yourself).1 Each of the astrological personality reports included the same 9-point accuracy rating scale used for the NEO-FFI summaries. Using this scale, the participants rated the accuracy of each of the 29 descriptions and, in a separate item, the accuracy of the chart as a whole.

Procedure

Testing occurred in two sessions that were 3 weeks apart. When participants volunteered for this study, we asked them to bring the verified date, time, and

Wyman & Vyse 291


location of their birth to the first session and encouraged them to contact their parents to obtain accurate birth information. In the first session, we gave partici-pants a packet containing the consent form, NEO-FFI questionnaire, and knowl-edge and beliefs questionnaire, respectively. We instructed participants to fill out the three forms in the order that they were presented. We verbally reminded participants to provide the date, location, and time of their birth and that provid-ing inaccurate information would be a violation of the honor code of the college. After they had completed all forms and returned the packet, we told participants that they would be contacted in approximately two weeks concerning the date of the next session.

During the second session, an experimenter who was blind to the hypothesis and had no knowledge of the real and bogus summaries presented each participant with his or her specific packet containing an instruction sheet and four personal-ity summaries, a real natal chart, a bogus natal chart, a real NEO-FFI summary, and a bogus NEO-FFI summary. We randomly selected the bogus astrological and NEO-FFI summaries from other participants in the study. We randomized the order of the summaries so that each participant had an equal chance of being presented with each summary in each ordinal position. We presented half of the participants with the astrological summaries first and the other half with the NEO-FFI summaries first. Each report was labeled as either Psychological Report or Astrological Report.

Participants were asked to attend to everything in the packet in the specific order that it was presented, first reading the directions and then rating each state-ment of all four personality summaries on the 9-point scale. In addition, partici-pants provided a single overall accuracy rating for each summary, and we asked them to identify which of the two NEO-FFI reports they believed was their own and which of the two astrological summaries was their own. Last, participants considered all four of the personality reports and identified the one that they thought was the most accurate description of their personality. On completion of all items, participants returned the packet to the experimenter. In return, the experimenter handed each participant a debriefing form and the participants real astrological natal chart, which was labeled with the participants name on its cover sheet and kept in a separate box from the other experimental materials.

Results

Personal Validation of Personality Descriptions

As we hypothesized, participants were able to correctly identify their NEO-FFI profiles at greater-than-chance levels, 2(1, N = 52) = 17.31, p < .001 (78.8% correct identifications). In contrast, only 46.2% of participants were able to iden-tify their real astrological summaries, which was not significantly better than the percentage of chance, 2(1, N = 52) = 0.31, p > .05.

Accuracy of Personality Reports

Participants rated the accuracy of psychological and astrological personality descriptions in two ways. For each of the five dimensions of the NEO-FFI and each of the 29 personality descriptions of the astrological reports, participants made individual ratings on a 9-point scale. In addition, they assessed the accuracy of each of the four reports in a single overall rating on the same 9-point scale. For the purposes of this analysis, we summed the individual ratings of items on the reports and divided each sum by 5 for the NEO-FFI reports and by 29 for the astrological reports, resulting in mean ratings for each type of report.

To assess the relative accuracy ratings of the four personality reports, we conducted a 2 (gender) 4 (personality report) mixed-design multivariate analy-sis of variance (MANOVA) with repeated measures across the four personal-ity descriptions and dependent variables being the overall ratings of accuracy and the mean accuracy ratings of the individual personality descriptions. We included gender as an independent variable because previous researchers have shown that women have significantly higher levels of belief in astrology (e.g., Wunder, 2003). In this case, neither the main effect of gender nor the interaction of personality report and gender was significant, but the main effect of personal-ity report was significant, Wilkss = .489, F(6, 43) = 7.57, p < .001, 2 = .51. Follow-up univariate analyses of variance revealed that there were significant effects of personality report on both overall ratings, F(3, 144) = 7.23, p < .001, 2 = .13, and mean individual item ratings, F(3, 144) = 15.01, p < .001, 2 = .24. The means and standard deviations for the mean-item and overall ratings for each of the four personality summaries are shown in Table 1. Tukey HSD tests revealed

Wyman & Vyse 293

TABLE 1. Means and Standard Deviations of Accuracy Ratings of Real and Bogus NEO-Five Factor Inventory (NEO-FFI; P. T. Costa, Jr., & R. R. McCrea, 1992a) Summaries and Real and Bogus Astrological Charts (N = 50)

Measure Mean-item ratings Overall ratings

Real NEO-FFI M 7.19 7.23 SD 1.14 1.27Bogus NEO-FFI M 6.21 6.09 SD 1.19 1.53Real astrological M 6.18 6.17 SD 0.91 1.17Bogus astrological M 6.05 6.48 SD 1.16 1.39


that, for both dependent variables, the real NEO-FFI was rated as significantly more accurate than the other personality reports (p < .05) and that there were no significant differences in accuracy ratings among the other three reports.

Last, when asked to choose which of the four personality reports provided the most accurate description of themselves, 54.9% of participants chose the real NEO-FFI, 19.6% chose the real astrological report, 15.7% chose the bogus astrological report, and only 9.8% chose the bogus NEO-FFI as the most accu-rate personality summary. Participants selected real NEO-FFI significantly more often than they did other summaries, 2(3, N = 51) = 25.31, p < .001.

P. T. Barnum Effect

The hypothesis that both psychological and astrological assessments would show the P. T. Barnum effect was supported. Each of the four personality reports was submitted to a one-sample t test for which the null hypothesis was that accuracy ratings in Table 1 were not significantly different from 5 (This is right as much as it is wrong). Both for overall ratings and for mean individual item ratings, the t tests revealed that all four personality reports were significantly dif-ferent from this neutral rating. Thus, both bogus NEO-FFI and bogus astrological reports showed evidence of the P. T. Barnum effect.

Sun-Sign Bias and Odd-Numbered Sign Bias

The hypothesis that participants prior knowledge of their sun sign would bias them in favor of accepting the astrological descriptions was partially sup-ported. To test this hypothesis, we divided participants into two groupsthose who correctly reported their sun sign and those who did not. Then we divided the statements of the real astrological summaries into statements related to the partic-ipants sun sign and statements unrelated to the participants sun sign. Analysis of these items showed evidence of sun-sign bias. Participants who knew their sun sign gave significantly higher accuracy ratings to the sun-sign-related statements than to the non-sun-sign statements, t(34) = 2.36, p < .05, whereas participants who did not know their sun sign did not, t(15) = 0.12, p > .05. Last, to determine whether the sun-sign bias affected accuracy ratings of the real astrological charts, we conducted a one-way MANOVA across these two groups on the mean-item and overall ratings of the participants real astrological charts. There were no significant differences in the accuracy ratings of these two groups, Wilkss = .996, F(2, 49) = 0.09, p > .05.

To analyze whether there was a bias in favor of the odd-numbered signs (Aries, Gemini, Leo, Libra, Sagittarius, Aquarius), which are often considered more favorable than the even-numbered ones (Taurus, Cancer, Virgo, Scorpio, Capricorn, Aquarius), we grouped participants by odd and even signs and con-ducted a one-way MANOVA across these groups with the overall and mean-item

accuracy ratings of both the real and bogus astrological charts as the dependent variables. The analysis was not significant, indicating that there was no bias pro-duced by the favorableness of the participants sun sign, Wilkss = .991, F(4, 47) = 0.48, p > .05.

Supplemental Exploratory Analyses

To further examine the results, we performed a number of additional analy-ses. We constructed a correlation matrix to examine the interrelationships of knowledge and belief in astrology, belief in psychological measures, and accu-racy ratings of the various personality measures (see Table 2). Belief in astrology was not correlated with accuracy ratings for either the real or bogus astrological summaries, but participants with greater knowledge of astrology gave the bogus chart a significantly lower total accuracy score. There was no correlation of knowledge or belief with overall ratings of the astrological summaries.

Despite the negative relation between astrological knowledge and accuracy ratings of the bogus astrological profile, a MANOVA revealed that participants who correctly identified their real astrological chart did not differ from those who did not do so in their knowledge of or belief in astrology, Wilkss = .972, F(2, 49) = 0.71, p > .05. Last, participants who knew their sun signs were less likely to correctly identify their real astrological profiles than those who did not know their sun signs (42% vs. 56%), although a chi-square test showed that this differ-ence was not statistically significant, 2(1, N = 52) = 0.95, p < .05.

In the case of psychological measures, participants who had greater belief in psychologically based measures gave the bogus NEO-FFI a higher overall rating; however, we did not see this pattern in either the total rating of the bogus NEO-FFI or the mean-item ratings of the real NEO-FFI. Similar to the results with the astrological profile, those who correctly identified their real NEO-FFI results did not differ significantly from those who did not in their degree of belief in psychological measures of personality, t(50) = 0.89, p > .05.

There were a number of positive correlations among the ratings of the real and bogus astrological summaries and the real and bogus NEO-FFI summaries, suggesting that some participants gave consistently higher ratings to all of the reports than others. These correlations were more often seen in the total ratings of the astrological and psychological summaries than in the overall ratings, perhaps because of the presumably greater reliability of the total ratings in comparison with the single item of the overall ratings.

Discussion

Consistent with previous research, we found that participants were unable to identify their own astrological charts at a greater-than-chance level, a result that extended the string of failure using the personal validation technique. Unlike

Wyman & Vyse 295


TA

BL

E 2

. Int

erco

rrel

atio

ns A

mon

g A

stro

logi

cal B

elie

f an

d K

now

ledg

e, B

elie

f in

the

Acc

urac

y of

Psy

chol

ogic

al P

erso

nalit

y M

easu

res,

and

Acc

urac

y R

atin

gs o

f R

eal a

nd B

ogus

Ast

rolo

gica

l and

NE

O-F

ive

Fac

tor

Inve

ntor

y (N

EO

-FF

I; P

. T. C

osta

, Jr.

, &

R. R

. McC

rea,

199

2a)

Sum

mar

ies

Var

iabl

e 1

2 3

4

5 6

7 8

9

10

11

1. A

stro

logi

cal b

elie

f

.2

5 .0

5 .0

5 .2

4 .2

7 .1

8 .0

3 .1

9 .2

4 .2

7 2

. Ast

rolo

gica

l kno

wle

dge

.

06

.10

.0

1 .0

3 .

27*

.14

.10

.10

.07

3. B

elie

f in

psy

chol

ogic

al m

easu

res

.01

.0

7 .0

4 .0

3 .

05

.01

.28*

.2

6 4

. Ove

rall

ratin

g of

rea

l ast

rolo

gica

l

s

umm

ary

.7

7***

.0

5 .2

2 .1

9 .2

2 .

02

.12

5. M

ean-

item

rat

ing

of r

eal a

stro

logi

cal

sum

mar

y

.1

3 .3

8**

.33*

.4

3**

.13

.34*

*

6. O

vera

ll ra

ting

of b

ogus

ast

rolo

gica

l

s

umm

ary

.4

6**

.20

.28*

.2

3 .3

6*

7. M

ean-

item

rat

ing

of b

ogus

a

stro

logi

cal s

umm

ary

.22

.33*

.2

2 .3

5*

8. O

vera

ll ra

ting

of r

eal N

EO

-FFI

s

umm

ary

.8

6***

.

06

.16

9. M

ean-

item

rat

ing

or r

eal N

EO

-FFI

s

umm

ary

.00

.29*

10. O

vera

ll ra

ting

of b

ogus

NE

O-F

FI

sum

mar

y

.81*

**

11. M

ean-

item

rat

ing

of b

ogus

NE

O-F

FI

sum

mar

y

* p

Documents

ContentServer (10).pdf