Progressing Measurement in Mental Toughness:A Case Example of the Mental Toughness Questionnaire 48

Daniel F. GucciardiThe University of Queensland, St. Lucia,

Queensland, Australia

Sheldon HantonCardiff Metropolitan University, Cardiff, Wales

Clifford J. MallettThe University of Queensland, St. Lucia, Queensland, Australia

Mental toughness has received increasing attention in the field of performance psy-chology, yet issues remain about its measurement by self-report. In this article, we havesummarized mental toughness measurement issues and, as an example, provided apsychometric examination of the most frequently used measure. In an effort to opera-tionalize mental toughness, Clough, Earle, and Sewell (2002) developed the MentalToughness Questionnaire 48 (MTQ 48) and provided initial evidence for its reliabilityand validity. Subsequent research has partially supported the internal reliability andvalidity of the MTQ 48. However, no research has rigorously tested the factorialstructure of the hypothesized model underlying this scale. Using two independentsamples of performers from various sports (n � 686) and the workplace (n � 639), wesought to examine the factorial validity of the MTQ 48 using confirmatory factoranalysis (CFA) and exploratory structural equation modeling (ESEM). Both CFA andESEM indicated that the hypothesized correlated four factor model did not fit the datawell in the athlete and workplace samples. Our overview of measurement issues andempirical case study of the MTQ 48 underscore the importance of having a clearlyarticulated conceptual underpinning combined with rigorous statistical procedures inattempting to develop a mental toughness inventory.

Keywords: exploratory structural equation modeling, factorial validity, mentally tough, personalresources, scale development

Regardless of the achievement context (e.g.,sport, workplace, education), individuals mustsuccessfully negotiate a variety of differentstressors, challenges, and adversities (e.g., in-jury, performance expectations and targets,work�life balance) if they are to perform totheir potential and reach their goals. What is it

that enables some performers to thrive underpressure situations, to overcome setbacksquickly, and to maintain a high level of func-tioning in the face of continuous challenges?Many suggest the answer lies in their psycho-logical makeup, which is commonly concep-tualized under the umbrella term mentaltoughness. It is not surprising then that mentaltoughness has attracted increasing empiricalattention in recent years, with much of thiswork devoted to its conceptualization and defi-nition (for a review, see Gucciardi & Gordon,2011). Coinciding with this increased attentionhas been the development of psychometric toolsdesigned to operationalize these different con-ceptualizations of mental toughness.

In a recent review of the mental toughnessmeasurement literature, Gucciardi, Mallett,Hanrahan, and Gordon (2011) concluded that,at present, no comprehensively sound measureexists. Gucciardi et al. considered several tradi-

This article was published Online First February 13, 2012.Daniel F. Gucciardi and Clifford J. Mallett, School of

Human Movement Studies, The University of Queensland,St. Lucia, Queensland, Australia; Sheldon Hanton, CardiffSchool of Sport, Cardiff Metropolitan University, Cardiff,Wales.

Gucciardi is supported by a University of QueenslandPostdoctoral Research Fellowship. Appreciation is extendedto Denise Hill, Rich Neil, Steve Mellalieu, Ross Wadey, andChris Wagstaff for their assistance with data collection.

Correspondence concerning this article should be addressedto Daniel F. Gucciardi, School of Human Movement Studies,The University of Queensland, St. Lucia, Queensland, Austra-lia, 4072. E-mail: d.gucciardi@uq.edu.au

Sport, Exercise, and Performance Psychology © 2012 American Psychological Association2012, Vol. 1, No. 3, 194–214 2157-3905/12/$12.00 DOI: 10.1037/a0027190

194

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

tional indicators of reliability (e.g., internal con-sistency, test�retest) and validity (e.g., content,factorial, predictive), as well as three fundamen-tal issues related to conceptual underpinning,statistical procedures, and practical utility inassessing the utility of the available measures.Although strengths of each instrument wereidentified, several limitations existed with eachquestionnaire according to the guiding criteria.These strengths and weaknesses are briefly re-viewed in the following section; interested read-ers are encouraged to consult Gucciardi et al.for a detailed discussion of these issues.

Gucciardi et al. (2011) reviewed both sport-general (i.e., developed with and for use withmultiple sports and athletes) and sport-specific(i.e., developed with and for use in a singlesport) measures of mental toughness. With re-gard to sport-general measures, the Psycholog-ical Performance Inventory (Loehr, 1986) wasdeemed inadequate according statistical (i.e.,lack of psychometric support) and conceptualcriteria (i.e., lack of a detailed conceptual un-derpinning), although the intuitive appeal of thismeasure in capturing some of the primary com-ponents of mental toughness was noted as a keystrength. Similar concerns about the conceptualunderpinning were made of its amended ver-sion, namely, the Psychological PerformanceInventory�A (Golby, Sheard, & van Wersch,2007), although strengths existed with regard topractical utility (i.e., item brevity) and the pre-liminary evidence supported its factorial valid-ity. The Sport Mental Toughness Questionnaire(Sheard, Golby, & van Wersch, 2009) was con-sidered strong in terms of the statistical proce-dures employed to support its psychometricproperties and practical utility (i.e., item brev-ity), yet it lacked an explicit conceptual modelunderpinning its factor structure. Finally, theMental Toughness Questionnaire 48 (MTQ 48;Clough et al., 2002), which has also been usedto examine mental toughness in nonsport con-texts (e.g., workplace, rehabilitation), wasdeemed inadequate according to statistical (i.e.,psychometric support for the hypothesizedmodel is currently unavailable) and conceptualcriteria (i.e., little information about the ratio-nale for adopting hardiness theory), althoughhaving a conceptual model for its developmentwas considered a key strength.

Gucciardi et al. (2011) also reviewed twosport-specific measures. The conceptual under-

pinning of the Australian Football MentalToughness Inventory (Gucciardi, Gordon, &Dimmock, 2009) was considered a key strength,despite the identification of weaknesses withregard to both statistical (i.e., cross-validationof the hypothesized measurement model wasnot supported) and practical utility (i.e., limitedusefulness beyond Australian football) criteria.The adoption of rigorous, hypothesis-testingstatistical procedures was considered a keystrength of the Cricket Mental Toughness In-ventory (Gucciardi & Gordon, 2009), yet con-cerns about the generalizability of the modelremained (i.e., practical utility).

From this brief review of the available mentaltoughness measures, it becomes apparent thatresearchers have tended to place greater impor-tance on either a strong conceptual underpin-ning or rigorous statistical procedures to de-velop and validate tools. However, both of thesecriteria are important for scale developmentsuch that the marginalization of one criterioncan have significant consequences for the integrityof an instrument (for a review, see MacKenzie,Podsakoff, & Podsakoff, 2011). In this article,we examined the extent to which a preferencefor a conceptual underpinning over rigorousstatistical analyses might compromise the psy-chometric integrity of the most commonly em-ployed mental toughness inventory.

The MTQ 48 (Clough et al., 2002), whichevolved from a noteworthy body of researchthat examined the stress�illness relationship, isthe most widely employed tool for assessingmental toughness both in sport and nonsportcontexts (for an overview, see Table 1). Emerg-ing from research on stress reactions in thehealth psychology literature is the hardinessconstruct, which is conceptualized as a combi-nation of three attitudes—commitment, control,and challenge (3Cs)—that provide an individ-ual with existential courage and motivation toappraise stressful situations as opportunities forgrowth (i.e., choose to approach the unknownrealms of the future, rather than repeating past,familiar experiences) (Maddi, 2004). ForClough et al. (2002), the three hardiness atti-tudes closely resembled but did not fully encap-sulate mental toughness. Accordingly, theyadded a fourth dimension, confidence, to ac-count for the ecologically valid views of keystakeholders (i.e., athletes, coaches, sport psy-chologists). Within the context of their 4Cs

195MEASURING MENTAL TOUGHNESS

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Tab

le1

Ove

rvie

wof

Stud

ies

Em

ploy

ing

the

MT

Q48

asa

Mea

sure

ofM

enta

lT

ough

ness

Stud

yPa

rtic

ipan

tsM

etho

dsPr

imar

yfin

ding

s

Clo

ugh,

Ear

le,

&Se

wel

l(2

002)

23pa

rtic

ipan

ts(d

emog

raph

icin

form

atio

nun

avai

labl

e,e.

g.,

age,

sex,

skill

leve

l)Se

lf-r

atin

gs(p

hysi

cal

dem

ands

,m

enta

lde

man

ds,

effo

rt,

MT

Q48

),V

O2m

ax,

and

cycl

ing

Inte

rnal

relia

bilit

yes

timat

esw

ere

not

prov

ided

.M

edia

nsp

litba

sed

onm

enta

lto

ughn

ess;

diff

eren

ces

inpe

rcei

ved

phys

ical

dem

ands

exis

ted

only

whe

nw

orkl

oad

was

high

(70%

VO

2m

ax),

but

not

whe

nlo

w(3

0%V

O2m

ax)

orm

oder

ate

(50%

VO

2m

ax).

Clo

ugh

etal

.(2

002)

79pa

rtic

ipan

ts(d

emog

raph

icin

form

atio

nun

avai

labl

e,e.

g.,

age,

sex,

skill

leve

l)R

ecei

ved

posi

tive

orne

gativ

efe

edba

ckaf

ter

com

plet

ing

anu

mbe

rof

mot

orta

sks,

then

com

plet

eda

plan

ning

task

(i.e

.,co

gniti

veex

erci

se)

Inte

rnal

relia

bilit

yes

timat

esw

ere

not

prov

ided

.M

enta

llyto

ughe

rpa

rtic

ipan

tspe

rfor

med

bette

ron

the

plan

ning

task

;in

tera

ctio

nef

fect

betw

een

men

tal

toug

hnes

san

dfe

edba

ck,

such

that

type

offe

edba

ckw

asir

rele

vant

for

men

tally

toug

her

part

icip

ants

,bu

tle

ssm

enta

llyto

ughe

rpa

rtic

ipan

tspe

rfor

med

wor

seaf

ter

nega

tive

feed

back

.C

rust

(200

9)55

mal

e(M

age

�22

.58

year

s)an

d57

fem

ale

(Mage

�21

.11

year

s)at

hlet

esfr

oma

vari

ety

ofsp

orts

Cro

ss-s

ectio

nal

surv

eyco

ntai

ning

the

Aff

ect

Inte

nsity

Mea

sure

(Lar

sen,

1984

)an

dth

eM

TQ

48

Inte

rnal

relia

bilit

yes

timat

epr

ovid

edfo

rto

tal

men

tal

toug

hnes

son

ly(�

�.8

6).

No

sign

ifica

ntdi

ffer

ence

sin

men

tal

toug

hnes

sbe

twee

nm

ale

and

fem

ale

athl

etes

,an

dre

crea

tiona

lan

dcl

ubor

high

erpa

rtic

ipan

tle

vel.

No

rela

tions

hips

obse

rved

betw

een

men

tal

toug

hnes

san

daf

fect

inte

nsity

.C

rust

&A

zadi

(200

9)66

mal

e(M

age

�30

.1ye

ars)

and

37fe

mal

e(M

age

�28

.6ye

ars)

athl

etes

from

club

orun

iver

sity

(n�

36)

and

coun

ty(n

�67

)st

anda

rd

Cro

ss-s

ectio

nal

surv

eyco

ntai

ning

the

Lea

ders

hip

Scal

efo

rSp

orts

(Che

lladu

rai

&Sa

leh,

1978

)an

dth

eM

TQ

48

Inte

rnal

relia

bilit

yes

timat

esw

ere

not

prov

ided

.C

orre

latio

nsev

iden

ced

betw

een

trai

ning

and

inst

ruct

ion

and

tota

lm

enta

lto

ughn

ess

(r�

.40)

,and

all

othe

rfa

cets

ofm

enta

lto

ughn

ess

(rra

nge:

.22�

.36)

exce

ptfo

rin

terp

erso

nal

confi

denc

e;de

moc

ratic

beha

vior

san

dco

nfide

nce

inab

ilitie

s(r

��

.27)

;so

cial

supp

ort

and

confi

denc

ein

abili

ties

(r�

�.2

0).

Reg

ress

ion

anal

yses

reve

aled

rela

tions

hips

betw

een

com

mitm

ent

(��

.26)

and

chal

leng

e(�

�.2

4)w

ithtr

aini

ngan

din

stru

ctio

n(1

8%va

rian

ceex

plai

ned)

;em

otio

nal

cont

rol

(��

�.2

1),c

onfid

ence

inab

ilitie

s(�

��

.29)

,and

life

cont

rol

(��

.28)

with

dem

ocra

ticbe

havi

ors

(14%

);ch

alle

nge

(��

.25)

and

com

mitm

ent

(��

�.2

2)w

ithau

tocr

atic

beha

vior

s(7

%);

chal

leng

e(�

�.1

9),

emot

iona

lco

ntro

l(�

��

.19)

,and

confi

denc

ein

abili

ties

(��

�.2

1)w

ithso

cial

supp

ort

196 GUCCIARDI, HANTON, AND MALLETT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Tab

le1

(con

tinu

ed)

Stud

yPa

rtic

ipan

tsM

etho

dsPr

imar

yfin

ding

s

(8%

);an

dem

otio

nal(

��

�.2

1)an

dlif

eco

ntro

l(�

�.2

7)w

ithpo

sitiv

efe

edba

ck(6

%).

Cru

st&

Aza

di(2

010)

67m

ale

(Mage

�22

.6ye

ars)

and

40fe

mal

e(M

age

�21

.1ye

ars)

athl

etes

from

club

orun

iver

sity

(n�

36)

and

coun

ty(n

�71

)st

anda

rd

Cro

ss-s

ectio

nal

surv

eyco

ntai

ning

the

Tes

tof

Perf

orm

ance

Stra

tegi

es(T

hom

as,

Mur

phy,

&H

ardy

,19

99)

and

the

MT

Q48

Inte

rnal

relia

bilit

yes

timat

esw

ere

notp

rovi

ded.

Com

mitm

ente

vide

nced

the

grea

test

num

ber

ofst

atis

tical

lysi

gnifi

cant

corr

elat

ions

with

psyc

holo

gica

lski

llus

age

inpr

actic

e(n

�7,

rra

nge:

.19–

.31)

and

com

petit

ion

(n�

6,r

rang

e�

�.3

2to

.40)

,fol

low

edby

tota

lmen

tal

toug

hnes

s(n

pra

ctic

e�

4,r

rang

e:.2

4�.3

5;n c

om

p�

6,r

rang

e:�

.47

to.2

4)an

dch

alle

nge

(npra

ctic

e�

3,r

rang

e:.1

9�.2

2;n c

om

p�

3,r

rang

e:�

.37

to.2

4).R

egre

ssio

nan

alys

esre

veal

edth

esu

perio

rity

ofco

mm

itmen

tin

term

sof

the

grea

test

num

ber

ofst

atis

tical

lysi

gnifi

cant

rela

tions

hips

with

psyc

holo

gica

lski

llsus

age

(n�

9,�

rang

e:.1

9�.4

6),f

ollo

wed

byem

otio

nalc

ontro

l(n

�9,

�ra

nge:

.21�

.42)

and

confi

denc

ein

abili

ties

(n�

4,�

rang

e:�

.24

to.4

2).

Cru

st&

Kee

gan

(201

0)69

mal

e(M

age

�22

.2ye

ars)

and

36fe

mal

e(M

age

�24

.6ye

ars)

stud

ent

athl

etes

from

recr

eatio

nal

(n�

32)

club

orun

iver

sity

(n�

55)

and

coun

ty(n

�18

)st

anda

rd

Cro

ss-s

ectio

nal

surv

eyco

ntai

ning

the

Atti

tude

sT

owar

dsR

isks

Que

stio

nnai

re(F

rank

en,

Gib

son,

&R

owla

nd,

1992

)an

dth

eM

TQ

48

Tot

alm

enta

ltou

ghne

ss,c

onfid

ence

inab

ilitie

s,an

din

terp

erso

nalc

onfid

ence

had

acce

ptab

lele

vels

ofin

tern

alre

liabi

lity

(i.e.

,��

.70)

,whe

reas

chal

leng

e(�

�.6

8),c

omm

itmen

t(�

�.6

2),

emot

iona

lcon

trol(

��

.60)

,and

life

cont

rol

(��

.56)

did

not.

With

rega

rdto

phys

ical

risks

,to

talm

enta

ltou

ghne

ss(r

�.3

0),c

halle

nge

(r�

.43)

,com

mitm

ent(

r�

.20)

,and

confi

denc

ein

abili

ties

(r�

.21)

had

stat

istic

ally

sign

ifica

ntre

latio

nshi

ps.O

nly

inte

rper

sona

lcon

fiden

ce(r

�.2

4)w

assi

gnifi

cant

lyre

late

dto

psyc

holo

gica

lris

ks.M

endi

spla

yed

high

erle

vels

ofto

talm

enta

lto

ughn

ess

(p�

.04)

and

confi

denc

ein

abili

ties

(p�

.04)

than

wom

en.

Cru

st,

Nes

ti,&

Litt

lew

ood

(201

0a)

112

mal

efo

otba

llers

aged

12to

18ye

ars

Cro

ss-s

ectio

nal

surv

eyco

ntai

ning

the

MT

Q18

Inte

rnal

relia

bilit

yes

timat

epr

ovid

edfo

rto

talm

enta

lto

ughn

ess

(��

.69)

.No

sign

ifica

ntdi

ffer

ence

sin

men

talt

ough

ness

wer

ere

veal

edbe

twee

npl

ayer

sw

how

ere

reta

ined

and

rele

ased

atth

een

dof

the

seas

on,o

rag

egr

oups

(i.e.

,Und

er13

/Und

er14

/Und

er16

/Und

er19

).

197MEASURING MENTAL TOUGHNESS

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Tab

le1

(con

tinu

ed)

Stud

yPa

rtic

ipan

tsM

etho

dsPr

imar

yfin

ding

s

Cru

st,

Nes

ti,&

Litt

lew

ood

(201

0b)

21m

ale

foot

ball

play

ers

aged

16to

18ye

ars;

2co

ache

s(d

emog

raph

ics

unav

aila

ble)

Pros

pect

ive

surv

eyof

the

MT

Q18

attw

otim

epo

ints

disp

erse

dby

thre

em

onth

s

Inte

rnal

relia

bilit

yes

timat

esw

ere

not

prov

ided

.Pl

ayer

sre

port

edsi

gnifi

cant

lyhi

gher

leve

lsof

men

tal

toug

hnes

sth

anon

eof

the

coac

hes

(p�

.05)

but

not

the

othe

rco

ach.

Cor

rela

tions

betw

een

the

thre

era

ting

sour

ces

wer

eno

tsi

gnifi

cant

.M

enta

lto

ughn

ess

ratin

gsfo

rea

chra

ting

sour

cew

ere

stab

leov

erth

e3-

mon

thpe

riod

(rra

nge:

.94�

.99)

.L

evel

ofag

reem

ent

betw

een

ratin

gso

urce

sw

aslo

w(i

ntra

clas

sr

rang

e:.0

8�.2

9).

Cru

st&

Swan

n(2

011)

110

mal

est

uden

tat

hlet

es(M

age

�20

.8ye

ars)

from

ava

riet

yof

spor

tsC

ross

-sec

tiona

lsu

rvey

cont

aini

ngth

eSp

orts

Men

tal

Tou

ghne

ssQ

uest

ionn

aire

(She

ard,

Gol

by,

&va

nW

ersc

h,20

09)

and

the

MT

Q48

All

subs

cale

sof

the

MT

Q48

,ex

cept

for

emot

iona

lco

ntro

l(�

�.4

5)an

dlif

eco

ntro

l(�

�.5

0),

show

edad

equa

tele

vels

ofin

tern

alre

liabi

lity

(i.e

.,�

�.7

0).

Sign

ifica

ntco

rrel

atio

nsw

ere

evid

ence

dbe

twee

nth

ehi

gher

-ord

er(i

.e.,

glob

alor

tota

lsc

ores

)fa

ctor

s(r

�.7

5).

Subs

cale

sw

ithov

erla

ppin

gco

ncep

tual

desc

ript

ions

(i.e

.,co

nfide

nce,

cont

rol,

com

mitm

ent/c

onst

ancy

)w

ere

mod

erat

ely

corr

elat

ed(r

rang

e:.4

9�.6

1).

Kai

sele

r,Po

lman

,&

Nic

holls

(200

9)48

2at

hlet

es(3

05m

ales

)ag

ed16

to45

year

s(M

age

�20

.44

year

s)fr

omin

tern

atio

nal

tocl

ubor

univ

ersi

tyle

vels

Cro

ss-s

ectio

nal

surv

eyco

ntai

ning

ast

ress

orty

pean

dap

prai

sal,

copi

ngan

dco

ping

effe

ctiv

enes

s(C

rock

er&

Gra

ham

,19

95),

and

the

MT

Q48

Inte

rnal

relia

bilit

yes

timat

esfo

rto

tal

men

tal

toug

hnes

s(�

�.9

2)an

dfiv

efa

cets

wer

ead

equa

te(�

�.6

9),

exce

ptfo

rem

otio

nal

cont

rol

(��

.55)

.M

enta

lto

ughn

ess

exhi

bite

dlo

w-t

o-m

oder

ate

corr

elat

ions

(rra

nge:

�.6

7to

.30)

with

copi

ngan

dco

ping

effe

ctiv

enes

s.T

otal

men

tal

toug

hnes

san

dits

six

face

tsac

coun

ted

for

am

inim

alam

ount

ofth

eva

rian

cein

stre

ssin

tens

ity(3

%an

d7%

,re

spec

tivel

y)an

dpe

rcei

ved

cont

rol

(4%

and

5%,

resp

ectiv

ely)

.T

otal

men

tal

toug

hnes

san

dits

six

face

tsac

coun

ted

for

alo

w-t

o-m

oder

ate

amou

ntof

the

vari

ance

inex

tent

ofus

e(0

–24%

and

3–48

%,

resp

ectiv

ely)

and

perc

eive

def

fect

iven

ess

(0–7

%an

d1–

10%

,re

spec

tivel

y)of

copi

ngst

rate

gies

.

198 GUCCIARDI, HANTON, AND MALLETT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Tab

le1

(con

tinu

ed)

Stud

yPa

rtic

ipan

tsM

etho

dsPr

imar

yfin

ding

s

Lev

y,Po

lman

,C

loug

h,M

arch

ant,

&E

arle

(200

6)70

reha

bilit

atio

npa

tient

s(4

4m

ale;

Mage

�32

.5ye

ars)

Cro

ss-s

ectio

nal

surv

eyco

ntai

ning

am

easu

reof

adhe

renc

e,Sp

ort

Inju

ryR

ehab

ilita

tion

Bel

iefs

Surv

ey(D

aly,

Bre

wer

,&

Van

Raa

lte,

1995

),Sp

ort

Inve

ntor

yfo

rPa

in(M

eyer

s,B

ourg

eois

,St

ewar

t,&

LeU

nes,

1992

)an

dth

eM

TQ

18

Inte

rnal

relia

bilit

yes

timat

efo

rm

enta

lto

ughn

ess

was

less

than

adeq

uate

(��

65).

Thr

eem

enta

lto

ughn

ess

grou

ps(l

ow,

med

ium

,hi

gh)

wer

ecr

eate

d(i

nfor

mat

ion

onho

wth

ese

grou

psw

ere

form

edw

asno

tpr

ovid

ed).

The

high

men

tal

toug

hnes

sgr

oup

repo

rted

sign

ifica

ntly

low

erle

vels

ofpe

rcei

ved

susc

eptib

ility

than

both

the

med

ium

(p�

.05)

and

the

low

(p�

.01)

grou

ps;

the

med

ium

men

tal

toug

hnes

sgr

oup

repo

rted

sign

ifica

ntly

high

erle

vels

ofpe

rcei

ved

seve

rity

than

the

high

men

tal

toug

hnes

sgr

oup

(p�

.01)

;th

ehi

ghm

enta

lto

ughn

ess

grou

pre

port

edsi

gnifi

cant

lyhi

gher

leve

lsof

copi

ngw

ithpa

inth

anbo

thth

em

ediu

m(p

�.0

1)an

dth

elo

w(p

�.0

01)

grou

ps;

the

low

men

tal

toug

hnes

sgr

oup

repo

rted

high

erle

vels

ofpa

inca

tast

roph

eth

anth

ehi

ghm

enta

lto

ughn

ess

grou

p(p

�.0

1);

the

high

men

tal

toug

hnes

sgr

oup

repo

rted

sign

ifica

ntly

low

erle

vels

ofcl

inic

reha

bilit

atio

nad

here

nce

than

both

the

med

ium

and

the

low

(p�

.01)

grou

ps,

and

high

erat

tend

ance

atre

habi

litat

ion

than

the

low

men

tal

toug

hnes

sgr

oup

(p�

.05)

.

199MEASURING MENTAL TOUGHNESS

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Tab

le1

(con

tinu

ed)

Stud

yPa

rtic

ipan

tsM

etho

dsPr

imar

yfin

ding

s

Mar

chan

tet

al.

(200

8)52

2m

ange

rs(2

10m

en)

from

orga

niza

tions

base

din

the

Uni

ted

Kin

gdom

Cro

ss-s

ectio

nal

surv

eyco

ntai

ning

the

MT

Q48

Inte

rnal

relia

bilit

yes

timat

efo

rto

tal

men

tal

toug

hnes

s(�

�.8

9)w

asad

equa

te,

with

the

subs

cale

sre

port

edas

bein

gab

ove

.70

(spe

cific

valu

esno

tre

port

ed).

Seni

orm

anag

ers

scor

edsi

gnifi

cant

lyhi

gher

than

mid

dle

and

juni

orm

anag

ers

onal

lM

TQ

48su

bsca

les

(p�

.01)

;m

iddl

em

anag

ers

scor

edhi

gher

leve

lsof

tota

lm

enta

lto

ughn

ess

(p�

.05)

,lif

eco

ntro

l,an

din

terp

erso

nal

confi

denc

e(p

�.0

1)th

anju

nior

man

ager

san

dcl

eric

alst

aff;

mid

dle

man

ager

ssc

ored

high

erle

vels

ofch

alle

nge

and

com

mitm

ent

(p�

.01)

than

cler

ical

staf

f,an

dco

nfide

nce

inab

ilitie

s(p

�.0

1)th

anju

nior

man

agem

ent.

Ana

lyse

sre

veal

edm

ain

effe

cts

ofag

efo

rto

tal

men

tal

toug

hnes

s(p

�.0

1),

com

mitm

ent

(p�

.01)

,em

otio

nal

cont

rol

(p�

.05)

,an

dlif

eco

ntro

l(p

�.0

01).

Tre

nds

(i.e

.,no

sign

ifica

nce

leve

lsre

port

ed)

inpo

stho

cco

mpa

riso

nsof

age

sugg

este

dol

der

part

icip

ants

wer

ege

nera

llyhi

gher

inm

enta

lto

ughn

ess

than

youn

ger

part

icip

ants

.N

icho

lls,

Lev

y,Po

lman

,&

Cru

st(2

011)

206

athl

etes

(182

mal

es;

Mag

e�

17.7

5ye

ars)

from

inte

rnat

iona

lto

club

orun

iver

sity

leve

ls.

Com

plet

edth

eM

TQ

48an

dC

opin

gSe

lf-E

ffica

cySc

ale

(Che

sney

,N

eila

nds,

Cha

mbe

rs,

Tay

lor,

&Fo

lkm

an,

2006

)on

the

day

ofa

com

petit

ion,

and

am

easu

reof

copi

ngef

fect

iven

ess

(Got

tlieb

&R

oone

y,20

04)

30m

inaf

ter

the

com

petit

ion

Inte

rnal

relia

bilit

yes

timat

esfo

rfiv

efa

cets

wer

ead

equa

te(�

�.6

9),

exce

ptfo

rch

alle

nge

(��

.54)

.A

step

wis

ere

gres

sion

reve

aled

that

tota

lm

enta

lto

ughn

ess

(��

.12)

expl

aine

dan

addi

tiona

l3%

vari

ance

inco

ping

effe

ctiv

enes

sth

anco

ping

self

-ef

ficac

y(6

%,

��

.18)

.A

seco

ndst

epw

ise

regr

essi

onw

ithth

esi

xm

enta

lto

ughn

ess

subs

cale

sre

veal

edco

mm

itmen

t(�

�.2

0)as

the

only

sign

ifica

ntfa

cet,

whi

chex

plai

ned

anad

ditio

nal

5%va

rian

cein

copi

ngef

fect

iven

ess

than

copi

ngse

lf-e

ffica

cy(6

%,

��

.18)

.

200 GUCCIARDI, HANTON, AND MALLETT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Tab

le1

(con

tinu

ed)

Stud

yPa

rtic

ipan

tsM

etho

dsPr

imar

yfin

ding

s

Nic

holls

,Po

lman

,L

evy,

&B

ackh

ouse

(200

8)67

7at

hlet

es(4

54m

ales

)ag

ed15

to58

year

s(M

age

�22

.66)

from

inte

rnat

iona

lto

begi

nner

leve

ls

Cro

ss-s

ectio

nal

surv

eyco

ntai

ning

the

Cop

ing

Inve

ntor

yfo

rC

ompe

titiv

eSp

ort

(Gau

drea

u&

Blo

ndin

,20

02),

Lif

eO

rien

tatio

nT

est

(Sch

eier

&C

arve

r,19

85),

and

the

MT

Q48

Inte

rnal

relia

bilit

yes

timat

efo

rto

tal

men

tal

toug

hnes

sw

asad

equa

te(�

�.8

7),

with

subs

cale

relia

bilit

ies

rang

ing

from

.58

to.7

1.M

enta

lto

ughn

ess

was

posi

tivel

yre

late

dw

ithpr

oble

mor

appr

oach

copi

ngst

rate

gies

(rra

nge:

�.1

5to

.32)

and

nega

tivel

yre

late

dw

ithav

oida

nce

copi

ngst

rate

gies

(rra

nge:

�.2

8to

.03)

,al

thou

ghex

cept

ions

inth

ere

latio

nshi

pdi

rect

ion

did

exis

t.M

enta

lto

ughn

ess

was

posi

tivel

yre

late

dto

optim

ism

(rra

nge:

.08�

.56)

and

nega

tivel

yre

late

dto

pess

imis

m(r

rang

e:�

.16

to�

.49)

.N

icho

lls,

Polm

an,

Lev

y,&

Bac

khou

se(2

009)

677

athl

etes

(454

mal

es)

aged

15to

58ye

ars

(Mage

�22

.66)

from

inte

rnat

iona

lto

begi

nner

leve

ls

Cro

ss-s

ectio

nal

surv

eyco

ntai

ning

the

MT

Q48

Inte

rnal

relia

bilit

yes

timat

efo

rto

tal

men

tal

toug

hnes

sw

asad

equa

te(�

�.8

7),

with

subs

cale

relia

bilit

ies

rang

ing

from

.58

to.7

1.M

ensc

ored

sign

ifica

ntly

high

er(p

�.0

5)th

anw

omen

onch

alle

nge,

emot

iona

lco

ntro

l,lif

eco

ntro

l,an

dco

nfide

nce

inab

ilitie

s.N

osi

gnifi

cant

diff

eren

ces

inm

enta

lto

ughn

ess

acro

ssac

hiev

emen

tle

vels

and

spor

tty

pe(t

eam

/indi

vidu

al,

cont

act/n

onco

ntac

t)w

ere

foun

d.A

gean

dye

ars

ofpl

ayin

gex

peri

ence

cont

ribu

ted

toth

eam

ount

ofva

rian

ceex

plai

ned

into

tal

men

tal

toug

hnes

s(3

%,

��

.18;

3%,

��

.17)

,ch

alle

nge

(2%

,�

�.1

4;3%

,�

�.1

8),

com

mitm

ent

(4%

,�

�.2

1;3%

,�

�.1

7),

and

life

cont

rol

(2%

,�

�.1

6;3%

,�

�.1

8).

Not

e.M

TQ

48�

Men

tal

Tou

ghne

ssQ

uest

ionn

aire

48;

VO

2m

ax�

max

imal

oxyg

enco

nsum

ptio

n;M

TQ

18�

Men

tal

Tou

ghne

ssQ

uest

ionn

aire

18.

201MEASURING MENTAL TOUGHNESS

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

model of mental toughness, mentally tough in-dividuals (a) view negative experiences (e.g.,stress and anxiety) as a challenge that they canovercome but also a natural and essential cata-lyst for growth and development; (b) believethat they are influential in dealing with andcontrolling negative life experiences; (c) aredeeply involved in what they are doing andcommitted to achieving their goals; and (d) areconfident in their ability to deal with and over-come negative life experiences. The MTQ 48 isdesigned to measure the 4Cs’ conceptualizationof mental toughness (Clough et al., 2002).

Although having a conceptual foundation forits development represents a key strength of theMTQ 48, psychometric examinations of the ro-bustness of its factor structure are required tostatistically substantiate the hypothesizedmodel. Construct validation is an ongoing pro-cess (e.g., Marsh, 1997), and central to thisprogress is the adoption of methodologies thatdemonstrate rigor, reliability, and validity. Fac-torial validity, in particular, has implicationsboth for practice (e.g., how an instrument isscored, defining subscales based on item con-tent) and theory (e.g., dimensionality, hierarchi-cal representation), and it is important to ascer-tain this type of validity before other forms,such as predictive and concurrent validity (Gig-nac, 2009; Marsh, Martin, & Jackson, 2010). Asthe measure of choice for most mental tough-ness researchers and practitioners, it is impor-tant that we have confidence in the psychomet-ric integrity of this inventory.

It appears that the MTQ 48 has been uncrit-ically adopted as a preferred tool for mentaltoughness measurement before a thorough ex-amination of its dimensionality has been under-taken. Indeed, several of the available studiesthat have employed the MTQ 48 as a measure ofmental toughness have included samples sizesin excess of 400 (e.g., Kaiseler, Polman, &Nicholls, 2009; Marchant et al., 2009; Nicholls,Polman, Levy, & Backhouse, 2008, 2009), yetwith one exception (Horsburgh, Schermer, Ve-selka, & Vernon, 2009) its factor structure has notbeen rigorously examined. As detailed in Table 1,internal reliability estimates are sometimes ig-nored, and when reported reveal inadequacieswith several MTQ 48 subscales, according to rec-ommended minimum levels for exploratory re-search (i.e., Cronbach’s alpha � .70; Nunnally &Bernstein, 1994). The lack of information about

the psychometric procedures employed to de-velop the MTQ 48 becomes even more prob-lematic when one considers the differing factorstructures reported in recent research, namely,four (e.g., Clough et al., 2002; Veselka,Schermer, Petrides, & Vernon, 2009), six (e.g.,Crust & Azadi, 2009, 2010; Nicholls, Levy,Polman, & Crust, 2011), and nine factor models(e.g., Horsburgh et al., 2009). Regardless of thetype of measurement model adopted, only onestudy to date has reported an examination of thefactorial validity of the MTQ 48. Unfortunately,however, Horsburgh et al. did not report anyempirical data (i.e., fit indices, parameter esti-mates) to support their conclusion about thesuperiority of the correlated, four factor modelwhen compared with a unidimensional modelwith a sample of the general population. Theinclusion of such empirical data and a descrip-tion of the criteria on which the adequacy of themodel�data fit is evaluated are importantpieces of information to support the veracity ofone’s conclusions.

Reexaminations of the factor structure ofmeasurement instruments are an important con-sideration for the robustness of theoretical mod-els, especially when examining multidimen-sional constructs across different populations tothose employed in the initial validation of aquestionnaire. Numerous examples exist bothwithin the mental toughness field (e.g., Guc-ciardi, 2009) and beyond (e.g., Lane, Harwood,Terry, & Karageorghis, 2004; Martens & Web-ber, 2002) in which measurement instrumentsthat were developed within one context or sam-ple failed to generalize to others. Despite itsongoing influence on research and practice asthe most frequently adopted measure for mostresearchers and practitioners, the MTQ 48 hasyet to be subjected to a rigorous psychometricexamination. According to Marsh, Martin, etal., “To move too quickly to potentially super-ficial between-construct research is to risk with-in-construct problems that characterize manypsychological measures” (2010, p. 464). Re-searchers (e.g., Connaughton & Hanton, 2009;Gucciardi et al., 2011) have questioned the use-fulness of the MTQ 48 as a measure of mentaltoughness according to both empirical (i.e., de-tailed information on the scale construction pro-cess and factorial validity is unavailable; inad-equate internal reliability) and conceptual con-siderations (i.e., 75% of the underlying model is

202 GUCCIARDI, HANTON, AND MALLETT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

hardiness theory, little information on the ratio-nale for the underlying theoretical model), al-though researchers have reported evidence tosupport its concurrent validity (see Table 1).

In response to these untested concerns, theprimary aim of this study was to examine thefactorial validity of the MTQ 48 in two broadachievement contexts. Although four (e.g.,Clough et al., 2002; Veselka et al., 2009), six(e.g., Crust & Azadi, 2009, 2010), and ninefactor models (e.g., Horsburgh et al., 2009) ofthe MTQ 48 have been employed in previousresearch, our primary focus was on the fourfactor model, which is consistent with the orig-inal conceptualization of mental toughness for-warded by Clough and his colleagues (i.e., 4Csmodel). An athlete sample was chosen becausethe majority of published research has used theMTQ 48 as a measure of mental toughness insport contexts (see Table 1). It was also deemedimportant to test the veracity of the hypothe-sized model in workplace contexts because har-diness theory emerged primarily from a 12-yearlongitudinal study of stress reactions amongmanagers at a telephone company (cf. Maddi &Kobasz, 1984). Because this study is the first toexamine to factorial validity of the MTQ 48 ina sample of athletes or workplace performers, anull hypothesis was adopted; that is, it washypothesized that the original, four-factormodel would evidence an adequate level of fitwith the data.

Because there is a hypothesized conceptualmodel underlying the MTQ 48 (Clough et al.,2002), it may be argued that state-of-the-artanalytical techniques such as confirmatory fac-tor analysis (CFA; Hagger & Chatzisarantis,2009, p. 513), which test an a priori structureagainst the data, should be employed to exam-ine the robustness of its measurement model(i.e., factorial validity). Reflecting the hypothe-sis that a specific number of factors are influ-enced by certain indicators, each item is al-lowed to load on one factor only (i.e., no crossloadings) and all nontarget loadings are con-strained to be zero in this highly restrictive dataanalytical approach (Thompson, 2004). To takea strictly confirmatory approach to analysis,therefore, it is important that psychometric in-struments are developed from a clearly articu-lated theoretical model and have simple mea-surement structure (Asparouhov & Muthen,2009). Owing to the limited published informa-

tion on the rationale for the conceptual modeland empirical evidence on its psychometricproperties (e.g., model fit, parameter estimates),CFA may not be suitably justifiable as an ana-lytical approach for the assessment of theMTQ 48.

When a strictly confirmatory approach toanalysis is not well suited, exploratory struc-tural equation modeling (ESEM) offers an al-ternative method for evaluating the psychomet-ric integrity of measurement instruments whenmodel�data fit is of primary interest (Marsh etal., 2009). Recently introduced to the academiccommunity, ESEM is a novel methodologicalextension of traditional factor analyses in whichthe strengths of both CFA and exploratory fac-tor analysis (EFA) are integrated within a struc-tural equation modeling framework (Asp-arouhov & Muthen, 2009). Specifically, ESEMavoids the strict requirements of CFA (i.e., onlycertain items load onto certain factors, nontargetloadings are constrained to be zero) by allowingall item indicators to be directly influenced byall common factors as in EFA, while at the sametime providing access to robust indicators ofmodel adequacy (e.g., parameter estimates,goodness-of-fit statistics, standard errors) thatare typically associated with CFA. When com-pared with CFA, ESEM is less likely to distort(i.e., inflate and bias) factors and structural re-lations and thereby improve the likelihood ofadequate model�data fit because it does notinappropriately impose nontarget loadings to beconstrained to zero (Asparouhov & Muthen,2009; Marsh et al., 2009). An emerging body ofresearch has supported the superiority of ESEMfor the examination of psychological constructssuch as the “Big Five” (e.g., Marsh, Ludtke, etal., 2010), student teacher evaluations (Marsh etal., 2009), motivation and engagement (Marsh,Liem, Martin, Morin, & Nagengast, 2011), bul-lying and victimization (Marsh, Nagengast, etal., 2011), physical self (Morin & Maıano,2011), and coaching efficacy (Myers, Chase,Pierce, & Martin, 2011). As a secondary objec-tive, therefore, we also sought to examine theutility of ESEM in offering a viable alternativeto assessing model�data fit of the MTQ 48(Clough et al., 2002) than the restrictive CFAapproach. We hypothesized that ESEM wouldresult in more favorable indices of model fit andmeasurement properties (e.g., factor correla-tions) than CFA.

203MEASURING MENTAL TOUGHNESS

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Methods

Participants

Participants from two achievement contextswere recruited to participate in the currentstudy.1 A total of 686 athletes (men � 354,women � 328, missing � 4) aged 17�47 years(M � 19.79, SD � 3.27) participated. Thesports represented included a variety of team(team � 466; e.g., basketball, hockey, netball,rugby) and individual sports (individual � 209;e.g., tennis, athletics, triathlon); 11 participantsdid not report their sport. At the time of com-pleting the questionnaire package, these athleteshad been competing in their sport for betweenone and 35 years (M � 9.11, SD � 4.41).Athletes’ highest level of participation includedinternational (9%), national (26%), state orcounty (25%), or district or local (39%) com-petition; a small portion (1%) did not reporttheir playing level.

A total of 639 full-time employees (men �369, women � 269, missing � 1) aged 20�65years (M � 30.02, SD � 8.69) participated.They were employed primarily in the informa-tion technology and communications (30%), ed-ucation (16%), business or finance (16%),health care (11%), academia or research (10%),and consumer or retail (10%) industries. At thetime of completing the survey, participants hadbeen engaged in their current role (M � 4.77,SD � 5.34) and industry (M � 6.31,SD � 6.26) for between 0 and 36 years.Participants’ highest level of education in-cluded an associate degree (n � 36), bache-lor’s degree (n � 426), master’s degree (n �152), and doctorate (n � 25).

Measure

Mental Toughness Questionnaire 48.The MTQ 48 (Clough et al., 2002) is a 48-itemscale that was developed to assess the 4Csmodel of mental toughness, comprising fourkey dimensions or subscales as defined by thedevelopers of the MTQ 48, namely, Control(e.g., “I generally feel in control”), Commit-ment (e.g., “I generally try to give 100%”),Challenge (e.g., “I usually enjoy a challenge”),and Confidence (e.g., “I am generally confidentin my own abilities”). Participants rated them-selves on a scale from 1 (strongly disagree) to 5

(strongly agree). There is some evidence for itsfactorial validity in nonsports contexts (e.g.,Horsburgh et al., 2009), and satisfactory inter-nal reliability and construct validity within sportcontexts (see Table 1).

Procedures

Participants completed an online survey con-taining the aforementioned measure at a timeand place most convenient to them. Studentathletes enrolled in undergraduate courses inpsychology and sports-related subjects at uni-versities in Australia and the United Kingdomwere invited to participate; Australian studentswere offered course credit for their participa-tion. For Australian students, study invitationswere distributed using an established researchparticipation scheme or during the first week ofclasses during a lecture. Email invitations fromcourse coordinators were distributed to studentsin the United Kingdom. The workplace samplewas recruited using an online survey panel (So-cialSci) that links researchers with participantswho have agreed to complete surveys for aca-demic research. Prior to completing the pack-age, all participants were assured of confidenti-ality and anonymity in responses, and informedof their right to withdraw participation at anytime before obtaining their consent. Ethics ap-proval was obtained from both coordinating in-stitutions prior to the commencement of datacollection.

Data Analyses

Our data analyses involved two stages. In thefirst stage, we screened the data for missingresponses in SPSS, Version 18.0 (SPSS Inc.). Inthe second stage, we examined the degree ofmodel�data fit of the MTQ 48 (Clough et al.,2002) using both CFA and ESEM.2 Both factorvalidity analyses were performed withMplus 6.12 (Muthen & Muthen, 2010). Weemployed the robust maximum likelihood esti-mator (MLR), which produces standard errors

1 Following reviewer recommendations, we collected ad-ditional data in an attempt to alleviate limitations associatedwith inadequate statistical power (i.e., increase our athletesample size) and the robustness of the findings (i.e., acrosstwo achievement contexts).

2 We thank an anonymous reviewer for the recommen-dation of ESEM.

204 GUCCIARDI, HANTON, AND MALLETT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

and tests of fit that are robust in relation tononnormality of observations and the use ofcategorical variables when there are at least fouror more response categories (e.g., Beauducel &Herzberg, 2006; Dolan, 1994; Muthen & Ka-plan, 1985). With regard to CFA, it was hypoth-esized that responses to the MTQ 48 would beexplained by four correlated factors in whicheach item would load on one factor only (i.e.,nonzero loading on its intended factor, withzero loadings on all other factors) and errorterms would be uncorrelated. With regard toESEM, it was hypothesized that responses tothe MTQ 48 would be explained by four corre-lated factors in which each item would show astatistically significant loading on its intendedfactor as well as a small, nonsignificant loadingon the other factors. As recommended (e.g.,Marsh et al., 2009; Marsh, Ludtke, et al., 2010),we used an oblique geomin rotation3 (the de-fault in Mplus) with an epsilon value of 0.5 forESEM. Default constraints that are built into theMplus estimation process to achieve modelidentification were employed (for further de-tails, see Asparouhov & Muthen, 2009; Marsh,Liem, et al., 2011).

In addition to the chi-square goodness-of-fitstatistic, several other traditional criteria (com-parative fit index [CFI] and Tucker�Lewis in-dex [TLI] � .90; root mean square error ofapproximation [RMSEA] scores and standard-ized root-mean-square residual [SRMR] � .08;Browne & Cudeck, 1992) were adopted as in-dicators of adequate model�data fit with Huand Bentler’s (1999) criteria (CFI and TLI�.95, and RMSEA and SRMR scores � .06) asevidence of good fit. Collectively, these indicesprovide a more conservative and comprehen-sive evaluation of model fit than any singleindex alone. However, caution has been urgedin the strict adherence to such recommendationsin psychometric evaluations of measures com-prising 50 or more items loading onto five ormore factors (e.g., Marsh, Hau, & Grayson,2005), and their relevance to ESEM is not en-tirely clear (Marsh et al., 2009; Marsh, Ludtke,et al., 2010). Thus, we also examined standard-ized solutions to evaluate the significance andstrength of parameter estimates. Standardizedfactor loadings were interpreted using Comreyand Lee’s (1992) recommendations (i.e., �.71 � excellent, � .63 � very good, � .55 �good, � .45 � fair, �. 32 � poor). Finally, a

composite reliability coefficient (Raykov, 1997)was calculated to estimate the level of internalreliability for each factor, because there arelimitations associated with Cronbach’s equation(Bentler, 2009; Sijtsma, 2009). With regard tothe ESEM findings, items that evidenced a sta-tistically significant loading on a latent factor atthe p � .01 level were included in the assess-ment of composite reliability.

Results

Preliminary Analyses

The amount of missing data was negligible(athletes � .02%, workplace � .01%) andtherefore were replaced using the expectation-maximization method prior to the factorial va-lidity analyses (Graham, 2009).

Factorial Validity Analyses

Athlete sample. The CFA revealed thatthe hypothesized correlated four factor modelof the MTQ 48 was unsatisfactory, according tothe multiple indices of model fit, �2(1074) �5511.88, p � .001, CFI � .487, TLI � .462,SRMR � .104, RMSEA � .078, 90% confi-dence interval [CI] [.076, .080]. In addition tothe poor model fit, the solution was improper, asindicated by a factor correlation between theControl and Confidence dimensions that ex-ceeded 1.0. The factor loadings, and factor cor-relations and composite reliabilities are detailedin Tables 2 and 3, respectively. All factors dem-onstrated an adequate level of internal reliabil-ity (i.e., composite reliability � .70), with theexception of Control. Collectively, CFA modelfit indices and parameters estimates did not sup-port the hypothesized correlated four factormodel of the MTQ 48 with the athlete sample.

The ESEM revealed that the hypothesized cor-

3 Although a geomin rotation with an epsilon value of 0.5is consistent with previous applications of ESEM (e.g.,Marsh et al., 2009; Marsh, Nagengast et al., 2011), anequally strong case could have been made for a targetrotation in which the analyst freely estimates a priori factorloadings and specifies cross-loadings with a target value ofzero (for further details, see Asparouhov & Muthen, 2009;Browne, 2001). As recommended (Morin & Maıano, 2011),therefore, we also explored solutions based on this alterna-tive rotation procedure. The results of these analyses can beobtained from coauthor Gucciardi.

205MEASURING MENTAL TOUGHNESS

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Table 2Standardized Parameter Estimates for the CFA and ESEM of the MTQ 48 With the Athlete Sample(n � 686)

Factor 1(Challengesubscale)

Factor 2(Commitment

subscale)Factor 3

(Control subscale)

Factor 4(Confidence

subscale)

ESEM (R2) CFA (R2)ESEM CFA ESEM CFA ESEM CFA ESEM CFA

Mt4 .23� .55�� �.04 .18 .33� .29�� .31��

Mt6 .07 .12� .31�� �.23� .24�� .24�� .02Mt14 .13� .12� .31�� �.23�� .17� .21�� .01Mt23 .47�� .69�� �.03 .19 .22 .43�� .47��

Mt30 .37�� .60�� �.18 .24 .14 .35�� .37��

Mt40 .29� .45�� �.03 .14 .06 .15�� .20��

Mt44 .20 .59�� �.06 .17 .43�� .34�� .35��

Mt48 .44�� .67�� �.22�� .12 .30 .45�� .45��

Mt1 .30�� �.03 .38�� .00 .20 .16� .14��

Mt7 .22� �.19�� .51�� .15� .37�� .30�� .26��

Mt11 �.17�� .46�� .32�� .05 .34� .40�� .10�

Mt19 .19� �.28� .47�� .21� .33�� .32�� .22��

Mt22 �.13� .14 .39�� �.07 .51�� .30�� .15��

Mt25 .12 �.12 .58�� .12 .47�� .30�� .34��

Mt29 .11 .13 .50�� �.07 .46�� .29�� .25��

Mt35 �.29�� .24 .26�� .02 .44�� .31�� .07�

Mt39 .07 �.17 .57�� .28�� .48�� .39�� .33��

Mt42 .03 .29 .39�� �.05 .50�� .41�� .24��

Mt47 .04 .25� .36�� .06 .27�� .19�� .13��

Mt2 .58�� .04 .08 .58�� .05 .41�� .34��

Mt5 .11 �.13 .50�� .49�� .13 .38�� .24��

Mt9 �.20�� .31�� �.12� �.07 .23 .22�� .01Mt12 .60�� .14�� .17�� .66�� .00 .49�� .44��

Mt15 �.02 .45�� �.07 .15�� .25 .33�� .02Mt33 .10 .33�� �.17 .04 �.01 .14�� .00Mt41 .21�� .41�� .11 .46�� .23 .38�� .21��

Mt21 .14 .40�� .00 .16�� �.07 .17� .03Mt26 .00 .07 �.58�� �.30�� .17�� .35�� .09��

Mt27 .10 .53�� �.09 .13�� �.03 .29�� .23��

Mt31 .35� �.07 .12 .48�� .21 .26�� .02Mt34 .08 �.33�� �.35�� �.16�� .17 .19 .02Mt37 �.20 .26� .00 .10 .41�� .28�� .01Mt45 .39�� .13 .23� .55�� .06 .31�� .31��

Mt3 .59�� .09 .20� .00 .67�� .49�� .45��

Mt8 .44�� .10� .36�� .07 .70�� .49�� .48��

Mt10 �.07 .50�� �.07 .07 .05 .28�� .00Mt13 .68�� .03 �.02 .00 .55�� .45�� .31��

Mt16 .55�� .09 .07 .10 .59�� .40�� .35��

Mt18 .20�� .49�� .00 .19� .39�� .39�� .15��

Mt24 .17� .15 .16� �.19 .20�� .09 .04�

Mt32 .15 .51�� �.01 .10 .29�� .33�� .08��

Mt36 .09 .54�� .04 .06 .25�� .32�� .06�

Mt17 �.03 .08 .67�� .01 .45�� .43�� .20��

Mt20 .20�� �.14 .45�� .17� .54�� .40�� .29��

Mt28 �.02 .38�� .20�� .04 .23�� .18�� .05Mt38 .08 .10 .54�� .05 .51�� .35�� .26��

Mt43 �.08 .03 .61�� .02 .38�� .35�� .14��

Mt46 �.09 .35�� .32�� .12 .28�� .24�� .08��

Note. CFA � confirmatory factor analysis; ESEM � exploratory structural equation modeling; MTQ 48 � MentalToughness Questionnaire 48; Mt � item number of the MTQ 48.� Statistically significant parameter estimates at p � .05. �� statistically significant parameter estimates at p � .01.

206 GUCCIARDI, HANTON, AND MALLETT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

related four factor model of the MTQ 48 wasunsatisfactory, according to the multiple indices ofmodel fit, �2(942) � 2970.25, p � .001, CFI �.766, TLI � .719, SRMR � .045, RMSEA �.056, 90% CI [.054, .058]. The factor loadings,and factor correlations and composite reliabili-ties are detailed in Tables 2 and 3, respectively.All factors demonstrated an adequate level ofinternal reliability (i.e., composite reliability �.70), with the exception of Factor 3. Inspectionof the factor loadings revealed a large degree ofinconsistency between the hypothesized struc-ture, according to the correlated four factormodel proposed by Clough et al. (2002), and thecurrent data. Collectively, ESEM model fit in-dices and parameters estimates did not supportthe hypothesized correlated four factor model ofthe MTQ 48 with the athlete sample.

Workplace sample. The CFA revealedthat the hypothesized correlated four factormodel of the MTQ 48 was unsatisfactory, ac-cording to the multiple indices of model fit,�2(1074) � 4928.95, p � .001, CFI � .521,TLI � .497, SRMR � .093, RMSEA � .075,90% CI [.073, .077]. In addition to the poormodel fit, the solution was improper (i.e., notpositive definite), as indicated by a factor cor-relation between the Control and Confidencedimensions that approached 1.0. The factor cor-relations and composite reliabilities, and factorloadings are detailed in Tables 3 and 4, respec-

tively. Collectively, CFA model fit indices andparameters estimates did not support for hy-pothesized correlated four factor model of theMTQ 48 with the workplace sample.

The ESEM revealed that the hypothesized cor-related four factor model of the MTQ 48 wasunsatisfactory, according to the multiple indices ofmodel fit, �2(942) � 2744.20, p � .001, CFI �.776, TLI � .732, SRMR � .045, RMSEA �.055, 90% CI [.052, .057]. The factor correlationsand composite reliabilities, and factor loadingsare detailed in Tables 3 and 4, respectively. Allfactors demonstrated an adequate level of inter-nal reliability (i.e., composite reliability � .70),with the exception of Factor 1. Collectively,ESEM model fit indices and parameters esti-mates did not support the hypothesized corre-lated four factor model of the MTQ 48 with theworkplace sample.4

4 We also tested a variety of other MTQ 48 models thathave appeared in the literature, such as the six (e.g., Crust &Azadi, 2009, 2010) and nine factor models (e.g., Horsburghet al., 2009). Both CFA and ESEM revealed that thesemodels were unsatisfactory, according to the multiple cri-teria of model fit. The results of these analyses can beobtained from coauthor Gucciardi.

Table 3Latent Factor Correlations and Composite Reliability Estimates for the CFA and ESEM of the MTQ 48

ESEM CFA

Factor 1 Factor 2 Factor 3 Factor 4

Factor 1(Challengesubscale)

Factor 2(Commitment

subscale)

Factor 3(Controlsubscale)

Factor 4(Confidence

subscale)

Athlete sample (n � 686)

Factor 1 (.73) (.75)Factor 2 .02 (.72) .68�� (.74)Factor 3 .38�� �.06 (.50) .85�� .67�� (.50)Factor 4 .28�� .25�� .20�� (.76) .81�� .69�� 1.01�� (.77)

Workplace sample (n � 639)

Factor 1 (.39) (.71)Factor 2 .23�� (.79) .67�� (.78)Factor 3 .17�� .27�� (.74) .73�� .90�� (.67)Factor 4 .38�� .13�� .10� (.71) .78�� .76�� .96�� (.79)

Note. Composite reliability estimates are enclosed in parentheses. CFA � confirmatory factor analysis; ESEM �exploratory structural equation modeling; MTQ 48 � Mental Toughness Questionnaire 48.� Statistically significant parameter estimates at p � .05. �� statistically significant parameter estimates at p � .01.

207MEASURING MENTAL TOUGHNESS

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Table 4Standardized Parameter Estimates for the CFA and ESEM of the MTQ 48 With the Workplace Sample(n � 639)

Factor 1(Challengesubscale)

Factor 2(Commitment

subscale)Factor 3

(Control subscale)

Factor 4(Confidence

subscale)

ESEM (R2) CFA (R2)ESEM CFA ESEM CFA ESEM CFA ESEM CFA

Mt4 .47� .55�� �.05 .06 .20 .34�� .30��

Mt6 .06 .37�� .35�� .12� .08 .20�� .14��

Mt14 �.07 .26�� .18 .39�� .01 .21�� .07��

Mt23 .38 .57�� .26�� �.04 .15 .33�� .32��

Mt30 .28 .54�� .01 .10 .29 .25�� .39��

Mt40 .18 .34�� .12 �.20�� .21 .14�� .11��

Mt44 .40 .65�� �.07 .08 .37 .42�� .42��

Mt48 .47 .61�� .06 .01 .19 .35�� .38��

Mt1 .54 �.07 .42�� .10 .06 .33 .18��

Mt7 .36 �.08 .40�� .19�� .16 .24�� .16��

Mt11 .01 .37�� .53�� .34�� .06 .34�� .28��

Mt19 .39� �.16� .48�� .32�� .14 .33�� .23��

Mt22 .20 .33�� .54�� .25�� �.08 .29�� .29��

Mt25 .49 �.02 .43�� .07 .06 .27 .18��

Mt29 .18 .01 .61�� .63�� .01 .47�� .38��

Mt35 .10 .36�� .46�� .27�� �.15 .28�� .21��

Mt39 .34�� .10 .46�� .07 .19 .26�� .21��

Mt42 .15 .12 .63�� .62�� �.05 .50�� .39��

Mt47 .01 .05 .48�� .53�� .10 .32�� .23��

Mt2 .36 .17 �.01 .49�� .22 .30�� .24��

Mt5 .24�� �.06 .05 .37�� .38 .27�� .14��

Mt9 .10 .05 .61�� .47�� �.05 .41�� .22��

Mt12 .30 .21�� �.04 .44�� .21 .26�� .20��

Mt15 �.04 .26�� .46�� .42�� �.02 .33�� .17��

Mt33 �.08 .06 .53�� .33�� .02 .29�� .11��

Mt41 .16 .08 .57�� .54�� �.01 .42�� .29��

Mt21 �.04 .47�� .16�� .36�� �.04 .28�� .13��

Mt26 .30 .08 .10 .01 �.55 .29 .00Mt27 �.07 .61�� .01 .35�� �.02 .36�� .12��

Mt31 .43 .33�� �.08 .45�� .03 .35�� .20��

Mt34 .28 .02 �.32�� �.18�� �.30 .20� .03�

Mt37 .12 .39�� .21�� .40�� �.13 .27�� .16��

Mt45 .19 .36�� �.11� .45�� .31 .35�� .21��

Mt3 .36 .03 .24�� .21 .58�� .34�� .33��

Mt8 .39�� .10 .07 .30�� .58�� .39�� .34��

Mt10 �.06 .50�� .02 .02 .34�� .24�� .11��

Mt13 .33 .22� �.06 .13 .46�� .24�� .21��

Mt16 .45 .28�� �.12� .08 .51�� .36�� .26��

Mt18 .19 .44�� .16�� .02 .54�� .35�� .29��

Mt24 .06 .52�� �.38�� .11 .26�� .34�� .07��

Mt32 �.07 .33�� .34�� �.07 .33�� .27�� .11��

Mt36 �.06 .62�� .13 .08 .48�� .45�� .23��

Mt17 �.04 .04 �.03 .67�� .40�� .44 .16��

Mt20 .26 �.14�� .11�� .46�� .44�� .39�� .19��

Mt28 �.03 .31�� .27�� .23� .50�� .29�� .24��

Mt38 .12 .17�� �.07 .52�� .48�� .37�� .23��

Mt43 �.06 �.04 �.05 .62�� .28�� .36�� .08��

Mt46 �.15 .10 .57�� .38�� .50�� .51�� .25��

Note. CFA � confirmatory factor analysis; ESEM � exploratory structural equation modeling; MTQ 48 � MentalToughness Questionnaire 48; Mt � item number of the MTQ 48.� Statistically significant parameter estimates at p � .05. �� statistically significant parameter estimates at p � .01.

208 GUCCIARDI, HANTON, AND MALLETT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Discussion

The MTQ 48 (Clough et al., 2002) has beenthe most popular measure for many researchersinterested in examining mental toughness overthe past decade. Despite the increasing popular-ity of the MTQ 48 as a measure of mentaltoughness, its factor structure has not yet beensubjected to a rigorous psychometric analysis inan athletic or a workplace sample. Both CFAand ESEM failed to support the hypothesizedcorrelated four factor model (i.e., Control,Commitment, Challenge, and Confidence) ofmental toughness in two independent samples,thereby revealing incongruence between theMTQ 48 and its measurement of the underlyingtheoretical model.

Although evidence of concurrent validity ofthe MTQ 48 is amassing (see Table 1), evidenceof nomological validity relies on a sound inter-nal structure (i.e., factorial validity, internal re-liability; Gignac, 2009). Some researchers havegone so far to say that:

. . . the representational effectiveness [correspondencebetween indicators and hypothesized constructs] of atest is in fact far more important to a scale’s scientificvalue [than predictive validity]. . . If the meaning of thescores on a scale is unclear, the accuracy of any infer-ences about constructs made on the basis of that scaleis in doubt. (McGrath, 2005, p. 113)

Additionally, correlational analyses involv-ing observed variables have been the primarymeans by which researchers have sought toestablish the construct validity of the MTQ 48.However, limitations associated with these tra-ditional analyses mean they are suboptimal forexaminations of theoretical models includingmultiple latent constructs. For example, rela-tionships between multiple antecedent (e.g.,stress), intervening (e.g., mental toughness),and outcome (e.g., behavior, performance) vari-ables cannot be simultaneously estimated, andparameter estimates do not take measurementerror into consideration (i.e., observed variablesare assumed to be measured without error).Structural equation modeling is ideally suitedfor such substantive inquiries, yet they requirepsychometrically sound instruments (Byrne,2010).

The utility of an instrument for research (e.g.,validity of conclusions, scoring, defining sub-scales), theory (e.g., dimensionality, hierarchi-cal representation), and practice (e.g., appropri-

ateness for different populations) is under-pinned by the degree to which that measurevalidly captures the construct in its intendedmanner (Gignac, 2009; Marsh, Martin, et al.,2010; McGrath, 2005). Preliminary researchwith individuals from the general populationhas supported the adequacy of hypothesizedfour factor model of the MTQ 48 (Horsburgh etal., 2009). Despite these initial findings fromnonsport contexts, researchers (e.g., Con-naughton & Hanton, 2009; Gucciardi et al.,2011) have expressed both empirical (i.e., de-tailed information on the scale constructionprocess and factorial validity is unavailable;inadequate internal reliability) and conceptualconcerns (i.e., 75% of the underlying model ishardiness theory, little information on the ratio-nale for the underlying theoretical model) withthe MTQ 48 as a measure of mental toughness.Aligned with these concerns, but contrary toprevious factor analytical research (Horsburghet al., 2009), empirical evidence detailed hereraises questions about the viability of the cor-related four factor model hypothesized to un-derpin the MTQ 48. Although there is someevidence to support the equivalency of the fac-tor structure or latent mean structures of psy-chometric tools collected via online or tradi-tional paper and pencil methods (e.g., Lonsdale,Hodge, & Rose, 2006), the difference in datacollection methods between the studies (i.e.,online here vs. hardcopy with Horsburgh et al.,2009) may provide an explanation for the dis-crepancy in findings.

Even when a psychological instrument has awell-defined EFA structure, its psychometricintegrity sometimes fails to replicate within ahighly restrictive CFA framework (Marsh et al.,2009). The CFA findings of the current studyare consistent with this general observation, be-cause EFA has been employed as the focalanalysis in the original development of theMTQ 48 (Clough et al., 2002) and its onlysubsequent psychometric evaluation (Hors-burgh et al., 2009). ESEM has recently beenproposed as a flexible alternative when there isinsufficient theory to guide a strictly confirma-tory approach (e.g., Asparouhov & Muthen,2009; Marsh et al., 2009; Myers, Chase et al.,2011). Consistent with an emerging body ofresearch from both nonsport (e.g., Marsh et al.,2009; Marsh, Nagengast et al., 2011) and sportcontexts (e.g., Morin & Maıano, 2011; Myers,

209MEASURING MENTAL TOUGHNESS

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Chase et al., 2011), the current study supportedthe superiority of ESEM when compared withCFA in terms of model fit indices and parame-ters estimates. Nevertheless, ESEM failed tosupport the hypothesized correlated four factormodel of mental toughness proposed to encap-sulate the MTQ 48 (Clough et al., 2002) in boththe athlete and the workplace samples, as evi-denced by a significant degree of model misfitand inconsistent parameter estimates (i.e., non-significant loadings for hypothesized indicator-�factor relationships, significant item cross-loadings on unintended factors, statistically sig-nificant indicator�factor associations in anonhypothesized direction) in both the athleteand the workplace samples.

Conceptual and empirical limitations with thecontent of several items on the MTQ 48 mayhave contributed to the empirical problems ob-served in the current study. With regard to CFA,empirical weaknesses were observed at the in-dividual item level. For example, only 14 (29%)of the 48 items evidenced very good to excellentloadings on their hypothesized factor (i.e., �.55) in both the athlete and the workplace sam-ples. With regard to the athlete sample, perhapsmost troubling is that 17 (35%) of the 48 itemswere very poor, in that they did not load morethan .32 on their hypothesized factor. ESEMalso identified a number of problematic items inboth samples (e.g., nonsignificant loadings onintended factors according to hypothesizedmodel, statistically significant associations in anonhypothesized direction). Problems at theitem level might indicate issues with partici-pants’ comprehension of the meaning of an itemin the context of its hypothesized subscale.From a conceptual standpoint, the Confidencesubscale, for example, contains several itemswhose content is not entirely consistent with a“high sense of self-belief” (Clough et al., 2002,p. 38). Items such as “I generally feel that I ama worthwhile person” and “At times I feel com-pletely useless” appear to be capturing aspectsof one’s self-esteem (i.e., self-evaluation or ap-praisal of one’s own worth; Harter, 1999),whereas other items such as “However badthings are, I usually feel they will work outpositively in the end” and “I generally look onthe bright side of life” are capturing one’s op-timistic outlook on life (i.e., tendency to per-ceive, react, and adapt to challenges in one’s lifein a positive manner; Scheier & Carver, 1985).

Collectively, our empirical data and conceptualconcerns detailed here and elsewhere (e.g.,Connaughton & Hanton, 2009; Gucciardi et al.,2011) support the idea that the entire frameworkof the MTQ 48 needs to be reconsidered andthat the magnitude of the problem is beyond forwhat post hoc modifications are intended. Asone of the most neglected stages of scale devel-opment, it is important that a clear articulationof the construct domain—both what is intendedto be captured as well as how it can be distin-guished from related constructs—is providedfrom the outset to reduce the likelihood of po-tential problems at later stages in the validationprocess (MacKenzie et al., 2011).

The acquisition of knowledge in a particulartopic area is in part dependent on psychometri-cally sound measures or inventories. Instru-ments that are theoretically derived and psycho-metrically sound according to multiple criteria(for reviews, see Gignac, 2009; Gucciardi et al.,2011) underpin substantive research questionsand theory development. Thus, it is important toascertain this type of validity before otherforms, such as predictive and concurrent valid-ity. According to Gignac, “. . .factorial validityis crucially important to the validity enterprise,as it helps determine what composite scoresderived from an inventory measure from a di-mensional perspective, or more specifically,how many dimensions are measured by thescores of an inventory?” (2009, p. 25). Both theCFA and ESEM findings of the current studycast doubt on the notion that the MTQ 48 ade-quately captures the 4Cs of mental toughness inathlete and workplace samples (Clough et al.,2002; Horsburgh et al., 2009). In particular, thelack of support in the workplace sample is aconcern because much of the original hardinessresearch came from a 12-year longitudinal studyof managers at a telephone company (cf. Maddi& Kobasa, 1984), and one would have expectedthose data to align well with the hypothesizedmodel. Although traditional psychometric eval-uations (e.g., internal consistency, predictivevalidity) are important, they are not sufficient indetermining the factorial validity (i.e., dimen-sionality, strength and direction of parameters)of the indicators. In this study, we have alsodemonstrated the potential problems of usinginternal reliability estimates (e.g., composite re-liability) as the sole indicator of an instrument’sappropriateness in a given sample. Thus, al-

210 GUCCIARDI, HANTON, AND MALLETT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

though preliminary and requiring replication inindependent samples, the current findings ques-tion some of the conclusions of previous re-search using the MTQ 48 as its central measureof mental toughness (cf. Hogan & Nicholson,1998).

Study Limitations

The key strength of this study is the use ofrigorous statistical analyses that accounted formeasurement error and the inclusion of moder-ately sized samples of performers from twoachievement contexts (i.e., sports and the work-place). Nevertheless, the results of the currentstudy should also be interpreted within thecontext of study limitations, which included areliance on self-reported mental toughness, across-sectional design, and online survey meth-odology. Future research that integrates multi-ple methods of assessment (e.g., self and coachor supervisor ratings), experimental manipula-tions, applies prospective longitudinal designs,and/or examines the impact of survey adminis-tration (i.e., online vs. paper-and-pencil, com-mon method bias) and nonindependence ofobservations (e.g., inclusion of team sports ath-letes) on factorial validity would prove fruitfulin addressing these concerns. It is particularlyimportant that the results of the current studyare verified on a larger sample of performers.Myers, Ahn, and Ying (2011) have recentlydemonstrated the usefulness of Monte Carlomethods for making an a priori decision aboutthe required sample size or for estimating thepower of a given sample size post hoc (see alsoMyers, Chase et al., 2011). Owing to the largedegree of model misfit, it was deemed inappro-priate to pursue a post hoc assessment of thepower with our sample using Monte Carlomethods (cf. Hancock, 2006), thereby suggest-ing the need for future research with even largersamples than the current study.

Conclusion

The current findings provide preliminary ev-idence that the psychometric properties of theMTQ 48 may not be adequate, particularly withrespect to its hypothesized underlying concep-tual model. In light of the current findings, it isimportant that researchers report empirical datapertaining to the factorial validity of the

MTQ 48 when this tool has been employed ameasure of mental toughness, as well as com-plete validation studies to address the within-network measurement issues (i.e., factorial va-lidity, internal reliability) revealed in both ourathlete and workplace samples. Until evidenceto support the factorial validity of the MTQ 48has been shown, researchers and practitionersshould proceed with caution when using theMTQ 48 as a measure of mental toughness.More broadly for the future of mental toughnessmeasurement research, the current findingshighlight the importance of having a clearlyarticulated definition and conceptual model thatunderpins item development, as well as apply-ing rigorous statistical procedures to purify andrefine the item pool before the underlying modelis cross-validated and assessed for its validity(cf. MacKenzie et al., 2011). Nevertheless, it isimportant to acknowledge that a number of con-ceptual and rhetoric debates still exist as to whatmental toughness is and of what it is made up(Connaughton & Hanton, 2009; for recent re-views, see Gucciardi & Gordon, 2011). Thus,because mental toughness is a construct with asparse and diverse theoretical and empirical no-mological network, it might be more prudent forresearchers to pursue a common understandingto formulate a consensual definition and theprimary facets that do and do not belong to it,rather than taking a statistical approach to val-idate a specific measurement model.

References

Asparouhov, T., & Muthen, B. (2009). Exploratory struc-tural equation modeling. Structural Equation Model-ing, 16, 397–438. doi:10.1080/10705510903008204

Beauducel, A., & Herzberg, P. Y. (2006). On theperformance of maximum likelihood versus meansand variance adjusted weighted least squares esti-mation in CFA. Structural Equation Modeling, 13,186–203. doi:10.1207/s15328007sem1302_2

Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psy-chometrika, 74, 137–143. doi:10.1007/s11336-008-9100-1

Browne, M. W. (2001). An overview of analyticrotation in exploratory factor analysis. Multivari-ate Behavioral Research, 36, 111–150. doi:10.1207/S15327906MBR3601_05

Browne, M. W., & Cudeck, R. (1992). Alternativeways of assessing model fit. Sociological Methods

211MEASURING MENTAL TOUGHNESS

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

& Research, 21, 230 –258. doi:10.1177/0049124192021002005

Byrne, B. M. (2010). Structural equation modelingwith AMOS: Basic concepts, applications, andprogramming (2nd ed.). Mahwah, NJ: LawrenceErlbaum Associates.

Chelladurai, P., & Saleh, S. (1978). Preferred lead-ership in sports. Canadian Journal of AppliedSport Sciences, 3, 85–92.

Chesney, M. A., Neilands, T. B., Chambers, D. B.,Taylor, J. M., & Folkman, S. (2006). A validityand reliability study of coping self-efficacy scale.British Journal of Health Psychology, 11, 421–437. doi:10.1348/135910705X53155

Clough, P., Earle, K., & Sewell, D. (2002). Mentaltoughness: The concept and its measurement. In I.Cockerill (Ed.), Solutions in sport psychology (pp.32–45). London, England: Thomson.

Comrey, A. L., & Lee, H. B. (1992). A first course infactor analysis. Hillsdale, NJ: Erlbaum.

Connaughton, D., & Hanton, S. (2009). Mentaltoughness in sport: Conceptual and practical is-sues. In S. D. Mellalieu & S. Hanton (Eds.), Ad-vances in applied sport psychology: A review (pp.317–346). London, England: Routledge.

Crocker, P. R. E., & Graham, T. R. (1995). Copingby competitive athletes with performance stress:Gender differences and relationships with affect.The Sport Psychologist, 9, 325–338.

Crust, L. (2009). The relationship between mentaltoughness and affect intensity. Personality andIndividual Differences, 47, 959–963. doi:10.1016/j.paid.2009.07.023

Crust, L., & Azadi, K. (2009). Leadership prefer-ences of mentally tough athletes. Personality andIndividual Differences, 47, 326–330. doi:10.1016/j.paid.2009.03.022

Crust, L., & Azadi, K. (2010). Mental toughnessathletes’ use of psychological strategies. EuropeanJournal of Sport Science, 10, 43–51. doi:10.1080/17461390903049972

Crust, L., & Keegan, R. (2010). Mental toughnessand attitudes to risk-taking. Personality and Indi-vidual Differences, 49, 164–168. doi:10.1016/j.paid.2010.03.026

Crust, L., Nesti, M., & Littlewood, M. (2010a). A cross-sectional analysis of mental toughness in a professionalfootball academy. Athletic Insight: The Online Journalof Sport Psychology, 2, 165–174.

Crust, L., Nesti, M., & Littlewood, M. (2010b). Player andcoach ratings of mental toughness in an elite associationfootball academy. Athletic Insight: The Online Journalof Sport Psychology, 2, 239–250.

Crust, L., & Swann, C. (2011). Comparing two mea-sures of mental toughness. Personality and Indi-vidual Differences, 50, 217–221. doi:10.1016/j.paid.2010.09.032

Daly, J. M., Brewer, B. W., & Van Raalte, J. L.(1995). Cognitive appraisal, emotional adjustment,and adherence to rehabilitation following knee sur-gery. Journal of Sport Rehabilitation, 4, 23–30.

Dolan, C. V. (1994). Factor analysis of variableswith 2, 3, 5 and 7 response categories: A compar-ison of categorical variable estimators using sim-ulated data. British Journal of Mathematical andStatistical Psychology, 47, 309–326. doi:10.1111/j.2044-8317.1994.tb01039.x

Franken, R., Gibson, K., & Rowland, G. (1992). Sensationseeking and the tendency to view the world as threat-ening. Personality and Individual Differences, 13, 31–38. doi:10.1016/0191-8869(92)90214-A

Gaudreau, P., & Blondin, J. P. (2002). Development of aquestionnaire for the assessment of coping strategiesemployed by athletes in competitive sport settings. Psy-chology of Sport and Exercise, 3, 1–34. doi:10.1016/S1469-0292(01)00017-6

Gignac, G. E. (2009). Psychometrics and the mea-surement of emotional intelligence. In C. Stough,D. H. Saklofske, & J. D. A. Parker (Eds.), Assess-ing emotional intelligence: Theory, research, andapplications (pp. 9–40). New York, NY: Springer.doi:10.1007/978-0-387-88370-0_2

Golby, J., Sheard, M., & van Wersch, A. (2007). Evalu-ating the factor structure of the psychological perfor-mance inventory. Perceptual and Motor Skills, 105,309–325. doi:10.2466/PMS.105.5.309-325

Gottlieb, B. H., & Rooney, J. A. (2004). Copingeffectiveness: Determinants and relevance to themental health and affect of family caregivers ofpersons with dementia. Aging & Mental Health, 8,364–373. doi:10.1080/13607860410001709719

Graham, J. W. (2009). Missing data analysis: Makingit work in the real world. Annual Review of Psy-chology, 60, 549 –576. doi:10.1146/annurev.psych.58.110405.085530

Gucciardi, D. F. (2009). Do developmental differ-ences in mental toughness exist between special-ized and invested Australian footballers? Person-ality and Individual Differences, 47, 985–989. doi:10.1016/j.paid.2009.08.001

Gucciardi, D. F., & Gordon, S. (2009). Development andpreliminary validation of the Cricket Mental ToughnessInventory. Journal of Sports Sciences, 27, 1293–1310.doi:10.1080/02640410903242306

Gucciardi, D. F., & Gordon, S. (Eds.). (2011). Mentaltoughness in sport: Developments in research andtheory. London, England: Routledge.

Gucciardi, D. F., Gordon, S., & Dimmock, J. A.(2009). Development and preliminary validationof a mental toughness inventory for Australianfootball. Psychology of Sport and Exercise, 10,201–209. doi:10.1016/j.psychsport.2008.07.011

Gucciardi, D. F., Mallett, C. J., Hanrahan, S. J., &Gordon, S. (2011). Measuring mental toughness insport: Current status and future directions. In D. F.

212 GUCCIARDI, HANTON, AND MALLETT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

Gucciardi & S. Gordon (Eds.), Mental toughnessin sport: Developments in research and theory (pp.108–132). London, England: Routledge.

Hagger, M. S., & Chatzisarantis, N. L. D. (2009).Assumptions in research in sport and exercise psy-chology. Psychology of Sport and Exercise, 10,511–519. doi:10.1016/j.psychsport.2009.01.004

Hancock, G. R. (2006). Power analysis in covariancestructure modeling. In G. R. Hancock, & R. O.Mueller (Eds.), Structural equation modeling: Asecond course (pp. 69–115). Greenwich, CT: In-formation Age Publishing.

Harter, S. (1999). The construction of the self. NewYork, NY: Guilford Press.

Hogan, R., & Nicholson, R. A. (1988). The meaning ofpersonality test scores. American Psychologist, 43, 621–626. doi:10.1037/0003-066X.43.8.621

Horsburgh, V. A., Schermer, J. A., Veselka, L., &Vernon, P. A. (2009). A behavioral genetic studyof mental toughness and personality. Personalityand Individual Differences, 46, 100–105. doi:10.1016/j.paid.2008.09.009

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria forfit indexes in covariance structure analysis: Con-ventional criteria versus new alternatives. Struc-tural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118

Kaiseler, M., Polman, R., & Nicholls, A. (2009).Mental toughness, stress, stress appraisal, copingand coping effectiveness in sport. Personality andIndividual Differences, 47, 728–733. doi:10.1016/j.paid.2009.06.012

Lane, A. M., Harwood, C., Terry, P. C., & Karageorghis,C. I. (2004). Confirmatory factor analysis of the Test ofPerformance Strategies (TOPS) among adolescent ath-letes. Journal of Sports Sciences, 22, 803–812. doi:10.1080/02640410410001716689

Larsen, R. J. (1984). Theory and measurement ofaffect intensity as an individual difference charac-teristic. Dissertation Abstracts International: Sec-tion B. Sciences and Engineering, 45(7), 2297.

Levy, A. R., Polman, R. C. J., Clough, P. J., March-ant, D. C., & Earle, K. (2006). Mental toughness asa determinant of beliefs, pain, and adherence insport injury rehabilitation. Journal of Sport Reha-bilitation, 15, 246–254.

Loehr, J. E. (1986). Mental toughness training forsports: Achieving athletic excellence. Lexington,MA: Stephen Greene Press.

Lonsdale, C., Hodge, K., & Rose, E. A. (2006).Pixels vs. paper: Comparing online and traditionalsurvey methods in sport psychology. Journal ofSport & Exercise Psychology, 28, 100–108.

MacKenzie, S. B., Podsakoff, P. M., & Podsakoff,N. P. (2011). Construct measurement and valida-tion procedures in MIS and behavioral research:Integrating new and existing techniques. MISQuarterly, 35, 293–334.

Maddi, S. R. (2004). Hardiness: An operationaliza-tion of existential courage. Journal of HumanisticPsychology, 44, 279 –298. doi:10.1177/0022167804266101

Maddi, S. R., & Kobasa, S. C. (1984). The hardyexecutive: Health under stress. Homewood, IL:Dow-Jones Irwin.

Marchant, D. C., Polman, R. C. J., Clough, P. J.,Jackson, J. G., Levy, A. R., & Nicholls, A. R.(2009). Mental toughness: Managerial and age dif-ferences. Journal of Managerial Psychology, 24,428–437. doi:10.1108/02683940910959753

Marsh, H. W. (1997). The measurement of physical self-concept: A construct validation approach. In K. Fox(Ed.), The physical self: From motivation to well-being(pp. 27–58). Champaign, IL: Human Kinetics.

Marsh, H. W., Hau, K.-T., & Grayson, D. (2005).Goodness of fit evaluation in structural equationmodeling. In A. Maydeu-Olivares & J. McArdle(Eds.), Contemporary psychometrics: A festschriftfor Roderick P. McDonald (pp. 275–340). Mah-wah, NJ: Erlbaum.

Marsh, H. W., Liem, G. A. D., Martin, A. J.,Morin, A. J. S., & Nagengast, B. (2011). Meth-odological measurement fruitfulness of explor-atory structural equation modeling (ESEM):New approaches to key substantive issues inmotivation and engagement. Journal of Psy-choeducational Assessment, 29, 322–346. doi:10.1177/0734282911406657

Marsh, H. W., Ludtke, O., Muthen, B., Asparouhov,T., Morin, A. J. S., Trautwein, U., & Nagengast, B.(2010). A new look at the big-five factor structurethrough exploratory structural equation modeling.Psychological Assessment, 22, 471– 491. doi:10.1037/a0019227

Marsh, H. W., Martin, A. J., & Jackson, S. A. (2010).Introducing a short version of the Physical SelfDescription Questionnaire: New strategies, short-form evaluation evaluative criteria, and applica-tions of factor analyses. Journal of Sport & Exer-cise Psychology, 32, 438–482.

Marsh, H. W., Muthen, B., Asparouhov, T., Ludtke,O., Robitzsch, A., Morin, A. J. S., & Trautwein, U.(2009). Exploratory structural equation modeling,integrating CFA and EFA: Application to students’evaluations of university teaching. StructuralEquation Modeling, 16, 439–476. doi:10.1080/10705510903008220

Marsh, H. W., Nagengast, B., Morin, A. S. J., Parada,R. H., Craven, R. G., & Hamilton, L. R. (2011).Construct validity of the multidimensional struc-ture of bullying and victimization: An applicationof exploratory structural equation modeling. Jour-nal of Educational Psychology, 103, 701–732. doi:10.1037/a0024122

Martens, M. P., & Webber, S. N. (2002). Psychometricproperties of the sport motivation scale: An evaluation

213MEASURING MENTAL TOUGHNESS

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.

with college varsity athletes from the US. Journal ofSport & Exercise Psychology, 24, 254–270.

McGrath, R. E. (2005). Conceptual complexity and con-struct validity. Journal of Personality Assessment, 85,112–124. doi:10.1207/s15327752jpa8502_02

Meyers, M. C., Bourgeois, A. E., Stewart, S., &LeUnes, A. (1992). Predicting pain response inathletes: Development and assessment of theSports Inventory for Pain. Journal of Sport &Exercise Psychology, 14(3), 249–261.

Morin, A. J. S., & Maıano, C. (2011). Cross-validation ofthe short form of the physical self-inventory (PSI-S)using exploratory structural equation modeling(ESEM). Psychology of Sport and Exercise, 12, 540–554. doi:10.1016/j.psychsport.2011.04.003

Muthen, B. O., & Kaplan, D. (1985). A comparisonof some methodologies for the factor analysis ofnonnormal Likert variables. British Journal ofMathematical and Statistical Psychology, 38, 171–189. doi:10.1111/j.2044-8317.1985.tb00832.x

Muthen, L. K., & Muthen, B. O. (2010). Mplus user’sguide (6th ed.). Los Angeles, CA: Authors.

Myers, N. D., Ahn, S., & Ying, J. (2011). Samplesize and power estimates for a confirmatory factoranalytic model in exercise and sport: A MonteCarlo approach. Research Quarterly for Exerciseand Sport, 82, 412–423.

Myers, N. D., Chase, M. A., Pierce, S. W., & Martin,E. (2011). Coaching efficacy and exploratorystructural equation modeling: A substantive-methodological synergy. Journal of Sport & Exer-cise Psychology.

Nicholls, A. R., Levy, A. R., Polman, R. C. J., &Crust, L. (2011). Mental toughness, coping self-efficacy, and coping effectiveness among athletes.International Journal of Sport Psychology, 42,513�524.

Nicholls, A. R., Polman, R. C. J., Levy, A. R., &Backhouse, S. H. (2008). Mental toughness, opti-mism, pessimism, and coping among athletes. Per-sonality and Individual Differences, 44, 1182–1192. doi:10.1016/j.paid.2007.11.011

Nicholls, A. R., Polman, R. C. J., Levy, A. R., &Backhouse, S. H. (2009). Mental toughness in

sport: Achievement level, gender, age, experience,and sport type differences. Personality and Indi-vidual Differences, 47, 73–75. doi:10.1016/j.paid.2009.02.006

Nunnally, J., & Bernstein, I. (1994). Psychometrictheory. New York, NY: McGraw-Hill.

Raykov, T. (1997). Estimation of composite reliabil-ity for congeneric measures. Applied Psychologi-cal Measurement, 21, 173–184. doi:10.1177/01466216970212006

Scheier, M. F., & Carver, C. S. (1985). Optimism,coping, and health: Assessment and implicationsof generalized outcome expectancies. Health Psy-chology, 4, 219 –247. doi:10.1037/0278-6133.4.3.219

Sheard, M., Golby, J., & van Wersch, A. (2009).Progress toward construct validation of the SportsMental Toughness Questionnaire (SMTQ). Euro-pean Journal of Psychological Assessment, 25,186–193. doi:10.1027/1015-5759.25.3.186

Sijtsma, K. (2009). On the use, the misuse, and thevery limited usefulness of Cronbach’s alpha. Psy-chometrika, 74, 107–120. doi:10.1007/s11336-008-9101-0

Thomas, P. R., Murphy, S. M., & Hardy, L. (1999).Test of performance strategies: Development andpreliminary validation of a comprehensive mea-sure of athletes’ psychological skills. Journal ofSports Sciences, 17, 697–711. doi:10.1080/026404199365560

Thompson, B. (2004). Exploratory and confirmatoryfactor analysis: Understanding concepts and ap-plications. Washington, DC: American Psycholog-ical Association. doi:10.1037/10694-000

Veselka, L., Schermer, J. A., Petrides, K. V., &Vernon, P. A. (2009). Evidence for a heritablegeneral factor of personality in two studies. TwinResearch and Human Genetics, 12, 254–260. doi:10.1375/twin.12.3.254

Received August 23, 2011Revision received December 22, 2011

Accepted December 27, 2011 �

E-Mail Notification of Your Latest Issue Online!

Would you like to know when the next issue of your favorite APA journal will beavailable online? This service is now available to you. Sign up at http://notify.apa.org/ andyou will be notified by e-mail when issues of interest to you become available!

214 GUCCIARDI, HANTON, AND MALLETT

This

doc

umen

t is c

opyr

ight

ed b

y th

e A

mer

ican

Psy

chol

ogic

al A

ssoc

iatio

n or

one

of i

ts a

llied

pub

lishe

rs.

This

arti

cle

is in

tend

ed so

lely

for t

he p

erso

nal u

se o

f the

indi

vidu

al u

ser a

nd is

not

to b

e di

ssem

inat

ed b

road

ly.