ARTICLE IN PRESS

wwwelseviercomlocateynimg

YNIMG-03679 No of pages 14 4C 3 6 7 8

DTD 5

NeuroImage xx (2006) xxx ndash xxx

Facial expressions and complex IAPS pictures Common and

differential networks

Jennifer C Brittona Stephan F Taylorb Keith D Sudheimera and Israel Liberzonbc

aDepartment of Neuroscience University of Michigan Ann Arbor MI 48109 USAbDepartment of Psychiatry University of Michigan Ann Arbor MI 48109 USAcPsychiatry Service Ann Arbor VAMC Ann Arbor MI 48105 USA

Received 20 July 2005 revised 11 December 2005 accepted 16 December 2005

Neuroimaging studies investigating emotion have commonly used two

different visual stimulus formats facial expressions of emotion or

emotionally evocative scenes However it remains an important

unanswered question whether or not these different stimulus formats

entail the same processes Facial expressions of emotion may elicit more

emotion recognitionperception and evocative pictures may elicit more

direct experience of emotion In spite of these differences common areas

of activation have been reported across different studies but little work

has investigated activations in response to the two stimulus formats in

the same subjects In this fMRI study we compared BOLD activation

patterns to facial expression of emotions and to complex emotional

pictures from the International Affective Picture System (IAPS) to

determine if these stimuli would activate similar or distinct brain

regions Healthy volunteers passively viewed blocks of expressive faces

and IAPS pictures balanced for specific emotion (happy sad anger fear

neutral) interleaved with blocks of fixation Eye movement reaction

times and off-line subjective ratings including discrete emotion valence

and arousal were also recorded Both faces and IAPS pictures activated

similar structures including the amygdala posterior hippocampus

ventromedial prefrontal cortex and visual cortex In addition expres-

sive faces uniquely activated the superior temporal gyrus insula and

anterior cingulate more than IAPS pictures despite the faces being less

arousing For the most part these regions were activated in response to

all specific emotions however some regions responded only to a subset

D 2006 Elsevier Inc All rights reserved

Introduction

Emotion research utilizes different types of stimuli (eg

expressive faces and complex evocative pictures) to probe affective

processing however the two lines of investigation have remained

1053-8119$ - see front matter D 2006 Elsevier Inc All rights reserved

doi101016jneuroimage200512050

Corresponding author Massachusetts General Hospital Psychiatry

Department Building 149 Thirteenth Street Charlestown MA 02129

USA Fax +1 617 726 4078

E-mail address jbrittonnmrmghharvardedu (JC Britton)

Available online on ScienceDirect (wwwsciencedirectcom)

relatively separate Facial expressions are often viewed as external

signals of experienced emotions that communicate information to

the observer (Frank and Stennett 2001) Facial expressions

portraying specific emotions (eg happy sad anger fear) are

universally recognized (Ekman 1992 1994 Izard 1994) and each

expression of discrete emotion has meaning targeting a specific

response (Halberstadt and Niedenthal 1997) Even though facial

expressions are used frequently as probes of emotion recognition

some studies have shown that faces can be inducers of emotion

(Hatfield et al 1992 Wild et al 2001) Facial expressions have

been also shown to evoke physiological changes (Clark et al 1992

Esteves and Ohman 1993) and autonomic activity in response to

facial expressions has been shown to correlate with neural activation

(Williams et al 2004) Complex pictures from the International

Affective Picture System (IAPS) another common emotional probe

depict emotion-laden scenes to induce affective states The

standardized set of IAPS pictures has been rated in terms of their

ability to induce valence (unpleasantpleasant) and arousal (calm

excited) changes These measures have also been correlated with

viewerrsquos heart rate and skin conductance changes respectively

providing physiological validity to subjectively reported emotion

induction (Lang et al 1993) However little work has been done to

identify the discrete emotions elicited by these pictures Although

both emotional faces and IAPS pictures target emotional processing

these two stimuli sets may preferentially engage certain brain

structures involved in emotion In addition it is not known whether

facial expressions and IAPS pictures of specific emotions (happy

sad anger and fear) would activate similar or discrete circuits

Studies of expressive faces and IAPS pictures suggest that a similar

set of regions is involved in processing both emotional stimulus types

Expressive faces and IAPS pictures activate regions involved in

emotion processing including the amygdala (Breiter et al 1996

Liberzon et al 2003 Morris et al 1996) hippocampus (Gur et al

2002 Lane et al 1997c) insula (Phan et al 2004 Phillips et al 1997)

anterior cingulate (ACC Killgore and Yurgelun-Todd 2004 Morris et

al 1998) medial prefrontal cortex (mPFC Kim et al 2003 Taylor et

al 2003 Winston et al 2003) ventromedial prefrontal cortex

(vMPFC Phan et al 2004)orbitofrontal cortex (OFC Blair et al

ARTICLE IN PRESS

Table 1

Demographics

Behavioral Group 1 Behavioral Group 2 fMRI

Participants 60 60 12

Gender (males) 30 30 6

Age (years) 216 T 30 (SD) 221 T 29 214 T 22

Caucasian 43 44 9

African American 3 2 1

Asian 8 6 2

Indian 4 4 0

Hispanic 2 4 0

JC Britton et al NeuroImage xx (2006) xxxndashxxx2

1999) and visual cortex (Liberzon et al 2003 Morris et al 1998)

Both stimulus types may recruit similar structures due to the underlying

emotional processes activated within those regions (eg amygdala

activation reflecting fear (LeDoux 2000) or stimulus salience (Liberzon

et al 2003) insula activation reflecting somaticvisceral responses

(Damasio 1999) and disgust perception (Phillips et al 1997) anterior

cingulate activation reflecting attention and self-awareness (Lane et al

1997a) and medial prefrontal activation reflecting emotion regulation

(Davidson et al 2000)) However few studies have compared these

stimuli directly In a single study comparing threat-related stimuli

bilateral amygdala activation was found in response to both expressive

faces and IAPS pictures (Hariri et al 2003) however the low Z-scores

and the cognitive matching task in this study prevent any definitive

conclusions regarding the common and differential emotional networks

activated by these emotional stimuli

Even though expressive faces and complex pictures may activate

a similar set of regions given the role of emotional facial expressions

in transacting social behavior emotional perception of faces is

thought to be processed by a distinct circuitry (Calder et al 2001)

including superior temporal gyrus (STG) and amygdala (Adolphs et

al 2002 Winston et al 2003) Facial expressions of emotion have

characteristic profiles (eg protruded tongue when disgusted

contracted eyebrows when angry) (Darwin 1998) and the STG

has been shown to respond to variable aspects in facial expressions

(Narumoto et al 2001) In some studies superior temporal gyrus

has also been shown to respond preferentially to faces relative to

pictures (Geday et al 2003) Lesion and neuroimaging studies

highlight the robustness of the amygdala response to faces

Amygdala lesions have been shown to impair fear recognition

(Yang et al 2002a) Neuroimaging studies have shown increased

amygdala activity when viewing fear (Breiter et al 1996 Hariri et

al 2003 Morris et al 1996 Phillips et al 1997 Whalen et al

2001) angry (Whalen et al 2001) sad (Blair et al 1999) and

happy facial expressions (Breiter et al 1996 Dolan et al 1996)

Even though IAPS pictures also activate these regions processing

emotional information from facial expressions may be processed

preferentially by superior temporal gyrus and amygdala

In the current study we aimed to examine the neural correlates of

responses to expressive faces and IAPS pictures Do these emotional

probes elicit similar or distinct activation patterns In order to

effectively compare BOLD responses to expressive faces and IAPS

pictures stimulus properties (eg specific emotion valence and

arousal) had to be balanced but only few studies have examined the

emotion induction capability of facial expressions (Wild et al 2001)

or the profiles of specific emotions induced by the IAPS pictures

(Davis et al 1995) Therefore a behavioral experiment was

conducted to match stimuli based on these features Subsequently

a block design fMRI study was conducted to examine the neural

correlates of processing facial expressions and IAPS pictures

balanced on specific emotion We hypothesized that facial expres-

sions and IAPS pictures would activate a similar emotional network

and that some brain regions (superior temporal gyrus and amygdala)

would preferentially respond to facial expressions

Methods

Participants

Healthy volunteers were recruited from advertisements placed

at local universities Demographics are outlined in Table 1 All

participants were between 18 and 30 years right-handed

English speaking and had normal or corrected-to-normal visual

acuity Participants did not have a current or prior history of

head injury learning disability psychiatric illness medical

illness or substance abusedependence (gt 6 months) For the

fMRI study a formal screening assessment (Mini SCID) was

used (Sheehan et al 1998) After explanation of the experi-

mental protocol all participants gave written informed consent

as approved by the University of Michigan Institutional Review

Board Participants were paid for their participation

Experiment 1 Behavioral study

Stimuli

The image set included 150 facial expressions of specific

emotions posed and evoked by actors balanced for gender and

ethnicity (Gur et al 2002) and 200 IAPS pictures (Lang et al

1997) These images were selected to target the emotions of

happiness (babies Mickey Mouse sporting events) sadness

(funeral scenescemeteries premature babies wounded bodies)

anger (human violence guns KKK images) and fear (snakes

spiders sharks medical procedures) in equal quantities In

addition neutral or nonemotional images (mushrooms household

items) were also selected All images were converted from color to

gray scaleblack and white using Photoshop 60 (Adobe Systems

San Jose CA) and matched on luminance

Procedure

Volunteers participated in separate rating-task experiments

(Group 1 IAPS rating task Group 2 Face rating task) For both

participants were seated in front of a laptop computer (Dell PC

Inspiron 2650) in a quiet experimental room

After viewing an image for 3 s participants were prompted to

rate each image IAPS pictures were rated only on (1)

predominant emotion and (2) emotion intensity because stan-

dardized ratings of valence and arousal for each picture have

been published (Lang et al 1997) Facial expressions were rated

on (1) predominant emotion (2) emotion intensity and also on

(3) valence and (4) arousal The predominant emotion rating

instructions were lsquolsquoIndicate the predominant emotion that is

depicted in the image given the following options happy neutral

sad anger fear and disgustrsquorsquo The emotion intensity rating

instructions were lsquolsquoIndicate the degreeintensity of the selected

emotion (1 = not at all 2 = mildly 3 = moderately 4 = strongly

5 = extremely)rsquorsquo The valence rating instructions were lsquolsquoRate how

unpleasant or pleasant the image makes you feel using a 1ndash9

scale (1 = very unpleasant 5 = neutral 9 = very pleasant)rsquorsquo The

arousal rating instructions were lsquolsquoRate how emotionally intense or

ARTICLE IN PRESSJC Britton et al NeuroImage xx (2006) xxxndashxxx 3

arousing the image makes you feel using a 1ndash9 scale (1 = calm

5 = somewhat aroused 9 = excited)rsquorsquo

Analysis

For each image the frequencies of the two most reported

emotions were compared using chi-squared analysis For images

with significant chi-squared values (P lt 005) the image was

classified according to the predominant emotion The number of

images in each specific emotional category was compiled and the

percentage agreement across participants was calculated

The valence and arousal ratings obtained for the face and

standardized IAPS picture ratings were compared using t tests A

series of t tests compared valence and arousal ratings within each

specific emotion category as well

Results

Using the criteria described above (significance on a chi-square

test) a proportion of the 150 facial expressions stimuli (826) were

classified according to specific emotions (happy 193 neutral

180 sad 173 anger 12 fear 16) From the set of 200

IAPS pictures 675were classified according to specific emotions

(happy 195 neutral 160 sad 135 anger 75 fear

110) (Fig 1)

After assigning the images in a particular specific emotion

category the percent agreement was analyzed (Fig 1A) In

Fig 1 Behavioral ratings Each stimulus set (expressive faces and complex IAPS

agreeing with predominant emotion assigned to each image (B) Valence (1 = ve

excited) ratings plotted for each image (C) Mean and standard error of valence ra

and standard error of arousal ratings for images within each assigned discrete em

general more agreement was detected in the emotional faces

(8370) than IAPS pictures (752) In addition happy images

showed most agreement (gt90)

Standardized valence and arousal ratings for the IAPS picture set

(Lang et al 1997) were compared to the ratings of facial expressions

obtained from our participants (Figs 1BndashD) The IAPS pictures

were rated higher on valence (ie more pleasant or more unpleasant)

as compared to faces in each specific emotion category (post hoc

pairwise t tests P lt 0001) except anger (P gt 0241) Happy and

neutral IAPS pictures were rated more positively than happy and

neutral facial expressions respectively Sad and fear IAPS pictures

were rated more negatively than sad and fear faces respectively The

arousal rating for the faces (319 T 006 (SEM)) was lower than for

IAPS pictures (507 T 009) for all specific emotion categories

[t(3349) = 1756 P lt 0001 post hoc pairwise t tests P lt 0001]

Experiment 2 fMRI study

Procedure

Volunteers were placed comfortably within the scanner A light

restraint was used to limit head movement during acquisition

While lying inside the scanner stimuli were presented to

participants via MRI-compatible display goggles (VisuaStimXGA

Resonance Technology) mounted on the RF head coil and adjusted

to ensure an unobstructed field of view Stimuli were displayed

pictures) was rated on several dimensions (A) Percentage of participants

ry unpleasant 5 = neutral 9 = very pleasant) and arousal (1 = calm 9 =

tings for images within each assigned discrete emotion category (D) Mean

otion category

ARTICLE IN PRESSJC Britton et al NeuroImage xx (2006) xxxndashxxx4

using Eprime software (Psychology Software Tools Inc

Schneider et al 2002ab) In addition Eprime recorded partic-

ipantsrsquo subjective responses via right-handed button-glove

Using a block design expressive faces and IAPS pictures were

interleaved with control periods Images for each specific emotion

block (happy neutral sad anger fear) were identified using the

emotional ratings and emotional intensities obtained in the

behavioral experiment Emotion block order was counterbalanced

across the entire scanning session Four emotional stimuli were

presented in each face and IAPS picture block Each image within

the block was shown for 4 s with no interstimulus interval Two gray

scale fixation images were presented during control periods The

sequence of face and picture blocks was repeated eight times within

each run Eight runs were acquired Each stimulus block was

repeated in the second half of the experiment however while the

stimuli within each block were maintained the block order within

each run was counterbalanced

Participants passively viewed each image and responded via

button-press using the right index finger to indicate when a new

image appeared on the screen The reaction time of this response was

recorded and used to monitor task performance In addition eye

movements were monitored with infrared camera within the display

goggles that sampled pupil location at 30 Hz with an accuracy of

10- of visual arc (ViewPoint Eyetracker Arrington Research)

Before scanning participants were introduced to a brief version

of the task consisting of one block of neutral expressive faces and

one block of neutral complex pictures interspersed by fixation The

images displayed in this practice session were not repeated during

image acquisition Immediately following scanning participants

completed a self-paced rating task outside the scanner similar to the

procedure of the behavioral experiment Maintaining image order

within each block participants rated each image on several

dimensions predominant emotion (forced-choice selection be-

tween happy neutral sad anger fear and disgust) associated

emotional intensity (1 = not at all 5 = extremely) valence (1 =

most unpleasant 5 = neutral 9 = most pleasant) and arousal (1 =

calm 9 = very excited) The block order was counterbalanced

between subjects

fMRI acquisition

Scanning was performed on a 30 T GE Signa System

(Milwaukee WI) using a standard radio frequency coil A T1-

weighted image was acquired for landmark identification to

position subsequent scans After initial acquisition of T1 structural

images functional images were acquired To minimize suscepti-

bility artifact (Yang et al 2002b) whole-brain functional scans

were acquired using T2-weighted reverse spiral sequence with

BOLD (blood oxygenation level-dependent) contrast (echo time

TE = 30 ms repetition timeTR of 2000 ms frequency of 64

frames flip angle of 90- field of viewFOV of 20 cm 40

contiguous 3 mm oblique axial slicesTR approximately parallel to

the ACndashPC line) Each run began with 6 Fdummy_ volumes

(subsequently discarded) to allow for T1 equilibration effects

After 8 functional runs were collected a high-resolution T1 scan

was also acquired to provide precise anatomical localization (3D-

SPGR TR of 27 ms minimum TE flip angle of 25- FOV of 24

cm slice thickness of 10 cm 60 slicesTR) Co-images were

reconstructed off-line using the gridding approach into a 128 128 display matrix with an effective spatial resolution of 3 mm

isotropic voxels

Analysis

Participants responded when a new image appeared on the

screen in this passive viewing task to monitor on-task performance

To test on-task performance the number of responses and the

reaction time was examined The number of responses to face

IAPS picture and fixation images was examined The reaction

times were examined using 2 (image face IAPS picture) 5

(emotion happy neutral sad anger fear) Repeated Measures

ANOVA and post hoc analysis Separate paired t tests were used to

test differences between images (face IAPS picture and fixation)

In addition paired t tests examined differences between the

reaction times during the first and last part of the experiment for

each image type

Preprocessing of eye movement occurred offline beginning

with the identification of eye blinks Linear interpolation was then

performed to correct for missing data points The standard

deviation of the eye position in horizontal and vertical directions

was calculated for each stimulus block using MATLAB (Math-

works Inc Sherborn MA) The eye movement data in the

horizontal and vertical directions were examined using separate 2

(image faces IAPS pictures) 5 (emotion happy neutral sad

anger fear) Repeated Measures ANOVA Paired t tests examined

the differences between faces IAPS pictures and fixation images

The postscan ratings (valence and arousal) were examined

using separate 2 (image type faces IAPS pictures) 5 (emotion

happy neutral sad anger fear) Repeated Measures ANOVA Post

hoc analysis determined significant main effects of image type and

emotion Paired t tests examined the differences between faces and

IAPS pictures in each discrete emotion category

fMRI analysis

Images were slice-time corrected realigned co-registered

normalized and smoothed according to standard methods Scans

were slice-time corrected using sinc interpolation of the eight

nearest neighbors in the time series (Oppenheim and Schafer

1989) and realigned to the first acquired volume using AIR 308

routines (Woods et al 1998) Additional preprocessing and image

analysis of the BOLD signal were performed with Statistical

Parametric Mapping (SPM99 Wellcome Institute of Cognitive

Neurology London UK wwwfilionuclacukspm) implemented

in MATLAB Images were co-registered with the high-resolution

SPGR T1 image This high-resolution image was then spatially

normalized to the Montreal Neurological Institute (MNI152)

template brain and transformation parameters were then applied

to the co-registered functional volumes resliced and spatially

smoothed by an isotropic 6 mm full-width-half-maximum

(FWHM) Gaussian kernel to minimize noise and residual differ-

ences in gyral anatomy Each normalized image set was band pass-

filtered (high pass filter = 100 s) to eliminate low frequency signals

(Ashburner et al 1997 Friston et al 1995) The data were

analyzed using a general linear model with parameters

corresponding to each specific emotion (happy neutral sad anger

and fear) and image type (expressive faces IAPS pictures and

fixation images) modeling each run separately Each stimulus

block was convolved with a canonical hemodynamic response

function (HRF)

For each participant parameter estimates of block-related

activity were obtained at each voxel Contrast images were

calculated by applying appropriate linear contrasts to the parameter

estimates of each block to produce statistical parametric maps

ARTICLE IN PRESSJC Britton et al NeuroImage xx (2006) xxxndashxxx 5

(SPMt) which were transformed to a normal distribution

(SPMZ) Relevant linear contrasts included image type main

effects (eg Face-Fixation IAPS-Fixation) specific emotion main

effects within each image type (eg Happy Face-Fixation Happy

FacendashNeutral Face) and emotion image type interaction effects

(eg [Emotional FacendashNeutral Face]ndash[Emotional IAPS picturendash

Emotional IAPS picture] [Happy FacendashNeutral Face]ndash[Happy

IAPS picturendashNeutral IAPS picture]) To account for interindivid-

ual variability an additional 6-mm smoothing was performed on

the contrast images before incorporating the individual contrasts in

a random effects analysis

A second-level random effects analysis used one-sample t

tests on smoothed contrast images obtained in each subject for

each comparison of interest treating subjects as a random

variable (Friston 1998) This analysis estimates the error

variance for each condition of interest across subjects rather

than across scans and therefore provides a stronger generaliza-

tion to the population from which data are acquired In this

random effects analysis resulting SPM maps (df = 11) were

examined in a priori regions (amygdalasublenticular extended

amygdala hippocampus STG insula ACC mPFC vMPFC

OFC) Whole-brain analysis conducts comparisons in a voxel-

wise manner increasing the possibility of false positives unless

an appropriate correction for multiple comparisons is used To

restrict the number of comparisons a Small Volume Correction

(SVC) was applied for all activations in a priori regions SVC

was implemented in SPM across a two volumes of interest

[rectangular box 1 x = 0 T 70 mm y = 10 T 30 mm z = 5 T25 mm rectangular box 2 x = 0 T 20 mm y = 35 T 35 mm z = 15 T45 mm] defined using the Talaraich atlas to isolate central regions

(amygdalaSLEA hippocampus STG insula) and anterior midline

regions (mPFC ACC vMPFCOFC) Within each SVC a false

discovery rate [FDR] of 0005 was used to ensure that on average no

more than 05 of activated voxels for each contrast are expected to

be false positive results (Genovese et al 2002) In addition

activation foci were required to have a cluster sizeextent threshold

of greater than 5 contiguous voxels For activation foci detected

between modalities (eg Faces gt IAPS Pictures and IAPS Pictures gt

Faces) regions activated within each modality that fell just below

the cluster threshold are also denoted in the tables

Results

On-task performance

Participants responded via button-press to 98 of images

(100 accuracy to faces 99 accuracy to IAPS pictures and

96 accuracy to blanks) confirming on-task performance

Reaction times differed depending on modality [F(111) =

1787 P lt 0001] Reaction times to faces (6385 T 918 ms

(SEM)) were significantly faster than reaction times to IAPS

pictures (8597 T 1423 ms t(11) = 423 P lt 0001) No main

effect of specific emotion was detected (P gt 0477)

The first half of the experiment elicited slower reaction times

than the second half for all image types [1st half 8074 T 1244 ms

2nd half 6860 T 1147 ms paired t tests t(11) = 491 P lt 0001]

Different lateral eye movement patterns were detected for

different stimulus types [image effect F(18) = 801 P lt

0018] IAPS pictures (SD 0339 T 0079) elicited more eye

movement in the horizontal direction compared to eye move-

ments elicited by faces (SD 0235 T 0065 t(8) = 283 P lt

0018) and fixation (SD 0210 T 0058 t(8) = 180 P lt

0101) No differences between specific emotions were detected

(P gt 0556) No difference in vertical eye movements between

different images was detected (P gt 0319)

Postscan subjective ratings

The stimulus sets were examined to determine the percentage

agreement with the predominant emotion standards determined

by the behavioral experiment In general more agreement was

detected in the emotional faces (835) compared to the IAPS

pictures (789) In addition happy images were more

consistently identified by participants than any other emotion

(Fig 2)

Similar to Experiment 1 emotional IAPS pictures were rated

with higher valence (ie more pleasant or more unpleasant) for

all specific emotions [image type F(111) = 1246 P lt 0005

paired t tests P lt 0005] Happy and neutral IAPS pictures were

rated more positively than happy and neutral faces respectively

Sad anger and fear IAPS pictures were rated more negatively

than the sad anger and fear faces respectively [emotion

F(555) = 7673 P lt 0001 emotion image type interaction

F(555) = 4862 P lt 0001]

Emotional IAPS pictures were more arousing than emotional

faces for all specific emotions [image type F(111) = 5204

P lt 0001 paired t tests P lt 0001 emotion main effect

F(555) = 3594 P lt 0001 image type emotion interaction

F(555) = 1388 P lt 0001] Arousal ratings for neutral IAPS

pictures and neutral faces were not significantly different

[paired t test t(11) = 174 P lt 0110]

fMRI results

Effects of facial expressions and IAPS pictures

Facial expressions analyzed together (contrast all faces-

fixation) and picture stimuli analyzed together (contrast all

IAPS pictures-fixation) activated a similar network bilateral

amygdala posterior hippocampus ventral medial prefrontal

cortex and visual cortex (Table 2 Fig 3) In addition

dorsomedial prefrontal cortex activated in response to expressive

faces [(3 57 33) Z = 302 k = 19]

This pattern of activation was consistently present when several

different specific emotions (happy sad anger fear and neutral)

were analyzed separately (contrast specific emotion-fixation eg

happy face-fixation) The amygdala was activated in response to all

emotional facial expressions and sad and anger IAPS pictures

With the exception of happy facial expressions the pattern of

dorsomedial prefrontal cortex activation was similar to the

amygdala Hippocampus activated in response to all facial

expressions (except happy) and also to all IAPS pictures

Ventromedial prefrontal cortex activated in response to all facial

expressions (except happy) and all IAPS pictures (except fear)

Visual cortex was activated in response to all facial expressions and

all IAPS pictures

Effects of emotional faces and emotional IAPS pictures

To identify and compare emotionality in these stimulus types all

facial expressions and all IAPS pictures were analyzed relative to

neutral (eg contrast emotional facesndashneutral faces and [emotional

facesndashneutral faces]ndash [emotional IAPS picturesndashneutral IAPS

ARTICLE IN PRESS

Fig 2 fMRI postscan ratings Each stimulus set (expressive faces and

complex IAPS pictures) was rated on several dimensions (A) Percentage of

participants agreeing with predominant emotion assigned to each image (B)

Mean and standard error of valence (1 = very unpleasant 5 = neutral 9 = very

pleasant) ratings for images within each assigned discrete emotion category

(C) Mean and standard error of arousal (1 = calm 9 = excited) ratings for

images within each assigned discrete emotion category

JC Britton et al NeuroImage xx (2006) xxxndashxxx6

pictures]) The superior temporal gyrus insula and anterior

cingulate activated in response to emotional faces and showed

greater activity in these regions compared to emotional IAPS

pictures Visual cortex activated in response to emotional pictures

and showed greater activity compared to facial expressions Of note

the activations to neutral stimuli did not differ in any region other

than the visual cortex [IAPS pictures gt faces (9 93 3) Z =

536 k = 656] (Table 3)

Effects of specific emotional faces and emotional IAPS pictures

To identify the effects of each specific emotion each

specific emotion (happy sad anger and fear) was also

analyzed separately (eg contrast happy facesndashneutral faces)

While amygdala hippocampus vMPFC and visual cortex

were commonly activated among faces and pictures when all

specific emotions were analyzed together we observed

differential activation in these regions in response to specific

emotions suggesting that some emotions contributed more

substantially to these overall results Amygdala activated in

response neutral stimuli however anger faces showed signif-

icantly greater amygdala activity than neutral faces Similarly

while hippocampus activated in response to neutral stimuli

anger and fear stimuli showed significantly greater hippocam-

pal activity than neutral stimuli Ventromedial prefrontal cortex

activated in response to neutral stimuli and sad and anger

faces and anger IAPS pictures showed greater vMPFC activity

than neutral faces and pictures respectively Visual cortex

activated in response to neutral stimuli and happy sad and

fear IAPS pictures showed greater visual cortical activity than

neutral pictures Additionally fear and sad IAPS pictures

showed greater visual activity compared to fear and sad faces

Specific emotions (eg contrast [happy facesndashneutral faces]ndash

[happy IAPS picturesndashneutral IAPS pictures]) contributed to the

overall differences in activation between facial expressions and

IAPS pictures in STG insula and ACC (Fig 4) STG was

significantly activated in response to all specific emotional faces

relative to neutral faces (happy sad anger and fear) All these

activations (except sad) were also significantly larger than

corresponding activations elicited by specific emotional IAPS

relative to neutral IAPS pictures Similarly insula was activated in

response to all specific emotional faces (happy at a subthreshold

level) and all these activations (except anger) were significantly

larger than corresponding activations elicited by specific emotional

IAPS pictures Anterior cingulate was significantly activated in

response to fear and sad (sad at a subthreshold level) facial

expressions and these activations showed greater anterior cingulate

activity compared to corresponding IAPS pictures Anger and sad

faces also elicited greater rostral anterior cingulate activity compared

to corresponding IAPS pictures (Table 4)

Discussion

In this study we examined whether expressive faces and IAPS

pictures would activate similar brain regions Analyzed as set of

stimuli expressive faces and IAPS pictures activated a common

pattern of brain regions including the amygdala posterior

hippocampus ventromedial prefrontal cortex and visual cortex

These stimuli also activated superior temporal gyrus insula and

anterior cingulate differentially eg more activation in these

regions to expressive faces than to IAPS pictures For the most

part these regions were activated in response to each specific

emotion separately however some regions responded only to a

subset of specific emotions

Expressive faces and IAPS pictures common areas of activation

The amygdala posterior hippocampus ventromedial prefrontal

cortex and visual cortex were activated by both expressive faces

and IAPS pictures analyzed as two sets of emotional stimuli

ARTICLE IN PRESS

Table 2

Emotional faces and IAPS pictures activate a similar network relative to fixation

Region Faces IAPS pictures

(x y z)a Zb kc (x y z) Z k

L amygdala (21 6 18) 418 36 (21 6 15) 381 12

R amygdala (24 6 15) 405 44 (24 3 15) 306 6

Hippocampus (24 30 3) 373 30 (15 30 6) 457 207

(15 30 3) 290 12

Ventromedial prefrontalorbitofrontal cortex (0 45 24) 455 49 (3 45 21) 365 45

Visual (30 78 15) 568 2559 (33 60 15) 579 5607

a Stereotactic coordinates from MNI atlas leftright (x) anteriorposterior ( y) and superiorinferior (z) respectively R = right L = leftb Z score significant after Small Volume Correction (SVC thresholded using a false discovery rate [FDR] correction for multiple comparisons of 0005)c Spatial extent in cluster size threshold 6 voxels

JC Britton et al NeuroImage xx (2006) xxxndashxxx 7

suggesting that these regions are involved in general emotion

processing (ie not specific to stimulus type or a particular process

recognition vs induction) Consistent with previous findings

negative emotional faces and IAPS pictures activated the amygdala

(Hariri et al 2000 2003) In addition we found amygdala

activation to happy emotional faces Amygdala activation has been

reported to positive and negative facial expressions (Breiter et al

1996 Morris et al 1996 Somerville et al 2004) and IAPS

pictures (Liberzon et al 2003) therefore it is unclear why positive

IAPS pictures did not activate the amygdala as well Emotional

faces and IAPS pictures activated the hippocampus in concert with

Fig 3 Common regions of activation SPM t map showing activated visual cortex

(A) Expressive Faces and (B) IAPS pictures relative to fixation Posterior hippocam

0005 uncorrected [k] gt 5 voxels threshold

previous studies (Fried et al 1997 Lane et al 1997c) The

hippocampus has been shown to be involved in episodic memory

and declarative knowledge (Bechara et al 1995) and with its

extensive connections from extrastriate visual areas including

fusiform gyrus the hippocampal activation may reflect contextual

memory and visual processing triggered by our stimuli Negative

facial expressions and negative IAPS pictures with the exception

of fear activated the ventromedial prefrontal cortex The medial

prefrontal cortex is thought to be involved in emotional self-

awareness (Lane et al 1997b) and reexperiencing the Ffeelings_ ofonersquos emotional past (Damasio 1999) In concert ventromedial

(visual) ventromedial prefrontal cortex (vMPFC) and amygdala (Amy) to

pus was also activated (not shown) Activated voxels are displayed with P lt

ARTICLE IN PRESS

Table 2

Emotional faces and IAPS pictures activate a similar network relative to fixation

Region Faces IAPS pictures

(x y z)a Zb kc (x y z) Z k

L amygdala (21 6 18) 418 36 (21 6 15) 381 12

R amygdala (24 6 15) 405 44 (24 3 15) 306 6

Hippocampus (24 30 3) 373 30 (15 30 6) 457 207

(15 30 3) 290 12

Ventromedial prefrontalorbitofrontal cortex (0 45 24) 455 49 (3 45 21) 365 45

Visual (30 78 15) 568 2559 (33 60 15) 579 5607

a Stereotactic coordinates from MNI atlas leftright (x) anteriorposterior ( y) and superiorinferior (z) respectively R = right L = leftb Z score significant after Small Volume Correction (SVC thresholded using a false discovery rate [FDR] correction for multiple comparisons of 0005)c Spatial extent in cluster size threshold 6 voxels

JC Britton et al NeuroImage xx (2006) xxxndashxxx 7

suggesting that these regions are involved in general emotion

processing (ie not specific to stimulus type or a particular process

recognition vs induction) Consistent with previous findings

negative emotional faces and IAPS pictures activated the amygdala

(Hariri et al 2000 2003) In addition we found amygdala

activation to happy emotional faces Amygdala activation has been

reported to positive and negative facial expressions (Breiter et al

1996 Morris et al 1996 Somerville et al 2004) and IAPS

pictures (Liberzon et al 2003) therefore it is unclear why positive

IAPS pictures did not activate the amygdala as well Emotional

faces and IAPS pictures activated the hippocampus in concert with

Fig 3 Common regions of activation SPM t map showing activated visual cortex

(A) Expressive Faces and (B) IAPS pictures relative to fixation Posterior hippocam

0005 uncorrected [k] gt 5 voxels threshold

previous studies (Fried et al 1997 Lane et al 1997c) The

hippocampus has been shown to be involved in episodic memory

and declarative knowledge (Bechara et al 1995) and with its

extensive connections from extrastriate visual areas including

fusiform gyrus the hippocampal activation may reflect contextual

memory and visual processing triggered by our stimuli Negative

facial expressions and negative IAPS pictures with the exception

of fear activated the ventromedial prefrontal cortex The medial

prefrontal cortex is thought to be involved in emotional self-

awareness (Lane et al 1997b) and reexperiencing the Ffeelings_ ofonersquos emotional past (Damasio 1999) In concert ventromedial

(visual) ventromedial prefrontal cortex (vMPFC) and amygdala (Amy) to

pus was also activated (not shown) Activated voxels are displayed with P lt

ARTICLE IN PRESS

Table 3

Emotional faces and IAPS pictures activate a different network relative to neutral

Region Faces Faces gt IAPS pictures IAPS pictures IAPS pictures gt Faces

(x y z)a Zb kc (x y z)a Zb kc (x y z)a Zb kc (x y z)a Zb kc

Superior temporal gyrus (60 39 6) 417 284 (69 21 15) 322 134

(66 6 9) 350 113 (66 9 12) 364 91

Insular cortex (39 18 9) 356 69 (27 6 12) 314 37

Anterior cingulate (0 30 30) 410 35 (3 30 30) 364 43

Visual cortex (12 75 12) 304 9 (36 78 0) 381 88

(30 93 15) 324 25

a Stereotactic coordinates from MNI atlas leftright (x) anteriorposterior ( y) and superiorinferior (z) respectivelyb Z score significant after Small Volume Correction (SVC thresholded using a false discovery rate [FDR] correction for multiple comparisons of 0005)c Spatial extent in cluster size threshold 6 voxels

JC Britton et al NeuroImage xx (2006) xxxndashxxx8

prefrontal lesions lead to deficits in recognizing emotion from

facial expressions (Hornak et al 1996) In addition ventromedial

prefrontal cortical activation to IAPS was modulated by the extent

of self-association (Phan et al 2004) thus the ventromedial

prefrontal cortex activation may reflect personal association Both

expressive faces and IAPS pictures activated the visual cortex

which is expected given the reports of emotional content

modulating visual processing Two components of emotional

processing (eg arousal and valence) have been shown to

contribute to visual cortex activations (Mourao-Miranda et al

2003) and increased activation in the visual cortex may also reflect

the stimulusrsquo significance (Anderson and Phelps 2001 Pessoa et

al 2002) or increased attention (Lane et al 1999)

The dorsomedial prefrontal cortex is thought to be involved in

general emotional processing (ie emotional appraisalevaluation

and emotion regulation) (Phan et al 2002) Discrete emotions in

both types of stimuli activated dorsomedial prefrontal cortex but

this activation was detected in the main effect of expressive faces but

not IAPS pictures The less consistent dorsomedial prefrontal cortex

activation to these emotional stimuli might have been be due to our

choice of passive viewing task in this study Including a cognitive

task (eg rating) has shown to increase dorsomedial prefrontal

cortex activation (Taylor et al 2003) and while the passive viewing

task was chosen as to not bias the participants towards emotion

recognition or emotion induction it is possible that subjects were

labeling the emotion displayed on each face Although the dMPFC

and amygdala activation was consistent among all facial expres-

sions it was observed that negative pictures showed dMPFC

activation when amygdala was activated in those conditions as well

Given the anatomical connections the co-activation of these two

structures have been hypothesized to reflect possible influence of

cortical inhibitory control (Ongur and Price 2000) The MPFC has

been implicated in emotion regulation (Levesque et al 2003

Ochsner et al 2002 Taylor et al 2003) extinguished fear (Milad

and Quirk 2002 Milad et al 2004 Quirk et al 2003) and

cognitive-emotion interactions (Liberzon et al 2000 Simpson et

al 2000 Taylor et al 2003) In this study sad and anger IAPS

pictures show medial prefrontal cortex and amygdala co-activation

suggesting that dorsomedial prefrontal cortex activation may be

playing a role in reappraisal of negative emotion (Beauregard et al

2001 Ochsner et al 2002 Phan et al 2005)

Expressive faces and IAPS pictures differential areas of activation

As a group expressive faces activated superior temporal gyrus

insula and anterior cingulate more than IAPS pictures despite the

fact that expressive faces overall were subjectively rated to be

lower on valence and arousal Previous studies suggest that facial

expressions can evoke emotion portrayed to the viewer through

primitive emotion contagion (Wild et al 2001) The subjective

responses in this study indicate that IAPS pictures are even more

potent than facial expressions at inducing changes in the subjective

state of emotional valence and arousal Nevertheless expressive

faces elicited greater activation than IAPS pictures in several

regions Superior temporal gyrus has been shown to be involved in

processing variable components of the face such as eye gaze eye

brows and mouth gape (Haxby et al 2000) therefore it is not

surprising that expressive faces would activate this region more

than IAPS pictures The insula has been shown to be involved in

processing emotional expression in others (Haxby et al 2002) and

insular projections to inferior prefrontal cortex and amygdala may

convey motivation and social information from these stimuli

(Critchley et al 2000) Anterior cingulate has been posited to

reflect emotional awareness (Lane et al 1997a) and cognitive-

emotion interactions (Bush et al 2000 2002) Generally the

processing differences between emotion types detected in these

regions may be partially a reflection of the fact that faces and IAPS

pictures differ on novelty and complexity (Winston et al 2003) If

novelty and complexity of the stimuli do contribute faster

habituation to novelty effects in faces and slower habituation to

novelty in pictures could explain the significant effect in one

modality (eg expressive faces) and a lack of effect resulting from

sustained activation in the other (eg IAPS pictures)

With respect to novelty faces can be viewed as a relatively

unchanging stimulus having consistent facial features (eyes nose

mouth) despite feature changes (raised brows gaping mouth etc)

that depict particular emotional states whereas each IAPS picture

with complex contextual scenes is often more unique and novel

Decreased novelty and resulting habituation of responses to neutral

facial expressions may lead to detectable activations (Fischer et al

2003 Wright et al 2003) whereas sustained novelty (ie similar

levels of novelty) between emotional and nonemotional IAPS

pictures would lend itself to not detecting activation Some

evidence in the literature supports this idea Several regions

including superior temporal gyrus insula and anterior cingulate

have been shown to respond to novel relative to familiar stimuli

(Downar et al 2002 Tulving et al 1994) Insula activation to

fearful faces was detected during early but not later periods

reflecting initial orienting not sustained processing (Williams et al

2004) Even though IAPS pictures have been shown to habituate

with repeated exposure (Phan et al 2003) IAPS pictures depict

emotion-laden scenes portraying a variety of contexts thus

novelty in IAPS pictures may show reduced habituation (ie

sustained activation) effects relative to facial expressions

ARTICLE IN PRESS

Fig 4 Expressive faces activate anterior cingulate insula and superior temporal gyrus more than IAPS pictures SPM t map showing greater BOLD activity to

expressive faces than IAPS pictures in anterior cingulate (ACC) insular cortex (Ins) and superior temporal gyrus (STG) (A) Happy relative to neutral (B) Sad

relative to neutral (C) Fear relative to neutral Activated voxels are displayed with P lt 0005 uncorrected [k] gt 5 voxels threshold

JC Britton et al NeuroImage xx (2006) xxxndashxxx 9

With respect to complexity faces may be processed more

automatically whereas the complex scenes within IAPS pictures

may require additional cognitive processing leading to sustained

activation in all IAPS conditions (including neutral) but not in face

conditions Neuroimaging studies involving masked faces designs

elicit emotional networks despite subjective experience (Whalen et

al 1998) pointing to the automatic processing of facial expressions

In addition emotions in facial expressions are universally recog-

nized (Ekman 1992 1994 Izard 1994) In support in our study

expressive faces showed more agreement on discrete emotion labels

than IAPS pictures This finding is consistent with studies reporting

high agreement of discrete emotion in facial expressions (Carroll

and Russell 1996 Frank and Stennett 2001) Significant activation

may be more easily detected due to automatic but relatively

transient processing of facial expressions On the other hand

subjective reports indicate that IAPS pictures have higher valence

and arousal and increasing intensity may introduce ambiguity The

IAPS pictures have less percent agreement and increased reaction

times than expressive faces which may reflect increased cognitive

demands Processing the context in relation to past experience and

acquired knowledge (memories and associations with emotional

stimulus) may require additional cognitive load Right insula and

ARTICLE IN PRESS

Table 4

Activations to specific emotions relative to neutral

Region Faces Faces gt IAPS pictures IAPS pictures IAPS pictures gt faces

(x y z)a Zb kc (x y z) Z k (x y z) Z k (x y z) Z k

Happy

Superior temporal

gyrus

(63 18 9) 351 58 (57 18 3) 356 160

(63 15 6) 289 8 (60 15 6) 273 10

Insular cortex (30 6 9) 262 2d (39 3 12) 350 161

Visual cortex (54 75 3) 433 78

(45 72 30) 364 8

Sad

Superior temporal

gyrus

(63 27 12) 321 27

Insular cortex (51 12 3) 260 3d (54 12 6) 304 14

(42 21 21) 353 102 (39 24 0) 292 9

Anterior cingulate (0 30 30) 252d 5d (3 36 30) 321 24

Rostral anterior

cingulate

(0 30 12) 310 9

Ventromedial

prefrontal

orbitofrontal

cortex

(9 51 9) 321 65

Visual cortex (0 72 24) 371 43 (0 90 0) 374 211 (30 90 15) 367

Anger

R amygdala (33 3 24) 301 8

Hippocampus (21 21 12) 414 62 (21 12 12) 284 10

Superior temporal

gyrus

(63 6 12) 373 45 (60 6 12) 332 19

(57 24 12) 343 80 (57 15 6) 336 163

Insular cortex (39 18 9) 386 307 (42 24 6) 411 47

Rostral anterior

cingulate

(3 27 3) 296 7

Ventromedial

prefrontal

orbitofrontal

cortex

(15 60 12) 295 7 (12 69 6) 307 8

Fear

L hippocampus (18 27 9) 296 9 (18 27 3) 391 14

Superior temporal

gyrus

(60 33 0) 399 113 (66 33 9) 301 39

Insular cortex (36 15 6) 361 125 (45 9 0) 331 27

(39 9 9) 315 34

Anterior cingulate (3 18 21) 350 7 (0 30 30) 286 8

(0 6 30) 345 42

Visual cortex (9 90 3) 36 40 (9 93 3) 362 28

(0 72 15) 316 12

part of larger clustera Stereotactic coordinates from MNI atlas leftright (x) anteriorposterior ( y) and superiorinferior (z) respectively R = right L = leftb Z score significant after Small Volume Correction (SVC thresholded using a false discovery rate [FDR] correction for multiple comparisons of 0005)c Spatial extent in cluster size threshold 6 voxelsd Subthreshold activations

JC Britton et al NeuroImage xx (2006) xxxndashxxx10

anterior cingulate were activated to an explicit evaluation task

suggesting that these regions may be required to associate personal

reflections and memories in order to make an evaluation of the

stimulus due to increased complexity (Cunningham et al 2004)

Like the case of novelty a significant effect in one modality (eg

faces) and a lack of effect in another (eg IAPS pictures) may be due

to differences in complexity (ie automatic vs effortful response or

innate vs learned associations)

Specific emotions

Specific emotions influenced the subjective ratings and neuro-

imaging activation patterns Subjectively positive emotions were

more easily identified Few labels for positive emotions exist

whereas increased variability in labeling negative emotions result

from increased choices This interpretation is consistent with the

fact that positive emotions may be more general whereas negative

ARTICLE IN PRESSJC Britton et al NeuroImage xx (2006) xxxndashxxx 11

emotions may be more specific (Fredrickson 2001 2004) As seen

in this study recognition of anger and fear is often confused for

one another This variability could result from similarly high levels

of valence and arousal (Carroll and Russell 1996 Davis et al

1995) or the inability to assign agency Anger and fear may elicit a

more intense reaction and increasing emotional intensity may

result in a more complex profile illustrating the increased

difficulty in distinguishing emotions Variability in the reports

may also result because a complementary emotion is activated

rather than mimicked (ie an angry faces makes the observer

fearful)

Some specific emotions contributed more substantially to the

regions activated by both expressive faces and IAPS pictures

Amygdala activity has been most reported in response to fearful

stimuli (Breiter et al 1996 Downar et al 2001 Hariri et al

2003 Morris et al 1996 Phillips et al 1997 Whalen et al

2001) In this study amygdala activity relative to fixation was

detected to fearful faces like all other specific emotional faces

However relative to neutral amygdala activity was detected in

response to anger faces only and not fearful faces In studies

examining anger and fear faces only amygdala activation was

significantly greater for fearful faces compared to anger faces

however these studies did not incorporate additional specific

emotions or additional stimulus types (Whalen et al 2001) Some

studies have suggested that this discrepancy may be explained by

the inclusion of other conditions that will influence activation

within the amygdala (Somerville et al 2004) Hippocampus

activated in response to both types of anger and fear stimuli more

significantly than neutral stimuli Since this hippocampal activation

was present in both faces and pictures it appears that hippocampus

may be more responsive to specific emotion rather than stimulus

type Ventromedial prefrontal cortex activated in response to anger

stimuli and sad faces Since no significant difference between

modality was detected in this region for sad stimuli the

ventromedial prefrontal cortex like the hippocampus seems to

respond to specific emotion

All specific emotions contributed to activations in STG and

insula in response to faces and most specific emotions contributed

to the differences in activation patterns seen between expressive

faces and IAPS pictures Happy anger and fear emotions

contributed to differences between expressive faces and IAPS

pictures in the superior temporal gyrus This finding may reflect

sensitivity to detect differences between facial expressions when

presented in isolation rather than the variability introduced by

expressions within a greater context as presented in IAPS pictures

Insula activation is typically found when recognizing disgust faces

(Phillips et al 1997) however happy sad and fear contributed to

the insula differences between modalities in this study This finding

suggests that the insula may play a role in general emotional

processing with respect to specific emotion (Phan et al 2002) but

also points to its preference to processing faces

A subset of negative emotions contributed to the anterior

cingulate activation differences between modalities For more

dorsal regions of the anterior cingulate the specific emotions of

sad and fear showed preferentially processing to faces both

activating and showing significant difference compared to IAPS

pictures In previous studies when a rating task is compared to a

perceptually matching task ACC activation was detected (Hariri et

al 2003) Thus this ACC activation may indicate that the negative

emotions in faces are being evaluated to a greater extent compared

to IAPS pictures For more rostral regions sad and anger faces

show greater activity than IAPS pictures While dorsal anterior

cingulate showed activation to faces and neither activation nor

deactivation to pictures the rostral anterior cingulate showed a

differential activity pattern From region of interest (ROI) analysis

we noted that sad and anger faces tend to activate while sad and

anger pictures tend to deactivate in the rostral anterior cingulate

cortex Even though the rACC is implicated in self-induced

sadness and depression (Mayberg 1997 Mayberg et al 1999)

and it may not be surprising to find sadness differentially activated

by faces one must take caution interpreting these results given that

the findings within each modality are nonsignificant Overall

though these findings suggest an interaction between specific

emotion and emotion type influences activation within anterior

cingulate regions

Several limitations should be noted when interpreting the results

of these studies First expressive faces and IAPS pictures were

balanced in terms of the predominant emotion but valence and

arousal varied expressive faces had lower valence and arousal This

result however offers an advantage in analyzing differential results

because neural activation patterns showed greater activity to

expressive faces despite lower arousal ratings Secondly forced

choice methods to determine specific emotions may reflect response

bias and demand characteristics however this bias is present in both

IAPS and expressive faces Even though the presence of mixed

emotions was minimized this method may not have completely

eliminated this effect Thirdly the two stimulus sets may be

unbalanced with respect to complexity intentionality and sociality

Despite these differences critical comparisons attempted to lsquolsquosub-

tracted outrsquorsquo the effects of stimulus type by comparing the emotional

stimulus relative to its neutral (eg emotional facendashneutral face and

emotional IAPSndashneutral IAPS) isolating the contribution of

emotionality above and beyond the processing of the stimuli

properties contained within each Even though we attempted to

lsquolsquosubtract outrsquorsquo effects of stimulus type the expressive face set may

be more balanced due to the cohesive properties of faces but more

susceptible to habituation effects whereas IAPS pictures may have

increased variability due to contextual differences but more resistant

to habituation effects Future investigations are needed to tease apart

these components Additionally the faces both emotional and

neutral may be more inherently social than the IAPS pictures given

their role in social communication Alternatively IAPS pictures may

be characterized by more variable interpersonal interactions Future

studies need to determine how sociality influences these neural

activation patterns Next the limited number of TR volumes

collected per emotion condition may have yielded low power

resulting in a failure to detect additional differences However it

should be noted that even at similar levels of power the differences

between facial expressions and IAPS pictures were detected Finally

this analysis assumed a canonical hemodynamic response function

however emotions elicited by facial expressions and IAPS pictures

may have different temporal dynamics (Siegle et al 2002) and this

warrants further exploration

In summary this study begins to elucidate the underlying

functional regions that are common and different among emotional

stimulus types that emphasize emotion recognition or emotion

evocation in direct comparison in the same subjects Even though

expressive faces may predominantly involve emotion recognition

and IAPS pictures predominantly involve emotion evocation both

expressive faces and IAPS pictures recruit similar brain regions

reflected in a common pattern of activation which included

amygdala hippocampus ventromedial prefrontal cortex and visual

ARTICLE IN PRESSJC Britton et al NeuroImage xx (2006) xxxndashxxx12

cortex This common activation pattern further confirms the role

these regions have in general emotional processing Some brain

regions however respond preferentially to a particular emotional

stimulus type In this study a differential pattern of activation was

detected in superior temporal gyrus insula and anterior cingulate

with more activation to expressive faces compared to IAPS pictures

Inherent properties unique to the specific emotional stimuli (eg

novelty complexity sociality) may have yielded the differential

pattern of brain activation In addition the effects of specific

emotions and their interactions with stimulus types may also be

contributing to these differential patterns These findings may aid in

determining the optimal stimulus selection for probing general

emotion processing emotion recognition emotion induction and

specific emotions Although further replication using other emo-

tional stimulus probes (eg Ekman faces evocative films) is needed

Acknowledgments

Wewish to thank Ruben Gur and his colleagues at the University

of Pennsylvania for graciously sharing with us their stimuli set of

facial expressions and Margaret Bradley Peter Lang and the NIMH

Center for the Study of Emotion and Attention (CSEA) at the

University of Florida for providing us with the set of IAPS pictures

Supported by Veterans Education and Research Association of

Michigan and National Institutes of Mental Health (NIMH)

National Research Service Award (NRSA) F31MH069003 to JCB

References

Adolphs R Baron-Cohen S Tranel D 2002 Impaired recognition of

social emotions following amygdala damage J Cogn Neurosci 14 (8)

1264ndash1274

Anderson AK Phelps EA 2001 Lesions of the human amygdala impair

enhanced perception of emotionally salient events Nature 411 (6835)

305ndash309

Ashburner J Neelin P Collins DL Evans A Friston K 1997

Incorporating prior knowledge into image registration NeuroImage 6

(4) 344ndash352

Beauregard M Levesque J Bourgouin P 2001 Neural correlates of

conscious self-regulation of emotion J Neurosci 21 (18) RC165

Bechara A Tranel D Damasio H Adolphs R Rockland C Damasio

AR 1995 Double dissociation of conditioning and declarative

knowledge relative to the amygdala and hippocampus in humans

Science 269 (5227) 1115ndash1118

Blair RJ Morris JS Frith CD Perrett DI Dolan RJ 1999

Dissociable neural responses to facial expressions of sadness and anger

Brain 122 (Pt 5) 883ndash893

Breiter HC Etcoff NL Whalen PJ Kennedy WA Rauch SL

Buckner RL et al 1996 Response and habituation of the human

amygdala during visual processing of facial expression Neuron 17 (5)

875ndash887

Bush G Luu P Posner MI 2000 Cognitive and emotional influences

in anterior cingulate cortex Trends Cogn Sci 4 (6) 215ndash222

Bush G Vogt BA Holmes J Dale AM Greve D Jenike MA

et al 2002 Dorsal anterior cingulate cortex a role in reward-based

decision making Proc Natl Acad Sci U S A 99 (1) 523ndash528

Calder AJ Burton AM Miller P Young AW Akamatsu S 2001 A

principal component analysis of facial expressions Vision Res 41 (9)

1179ndash1208

Carroll JM Russell JA 1996 Do facial expressions signal specific

emotions judging emotion from the face in context J Pers Soc

Psychol 70 (2) 205ndash218

Clark BM Siddle DA Bond NW 1992 Effects of social anxiety and

facial expression on habituation of the electrodermal orienting response

Biol Psychol 33 (2ndash3) 211ndash223

Critchley H Daly E Phillips M Brammer M Bullmore E Williams

S et al 2000 Explicit and implicit neural mechanisms for processing

of social information from facial expressions a functional magnetic

resonance imaging study Hum Brain Mapp 9 (2) 93ndash105

Cunningham WA Raye CL Johnson MK 2004 Implicit and

explicit evaluation FMRI correlates of valence emotional intensity

and control in the processing of attitudes J Cogn Neurosci 16 (10)

1717ndash1729

Damasio AR 1999 The Feeling of What Happens Body and Emotion in

the Making of Consciousness first edR Harcourt Brace New York

Darwin C 1998 The Expression of the Emotions in Man and Animal

third edition Oxford Univ Press New York

Davidson RJ Jackson DC Kalin NH 2000 Emotion plasticity

context and regulation perspectives from affective neuroscience

Psychol Bull 126 (6) 890ndash909

Davis WJ Rahman MA Smith LJ Burns A Senecal L McArthur

D et al 1995 Properties of human affect induced by static color slides

(IAPS) dimensional categorical and electromyographic analysis Biol

Psychol 41 (3) 229ndash253

Dolan RJ Fletcher P Morris J Kapur N Deakin JF Frith CD

1996 Neural activation during covert processing of positive emotional

facial expressions NeuroImage 4 (3 Pt 1) 194ndash200

Downar J Crawley AP Mikulis DJ Davis KD 2001 The effect of

task relevance on the cortical response to changes in visual and auditory

stimuli an event-related fMRI study NeuroImage 14 (6) 1256ndash1267

Downar J Crawley AP Mikulis DJ Davis KD 2002 A cortical

network sensitive to stimulus salience in a neutral behavioral context

across multiple sensory modalities J Neurophysiol 87 (1) 615ndash620

Ekman P 1992 Are there basic emotions Psychol Rev 99 (3) 550ndash553

Ekman P 1994 Strong evidence for universals in facial expressions a

reply to Russellrsquos mistaken critique Psychol Bull 115 (2) 268ndash287

Esteves F Ohman A 1993 Masking the face recognition of emotional

facial expressions as a function of the parameters of backward masking

Scand J Psychol 34 (1) 1ndash18

Fischer H Wright CI Whalen PJ McInerney SC Shin LM Rauch

SL 2003 Brain habituation during repeated exposure to fearful and

neutral faces a functional MRI study Brain Res Bull 59 (5) 387ndash392

Frank MG Stennett J 2001 The forced-choice paradigm and the

perception of facial expressions of emotion J Pers Soc Psychol 80

(1) 75ndash85

Fredrickson BL 2001 The role of positive emotions in positive

psychology The broaden-and-build theory of positive emotions Am

Psychol 56 (3) 218ndash226

Fredrickson BL 2004 The broaden-and-build theory of positive

emotions Philos Trans R Soc London Ser B Biol Sci 359 (1449)

1367ndash1378

Fried I MacDonald KA Wilson CL 1997 Single neuron activity in

human hippocampus and amygdala during recognition of faces and

objects Neuron 18 (5) 753ndash765

Friston KJ 1998 Generalisability random effects and population

inference NeuroImage 7 S754

Friston KJ Holmes AP Worsley KJ Poline JB Frith CD

Frackowiak RS 1995 Statistical parametric maps in functional

imaging a general linear approach Hum Brain Mapp 2 189ndash210

Geday J Gjedde A Boldsen AS Kupers R 2003 Emotional valence

modulates activity in the posterior fusiform gyrus and inferior medial

prefrontal cortex in social perception NeuroImage 18 (3) 675ndash684

Genovese CR Lazar NA Nichols T 2002 Thresholding of statistical

maps in functional neuroimaging using the false discovery rate

NeuroImage 15 (4) 870ndash878

Gur RC Schroeder L Turner T McGrath C Chan RM Turetsky

BI et al 2002 Brain activation during facial emotion processing

NeuroImage 16 (3 Pt 1) 651ndash662

Halberstadt JB Niedenthal PM 1997 Emotional state and the use

ARTICLE IN PRESSJC Britton et al NeuroImage xx (2006) xxxndashxxx 13

of stimulus dimensions in judgment J Pers Soc Psychol 72 (5)

1017ndash1033

Hariri AR Bookheimer SY Mazziotta JC 2000 Modulating

emotional responses effects of a neocortical network on the limbic

system NeuroReport 11 (1) 43ndash48

Hariri AR Mattay VS Tessitore A Fera F Weinberger DR 2003

Neocortical modulation of the amygdala response to fearful stimuli

Biol Psychiatry 53 (6) 494ndash501

Hatfield E Cacioppo JT Rapson RL 1992 Primitive emotional

contagion Rev Person Soc Psychol 14 151ndash177

Haxby JV Petit L Ungerleider LG Courtney SM 2000 Distinguish-

ing the functional roles of multiple regions in distributed neural systems

for visual working memory NeuroImage 11 (5 Pt 1) 380ndash391

Haxby JV Hoffman EA Gobbini MI 2002 Human neural systems

for face recognition and social communication Biol Psychiatry 51 (1)

59ndash67

Hornak J Rolls ET Wade D 1996 Face and voice expression

identification in patients with emotional and behavioural changes

following ventral frontal lobe damage Neuropsychologia 34 (4)

247ndash261

Izard CE 1994 Innate and universal facial expressions evidence

from developmental and cross-cultural research Psychol Bull 115

(2) 288ndash299

Killgore WD Yurgelun-Todd DA 2004 Activation of the amygdala

and anterior cingulate during nonconscious processing of sad versus

happy faces NeuroImage 21 (4) 1215ndash1223

Kim H Somerville LH Johnstone T Alexander AL Whalen PJ

2003 Inverse amygdala and medial prefrontal cortex responses to

surprised faces NeuroReport 14 (18) 2317ndash2322

Lane RD Fink GR Chau PM Dolan RJ 1997a Neural activation

during selective attention to subjective emotional responses Neuro-

Report 8 (18) 3969ndash3972

Lane RD Reiman EM Ahern GL Schwartz GE Davidson RJ

1997b Neuroanatomical correlates of happiness sadness and disgust

Am J Psychiatry 154 (7) 926ndash933

Lane RD Reiman EM Bradley MM Lang PJ Ahern GL

Davidson RJ et al 1997c Neuroanatomical correlates of pleasant

and unpleasant emotion Neuropsychologia 35 (11) 1437ndash1444

Lane RD Chua PM Dolan RJ 1999 Common effects of emotional

valence arousal and attention on neural activation during visual

processing of pictures Neuropsychologia 37 (9) 989ndash997

Lang PJ Greenwald MK Bradley MM Hamm AO 1993 Looking

at pictures affective facial visceral and behavioral reactions

Psychophysiology 30 (3) 261ndash273

Lang PJ Bradley MM Cuthbert BN (1997) International Affective

Picture System (IAPS) Technical Manual and Affective Ratings

Gainesville FL NIMH Center for the Study of Emotion and Attention

University of Florida

LeDoux JE 2000 Emotion circuits in the brain Annu Rev Neurosci 23

155ndash184

Levesque J Joanette Y Mensour B Beaudoin G Leroux JM

Bourgouin P et al 2003 Neural correlates of sad feelings in healthy

girls Neuroscience 121 (3) 545ndash551

Liberzon I Taylor SF Fig LM Decker LR Koeppe RA

Minoshima S 2000 Limbic activation and psychophysiologic

responses to aversive visual stimuli Interaction with cognitive task

Neuropsychopharmacology 23 (5) 508ndash516

Liberzon I Phan KL Decker LR Taylor SF 2003 Extended

amygdala and emotional salience a PET activation study of

positive and negative affect Neuropsychopharmacology 28 (4)

726ndash733

Mayberg HS 1997 Limbic-cortical dysregulation a proposed model of

depression J Neuropsychiatry Clin Neurosci 9 (3) 471ndash481

Mayberg HS Liotti M Brannan SK McGinnis S Mahurin RK

Jerabek PA et al 1999 Reciprocal limbic-cortical function and

negative mood converging PET findings in depression and normal

sadness Am J Psychiatry 156 (5) 675ndash682

Milad MR Quirk GJ 2002 Neurons in medial prefrontal cortex signal

memory for fear extinction Nature 420 (6911) 70ndash74

Milad MR Vidal-Gonzalez I Quirk GJ 2004 Electrical stimulation of

medial prefrontal cortex reduces conditioned fear in a temporally

specific manner Behav Neurosci 118 (2) 389ndash394

Morris JS Frith CD Perrett DI Rowland D Young AW Calder

AJ 1996 A differential neural response in the human amygdala to

fearful and happy facial expressions Nature 383 (6603) 812ndash815

Morris JS Friston KJ Buchel C Frith CD Young AW Calder

AJ et al 1998 A neuromodulatory role for the human amygdala in

processing emotional facial expressions Brain 121 (Pt 1) 47ndash57

Mourao-Miranda J Volchan E Moll J de Oliveira-Souza R Oliveira

L Bramati I et al 2003 Contributions of stimulus valence and

arousal to visual activation during emotional perception NeuroImage

20 (4) 1955ndash1963

Narumoto J Okada T Sadato N Fukui K Yonekura Y 2001

Attention to emotion modulates fMRI activity in human right superior

temporal sulcus Brain Res Cogn Brain Res 12 (2) 225ndash231

Ochsner KN Bunge SA Gross JJ Gabrieli JD 2002 Rethinking

feelings an FMRI study of the cognitive regulation of emotion J Cogn

Neurosci 14 (8) 1215ndash1229

Ongur D Price JL 2000 The organization of networks within the orbital

and medial prefrontal cortex of rats monkeys and humans Cereb

Cortex 10 (3) 206ndash219

Oppenheim A Schafer R 1989 Discrete-Time Signal Processing

Englewood Cliffs Prentice Hall NJ

Pessoa L Kastner S Ungerleider LG 2002 Attentional control of the

processing of neural and emotional stimuli Brain Res Cogn Brain Res

15 (1) 31ndash45

Phan KL Wager T Taylor SF Liberzon I 2002 Functional

neuroanatomy of emotion a meta-analysis of emotion activation studies

in PET and fMRI NeuroImage 16 (2) 331ndash348

Phan KL Liberzon I Welsh RC Britton JC Taylor SF 2003

Habituation of rostral anterior cingulate cortex to repeated emotionally

salient pictures Neuropsychopharmacology 28 (7) 1344ndash1350

Phan KL Taylor SF Welsh RC Ho SH Britton JC Liberzon I

2004 Neural correlates of individual ratings of emotional salience a

trial-related fMRI study NeuroImage 21 (2) 768ndash780

Phan KL Fitzgerald DA Nathan PJ Moore GJ Uhde TW Tancer

ME 2005 Neural substrates for voluntary suppression of negative

affect a functional magnetic resonance imaging study Biol Psychiatry

57 (3) 210ndash219

Phillips ML Young AW Senior C Brammer M Andrew C Calder

AJ et al 1997 A specific neural substrate for perceiving facial

expressions of disgust Nature 389 (6650) 495ndash498

Quirk GJ Likhtik E Pelletier JG Pare D 2003 Stimulation of

medial prefrontal cortex decreases the responsiveness of central

amygdala output neurons J Neurosci 23 (25) 8800ndash8807

Schneider W Eschman A Zuccolotto A 2002a E-Prime Reference

Guide Psychology Software Tools Inc Pittsburgh

Schneider W Eschman A Zuccolotto A 2002b E-Prime Userrsquos Guide

Psychology Software Tools Inc Pittsburgh

Sheehan D Janavs J Baker R Harnett-Sheehan K Knapp E

Sheehan M 1998 Mini International Neuropsychiatric Interview

English Version 500 DSM-IV

Siegle GJ Steinhauer SR Thase ME Stenger VA Carter CS

2002 Canrsquot shake that feeling event-related fMRI assessment of

sustained amygdala activity in response to emotional information in

depressed individuals Biol Psychiatry 51 (9) 693ndash707

Simpson JR Ongur D Akbudak E Conturo TE Ollinger JM

Snyder AZ et al 2000 The emotional modulation of cognitive

processing an fMRI study J Cogn Neurosci 12 (Suppl 2)

157ndash170

Somerville LH Kim H Johnstone T Alexander AL Whalen PJ

2004 Human amygdala responses during presentation of happy and

neutral faces correlations with state anxiety Biol Psychiatry 55 (9)

897ndash903

ARTICLE IN PRESSJC Britton et al NeuroImage xx (2006) xxxndashxxx14

Taylor SF Phan KL Decker LR Liberzon I 2003 Subjective rating

of emotionally salient stimuli modulates neural activity NeuroImage 18

(3) 650ndash659

Tulving E Markowitsch HJ Kapur S Habib R Houle S 1994

Novelty encoding networks in the human brain positron emission

tomography data NeuroReport 5 (18) 2525ndash2528

Whalen PJ Rauch SL Etcoff NL McInerney SC Lee MB Jenike

MA 1998 Masked presentations of emotional facial expressions

modulate amygdala activity without explicit knowledge J Neurosci 18

(1) 411ndash418

Whalen PJ Shin LM McInerney SC Fischer H Wright CI

Rauch SL 2001 A functional MRI study of human amygdala

responses to facial expressions of fear versus anger Emotion 1 (1)

70ndash83

Wild B Erb M Bartels M 2001 Are emotions contagious Evoked

emotions while viewing emotionally expressive faces quality

quantity time course and gender differences Psychiatry Res 102

(2) 109ndash124

Williams LM Brown KJ Das P Boucsein W Sokolov EN

Brammer MJ 2004 The dynamics of cortico-amygdala and auto-

nomic activity over the experimental time course of fear perception

Brain Res Cogn Brain Res 21 (1) 114ndash123

Winston JS OrsquoDoherty J Dolan RJ 2003 Common and distinct

neural responses during direct and incidental processing of multiple

facial emotions NeuroImage 20 (1) 84ndash97

Woods RP Grafton ST Watson JD Sicotte NL Mazziotta

JC 1998 Automated image registration II Intersubject validation

of linear and nonlinear models J Comput Assist Tomogr 22 (1)

153ndash165

Wright CI Martis B Schwartz CE Shin LM Fischer HH

McMullin K et al 2003 Novelty responses and differential effects

of order in the amygdala substantia innominata and inferior temporal

cortex NeuroImage 18 (3) 660ndash669

Yang TT Menon V Eliez S Blasey C White CD Reid AJ et al

2002a Amygdalar activation associated with positive and negative

facial expressions NeuroReport 13 (14) 1737ndash1741

Yang Y Gu H Zhan W Xu S Silbersweig DA Stern E 2002b

Simultaneous perfusion and BOLD imaging using reverse spiral

scanning at 3T characterization of functional contrast and susceptibility

artifacts Magn Reson Med 48 (2) 278ndash289