Single-Trial P3 Amplitude and Latency Informed Event-related fMRI Models Yield

NeuroImage 47 (2009) 1532–1544

Contents lists available at ScienceDirect

NeuroImage

j ourna l homepage: www.e lsev ie r.com/ locate /yn img

Single-trial P3 amplitude and latency informed event-related fMRI models yielddifferent BOLD response patterns to a target detection task

T. Warbrick a,b,⁎, A. Mobascher a,b, J. Brinkmeyer a,b, F. Musso a, N. Richter a, T. Stoecker b, G.R. Fink b,c,N.J. Shah b, G. Winterer a,b

a Neuropsychiatric Research Laboratory, Department of Psychiatry, Heinrich-Heine University, Bergische Landstr. 2, 40629 Duesseldorf, Germanyb Institute of Neurosciences and Medicine, Helmholtz Research Center Juelich, Germanyc Department of Neurology, University of Cologne, Germany

⁎ Corresponding author. Neuropsychiatric ResearchPsychiatry, Heinrich-Heine University, Bergische LaGermany. Fax: +49 211 922 2759.

E-mail address: [email protected] (T. Warbric

1053-8119/$ – see front matter © 2009 Elsevier Inc. Aldoi:10.1016/j.neuroimage.2009.05.082

a b s t r a c t
a r t i c l e i n f o
Article history:Received 4 December 2008Revised 19 May 2009Accepted 26 May 2009Available online 6 June 2009

Keywords:Event Related Potentials (ERPs)functional Magnetic Resonance Imaging(fMRI)Single-trialP3 amplitudeP3 latencyMultimodal imaging

Using single-trial parameters as a regressor in the General Linear Model (GLM) is becoming an increasinglypopular method for informing fMRI analysis. However, the parameter used to characterise or to differentiatebrain regions involved in the response to a particular task varies across studies (e.g. ERP amplitude, ERPlatency, reaction time). Furthermore, the way in which the single-trial information is used in the fMRIanalysis is also important. For example, the single-trial parameters can be used as regressors in the GLM or tomodify the duration of the events modelled in the GLM. The aim of this study was to investigate the BOLDresponse to a target detection task when including P3 amplitude, P3 latency and reaction time parameters inthe GLM. Simultaneous EEG-fMRI was recorded from fifteen subjects in response to a visual choice reactiontime task. Including P3 amplitude as a regressor in the GLM yielded activation in left central opercular cortex,left postcentral gyrus, left insula, left middle frontal gyrus, left insula and left parietal operculum. Using P3latency and reaction time as an additional regressor yielded no additional activation in comparison with theconventional fMRI analysis. However, when P3 latency or reaction time was used to determine the durationof events at a single-trial level, additional activation was observed in the left postcentral gyrus, left precentralgyrus, anterior cingulate cortex and supramarginal gyrus. Our findings suggest that ERP amplitudes andlatencies can yield different activation patterns when used to modify relevant aspects of the GLM.

© 2009 Elsevier Inc. All rights reserved.

Introduction

The attraction of simultaneous electroencephalography (EEG) andfunctional Magnetic Resonance Imaging (fMRI) recording lies in thecombination of high spatial resolution of fMRI and superb temporalresolution of EEG for responses to the same task under the sameconditions. A number of approaches have been used to fuse EEG andfMRI data; for example fMRI informed sourcemodelling (Strobel et al.,2008) andmodulation of separate session EEG and fMRI by parametrictask manipulation (Horovitz et al., 2002). While these approacheshave yielded valuable findings, an approach receiving an increasingamount of attention is the use of single-trial parameters from the EEGsignal as a regressor in the General Linear Model (GLM) whenanalysing fMRI data. The individual components of Event RelatedPotentials (ERPs) represent different aspects of stimulus processing ona millisecond time scale, a temporal resolution that is not available infMRI data. Thus, using relevant ERP parameters in the analysis of fMRI

Laboratory, Department ofndstr. 2, 40629 Duesseldorf,

k).

l rights reserved.

data can identify a task/process related fMRI activation by exploitingthe temporal resolution of EEG (Debener et al., 2005; Eichele et al.,2005; Mulert et al., 2008). One particular advantage of the single-trialapproach is that it can utilise variability in responses that is lost duringstandard averaging. This variability is often viewed as noise but mayreflect differences in the response to individual trials (Bagshaw andWarbrick, 2007). Using single-trial ERP parameters in the fMRI modelis a way of utilising the information provided by this single-trialvariability to understand cognitive processes. This is particularlyimportant in studies where variability in responses may be indicativeof modulation of stimulus processing or task performance and notsimply irrelevant noise (Debener et al., 2005; Eichele et al., 2005).

Using single-trial ERP amplitude as a regressor when modellingthe Blood Oxygen Level Dependant (BOLD) response has yieldeddifferent activation patterns for different ERP components (Eicheleet al., 2005). Compared to conventional BOLD analysis, more specificactivation patterns (Debener et al., 2005) as well as additionalactivation in task related areas (Mulert et al., 2008) have beenreported. However, the parameters used to characterise or differenti-ate brain regions involved in the response to a particular task varyacross studies. For example, ERP amplitude (Bénar et al., 2007;Debener et al., 2005; Eichele et al., 2005; Mobascher et al., 2009b;

mailto:[email protected]

http://dx.doi.org/10.1016/j.neuroimage.2009.05.082

http://www.sciencedirect.com/science/journal/10538119

1533T. Warbrick et al. / NeuroImage 47 (2009) 1532–1544

Mulert et al., 2008), the latency of the ERP component of interest(Bénar et al., 2007), the ERP time course (Mantini et al., 2009),reaction time (Hagenbeek et al., 2007; Grinband et al., 2008), andelectrodermal activity (Mobascher et al., 2009a) have all been used toprovide additional information to include in the GLM. Naturally, thechoice of task related parameter should be guided by the researchquestion, task and available data. An a priori selection of theappropriate parameter, however, requires specific knowledge of theactivation patterns obtained as a result of those different parameters.In the case of detecting infrequent target stimuli, the amplitude andlatency of the ERP P3 component, the P3 time course and reactiontime may all be considered as possible task related parameters thatcould characterise or differentiate brain regions involved in targetdetection and decision making.

The way in which the single-trial information is used in the fMRIanalysis is also important. For example, an increasingly commonmethod is to use the single-trial amplitude, or in some cases latency,as an additional regressor in the GLM. This is desirable when one isinterested in brain activation that is sensitive to the single-trialvariability of response intensity. However, it has been suggested thatwhen using parameters related to the timing or duration ofcognitive processes, such as the latency of ERP components orreaction times, using the single-trial information to modify theduration of the haemodynamic response function, i.e., the timingaspects of the model is more powerful (Grinband et al., 2008). Toillustrate, the most common way to analyse event-related designs isto use an impulse function where the duration of the event is zero.When investigating cognitive processes, such as target discrimina-tion, the decision process of interest extends some hundreds ofmilliseconds after stimulus onset. So, by using an event durationthat captures this cognitive processing rather than just the instant ofevent onset, a more accurate representation of task related BOLDactivation might be achieved. For example, Grinband et al. (2008)showed that using reaction time as the duration of an event resultedin more powerful and more reliable models. This not only makessense from a cognitive perspective but also from an fMRI dataanalysis standpoint. The haemodynamic response to a decisionmaking process has been shown to vary with the time taken for thesubject to make a response (Connolly et al., 2005; Menon et al.,1998). This is a critical point for tasks where subjects are involved ina decision making process and provide a response. However,surveying recent fMRI studies published in a five month period(Jan–May 2007) Grinband et al. (2008) found that 84% of event-related studies with a decision component assumed that the timenecessary to process a stimulus or generate a response was constantfor all trials and all subjects.

The study presented here focuses on using the single-trial ERPparameters and reaction time to model the BOLD response to a visualtwo choice reaction time task with infrequent target stimuli similar tothat of an oddball task. The P3 component of the ERP represents targetdetection/event categorisation (Halgren et al., 1998; Picton, 1992), ormore broadly, is considered to reflect selective attention and workingmemory processes (Gur et al., 2007; Javitt et al., 2008). In oddballtasks, working memory is fundamental to top-down attention in thesense that whatever requires attention (for example, a visual targetstimulus) has to be maintained in working memory. The workingmemory then biases competition between the multiple bottom-upitems in the stimulus input; as a result, in the neuronal competitionbetween the multiple inputs, the item that receives top-down biasfrom the working memory has an advantage (Rolls et al., 2008).Attention to task related stimuli will vary over the course of anexperiment thus influencing the competition between the multipleinputs and which items are successfully held inworking memory. Thisvariability in attention and working memory processing acrossexperimental trials can be modelled by analysing data at the single-trial level.

The fMRI response to the same tasks that also evoke the ERP P3 inelectrophysiological experiments involves a large distributed networkincluding the supramarginal gyrus, frontal, insula, thalamus, cerebel-lum, occipital–temporal, superior temporal and cingulate regions(Bledowski et al., 2004; Kiehl et al., 2005; Gur et al., 2007;Musso et al.,2006; Winterer et al., 2007; Strobel et al., 2008). Single-trial ERPinformed fMRI analysis of responses to target detection tasks haspredominantly involved using the ERP parameters as a regressor inthe fMRI model (Bénar et al., 2007; Eichele et al., 2005; Mulert et al.,2008), with the exception of Mantini et al. (2009) who used P3timecourse to inform their analysis. Thus, they obtained a P3 responsetime course for each subject which was then convolved with acanonical haemodynamic response function and used as a predictor inthe GLM fMRI analysis. Using this method, they were able todifferentiate two functionally distinct networks, i.e., the ventral(temporo-parietal junction, inferior and middle frontal gyri, anteriorcingulate cortex) and dorsal (intraparietal sulcus, frontal eye field,middle frontal gyrus) attention systems, suggesting that using single-trial ERP timing parameters can provide valuable information formodelling the BOLD response. However, what remains unclear fromthe study of Mantini et al. is what specific information sub-processesare related to the ventral and dorsal attentional network.

The aim of this study is to use simultaneously acquired EEG, fMRIand behavioural data to assess whether different information sub-processes as reflected by single-trial ERP amplitude and latencyparameters can be used to differentiate brain regions of the ventraland dorsal attentional network involved in target detection processes.Specifically, the amplitude of the P3, the latency of the P3 and thereaction time to each target stimulus will be used to inform the fMRIanalysis.

Methods

Subjects

Twenty one healthy subjects participated in the study (10 male,mean age 35.1 years (SD=13.3)). Subjects were recruited from a largepopulation-based database in Germany with no history of medical,neurological or psychiatric illness or alcohol and drug abuse asassessed by a full medical interview and examination, routinelaboratory tests, a drug screening test, an electrocardiogram and astandardized psychiatric interview (SCID) (First et al., 1995). Elevensubjects were smokers and 10 had never smoked. Written informedconsent was obtained from all subjects. The study was conducted incompliance with the declaration of Helsinki and was approved by theethics committee of the Heinrich-Heine University Düsseldorf.

The data for one subject were discarded due to excessive motionartefact in the fMRI, one subject was excluded because of a lack ofsignificant BOLD response to the task, and four further subjects' datawere discarded due to poor EEG quality (inadequate ballistocardio-gram artefact correction for three subjects, and a generally noisysignal for one subject due to the inability to reduce impedance atrecording electrodes to b10 kΩ), resulting in the data from fifteensubjects (8 male, with a mean age of 33.7 years (SD=14.2), 7 subjectswere smokers) being included in the analysis.

Behavioural task

Subjects performed a hybrid two choice reaction time–oddball task(black and white checkerboard reversal) consisting of 256 frequentstimuli and 64 infrequent stimuli that is, the reversal of thecheckerboard from the frequent pattern was the target. The ratio offrequent to infrequent stimuli is similar to that of an oddball task andrequires the subject to identify the target stimuli and make a differentresponse to these stimuli. The frequent stimuli will be referred to asfrequent targets and the infrequent stimuli as infrequent targets.

1534 T. Warbrick et al. / NeuroImage 47 (2009) 1532–1544

Stimuli were presented using Presentation version 11.3 (Neuro-behavioural systems, Albany, CA, USA) via a screen situated behindthe scanner. Subjects were able to view the screen via a mirrormounted on the head coil. Subjects' responses were recorded usingLumitouch key pads (Photon Control Inc, Burnaby, BC, Canada). Forinfrequent target stimuli subjects responded with their right indexfinger and for frequent targets they responded with their left indexfinger; they were asked to respond quickly and accurately to eachstimulus and reaction time was recorded for each response. Stimuliwere presented with a duration of 1000 ms and a pseudorandomisedinterstimulus interval of 4000 (±2500) ms.

fMRI data acquisition

Functional MR-images were acquired using a 3 T scanner (Trio,Siemens, Erlangen, Germany). In order to avoid head movements, thehead of each subject was secured using sponge pads. Using echoplanar imaging (EPI), 630 volumes were obtained applying thefollowing EPI parameters: 33 slices, slice thickness 3 mm, FOV200×200 mm, 64×64 matrix, repetition time 2000 ms, echo time30 ms, and flip angle 90°. To facilitate localisation and co-registrationof functional data, structural scans were acquired using T1-weightedMRI sequences (Magnetization prepared rapid gradient echo (MP-RAGE): TR/TE=2250/3.03 ms, flip angle=9°, 176 sagittal slices, FOV200×200 mm, 64×64 matrix, and voxel size 1×1×1 mm.

fMRI analysis

fMRI analysis was performed with FSL (FMRIB's Software Library,www.fmrib.ox.ac.uk/fsl), employing different modules of the FSL-software package, non-brain removal using BET (Smith, 2002), spatialsmoothing using a Gaussian kernel of FWHM=8 mm, mean-basedintensity normalization of all volumes by the same factor, andhighpass temporal filtering (sigma=125 s). General linear model(GLM) time-series statistical analysis of individual data sets wascarried out using FILM (FMRIB's Improved Linear Model) with localautocorrelation correction (Woolrich et al., 2001). Registration offunctional images to high resolution structural images was donewith FLIRT (Forman et al., 1995; Jenkinson et al., 2002). Explanatoryvariables were convolved with a gamma haemodynamic responsefunction. A description of the explanatory variables constructed andthe different models used is provided below. Group level mixedeffect analyses were conducted using FLAME (FMRIB's LocalAnalysis of Mixed Effects) (Behrens et al., 2003) with spatialnormalization to MNI (Montreal Neurological Institute) space andapplying a cluster significance threshold of ZN2.3 (Forman et al.,1995; Friston et al., 1994; Worsley et al., 1992) with the exception ofthe models including single-trial parameters as an additionalregressor where a cluster significance threshold of ZN2.0 wasapplied. Functional data were imported to MRIcron (Rorden et al.,2007) for visual display purposes.

EEG data acquisition

EEG data were recorded using a 32-channel MR compatible EEGsystem (Brain Products, Gilching, Germany). The EEG cap (BrainCapMR, EasyCap GmbH, Breitbrunn, Germany) consisted of 30 scalpelectrodes distributed according to the 10–20 system and twoadditional electrodes, one of which was attached to the subjects'back for recording the electrocardiogram (ECG), while the other wasattached on the outer canthus of the left eye for detection of ocularartefacts. Datawere recorded relative to an FCz reference and a groundelectrode was located at Iz (10–5 electrode system, Oostenveld andPraamstra, 2001). Data were sampled at 5000 Hz, with a bandpass of0.016–250 Hz. Impedance at all recording electrodes was less than10 kΩ.

EEG data analysis

Raw EEG data were processed offline using BrainVision Analyzer 2(Brain Products, Gilching, Germany). Gradient artefact correction wasperformed using modified versions of the algorithms proposed byAllen et al. (2000), where a gradient artefact template is subtractedfrom the EEG using a baseline corrected sliding average of 20 MR-volumes. Data were then down-sampled to 250 Hz. Followinggradient artefact correction, the datawere corrected for cardioballisticartefacts. An average artefact subtraction method (Allen et al., 1998)was implemented in Brain Vision Analyzer. This method involvessubtracting the artefact on a second by second basis using heartbeatevents (R peaks) detected in the previous 10 s. As such it requiresaccurate detection of R peaks which is aided by the employment of amoving average low pass filter and a finite impulse response high passfilter (for details, see Allen et al., 1998). In this study, the R peaks weredetected semi-automatically, with manual adjustment for peaksmisidentified by the software. To average the artefact in the EEGchannels, the R peaks are transferred from the ECG to the EEG over aselectable time delay. The average artefact was then subtracted fromthe EEG. Once corrected for gradient and cardioballistic artefacts, thedata were segmented into 1000 ms epochs (−200 ms to 800 ms) forthe purposes of ERP (event-related potential) analysis. Datawere theninspected for artefacts resulting from an eye blink or other muscularsources, andanyepoch containing avoltage change ofmore than 150 μVwas rejected. The remaining epochs were then averaged separately forinfrequent and frequent targets (N50 for infrequent targets and N220for frequent targets for all subjects). Data from Cz were furtherprocessed for ST analysis using wavelet denoising. Wavelet denoisingwas accomplished using EP_den_v2, a freely available tool (http://www.vis.caltech.edu/~rodri/EP_den/EP_den_home.htm) that runs inMATLAB (TheMathworks Inc, Natick, MA, USA). Following themethodofQuianQuiroga andGarcia (2003) the dataweredecomposed intofivewavelet scales, wavelet coefficients were then chosen to optimise theaverage signal up to and including the P3. This optimised denoisingscheme was then applied to the single-trial data. Single-trial P3parameterswere extracted from the data for each subject by identifyingthe peak positive amplitude in a window from 280–550 ms poststimulus. The single-trial latencies and amplitudes were then screenedfor outliers using Z scores; any trials with a value more than 3 standarddeviations away from the mean were discarded; a mean of 89%(SD=6.7) of trials were kept for further analysis.

Combined fMRI/ERP analysis

First we performed a conventional analysis of the data to assessgroup level activation for the task by using one explanatory variablewith a constant impulse function (same height and duration)positioned at stimulus onset (Fig. 1a). It is worth noting at this pointthat in all models the input functions (the onset time, event duration,and event intensity) are convolved with the gamma haemodynamicresponse function which blurs and delays the waveform to match thedifference between the input function and the measured haemo-dynamic response. To test the effects of using ERP parameters tomodify the basis function we used a number of models. First weincluded P3 amplitude as a regressor to provide a basis function withvariable height (Fig. 1b). This results in the height of the basis functionbeing higher when P3 amplitude is larger. This reflects differences inthe intensity of responses, as measured by P3 amplitude. Wehypothesised that this would show activation in areas sensitive tosingle-trial variability in P3 amplitude. For P3 latency we used thesingle-trial latency value in two ways. First, we used single-trial P3peak amplitude latencies as an additional regressor (to modify theheight of the basis function, Fig. 1b). So, the height of the basisfunction increases as P3 latency increases. The idea here is that alonger response would reflect longer processing time and thus a

http://www.fmrib.ox.ac.uk/fsl

http://www.vis.caltech.edu/~rodri/EP_den/EP_den_home.htm

http://www.vis.caltech.edu/~rodri/EP_den/EP_den_home.htm

Fig. 1. Summary of how the basis function is modified by including single-trial parameters in the GLM. Part A shows the conventional analysis where a constant impulse (same heightand duration) is positioned at stimulus onset. Part B shows a variable impulse model where the height of the basis function is modified by single-trial parameters. The curves to theright of the model show how the HRF can be modified by including the parameters, the height changes but the overall shape remains the same. When P3 amplitude was included inthe model this resulted in the height of the basis function being higher when P3 amplitude was larger. This reflects differences in the intensity of responses, as reflected in P3amplitude. When P3 latency and reaction time values were used in this model the height of the basis function increased as P3 latency and reaction time increased. The idea here isthat a longer response would reflect longer processing time and thus a stronger BOLD response. Part C shows a variable epoch model where the duration of the boxcar in the basisfunction is modified by single-trial parameters. The curves to the right of the model show how the HRF can be modified by including the parameters, both the height and the overallshape change. When P3 latency and reaction time are included in this model the length of the boxcar increases with longer P3 latency and reaction times. So, whenwe have a longerP3 latency or reaction timewe expect the decisionmaking process to be longer in those trials, by using these single-trial parameters to determine the epoch length we aim to captureneural activity in an appropriate time window for that response. For all models the input functions (the onset time, event duration, and event intensity) were convolved with thegamma haemodynamic response function which blurs and delays the waveform to match the difference between the input function and the measured haemodynamic response.

Table 1Mean and standard deviation (SD) for the single-trial parameters included in the GLM:P3 amplitude and P3 latency at Cz and reaction time.

Reaction time P3 amplitude P3 latency

Mean SD Mean SD Mean SD

Target 647 117 14.3 9.2 438 92Non-target 598 128 12.5 6.99 451 101

Means and standard deviations were calculated from the single-trial values for eachparameter.


stronger BOLD response, so this approach should model activationsensitive to P3 latency variability. However, Bénar et al. (2007) usedsingle-trial latency values in this way and found an inconsistentpattern of results across subjects and Grinband et al. (2008) foundthat including reaction time as an additional regressor results in theleast powerful models. We therefore expected this model to be theleast informative. Second, we created a variable epoch model wherethe length of the box car in the basis function was determined at thesingle-trial level on the basis of the P3 peak amplitude latencies(Fig. 1c). So, the length of the boxcar increases with longer P3latencies. Increasing the boxcar length results in a change in the heightand overall shape of the HRF. The mean and standard deviation of theparameters included can be seen in Table 1. Whenwe have a longer P3latency we expect the decision making process to be longer in thosetrials; by using these single-trial parameters to determine the epochlength we aim to capture neural activity relating to the duration of thecognitive process of target detection. Previous studies have shownthat considering the time taken to make a decision can be morepowerful for detecting activation related to decision making (Menonet al., 1998; Grinband et al., 2008). Consequently, we hypothesise thatthis model will emphasise target detection related activity, whilebeing aware that time resolution of the BOLD-responses may not besufficient to disentangle different aspects on the basis of thisapproach. For comparison with the work of Grinband et al. (2008)we also used single-trial mean centred reaction times in the sameway.Firstly, by using them to determine the height of the basis function on

a single-trial level to test whether activation related to variability inresponse time could be found (Fig. 1b). Secondly, we used reactiontime to determine the length of the boxcar to look at the wholeduration of the response process (Fig. 1c).

For all variations of our models, we ran the analysis in two ways:firstly, with a single explanatory variable (containing the single-trialinformation) to test the overall effect of these parameters. Secondly, amodel containing two explanatory variables, the first being theconventional analysis (stimulus onset time course with no additionalsingle-trial information) and the secondexplanatory variable containingthe various single-trial parameters. The second EV was orthogonalisedto the first to determine any additional activation explained by theinclusion of single-trial parameters. Orthogonalisation was performedusing the Schmidt–Gram procedure as implemented in FSL's FEAT, theformula is as follows: Aorth=A−B (A.B)/(B.B).

Fig. 2. (A) BOLD activation for the conventional (uninformed) analysis (second-level mixed-effects FLAME. N=19, Cluster-corrected threshold Z=2.3, P=0.05) (B) Grand averageERP at Fz, Cz and Pz (N=15) for infrequent target vs frequent target stimuli. (C) Stacked plot of single-trial ERPs at Cz for one representative subject.


Our main aim was to detect activation related to identifying theinfrequent target stimuli as ‘targets’, this was achieved by includingonly response to infrequent target stimuli in the model. All analysesdescribed above were conducted for responses to infrequent targetstimuli. To detect activation that might be related to the general

Table 2Mean and standard error of the mean (SEM) for the amplitude and latency of the N2and P3 components of the ERP at Fz, Cz, and Pz for target and non target stimuli.

Fz Cz Pz

Target Non-target Target Non-target Target Non-target

N2 amplitude(μV)

Mean −4.01 −2.96 −3.6 −2.7 −2.8 −2.3SEM 0.57 0.41 0.51 0.38 0.45 0.41

N2 latency(ms)

Mean 119 118 120 123 118 116SEM 5.10 5.2 4.7 4.6 5.09 5.15

P3 amplitude(μV)

Mean 5.83 4.58 6.06 4.68 5.63 4.98SEM 6.67 2.59 5.78 2.08 2.59 .88

P3 latency(ms)

Mean 459 444 453 450 491 449SEM 22.4 20.74 23.4 20.98 25.7 20.57

Peak amplitude and latency values were extracted from the average ERP for eachsubject for each condition, these values were then used to calculate mean and SEM forthe group.

processing of the visual stimuli presented during the task we also ranthe P3 amplitude informed variable impulse model and P3 latencyinformed variable epoch model with responses to all stimuli includedin the model. This also allowed us to determine activation specificallyassociated with responding to infrequent target stimuli.

Table 3Brain regions and local maxima for the standard analysis of responses to target stimuli.

Region (Harvard-Oxford,maximum probability)

MNI coordinates oflocal maxima (x, y, z)

Maximum Z score

Left postcentral gyrus −40, −28, 68 5.74−40, −26, 60 5.67−40, −22, 60 5.64

Cerebelluma 22, −50, −26 5.246, −62, −20 4.76

Right precentral gyrus 42, 8, 28 4.0940, 4, 32 3.94

Right insula 34, 22, 2 4.03Left insula −42, 2, −2 4.45Left occipital pole −32, −96, −2 4.61

−26, −100, 4 4.54−26, −100, 4 4.49

a MNI label.


Region of interest analysis

Region of interest (ROI) analysis was performed for the modelsusing single-trial P3 amplitude as an additional regressor and themodel using P3 latency to create variable epochs for the infrequenttarget stimuli. Masks were created for the regions of interest based onthe additional group level activation in each orthogonalised model(“added value”). For each model one mask was created and includedall active voxels in the added value analysis (Z=2.0 for the modelincluding amplitude as a regressor, Z=2.3 for the variable epochmodel). For the P3 amplitudemodel themask included the left centralopercular cortex, left postcentral gyrus, left insula, left middle frontalgyrus, and left parietal operculum (see Fig. 3a). For the P3 latencyvariable epochmodel themask included the left postcentral gyrus andleft precentral gyrus (Fig. 3c).

Statistical analysis

Pearson's correlation coefficient r was used to test the relation-ships between measures using the SPSS (SPSS Inc, Chicago, IL, USA)statistical analysis programme.

Results

Behavioural data

All subjects performed the task with an average of 98.6% (SD 3.3%)correct responses. No false responses were recorded but an average of1.4% (SD 3.3%) stimuli was missed. Mean reaction time was 647 ms(SD=117) for infrequent target stimuli and 598 ms (SD=128) forfrequent target stimuli.

EEG data

The grand average ERP showed the typical morphology in responseto visual stimuli during a decision making task at central electrodeswith N1 (104 ms poststimulus) and P2 (164 ms poststimulus) to allstimuli and a P3 (412 ms poststimulus) that was more prominent forthe infrequent target stimuli than the infrequent target stimuli(Fig. 2b). A summary of the ERP data can also be seen in Supple-mentary Figure 1. P3 data for responses to infrequent target stimuli atthe single-trial level can be seen in the stacked plot in Fig. 2c andsummary statistics of the amplitude and latency of the N2 and P3peaks can be seen in Table 2.

fMRI data

Conventional fMRI analysisThe conventional BOLD analysis revealed activation in response to

infrequent target stimuli in the postcentral gyrus, precentral gyrus,cerebellum, supramarginal gyrus, insula, frontal operculum, inferiorfrontal gyrus, middle frontal gyrus, and occipital cortex (Fig. 2a; seeTable 3 for MNI coordinates andmean Z values). The models includingresponses to all stimuli also yielded activation in these areas(Supplementary Figure 2). Similarly, all single-trial informed modelswith a single EV revealed a similar pattern of activation (Fig. 3, leftcolumn).

P3 amplitude informed fMRI analysisFor the infrequent target stimuli the model including single-trial

P3 amplitude as a regressor orthogonalised to the conventionalanalysis, in other words the added value of P3 amplitude as aregressor, yielded a peak cluster in the central opercular cortex,including precentral gyrus, temporal pole and planum polare. Addi-tional activationwas observed in the insula, and left parietal operculum(Fig. 3, right column, Fig. 4 and Table 2). No additional activation was

gained for the P3 amplitude informed model when responses to allstimuli were included.

P3 latency informed fMRI analysisThe model using P3 latency as a second regressor did not result in

any additional activation (added value) (Fig. 3, left column). However,when using the single-trial P3 latency to create a variable epochmodel, additional activation was found for the analysis involvinginfrequent targets only and that involving all stimuli. For infrequenttarget stimuli activationwas observed in the left postcentral gyrus, leftprecentral gyrus, thalamus, parietal operculum and supramarginalgyrus (see Fig. 3, left column, Fig. 4, and Table 2). For all stimuliadditional activation was found in the right precentral gyrus, rightpostcentral gyrus, cerebellum, right lateral occipital cortex andanterior cingulate cortex (ACC) (Fig. 4). The activation observed inthe ACC for the analysis including all stimuli was not observed for theinfrequent target only analysis.

The peak clusters of additional activation gained for the P3latency informed variable epoch models could reflect the lateralisedmotor response required as part of the task; predominantly leftprecentral gyrus activation for the right handed response toinfrequent targets and right precentral gyrus activation for the lefthanded response associated with the majority of responses for allstimuli. This would be expected since the variable epoch modeldetermined by P3 latency would include the cognitive processesinvolved in identifying the stimuli and deciding which response tomake and would therefore overlap somewhat with the motorresponse to these stimuli. However, we also gain additionalactivation for infrequent target stimuli only in the parietaloperculum, supramarginal gyrus and Heschl's gyrus which areregions associated with target detection and not motor responses.To illustrate that this additional activation in these areas is specificto processing the infrequent target stimuli we also performed P3latency informed analysis for only the frequent target stimuli. Thisclearly illustrates the lateralised motor response associated witheach type of stimulus, but also shows that the activation in theparietal operculum and supramarginal gyrus is unique to theinfrequent target stimuli (Fig. 5). While this analysis shows thatthe lateralised motor response is modelled by our P3 informedvariable epoch models, it also shows that the ratio of frequent toinfrequent stimuli in our hybrid two choice reaction time–oddballtask is analogous to an oddball paradigm and responses toinfrequent target stimuli should be treated target detectionresponses.

To test whether the additional activation seen in the variable epochmodel was a result of choosing a longer epoch for analysis or thevariability of P3 latency at a single-trial level, we ran a constant epochmodel for the infrequent target stimuli only using the P3 latency fromthe grand average ERP (412 ms). We hypothesised that the temporalresolution of our fMRI datawould not be sensitive enough to the smallvariations (10 s of ms) on a trial to trial level and that it was our longerwindow that was responsible for the added value activation ratherthan single-trial variability. This constant epochmodel also resulted inadded value in similar regions to the variable epoch model (the leftpostcentral gyrus and left precentral gyrus) and an additional clusterin the right occipital cortex (Fig. 3, right column).

Reaction time informed fMRI analysisAnalyses including reaction time were performed for the infre-

quent target stimuli only. When using reaction time as an additionalregressor, no additional activation in relation to the conventional,uninformed analysis was observed (Fig. 3, right column). However,when using single-trial reaction time to create a variable epoch modeladditional activation was observed in the left precentral gyrus, leftpostcentral gyrus, and left and right lateral occipital cortices (Fig. 3,right column, and see Table 2 for details).



ROI analysisTo assesswhether the addedvalue in the EEG informed fMRImodels

(P3 amplitude and P3 latency variable epoch models) was driven byvariability in the single-trial ERP parameters, correlation analyses wereperformed between the standard deviation of the ERP parameters ofinterest and themean BOLD response Z values in each region of interest(see Methods section for the added value regions defined based on thegroup level activation). We predicted that single-trial ERP variabilitywould be driving the added value activation and so anticipated apositive correlation between the standard deviation of the ERPmeasures and the mean Z values in each region of interest. For themodel using P3 amplitude as a regressor themean Z value in the regionof interest showed a trend for a positive relationship with the standarddeviation of P3 amplitude (r(14)=0.48, P=0.03, one sided) (Fig. 6). Forthe model using a variable epoch model based on ST P3 latency thestandard deviation of P3 latency didnot correlatewith themean Zvaluein the region of interest (r(14)=−0.15, P=0.29, one sided) (Fig. 6).

Correlations between single-trial measures across subjects

Pearson's product moment correlation coefficient r was used totest the relationships between the single-trial measures: P3 ampli-tude, P3 latency and reaction time. Given the equivocal findings onthe relationship between reaction time and electrophysiologicalmeasures generally reported in literature (Musso et al., 2006), wemade no prediction about the direction of the relationship. Nosignificant relationship was found between reaction time and P3amplitude (r(14)=0.003, P=0.92, two sided) or reaction time andP3 latency (r(14)=−0.41, P=0.13, two sided).

Discussion

The conventional GLM analysis revealed activation in response toinfrequent target stimuli in brain regions consistent with previousfMRI, intracortical electrophysiological recording and EEG sourcelocalisation studies. The postcentral gyrus, precentral gyrus, cerebel-lum, supramarginal gyrus, insula, frontal operculum, inferior frontalgyrus, middle frontal gyrus, and occipital cortex have all been reportedin response to tasks involving the detection of infrequent targetstimuli (Halgren et al., 1995a,b; Gur et al., 2007; Strobel et al., 2008;Winterer et al., 2007). The activation pattern was similar for all singleexplanatory variable (EV) models (Fig. 1), with peak clusters beingobserved in the same regions with similar Z values. However, it is theso called ‘added value’ models that provide the most information. Byorthogonalising the single-trial informed explanatory variable to theconventional uninformed explanatory variable only additional activa-tion explained by including the single-trial parameters is observed,improving the specificity of the model.

Added value of single-trial parameter informed models

P3 amplitudeFor infrequent target stimuli including P3 amplitude as a regressor

in the GLM yielded additional activation in the central opercularcortex, precentral gyrus, temporal pole, frontal operculum and insulawhereby activation seen in these areas is consistent with thedetection of infrequent target stimuli in fMRI studies (Bledowskiet al., 2004; Gur et al., 2007). The activation pattern is obtained

Fig. 3. BOLD activation for all analyses (second-level mixed-effects FLAME. N=19, Cluster-coadded value models). The left column shows the single explanatory variable analysis (convincluding two explanatory variables, the first being the conventional analysis (stimulus onsvariable containing the various single-trial parameters, the second EV was orthogonalisedparameters. Part A shows the conventional model where a constant impulse model was useused as an additional regressor to modify the height of the basis function. Parts C and E use aPart G uses a constant epochmodel where the length of the boxcar is determined by the latenof each type of model.

without observing the occipital activation that is seen in theconventional analysis. This would suggest that the single-trial P3amplitude informed analysis is more effective for localising activationin areas related to the detection of infrequent targets. Furthermore,the added value in the P3 amplitude informed model (mean Z scorein ROI) shows a significant positive relationship with the standarddeviation of the single-trial P3 amplitudes. This suggests that theadditional fMRI activation is driven by the variability in the single-trial P3 amplitudes which supports the idea that using single-trialERP amplitude is a valid way to harness the variability in responses toindividual stimuli in order to maximise our analysis of fMRI data.However, while ST P3 amplitude informed analysis yields additionalactivation in this left fronto-central region, the activation in the moreparietal regions also associated with target detection and frequentlyseen in fMRI studies of target detection (Gur et al., 2007; Wintereret al., 2007) is lost in the added value analysis. The most plausibleexplanation is that the added value activation reflects areas that aremore closely associated with variability in the single-trial P3amplitudes. The resulting activation map is very similar to theventral attention system activation map obtained by Mantini et al.(2009) when differentiating the ventral attention system from thedorsal attention system. They suggest that the ventral attentionsystem is associated with the detection of behaviourally relevantstimuli, so an emphasis on this region in P3 amplitude informedanalysis is plausible. When all trials (frequent and infrequent) areincluded in the analysis there is no added value for the amplitudeinformed model. Given that the amplitude informed model (variableimpulse) yields BOLD activation that is related to the variability in theERP amplitude (Grinband et al., 2008) we can assume that theabsence of added value reflects one of two things: 1. there is lessvariability in the frequent ‘non targets’ compared to the infrequent‘targets’, 2. the variability in responses to the frequent stimuli is notrelated to a specific pattern of BOLD activation. In support of the firstexplanation, the variability in the single-trial P3 amplitudes (asmeasured by standard deviation) is marginally higher for the targetscompared to non target (T(14)=2.01, P=0.06). In support of thesecond explanation, the regions in which we get additional activationin the ERP amplitude model for the frequent stimuli only (centralopercular cortex, precentral gyrus, temporal pole, frontal operculumand insula) are consistent with the detection of behaviourallyrelevant stimuli (e.g. Mantini et al., 2009). We therefore concludethat the pattern of additional activation yielded by the inclusion ofsingle-trial P3 amplitude is related to the additional attentionalrequirements of detecting the infrequent target stimuli and thevariability in responses to these stimuli across trials.

An alternative explanation of our added value activation could berelated to the nature of our task as a two choice response task. As ourtask is not a true oddball paradigm, in that a response is also given tofrequent stimuli, our results could represent the switch cost betweenthe frequent and infrequent stimuli and corresponding responses.Task switching involves shifting attention and retrieving stimulusresponse rules (Goffaux et al., 2006) and this process could explainthe attention related regions activated when including single-trial P3amplitude as a regressor. However, imaging studies of task switchinghave identified the lateral prefrontal cortex and superior frontal gyrus(Cutini et al., 2008; Dove et al., 2000) as the key structures involved inswitch cost, this does not match our area of activation. This suggeststhat we are modelling activity related to a different process. This is

rrected threshold Z=2.3, P=0.05 for the single EV models and Z=2.0, P=0.05 for theentional and single-trial informed). The right column shows the results of the modelet timecourse with no additional single-trial information) and the second explanatoryto the first to determine any additional activation explained by including single-triald. Parts B, D and F show variable impulse models where the single-trial parameter wasvariable epoch model based on single-trial P300 latency and reaction time respectively.cy of P3 taken from the grand average ERP. See Fig.1 for more details on the construction

Fig. 4. Coronar slices showing the peak activation for the added value models (second-level mixed-effects FLAME. N=19, Cluster-corrected threshold Z=2.0, P=0.05). Part A showsthe additional activation found when using P300 amplitude as an additional regressor for infrequent target stimuli. The activation is in the central opercular cortex, precentral gyrusand insula (see Table 4 for details). Part B shows the additional activation when using a variable epoch model informed by single-trial P300 latency for infrequent target stimuli. Theactivation is posterior to that of the variable impulse model and involves the parietal operculum and supramarginal gyrus (see Table 4 for details). Part C shows the additionalactivation for the variable epoch model informed by single-trial P3 latency for all stimuli. This highlights the activation obtained in the anterior cingulate cortex. Part D shows theadditional activation when using a variable epoch model informed by single-trial reaction time for infrequent target stimuli. The activation is predominantly in the pre andpostcentral gyri (see Table 4 for details).


Table 4Brain regions and local maxima for single-trial informed analysis orthogonalised to thestandard analysis (added value).

Model Region (Harvard-Oxford,maximum probability)

MNI coordinates oflocal maxima (x, y, z)

MaximumZ score

P300 amplitude(infrequent stimuli)

Left centralopercular cortex

−52, 4, 2 4.02

Left insula −32, −8, 0 3.75Right insula 28, 18, 10 3.63Left caudate −16, 2, 20 3.55Left postcentral gyrus −48, −36, 60 4.21

P300 variable epoch(infrequent stimuli)

Postcentral gyrus −36, −30, 70 4.57−50, −28, 60 3.92−46, −28, 64 3.84

Postcentral gyrus,precentral gyrus

−38, −22, 36 4.34−36, −28, 64 4.25

Left thalamus −8, −10, 2 3.28Right thalamus 10, 0, 10 3.27Parietal operculum −54, −24, 16 2.83Supramarginal gyrus −62, −24,30 2.60Heschl's gyrus −44,−24, 12 2.10

P300 variable epoch(all stimuli)

Right postcentral gyrus 54, −20, 54 3.6948, −16, 58 3.62

Right precentral gyrus 28, −10, 70 3.4Right lateraloccipital cortex

22, −68, 60 3.48

Anterior cingulate cortex −8, 10, 36 2.4−8, 8, 38 2.8−2, 4, 42 2.5−44, −78, −14 3.07

Reaction timevariable epoch(infrequent stimuli)

Left postcentral gyrus −46, −26, 54 3.86Left precentral gyrus −34, −16, 50 3.40Left lateraloccipital cortex

−28, −72, 38 3.24−20, −70, 46 3.23

Right lateraloccipital cortex

30, −64, 40 3.7534, −66, 32 3.42


supported by electrophysiological data; for example in uncued taskswitching P3 amplitude and latency do not differ between repeat andswitch trials (Hsieh and Liu, 2005; Hsieh and Yu, 2003) suggestingthat the switch cost is after the stimulus identification stage.Differences in P3 are observed in cued switch tasks (Hsieh and Liu,2005), however, there is no cue in our task so it is likely that ourmarginal difference in P3 parameters between frequent and infre-quent stimuli (Supplementary Table 1) is related to target detectionprocesses involved in our paradigm rather than a result of switch cost.In support of this Dove et al. (2000) reported that a low ratio of switchtrials to repeat trials can make the task more like an oddball task.Given that the stimulus ratio in our paradigm is designed for anoddball task we believe that we are indeed looking at target detectionnetworks rather than switch cost.

P3 latency and reaction timeThe model using single-trial P3 latency as a regressor to modify

only the height of the basis function, did not result in any additionalactivation (added value). Our findings are consistent with those ofGrinband et al. (2008) who found that using timing information (intheir case reaction time) as a regressor in the model was the leastpowerful model. However, when using single-trial P3 latency to createa variable epoch model, modifying the duration/overall shape inaddition to the height of the basis function, we found additionalactivation in the left postcentral gyrus, left precentral gyrus, thalamus,parietal operculum, supramarginal gyrus and Heschl's gyrus forinfrequent targets only and in the right postcentral gyrus, rightprecentral gyrus and ACCwhen all stimuli are included in the analysis.While this additional activation does include some regions that areassociated with the motor response to the stimuli, there is additionalactivation in regions associated with the detection of stimuli andmaking decisions about those stimuli. This would suggest that byusing an appropriate epoch length and orthogonalising the explana-tory variable to the conventional timecourse we can determine

activation in brain regions specific to the cognitive processes involvedin the task. Furthermore, the non-motor activation detected differswhen only infrequent target stimuli and when all stimuli are includedin themodel. Additional activationwas only obtained in the ACCwhenall stimuli were included in the analysis. The ACC is associated withexecutive, evaluative and cognitive functions and its involvement inour task is consistent with other fMRI studies of target detection anddecision making tasks (Gur et al., 2007; Philiastides and Sajda, 2007).Since the ACC has an important role in processing top-down andbottom-up stimuli and assigning appropriate control to other areas inthe brain, and is associated with effort applied to a task (Mulert et al.,2008; Esposito et al., 2009), its involvement in responses to all stimuliin a decisionmaking task such as ours is expected. The fact that we didnot obtain additional ACC activation for infrequent targets only islikely to be due to there being too few trials in this condition to revealthis activation, exploiting the P3 information available in all trialsyields this additional activation.

The parietal regions identified using the P3 latency informedvariable epoch model for infrequent target stimuli only are consistentwith fMRI target detection studies (Gur et al., 2007; Strobel et al.,2008; Winterer et al., 2007) and may reflect working memoryprocesses. This is also consistent withwhat is known about the ventraland dorsal streams of information processing; the ventral stream isinvolved in the detection of salient events while the dorsal stream isinvolved in maintaining visuospatial attention once the salientstimulus has been identified (Corbetta et al., 2002; Corbetta andShulman, 2002). The dorsal stream projects to the parietal region, inline with our added value activation, andmediates the response to thesalient stimulus (Goodale and Milner, 1992). In other words the top-down influence of the ventral system in voluntarily directing attentionto the salient stimulus leads to the maintenance of attention to thatstimulus and the updating of working memory (Sridharan et al.,2007). So, in our paradigm attention is directed to the salientinfrequent target stimuli and thus an increase in BOLD activation isseen in these brain areas in response to infrequent target stimuli. Fromthe pattern of activation we observed in the added value analysis, itwould appear that we can model these working memory processes byusing a variable epoch model informed by single-trial P3 latency.Similar results were foundwhen using single-trial reaction time in theGLM; no additional activation was seen when single-trial reactiontime was used to create a variable impulse model but additionalactivation in parietal and occipital regions were observed when usingsingle-trial reaction time to create a variable epoch model.

Using single-trial parameters in the GLM

Responses to repeated stimuli do vary, not just from subject tosubject but on a trial to trial basis. By accounting for this variability inthe GLM by using single-trial parameters we can conduct a moreappropriate analysis of our data and gain a better understanding of therelationship between event-related BOLD and EEG (Debener et al.,2006; Fell, 2007; Makeig et al., 2005). The questions we aimed toanswer in this paper were what aspects of the ERP and behaviouralresponse to a stimulus aremost informativewhen included in the GLM,and what is the most appropriate way to use this single-trialinformation in the GLM, specifically for a choice reaction time/targetdetection (P3) paradigm. The summary above indicates that includingdifferent single-trial parameters in theGLMdoes provide varied results,whatwe now need to do is determinewhich parameter should be usedand in what way this information should be included in the GLM.

Our first approach was to include the ST information as a regressorin the GLM to modify the height of the basis function, this serves tomodify the height of the basis function at a trial by trial level (see Fig.1for how this was achieved in this paper, and Grinband et al. (2008) fora more detailed general discussion). This is analogous to modifyingthe intensity of each event. This proved to be effective for P3

Fig. 5. Coronar slices showing the peak activation for the added value models (second-level mixed-effects FLAME. N=19, Cluster-corrected threshold Z=2.0, P=0.05) Part A showsthe additional activationwhen using a variable epoch model informed by single-trial P300 latency for non target stimuli. Part B shows the additional activationwhen using a variableepoch model informed by single-trial P300 latency for target stimuli. Part C shows the binarised activation maps for additional activation for both the target (blue) and non target(red) stimuli. The binarised maps show the location of the activation only and give no indication of varying amplitude of activation. This illustrates that while the lateralised motorresponse is modelled by the P3 latency informed variable epoch model, the target stimuli elicit additional activation in the parietal operculum compared to the non target stimuli.


amplitude but not for P3 latency or reaction time. This suggests thatinformation gained from timing measures are not as informative asamplitude measures when used in this way. This makes sense giventhat reaction time and P3 latency vary in time, and while variableimpulse models changes the height of the basis function the durationof the function remains the same. It is a logical conclusion then, that avariable impulse model is not the most appropriate way to use single-trial latency measures. However, this method does appear to beappropriate for measures of event intensity, in our case P3 amplitude.The amplitude of P3 varies with the intensity/arousal associated witha response and so we would expect that the most appropriate way tomodel this variability would be to use a variable impulse model. Ourresults show that variability in responses on a trial by trial basis isreflected in the amplitude of the P3 and that the BOLD responseassociated with this variability can be determined by including theseparameters in a variable impulse model. This is in line with previousstudies using ERP amplitude as a regressor in the GLM (Debener et al.,2005; Eichele et al., 2005; Mulert et al., 2008). Furthermore, distinctactivation patterns were observed for infrequent targets and frequenttargets illustrating that this method can differentiate responses tostimuli with differing cognitive demands.

The second approach was to use timing information gained fromthe ERP or behavioural response to modify the overall shape andheight of the basis function in the GLM. The haemodynamic responseto a decision making process has been shown to vary with the timetaken for the subject to make a response (Menon et al., 1998). This is acritical point for paradigms such as choice reaction time tasks andoddball tasks where subjects are involved in a decision makingprocess and provide a response. This decision making process is likelyto persist after stimulus onset, so a zero duration impulse functionwould not be the most appropriate way to model this process. Wewould also argue that while reaction time provides some indication ofhow long this decision making process takes, it also includes theprocessing involved in producing the voluntary motor response. Forthis reason we suggest that P3 latency is a more accurate measure ofthe duration of the target detection/decision making process.

Using P3 latency to create a variable epoch model resulted inadditional activation in parietal regions associated with targetdetection. This would suggest that modelling the duration of thedecision making process results in a more specific localisation of brainactivation related to target detection. However, the added valueactivation (Z values in ROIs) and the standard deviation of P3 latency

Fig. 6. Scatter plots showing the relationship between mean Z values in added valueregions of interest (ROI) and single-trial parameter standard deviation for both P3amplitude and P3 latency.


did not correlate which would suggest that this finding is not drivenby a single-trial variability. A plausible alternative is that thesemethods just capture a more appropriate window for analysis thanusing an impulse function. To test this we conducted some additionalanalysis using a constant epochmodel using P3 latency from the grandaverage ERP to determine the epoch duration for each subject. Thismodel resulted in a similar activation pattern suggesting that it is amore appropriate time window that is important rather than latencyvariability on a single-trial level being the sole driving force behindthis finding. The temporal resolution of fMRI BOLD, particularly forwhole brain analysis, might not be sensitive enough to modulation bysingle-trial fluctuations in decision making time that will be in theorder of 10s ofmilliseconds. For example, some of the studies that showa detectable BOLD response with stimulus durations and onsetasynchronies as low as 50 ms, use a small number of slices coveringthe area of interest with short TR (Connolly et al., 2005; Menon et al.,1998). On the other hand, some studies usingwhole brain analysis havealso detected short latencydifferences in the BOLD response (Henson etal., 2002; Formisanoet al., 2002). Further investigation of the sensitivityof the BOLD response to small temporal variations on the single-triallevel is necessary to determine whether variable epoch models doindeed model single-trial variability or whether the gain is due to anevent duration that is more representative of cognitive processing.

Using reaction time to create a variable epoch model also yieldedadded value activity in left parietal regions, however the cluster wassmaller than for the model informed by P3 latency. The reducedadditional activation for the reaction time variable epoch modelcompared to the P3 latency variable epoch model could be related tothe fact that reaction time itself covers a longer window than the P3.This would include some processes irrelevant to the decision makingprocess resulting in more activation in general, perhaps obscuring thepotential added value of this parameter in detecting activity related todecision making. However, we have shown that using reaction time to

create a variable epoch model does localise some activation in targetdetection areas. This builds on the findings of Grinband et al. (2008)who showed that using reaction time simulations and varying lengthvisual stimuli as a proxy for varying reaction time could result in amore powerful model. We found that more voxels were activated inthe informedmodels compared to the conventional model and similarZ scores were also observed (with the exception of the P3 amplitudemodel which had the smallest maximum Z value). We also foundadded value in these models, showing that this finding holds for realreaction time data. While the variable epoch model informed byreaction time showed less additional activation than the P3 latencyinformed model, it can still be considered more powerful than theuninformed model, this illustrates that including basic single-trialinformation is useful even in the absence of single-trial EEG. However,as described above it is still unclear whether single-trial variability ismodelled by the variable epoch models or whether the moreappropriate HRF shape determined by an event duration extendingbeyond stimulus onset leads to the additional activation observed.Grinband et al. (2008) observed that while many studies measurereaction time very few actually use this information when analysingtheir fMRI data. Based on our findingswewould concurwith Grinbandet al. in that a measure reflecting the duration of cognitive processesinvolved in a task should be considered in order to optimise theanalysis of fMRI data.

We found different additional activations when includingresponses to all stimuli or only responses to infrequent target stimuliin the models. For the P3 latency informed variable epoch model theACC activationwas only apparentwhen all trialswere included and theleft parietal operculum/supramarginal gyrus activation was onlyobserved for infrequent target stimuli. We only obtained additionalactivation for the P3 amplitude informed model for infrequent targetstimuli. These differences in activation have been explained by thedemands of the task and the concordance of our activation patternswith previous research. These distinct patterns of activation show thatERP parameters can be used to differentiate regions involved indifferent aspects of cognitive processing however, this is dependent onthe information that is included in the model. This highlights the factthat while ERP informed fMRI analysis is potentially useful great careneeds to be taken in order to ensure that the single-trial informationentered into the model reflects the cognitive processes of interest.

In conclusion it would appear that both single-trial P3 amplitudeand P3 latency are informative when included in the GLM whenanalysing fMRI data. Single-trial amplitude should be used to modifythe height of the basis function whereas P3 latency is moreinformative when used to create a variable epoch model. Given thatP3 amplitude and latency yield different patterns of additional taskrelated BOLD activation we would advocate using both of these STparameters when analysing P3 data. While it is still unclear whetherthe P3 latency informed variable epochmodel accounts for single-triallatency fluctuations or reflects a more appropriate time window foranalysis, it would still appear that this model is more powerful fordetecting task related activity than the conventional constant impulsemode. By including parameters representing aspects of cognitiveprocessing BOLD activation in brain areas involved in the selectiveattention and working memory aspects of target detection can beinvestigated more specifically. Furthermore, given that P3 is acommonmeasure when investigating disorders such as schizophreniaand Alzheimer's, effects of substances such as nicotine and processessuch as ageing, understanding how single-trial P3 parameters can beused to inform fMRI analyses in these populations is potentially useful.Of particular relevance to our P3 latency related findings is that thehaemodynamic response can be delayed in specific populations, suchas in schizophrenics (Ford et al., 2005). Using indices of cognitiveprocessing to help model these difference in the haemodynamicresponse could make single-trial informed fMRI analyses a particu-larly useful tool in this context.


Acknowledgment

This study was conducted within the framework of the PriorityProgram SPP1226 of the German Research Foundation (DFG, DeutscheForschungsgemeinschaft): “Nicotine: Molecular and PhysiologialEffects in CNS” [http://www.nicotine-research.com], grant number:Wi1316/7-1.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, inthe online version, at doi:10.1016/j.neuroimage.2009.05.082.

References

Allen, P., Josephs, O., Turner, R., 2000. A method for removing imaging artifact fromcontinuous EEG recorded during functional MRI. NeuroImage 1, 230–239.

Allen, P., Polizz, G., Krakow, K., Fish, D.R., Lemieux, L., 1998. Identification of EEG eventsin the MR scanner: the problem of pulse artifact and a method for its subtraction.NeuroImage 8, 229–239.

Bagshaw, A.P., Warbrick, T., 2007. Single trial variability of EEG and fMRI responses tovisual stimuli. NeuroImage 38, 280–292.

Behrens, T., Woolrich, M.W., Smith, S., 2003. Multi-Testing Using a Fully Subject NullHypothesis Bayesian Framework: Theory. Human Brain Mapping Meeting, NewYork City.

Bénar, C-G., Schön, D., Grimault, S., Nazarian, B., Burle, B., Roth, M., Badier, J-M., Marquis,P., Liegeois-Chauvel, C., Anton, J-C., 2007. Single-trial analysis of oddball event-related potentials in simultaneous EEG-fMRI. Hum. Bain Mapp. 28, 602–613.

Bledowski, C., Prvulovic, D., Hoechstetter, K., Scherg, M., Wibral, M., Goebel, R., Linden,D.E.J., 2004. Localizing P300 generators in visual target and distractor processing: acombined event-related potential and functional magnetic resonance imagingstudy. J. Neurosci. 24 (42), 9353–9360.

Connolly, J.D., Goodal, M.A., Goltz, H.C., Munoz, D.P., 2005. fMRI activation in the humanfrontal eye field is correlatedwith saccadic reaction time. J. Neurophysiol. 94, 605–611.

Corbetta, M., Kincade, J.M., Shulman, G.L., 2002. Neural systems for visual orienting andtheir relationships to spatial working memory. J. Cogn. Neurosci. 14 (3), 508–523.

Corbetta, M., Shulman, G.L., 2002. Control of goal-directed and stimulus-drivenattention in the brain. Nat. Rev. Neurosci. 3, 201–215.

Cutini, S., Scatturin, P., Menon, E., Bisiacchi, P.S., Gamberini, L., Zorzi, M., Dell'Acqua, R.,2008. Selective activation of the superior frontal gyrus in task-switching: an eventrelated fNIRS study. NeuroImage 42, 945–955.

Debener, S., Ullsperger, M., Siegel, M., Fiehler, K., von Cramon, D.Y., Engels, A.K., 2005.Trial-by-trial coupling of concurrent electroencephalogram and functionalmagnetic resonance imaging identifies the dynamics of performance monitoring.J. Neurosci. 25, 11730–11737.

Debener, S., Ullsperger, M., Siegel, M., Engel, A.K., 2006. Single-trial EEG-fMRI revealsthe dynamics of cognitive function. Trends Cogn. Sci. 10, 463–558.

Dove, A., Pollman, S., Schubert, T., Wiggins, C.J., von Crammon, D.Y., 2000. Prefrontalcortex activation in task switching: an event related fMRI study. Cogn. Brain Res. 9,103–109.

Eichele, T., Specht, K., Moosman, M., Jongsma, M.L., Quiroga, R.Q., Nordby, H., Hugdahl, K.,2005. Assessing the spatiotemporal evolution of neuronal activationwith single trialevent-related potentials and functional MRI. Proc. Natl. Acad. Sci. 102, 17798–17803.

Esposito, F., Mulert, C., Goebel, R., 2009. Combined distributed source and single-trial EEG-fMRI modeling: application to effortful decision making processes.NeuroImage, doi:10.1016/j.neuroimage.2009.03.074.

Fell, J., 2007. Cognitive neurophysiology: beyond averaging. NeuroImage 37, 1069–1072.First, M.B., Spitzer, R.L., Giddon, M., Williams, J.B., 1995. The Structured Clinical

Interview for DCM-IV Axis I Disorders Research Version (SCID-1). AmericanPsychiatric Press, New York.

Ford, J.M., Johnson, M.B., Whitfield, S.L., Faustman, W.O., Mathalon, D.H., 2005. Delayedhemodynamic response in schizophrenia. NeuroImage 26, 922–931.

Forman, S.D., Cohen, J.D., Fitzgerald, M., Eddy, W.F., Mintun, M.A., Noll, D.C., 1995.Improved assessment of significant activation in functional magnetic resonanceimaging (fMRI): use of a cluster-size threshold. Magn. Reson. Med. 33, 636–647.

Formisano, E., Linden, D.E., Di Salle, F., Trojano, L., Esposito, F., Sack, A.T., Grossi, D.,Zanella, F.E., Goebel, R., 2002. Tracking the mind's image in the brain I: time-resolved fMRI during visuospatial mental imagery. Neuron 35, 185–194.

Friston, K.J., Worsley, K.J., Frackowiak, R.S.J., Mazziotta, J.C., Evans, A.C., 1994. Assessingthe significance of focal activations using their spatial extent. Hum. Brain Mapp. 1,214–220.

Goffaux, P., Phillips, N.A., Sinai, M., Pushkar, D., 2006. Behavioural and electrophysio-logical measures of task switching during single and mixed-task conditions. Biol.Psychol. 72, 278–290.

Goodale, M.A., Milner, A.D., 1992. Separate visual pathways for perception and action.Trends Neurosci. 15 (1), 20–25.

Gur, R.C., Turetsky, B.I., Loughead, J., Waxman, J., Snyder, W., Ragland, J.D., Elliott, M.A.,Bilker, W.B., Arnold, S.E., Gur, R.E., 2007. Hemodynamic responses in neuralcircuitries for detection of visual target and novelty: an event-related fMRI study.Hum. Brain Mapp. 28, 263–274.

Grinband, J., Wager, T.D., Lindquist, M., Ferrera, V.P., Hirsch, J., 2008. Detection of time-varying signals in event-related fMRI designs. NeuroImage 43 (3), 509–520.

Hagenbeek, R.E., Rombouts, S.A., Veltman, D.J., Van Strien, J.W., Witter, M.P., Scheltens,P., Barkhof, F., 2007. Nonlinear changes in brain activity during continuous wordrepetition: an event-related multiparametric functional MR imaging study. Am. J.Neuroradiol. 28 (9), 1715–1721.

Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Liégeois, C., Chauvel, P., Musolino, A.,1995a.Intracerebral potentials to rare target and distractor auditory and visual stimuli. I.Superior temporal plane and parietal lobe. Electroencephalogr. Clin. Neurophysiol.94, 191–220.

Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Marinkovic, K., Devaux, B., Vignal, J-P.,Biraben, A.,1995b. Intracerebral potentials to rare target and distractor auditory andvisual stimuli. II. Medial, lateral and posterior temporal lobe. Electroencephalogr.Clin. Neurophysiol. 94, 229–250.

Halgren, E., Marinkovic, K., Chauvel, K., 1998. Generators of the late cognitive potentialsin auditory and visual oddball tasks. Electroencephalogr. Clin. Neurophysiol. 106,156–164.

Henson, R.N.A., Price, C.J., Rugg, M.D., Turner, R., Friston, K.J., 2002. Detecting latencydifferences in event-related BOLD responses: application to words versus non-words and initial versus repeated face presentations. NeuroImage 15, 83–97.

Horovitz, S.G., Skudlarski, P., Gore, J.C., 2002. Correlations and dissociations betweenBOLD signal and P3 amplitude in an auditory oddball task: a parametric approach tocombining fMRI and ERP. Magn. Reson. Imaging 20, 319–325.

Hsieh, S., Liu, L.-C., 2005. The nature of switch cost: task set configuration or carry-overeffect. Cogn. Brain Res. 22, 165–175.

Hsieh, S., Yu, Y.-T., 2003. Exploring the nature of switch cost: inferences from the P300and the lateralised readiness potentials. Brain Res. Protocols 12, 49–59.

Javitt, D.C., Spencer, K.M., Thaker, G.K., Winterer, G., Hajós, M., 2008. Neurophysiologicalbiomarkers for drug development in schizophrenia. Nat. Rev. Drug Discov. 7 (1),68–83.

Jenkinson, M., Bannister, P., Brady, J., Smith, S., 2002. Improved optimization for therobust and accurate linear registration and motion correction of brain images.NeuroImage 17, 825–841.

Kiehl, K.A., Stevens, M.C., Laurens, K.R., Pearlson, G., Calhoun, V.D., Liddle, P.F., 2005. Anadaptive reflexive processing model of neurocognitive function: supportingevidence from a large scale (n=100) fMRI study of an auditory oddball task.NeuroImage 25, 899–915.

Makeig, S., Debener, S., Onton, J., Delorme, A., 2005. Mining event-related braindynamics. Trends Cogn. Sci. 8, 204–210.

Menon, R.S., Luknowsky, D.C., Gati, J.S., 1998. Mental chronometry using latency-resolved functional MRI. Proc. Natl. Acad Sci. U. S. A. 95, 10902–10907.

Mantini, D., Corbetta, M., Perucci, M.G., Romani, G.L., Del Gratta, C., 2009. Large-scalebrain networks account for sustained and transient activity during target detection.NeuroImage 44 (1), 265–274.

Mobascher, A., Brinkmeyer, J., Warbrick, T., Musso, F., Wittsack, H.J., Stoermer, R., Saleh,A., Schnitzler, A., Winterer, G., 2009a. Fluctuations in electrodermal activity revealvariations in single trial brain responses to painful laser stimulation — a fMRI/EEGstudy. NeuroImage 44 (3), 1081–1092.

Mobasher, A., Brinkmeyer, J., Warbrick, T., Musso, F., Wittsack, H.J., Stoermer, R., Saleh,A., Schnitzler, A., Winterer, G., 2009b. Laser-evoked potential P2 single-trialamplitudes covary with the fMRI BOLD response in the medial pain system andinterconnected subcortical structures. NeuroImage 45 (3), 917–926.

Mulert, C., Seifert, C., Leicht, G., Kirsch, V., Ertl, M., Karch, S., Moosman, M., Lutz, J.,Möller, H.-J., Hegerl, U., Pogarell, O., Jäger, L., 2008. Single-trial coupling of EEG andfMRI reveals the involvement of early anterior cingulate cortex activation ineffortful decision making. NeuroImage 42, 158–168.

Musso, F, Konrad, A, Vucurevic, G, Schäffner, C, Friedrich, B, Frech, P, Stoeter, P,Winterer, G., 2006. Distributed BOLD-response in association cortex vector statespace predicts reaction time during selective attention. NeuroImage 29 (4),1311–1318.

Oostenveld, R., Praamstra, P., 2001. The five percent electrode system for high-resolution EEG and ERP measurement. Clin. Neurophysiol. 112, 713–719.

Philiastides, M.G., Sajda, P., 2007. EEG-Informed fMRI Reveals SpatiotemporalCharacteristics of Perceptual Decision Making. J. Neurosci. 27 (48), 13082–13091.

Picton, T.W., 1992. The P300 wave of the human event related potential. J. Clin.Neurophysiol. 9, 456–479.

Quian Quiroga, R., Garcia, H., 2003. Single-trial event-related potentials with waveletdenoising. Clin. Neurophysiol. 114, 376–390.

Rolls, E.T., Loh, M., Deco, G., Winterer, G., 2008. Computational models of schizophreniaand dopamine modulation in the prefrontal cortex. Nat. Rev., Neurosci. 9, 696–709.

Rorden, C., Karnath, H.O., Bonilha, L., 2007. Improving lesion-symptommapping. J. Cogn.Neurosci. 19, 1081–1088.

Smith, S., 2002. Fast robust automated brain extraction. Hum. Brain Mapp. 17, 143–155.Sridharan, D., Levitin, D.J., Chafe, C.H., Berger, J., Menon, V., 2007. Neural dynamics of

event segmentation in music: converging evidence for dissociable ventral anddorsal networks. Neuron 55, 521–532.

Strobel, A., Debener, S., Sorger, B., Peters, J.C., Kranczioch, C., Hoechstetter, K., Engel, A.K.,Brocke, Burkhard, Goebel, R., 2008. Novelty and target processing during anauditory novelty oddball: a simultaneous event-related potential and functionalmagnetic resonance imaging study. NeuroImage 40, 869–883.

Winterer, G., Musso, F., Konrad, A., Vucurevic, G., Stoeter, P., Sander, T., Gallinat, J., 2007.2007. Association of attentional network function with exon 5 variations of theCHRNA4 gene. Hum. Molec. Genet. 16 (18), 2165–2174.

Woolrich, M.W., Ripley, B.D., Brady, M., Smith, S.M., 2001. Temporal autocorrelation inunivariate linear modelling of fMRI data.

Worsley, K.J., Evans, A.C., Marrett, S., Neelin, P., 1992. A three-dimensional statisticalanalysis for CBF activation studies in human brain. J. Cereb. Blood Flow Metab. 12,900–918.

http://www.nicotine-research.com

http://dx.doi.org/10.1016/j.neuroimage.2009.05.082

Documents

Single-Trial P3 Amplitude and Latency Informed Event-related fMRI Models Yield