The dynamics of attentional sampling during visual searchrevealed by Fourier analysis of periodic noise interference

Laura Dugue $

CNRS Centre de Recherche Cerveau et CognitionFaculte de Medecine de Purpan Toulouse France

Universite Paul Sabatier Toulouse France

Rufin VanRullen $

CNRS Centre de Recherche Cerveau et CognitionFaculte de Medecine de Purpan Toulouse France

Universite Paul Sabatier Toulouse France

What are the temporal dynamics of perceptualsampling during visual search tasks and how do theydiffer between a difficult (or inefficient) and an easy(or efficient) task Does attention focus intermittentlyon the stimuli or are the stimuli processedcontinuously over time We addressed thesequestions by way of a new paradigm using periodicfluctuations of stimulus information during a difficult(color-orientation conjunction) and an easy (thorn amongLs) search task On each stimulus we applied adynamic visual noise that oscillated at a givenfrequency (2ndash20 Hz 2-Hz steps) and phase (fourcardinal phase angles) for 500 ms We estimated thedynamics of attentional sampling by computing aninverse Fourier transform on subjectsrsquo d-primes Inboth tasks the sampling function presented asignificant peak at 2 Hz we showed that this peakcould be explained by nonperiodic search strategiessuch as increased sensitivity to stimulus onset andoffset Specifically in the difficult task however asecond higher-frequency peak was observed at 9 to10 Hz with a similar phase for all subjects thisisolated frequency component necessarily entailsoscillatory attentional dynamics In a secondexperiment we presented difficult search arrays withdynamic noise that was modulated by the previouslyobtained grand-average attention sampling function orby its converse function (in both cases omitting the 2Hz component to focus on genuine oscillatorydynamics) We verified that performance was higher inthe latter than in the former case even for subjectswho had not participated in the first experiment Thisstudy supports the idea of a periodic sampling ofattention during a difficult search task Althoughfurther experiments will be needed to extend thesefindings to other search tasks the present report

validates the usefulness of this novel paradigm formeasuring the temporal dynamics of attention

Introduction

Visual search tasks can be classified according totheir level of difficulty Some are lsquolsquoeasyrsquorsquo characterizedby near-zero slopes when measuring reaction time as afunction of set size and involve preattentive processeswhereas others are more lsquolsquodifficultrsquorsquo characterized bypositive slopes and specifically involve attention(Treisman amp Gelade 1980 Wolfe Cave amp Franzel1989) Nowadays one of the main debated questionsconcerning difficult visual search is whether attentionfocuses sequentially on the stimuli (Treisman amp Gelade1980 Wolfe 1998 Wolfe et al 1989) acting as alsquolsquospotlightrsquorsquo that switches from one stimulus (or groupof stimuli) to another (Vanrullen Carlson amp Cav-anagh 2007) or whether it processes them all at thesame time in a continuous or parallelmanner (EcksteinThomas Palmer amp Shimozaki 2000 Palmer Ames ampLindsey 1993) The first hypothesismdashbut a priori notthe secondmdashshould predict that difficult visual searchinvolves a periodic temporal dynamic of attentionalsampling (Vanrullen amp Dubois 2011)

Over the past decades interest in the potentialimplication of periodic processes in perceptual capa-bilities has grown steadily There are different lines ofevidence in favor of the idea of lsquolsquodiscrete perceptionrsquorsquo(Stroud 1956 VanRullen amp Koch 2003) wherebyperceptual experience would build on a series of rapidlytaken discrete snapshots giving us a false impression ofcontinuity Recent studies demonstrated that attention

Citation Dugue L amp VanRullen R (2014) The dynamics of attentional sampling during visual search revealed by Fourieranalysis of periodic noise interference Journal of Vision 14(2)11 1ndash15 httpwwwjournalofvisionorgcontent14211doi10116714211

Journal of Vision (2014) 14(2)11 1ndash15 1httpwwwjournalofvisionorgcontent14211

doi 10 1167 14 2 11 ISSN 1534-7362 2014 ARVOReceived March 11 2013 published February 13 2014

can drive this periodic dynamic (Busch amp Vanrullen2010 Vanrullen et al 2007 Vanrullen Reddy ampKoch 2005) This suggests that attention could alsoemploy such periodic dynamics during difficult atten-tional search tasks as predicted by sequential models ofvisual search (Treisman amp Gelade 1980 Wolfe 1994Wolfe et al 1989)

In this study we introduce a novel technique aimedat measuring the temporal dynamics of attentionalsampling during visual search tasks and we demon-strate its practical application to two standard searchtasks one easy search (L vsthorn) and one difficult search(color-orientation conjunction) We used an oscillatorymodulation of stimulus information by dynamic visualnoise and performed a complex decomposition anal-ysis using Fourier series similar to a recent study byGobell and collaborators (Gobell Tseng amp Sperling2004) but in the time domain instead of space Wereasoned that the presence of a sampling periodicity ata particular frequency would support the sequentialmodel of attention deployment and help constrain theunderlying neural mechanisms On the other hand anabsence of sampling periodicity would favor theparallel model of attention We found a periodicsampling behavior occurring in the difficult task at10 Hz but no corresponding periodicity in the easysearch task Such a periodic attentional samplingnaturally supports the sequential model of attentionalthough it can be a posteriori reconciled with theparallel model by postulating an oscillatory modula-tion of the efficiency of attention over time Weconclude that our new method can allow the charac-terization of the temporal dynamics of attentionduring visual search Future studies will be neededhowever to test if our finding of a 10-Hz attentionalperiodicity can be generalized to other difficult visualsearch tasks

Methods

Subjects

The age of the participants was between 20 and 36years Overall 29 subjects were included in theexperiments (15 women) Nineteen subjects partici-pated in Experiment 1 (14 in the difficult task and 14in the easy task nine subjects participated in bothtasks) consisting of the evaluation of attentionaldynamics during visual search Seven participantsfrom the difficult task in Experiment 1 also performedExperiment 2 This second experiment tested therelevance of the estimated attentional function of thedifficult search task Ten new subjects (who did notparticipate in Experiment 1) also performed Experi-

ment 2 Three participants were excluded at this stageof the analysis two had chance-level performance(percentage correct between 50 and 60) and onehad reaction times more than three standard devia-tions above the group average

Stimuli

Subjects were placed 57 cm from the screen andtheir heads were maintained using a chinrest and aheadrest Two tasks were performed an easy (or lsquolsquopop-outrsquorsquo) and a difficult visual search task In the easysearch subjects were asked to report the presence orabsence of a lsquolsquothornrsquorsquo sign among L distracting letters (228visual angle) Each letter could be presented randomlyin four orientations 08 908 1808 and 2708 fromupright The difficult task was a conjunction taskbetween color and orientation Subjects reported thepresence or absence of a red grating oriented 308 fromupright among red gratings oriented 3308 from uprightand green gratings oriented 308 from upright (and viceversa) The gratings with a spatial frequency of sevencycles per degree measured 38 of visual angle In bothtasks the target was present in half of the trialspseudo-randomly determined The search arrays weredisplayed for 500 ms followed by an empty responsescreen In a preliminary experiment with variable setsizes we determined a specific set size for each subjectto achieve approximately 75 correct using thisstimulus duration of 500 ms This fixed set size wasthen used in the present experiments During the easytask we presented 6 (621) elements (average 6standard deviation over subjects) and we presented 54(62) elements during the difficult task Subjects whoperformed the difficult search task in both Experi-ments 1 and 2 had the same set size in bothexperiments

Experimental procedure

Experimental logic

In a nutshell our logic was to apply Fourier seriesanalysis using sine- and cosine-modulated stimulusinformation at different frequencies and measureattentional dynamics using the inverse Fourier trans-form

In mathematical terms we assume that there existsan attentional sampling function A(t) that we aim tomeasure over the interval of 0 to 05 s Our workingdefinition of attention entails that stimulus informationS(t) at different moments in time will weigh more orless on the perceptual decision as a function of thevalue A(t) at that time in other words the subjectsrsquoperception can be approximated (potentially with an

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 2

additive or multiplicative constant) by a linear combi-nation of stimulus information and attentional sam-pling

P frac14Z 05

0

SethtTHORNAethtTHORNdt

For a given temporal frequency x we call

Pcosx frac14Z05

0

coseth2pxtTHORNAethtTHORNdt

andPsinx frac14Z05

0

sineth2pxtTHORNAethtTHORNdt

Pcosx frac14Z05

0

coseth2pxtTHORNAethtTHORNdt

andPsinx frac14Z05

0

sineth2pxsTHORNAethtTHORNdt

Pcosx Psinx P-cosx and P-sinx can be directly estimatedby presenting a visual stimulus whose temporal profileis modulated with a cosine sine ndashcosine or ndashsinefunction (respectively) and measuring the resultingperceptual performance (eg d-primes) This is thepurpose of Experiment 1 (see Figure 1) In theory someof these measures should be expected to provideredundant results because Pcosx frac14P-cosx and Psinx frac14P-sinx only one measurement for each pair should (intheory) be sufficient In practice however one canimagine that modulating the visual stimulus at aparticular frequency could induce performance changesindependently of the exact phase (for example if fastermodulation frequencies cause an increase of arousal orvigilance) Therefore we systematically measured theeffect of each phase modulation (cosine sine) and theirinverse (cosine sine) on performance Any unwant-ed factor that would equally affect all modulationphases at a given frequency would then be eliminated inthe following equations

Finally we define the complex coefficient Cx as Cxfrac14(Pcosx P-cosx)2 i (Psinx P-sinx)2 In other words

Figure 1 Experimental design of Experiment 1 Examples of four presentation conditions (cosine sine cosine and sine) at onegiven frequency (4 Hz) during the easy visual search task Observers searched for a thorn among Ls (the letters were presented

randomly in four orientations 08 908 1808 and 2708 from upright) A trial started when subjects pressed the space bar on the

keyboard The stimuli appeared after a random delay (between 15 and 25 s) and stayed on the screen during 500 ms On each of

the stimuli a visual noise patch composed by random dots was displayed by transparency and its opacity fluctuated according to 1

of 40 conditions randomly intermixed 10 different frequencies (2ndash20 Hz in 2-Hz steps) and four phases (cosine sine cosine andsine) For all of these conditions there was always the same amount of stimulus information and visual noise only the temporal

ordering of the display frames varied A similar procedure was applied to a difficult visual search task (color-orientation conjunction

task)

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 3

Cx frac14

Z05

0

coseth2pxtTHORNAethtTHORNdtZ05

0

coseth2pxtTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

i

Z05

0

sineth2pxtTHORNAethtTHORNdtZ05

0

sineth2pxsTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

frac14

Z05

0

coseth2pxtTHORNAethtTHORN thorn coseth2pxtTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

i

Z05

0

sineth2pxtTHORNAethtTHORN thorn sineth2pxtTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

frac14Z05

0

coseth2pxtTHORNAethtTHORNdt i

Z05

0

sineth2pxtTHORNAethtTHORNdt

frac14Z05

0

AethtTHORNei2pxtdt

Fourier series analysis implies that the attentionalsampling function should be proportional to

AethtTHORNrsquoXlsquo

nfrac140

Cn=05e2piethn=05THORNt

In other words measuring the complex coefficients Cx

every 2 Hz (from 2 to 20 Hz) will give us an estimate ofthe attentional function (note that values greater than20 Hz are not considered here for practical purposesalthough of course they may be relevant for attention)

Experiment 1 Measuring attentional sampling dynamics

This experiment was intended to put into practice thetheory developed in the preceding section in order tovalidate our new paradigm Subjects were placed infront of a gray screen and were asked to keep their gazeon a fixation point at the center When the subjectspressed the space bar on the keyboard a fixed number

of stimuli (determined separately for each subject asexplained above) appeared at random but equallyspaced positions on a circle at 148 eccentricity after arandom delay between 15 and 25 s On each stimuluselement a dynamic visual noise was superimposed bytransparency consisting of a square filled with dots ofrandom luminance the square was 15 times largerthan the stimuli The dynamic visual noise was appliedwith a temporal modulation function following 1 of 40possible conditions (Figure 1) 10 different frequencies(2ndash20 Hz by steps of 2 Hz) and four different phases(sine cosinesine andcosine) randomly interleavedin different trials of the same blocks When themodulation value was maximal the noise was opaqueand the stimulus was invisible when the modulationwas minimal the noise was fully transparent and thestimulus was thus unaffected Note that for all 40conditions there was always the same amount ofstimulus information and of visual noise over thecourse of each trial (in other words only the temporalordering of the display frames differed betweenconditions not the contents of the frames)

We evaluated the performances of the subjects bycomputing d-primes Based on a complex decompositionanalysis we used these d-primes to assess the dynamicsof attentional sampling occurring during one visualsearch trial for the difficult and the easy tasks (Figure 2)As explained in the preceding section this methodconsists in first combining for each frequency the d-primes for sine andsine phase conditions and(separately) for cosine andcosine phase conditions andsecond combining the two resulting estimators to obtaina vector in the complex domain For each subject wethus obtained 10 complex vectors for the 10 differentfrequencies These vectors were considered as Fouriercoefficients defined by their length (ie oscillatoryamplitude) and pointing angle (ie oscillatory phase)The complex coefficients were then used to compute aninverse Fourier transform thereby estimating theattentional sampling function in the time domain foreach of the subjects For both tasks we then computedthe average estimated attentional function over allsubjects Finally we analyzed the amplitude spectra ofthese two estimated sampling functions We firstcomputed a fast Fourier transform (FFT) on the grand-average attentional functions (ie averaged over allsubjects) and looked at the obtained amplitude spec-trum We also calculated this amplitude spectrum foreach subject based on his or her individual attentionalfunctions and recomputed the average amplitude spec-trum For both analyses we evaluated the significance ofthe measured amplitude spectra by using a Monte Carloprocedure Surrogate data were created for each subjectunder the null hypothesis that hit rate and false alarmrate are independent of phase and frequency Thecomplex decomposition analysis was recomputed foreach surrogate (nfrac1410000) and the amplitude spectra of

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 4

surrogate attentional functions (either based on grand-average attentional functions or on individual functions)were used to estimate significance To achieve this the10000 surrogate amplitude spectra were ranked inascending order separately for each frequency The9501th 9901th 9991th and 10000th values were

considered as the respective limits of four differentconfidence intervals (95 99 999 and 9999)which are represented with different colors in thebackground of the four corresponding graphs Anexperimentally observed spectral amplitude value wasconsidered significantly different from the corresponding

Figure 2 Estimation of the attentional sampling function by a complex decomposition analysis For both visual search tasks for each

subject we measured the performance modulation induced by visual noise across the 10 different frequencies for the four phase

conditions (sine cosinesine andcosine) We then combined the performances across the four phases for each frequency in the

complex domain For each subject we thus obtained 10 complex vectors characterized by their angle (phase in the complex domain)

and their length (amplitude) These vectors were used as Fourier coefficients on which we finally applied an inverse fast Fourier

transform to estimate the dynamics of attentional sampling in the time domain during one visual search trial All data in this figure

are fictitious and for illustration purposes only

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 5

null distribution with p 005 if it exceeded the 95confidence threshold (and with p 001 above the 99confidence threshold and so on) To take into accountthe possible problem of multiple statistical comparisonsacross the 10 frequencies used (2ndash20 Hz) we adopted aconservative statistical threshold of p 0001 For both

tasks we also ran the same analysis discarding thesignals from stimulus onset (0ndash83ms) and stimulus offset(417ndash500 ms) This was aimed at verifying that anyperiodicity highlighted by the previous analysis wouldnot be due merely to an increased sensitivity to the onsetandor the offset of the stimuli

Experiment 2 Testing the validity of estimatedattentional functions

Based on the previous estimates of attentionalsampling dynamics for the difficult visual search (seeResults and Figure 6) we tested the functionalsignificance of the sampling function on subjectsrsquoperformances For two groups of subjects seven whohad also performed the difficult search task inExperiment 1 and seven new subjects we presented thesearch arrays (color-orientation conjunction) embed-ded in dynamic noise with signal-to-noise ratios (SNRs)following the grand-average estimated attentionalfunction (ie maximal signal when attention isexpected to be maximal this was called the lsquolsquotestrsquorsquocondition) or the opposite signal (ie maximal signalwhen attention is expected to be minimal this wascalled the lsquolsquocontrolrsquorsquo condition) Whereas in the testcondition subjects should perform the search taskefficiently in the control condition their performanceswould be expected to decrease because stimulusinformation no longer matches their natural samplingdynamics We equalized the histograms of SNRs acrossthe two conditions to ensure they had comparable totalsignal energy and high-pass filtered the attentionalfunction excluding the 2 Hz component to ensure thatour results would not be driven by the 2 Hz componentwhich could have reflected a nonperiodic samplingstrategy (eg increased sensitivity for stimulus onsetandor offset as found in the easy search estimatedfunction) The two presentation conditions (test vscontrol) were randomly interleaved during the exper-iment We looked at the performances of the subjects inthese two conditions by computing d-primes Wecompared test versus control conditions using one-tailed t tests according to the prediction that a searcharray presented with an SNR following the estimatedattentional function should lead to better searchperformance than the opposite signal

Results

Experiment 1

In a preliminary experiment on the same group ofsubjects (nfrac1414 for both tasks) we verified that subjectsused the appropriate search strategy for the two tasks

Figure 3 Attentional sampling function estimated for an easy

visual search trial Subjects (n frac14 14) performed an easy task

consisting in finding athorn among Ls d-primes were measured for

the 40 phase and frequency visual noise presentation

conditions Performances are represented in the top panel as a

modulation of d-prime with respect to the average computed

over all conditions (dashed baseline) On these performance

modulations we applied the complex decomposition analysis

(Figure 2) This analysis was performed for each individual

subject Then the individual estimated attentional sampling

functions were grand-averaged over all subjects (bottom panel)

The standard error of the mean is represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 6

that we intended to use in Experiment 1 finding a thornamong Ls or a conjunction of color (red or green) andorientation (308 or 3308 from upright) We presentedthe stimuli for unlimited durations with a set sizerandomly drawn between four and eight elements andcomputed the RT middot set size slopes As expected theslopes were near zero for the L versusthorn task 77 ms 679 ms per element t(13)frac14 31 p frac14 001 for targetpresent and 116 ms 6 67 ms per element t(13)frac14 55p 001 for target absentmdasht tests performed under thenull hypothesis that the slopes are equal to zero (iethe task involved minimal attention) Slopes werestrongly positive for the conjunction task 70 ms 6 77ms per element t(13) frac14 256 p 00001 for targetpresent and 1846 ms 6 274 ms per element t(13) frac14191 p 00001 for target absent (ie the taskinvolved significant attentional resources)

In the main experiment we estimated the attentionalsampling dynamics for these two visual search tasksTo do so we modulated stimulus information byapplying an oscillatory visual noise (cf Figure 1)according to 40 different conditions (randomly inter-leaved) 10 different frequencies (2ndash20 Hz 2-Hz steps)and four different phases (sine cosine sine orcosine) at each frequency In both tasks we measuredfor each condition (ie for a given phase andfrequency) the performance of the subjects in detectingthe target by computing d-primes Then we applied acomplex decomposition analysis to estimate the dy-namics of attentional sampling during one trial ofvisual search (cf Methods section and Figure 2) ateach frequency d-primes for the different phaseconditions were combined to obtain a complex vectorin the Fourier domain An inverse Fourier transformallowed us to estimate attentional dynamics in the timedomain (see Methods)

For the easy search the resulting grand-averageattentional function over 14 subjects (Figure 3)presented large fluctuations in d-prime To determinethe relative power and the exact frequency of theseeffects we computed an FFT to obtain the amplitudespectrum of the grand-average attentional function(Figure 4A) The first four frequencies (2 4 6 and 8Hz) were significantly greater than chance (p 103 aconservative statistical threshold due to multiplecomparisons across frequencies) In Figure 4B theamplitude spectrum was computed first on eachindividual sampling function and subsequently aver-aged In this case the spectrum presented an overalldecreasing shape significantly different from chance atfrequencies of 2 4 and 6 Hz The existence of similarlow-frequency components (2 4 6 Hz) in the amplitudespectra from both the individual and grand-averagesampling functions implies that these components notonly present increased amplitude for each subject butalso are strongly phase locked between subjects As wewill see these low-frequency components mostly reflectincreased sensitivity to the onset and offset of thestimulus sequence

The same analysis was performed to reveal theattention sampling function during the difficult searchtask (Figure 5) This function was characterized by amore restricted range of oscillatory frequencies a rapidoscillation appeared to be superimposed on a slowerfluctuation In the amplitude spectra of Figure 6 threespecific frequency components stand out one at 2 Hzone at 10 Hz and one at 18 to 20 Hz These peaks wereapparent in the amplitude spectrum of the grand-average function (Figure 6A) as well as on the averageamplitude spectrum of individual sampling functions(Figure 6B) As previously we can thus conclude thatthese peaks were due to both higher amplitude and

Figure 4 Amplitude spectra of the attention function estimated for the easy search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) at the four lowest frequencies 2 Hz 4 Hz 6 Hz and 8 Hz (B) Average amplitude spectrum of the

individual attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude

reaches significance ( p value 103) at the three lowest frequencies 2 Hz 4 Hz and 6 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 7

stronger phase locking between subjects at thesefrequencies In other words this 10-Hz oscillation at aparticular phase likely corresponds to a temporalsearch strategy that is common to most subjects for thisparticular task

To summarize Experiment 1 revealed that percep-tual sampling during an easy search (L vs thorn) engagedmostly low temporal frequency components whereas adifficult search (color-orientation conjunction) involvedthree distinct frequency peaks 2 Hz 10 Hz and 18 to20 Hz One might wonder whether the slow frequencieswith performance rising and falling on a time scale ofseveral hundred milliseconds could reflect a voluntaryattentional strategy andor the effect of onset and offsettransients To test the latter possibility we recomputedamplitude spectra on the central 333 ms of each 500-ms-long sampling functions (ie after truncating thefirst 83 ms after stimulus onset and the last 83 msbefore stimulus offset Figure 7) For the easy searchno significant frequency remained in the amplitudespectrum of the grand-average sampling function Inthe average amplitude spectrum of individual samplingfunctions the first three frequencies (3ndash9 Hz) weresignificantly greater than chance As explained beforethe absence of corresponding effects in the grand-average sampling function implies that these low-frequency components were not phase locked anymorebetween subjects In other words the previouslyobserved low-frequency effects during the easy searchtask (Figure 4) appeared to be due in large part to theonset and offset portions of the sampling functionsduring which all observers systematically displayedincreased sensitivity For the difficult search both theamplitude spectrum of the average sampling functionand the average amplitude spectrum of individualsampling functions displayed a significant peak at 9 HzTogether the results in Figure 6 and Figure 7 suggestthat the attentional sampling function during thisdifficult search task fluctuated periodically at 9 to 10Hz not just at stimulus onset and offset but also duringthe central portion of stimulus presentation Inaddition we can speculate that the 18 to 20 Hzcomponent observed in Figure 6 and to some extent inFigure 7 could correspond to a harmonic of this 9 to 10Hz periodicity In the next experiment we aimed todemonstrate the functional significance of the periodicsampling observed in the difficult search task

Experiment 2

The attention function measured in the difficultsearch task of Experiment 1 is supposed to representthe fluctuations in attention efficiency over time for anaverage search trial in an average observer If this istrue one should predict that search patterns system-

Figure 5 Attentional sampling function estimated for a difficult

visual search trial Subjects (n frac14 14) were asked to report the

presence or absence of a red grating oriented 308 from upright

among red gratings oriented 3308 from upright and green

gratings oriented 308 from upright (and vice versa) d-primes

were measured for the 40 phase and frequency visual noise

presentation conditions Performances are represented in the

top panel as a modulation of d-prime with respect to the

average computed over all conditions (dashed baseline) On

these performance modulations we applied the complex

decomposition analysis (Figure 2) This analysis was performed

for each individual subject Then the individual estimated

attentional sampling functions were grand-averaged over all

subjects (bottom panel) The standard error of the mean is

represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 8

Figure 6 Amplitude spectra of the attention function estimated for the difficult search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for four frequencies 2 Hz 10 Hz and 18ndash20 Hz (B) Average amplitude spectrum of the individual

attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for six frequencies 2 Hz 8ndash10 Hz 16ndash20 Hz

Figure 7 Control for potential onset-offset (nonperiodic) effects The analysis from Figures 3 to 6 was replicated with estimated

attentional functions truncated for the first 83 ms and the last 83 ms (leaving only the central 333 ms portion of the original 500-ms-

long functions) (A) For the easy search task no frequency reached significance in the amplitude spectrum of the grand-average

attentional function (middle note that the frequency resolution of this analysis is 3 Hz instead of 2 Hz because of the shortened

duration of the sampling function) In the average amplitude spectrum of individual attentional functions (right) significance ( p value

103) was reached for three frequencies 3 Hz 6 Hz and 9 Hz (B) For the difficult search task three frequencies reached

significance in the amplitude spectrum of the grand-average attentional function 3 Hz 6 Hz and 9 Hz In the average amplitude

spectrum of individual attentional functions significance ( p value 103) was reached for five frequencies 3 Hz 6 Hz 9 Hz 15 Hz

and 18 Hz with a peak frequency at 9 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 9

atically presented with an SNR following the estimatedgrand-average attentional function (test condition)should lead to better performance than with theopposite signal-to-noise modulation (control condi-tion) We directly tested this prediction in Experiment2 Notice that we high-pass filtered the attentionsampling function by discarding the 2 Hz componentthought to correspond to an onset-offset transienteffect and therefore not reflecting a true samplingperiodicity Overall the aim of Experiment 2 was totest whether rhythmic attention sampling significantlycontributes to search performance This experimentwas performed on two sets of subjects seven subjectsfrom the lsquolsquoinitial subjectsrsquorsquo group (ie subjects who alsoperformed the difficult search task in Experiment 1)and seven subjects from the lsquolsquonaive subjectsrsquorsquo group (iesubjects who had not been included in Experiment 1)

As previously we first verified that naive observersused an appropriate search strategy by computing theRT middot set size slopes As expected (and as in Experiment1) we obtained strongly positive slopes 889 ms 6 413ms per element t(6)frac14 25 p frac14 004 for target presentand 1495 ms 6 35 ms per element t(6) frac14 47 pfrac1400034 for target absent

In the main comparison for Experiment 2 thesubjectsrsquo performance (as measured by d-primes) wassignificantly higher when stimulus presentation fol-lowed the estimated sampling function than its opposite(Figure 8 mixed design two-way analysis of variancewithin-subject factor lsquolsquotestcontrolrsquorsquo F(1 12)frac14 82 pfrac140014) Moreover no significant difference was foundbetween subject groups (between-subject factor lsquolsquoinitialnaıversquorsquo F(1 12)frac14 09 p frac14 037) nor any significant

interaction between the two factors (lsquolsquotestcontrolrsquorsquo middotlsquolsquoinitialnaiversquorsquo F(1 12) frac14 06 pfrac14 0471)

In summary these results reveal that the grand-average attention sampling function estimated for thedifficult search task (or more precisely its frequencycomponents 2 Hz) is representative of the subjectsrsquosampling strategy in this specific task This is true notonly for the initial subjects indicating that the resultobtained with this new method in Experiment 1 isrobust and replicable but also for the naive subjectsmeaning that the estimated attention sampling functionfor this specific task can also apply to a more generalpopulation of human observers

Discussion

In this study we proposed a novel experimentalmethod based on periodic noise interference andFourier series analysis to characterize the dynamics ofperceptual sampling occurring during a difficult task(color-orientation conjunction) and an easy task (L vsthorn) This method produces for each task a time courserepresenting the sampling efficiency of attention overthe 500 ms of stimulus display Our major result is thepresence of a significant oscillation in this time coursefor the difficult task but not for the easy one A simplecontinuous model of attention sampling in whichperceptual efficiency is constant over time (exceptpossibly for random fluctuations) would have pre-dicted flat attentional sampling functions with notemporal fluctuation and therefore no peak in thecorresponding amplitude spectrum This is not what we

Figure 8 Testing the behavioral relevance of the estimated attentional function A difficult search array was presented with dynamic

noise fluctuations following two conditions lsquolsquotest conditionrsquorsquo with a signal-to-noise ratio (SNR) following the previously estimated

grand-average attentional function (Figure 5) and lsquolsquocontrol conditionrsquorsquo with an SNR following the opposite of the estimated grand-

average attentional function We predicted significantly higher performance in the former than the latter case In both cases the 2 Hz

component of the function was omitted (ie the function was high-pass filtered) so as to focus on genuine periodic effects This

experiment was performed on subjects who had previously participated in the first experiment the so-called lsquolsquoinitial subjectsrsquorsquo (nfrac147)

but also on a new set of subjects the so-called lsquolsquonaive subjectsrsquorsquo (nfrac14 7) A two-way analysis of variance (mixed design) revealed a

significant difference between control and test conditions F(1 12)frac14 82 pfrac14 0014 but no significant difference between naive and

initial subjects F(1 12) frac14 09 p frac14 037 nor any interaction between the two factors F(1 12) frac14 06 p frac14 0471

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 10

observed Both tasks presented significant peaks intheir amplitude spectrum (Figures 4 and 6) Assumingthat perceptual efficiency is optimal at one or possiblytwo specific moments of stimulus presentation as onewould expect for example due to greater stimulus-onsetandor -offset sensitivity should have predicted in-creasing andor decreasing attentional sampling func-tions and therefore a steadily decreasing amplitudespectrum with a maximum at the lowest frequency (2Hz) We did observe such a 2 Hz peak in both tasks andconfirmed that it was mainly due to stimulus-onset and-offset effects (Figure 7) However a major unexpectedresult was the presence of another significant localmaximum in the difficult task with a similar phase forall subjects (9ndash10 Hz Figure 6) which was absent forthe easy task (Figure 4) this peak can be explained onlyby assuming rhythmic perceptual sampling by atten-tion

The presence of a 9 to 10 Hz oscillation in the grand-average temporal function (Figure 6A) could in theorybe explained either by the existence of an equivalentoscillation (with a similar phase) at the level of eachindividual subject or by a chance combination ofsubject-specific peaks and troughs at different momentsthat would spuriously create an oscillation whenaveraged over subjects However our finding thatindividual subjects also present a 9 to 10 Hz peak intheir temporal functions (Figure 6B) clearly rules outthe latter explanation Rather it seems that this specifictemporal function with its 9 to 10 Hz oscillation at aparticular phase captures rather well the attentiondeployment of a lsquolsquotypicalrsquorsquo subject in this search task (asshown also in Experiment 2)

Such rhythmic sampling in a difficult task (color-orientation conjunction) appears naturally compatiblewith the classic sequential models of attention deploy-ment a periodic focusing of attention on different items(or subsets of items) within the search array until thetarget is found (Treisman amp Gelade 1980 Treisman ampSouther 1985 Wolfe 1994 Wolfe et al 1989)However we cannot fully rule out the alternativeparallel model of attention deployment in which allitems are sampled at the same time (Eckstein et al2000 Palmer et al 1993) Indeed it is conceivable thatall of the items in the difficult search were processedtogether within each attentional cycle even as theefficiency of this processing fluctuated periodically at9 to 10 Hz Of course this would represent asubstantial alteration of the original parallel model(Eckstein et al 2000 Palmer et al 1993) yet such arevision appears necessary to reconcile this model withour experimental findings Whether attention sampledthe entire search array or only a subset of elements theimportant conclusion of our study is that it did soperiodically rather than continuously (at least for thespecific difficult search task that we used)

Our novel paradigm appears suitable for revealingthe temporal dynamics of attention in at least somevisual search tasks The method is not infalliblehowever and it is worth reviewing its main limitationsand the possible ways it could be improved in futurestudies

First and foremost our paradigm relies on Fourierseries decomposition which is an intrinsically linearframework (see the Appendix for further description ofthe model) As such we must assume that the searchperformance on each trial is the product of an attentionsampling function and the superimposed noise functionand that this attentional sampling function does notsystematically vary when the noise function changes(for example when we apply sinusoidal noise atdifferent phases and frequencies) This assumption oflinearity is both a reasonable first approximation in ourendeavor to measure the dynamics of attention and anobvious potential limitation of the proposed techniqueIndeed it implies that many nonlinear attentionalstrategies would not be appropriately captured by thetechnique It is worth pointing out however that ourmethod was explicitly designed to prevent one commonform of nonlinearity that would have resulted from asteady-state masking by the sinusoidal noise (ieperception fluctuating in phase with the noise) or froma perceptual entrainment to the stimulus (ie percep-tion fluctuating in counterphase with the noise) Inthese two forms of entrainment one would expect ageneral decrease (or increase respectively) of perfor-mance at specific frequencies regardless of the phaseYet in our experiments at each frequency we did notmerely measure a response to two sinusoids separatedby 908 (cosine and sine as Fourier analysis wouldminimally require) but to all four cardinal sinusoids(ie cosine and its opposite sine and its opposite)Therefore any nonlinear perceptual entrainment orsteady-state masking that would have equally affectedall four noise sinusoids at a given frequency would haveresulted in a null attention modulation at thisfrequency To conclude this type of nonlinear con-found alone cannot explain our finding of a 10 Hzattentional modulation in the difficult search task

Another potential issue with the current version ofour paradigm is the possibility that increased percep-tual sensitivity at stimulus onset andor offset couldhave produced edge effects that would have concealedthe true temporal fluctuations of attention Indeedthere was no noise on the screen before or after our500-ms sampling window meaning that the onset andoffset stimulus information were potentially free offorward and backward masking respectively Thispotential confound led us to perform an additionalanalysis in which we removed the edges of our samplingwindow (Figure 7) Although this removal stillproduced a 10 Hz periodicity in the difficult search

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 11

taskmdashthereby confirming that this periodicity was notmerely an artifact induced by edge effectsmdashit also leftno systematic remaining temporal fluctuation in theeasy search task It is plausible however that suchtemporal fluctuations maybe even periodic oneswould have occurred in the easy search task if the onsetand offset transient information had not been directlyavailable to the observers Therefore in future studieswe propose to improve the paradigm by starting thesinusoidal noise modulation well before (eg 500 ms)and ending it well after the 500-ms critical samplingwindow during which the search array is actuallydisplayed This would essentially avoid edge effects byseparating the onset and offset transients from ourwindow of analysis

Finally the potential influence of eye movementsmust be considered Even though we instructed thesubjects not to move their eyes one might argue thatsome of the temporal fluctuations observed in oursampling functions could be the result of unwantedsaccadic andor micro-saccadic eye movements Thiswould imply that these eye movements are time lockedto stimulus onset and that they would entail differenttemporal patterns depending on the exact search taskSaccades andor microsaccades could not have directlyproduced the temporal pattern observed in the difficultsearch task however because they tend to occur atlower frequencies typically less than 5 Hz (Martinez-Conde Macknik amp Hubel 2004) It appears morelikely therefore that the 10 Hz periodicity obtainedin our difficult search task was a reflection of covertrather than overt attention sampling dynamics

Conclusions

To summarize using an innovative psychophysicalmethod we were able to argue that an easy search taskwas processed in an apparently continuous modewhereas a difficult search task was processed periodi-cally at a frequency of 9 to 10 Hz Of course ourconclusions are limited to the two search tasks tested inthis study and further work would be needed todetermine if they could be generalized to other easy anddifficult search tasks In addition a potential limitationof our study is that we sampled attentional dynamicsfrom only 2 to 20 Hz Values greater than 20 Hz (egin the gamma frequency range) were not consideredalthough they may be relevant for attention (BauerOostenveld Peeters amp Fries 2006 Gruber MullerKeil amp Elbert 1999 Jensen Kaiser amp Lachaux 2007Treue 2001) A number of recent studies havesuggested that attention samples sensory informationperiodically at frequencies between 7 and 13 Hz invarious other experimental situations even when only

one object is attended and thus no sequentialexploration is required (Busch amp Vanrullen 2010Vanrullen et al 2007 Vanrullen amp Dubois 2011) Thepresent demonstration of an intrinsic periodicity duringa visual search task adds significant weight to thisemerging view of periodic attention

Keywords visual search attention temporal dynam-ics periodicity Fourier analysis dynamic noise

Acknowledgments

This research was funded by a EURYI Awardto RV

Commercial relationships noneCorresponding author Laura DugueEmail lauraduguecercoups-tlsefrAddress CNRSndashCerco Toulouse France

References

Bauer M Oostenveld R Peeters M amp Fries P(2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduceslow-frequency activity in parieto-occipital areasJournal of Neuroscience 26 490ndash501

Busch N A amp Vanrullen R (2010) SpontaneousEEG oscillations reveal periodic sampling of visualattention Proceedings of the National Academy ofSciences USA 107 16048ndash16053

Eckstein M P Thomas J P Palmer J ampShimozaki S S (2000) A signal detection modelpredicts the effects of set size on visual searchaccuracy for feature conjunction triple conjunc-tion and disjunction displays Perception amp Psy-chophysics 62 425ndash451

Gobell J L Tseng C H amp Sperling G (2004) Thespatial distribution of visual attention VisionResearch 44 1273ndash1296

Gruber T Muller M M Keil A amp Elbert T(1999) Selective visual-spatial attention altersinduced gamma band responses in the human EEGClinical Neurophysiology 110 2074ndash2085

Jensen O Kaiser J amp Lachaux J P (2007) Humangamma-frequency oscillations associated with at-tention and memory Trends in Neuroscience 30317ndash324

Lu Z L amp Dosher BA (2008) Characterizingobservers using external noise and observer models

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 12

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

additive or multiplicative constant) by a linear combi-nation of stimulus information and attentional sam-pling

P frac14Z 05

0

SethtTHORNAethtTHORNdt

For a given temporal frequency x we call

Pcosx frac14Z05

0

coseth2pxtTHORNAethtTHORNdt

andPsinx frac14Z05

0

sineth2pxtTHORNAethtTHORNdt

Pcosx frac14Z05

0

coseth2pxtTHORNAethtTHORNdt

andPsinx frac14Z05

0

sineth2pxsTHORNAethtTHORNdt

Pcosx Psinx P-cosx and P-sinx can be directly estimatedby presenting a visual stimulus whose temporal profileis modulated with a cosine sine ndashcosine or ndashsinefunction (respectively) and measuring the resultingperceptual performance (eg d-primes) This is thepurpose of Experiment 1 (see Figure 1) In theory someof these measures should be expected to provideredundant results because Pcosx frac14P-cosx and Psinx frac14P-sinx only one measurement for each pair should (intheory) be sufficient In practice however one canimagine that modulating the visual stimulus at aparticular frequency could induce performance changesindependently of the exact phase (for example if fastermodulation frequencies cause an increase of arousal orvigilance) Therefore we systematically measured theeffect of each phase modulation (cosine sine) and theirinverse (cosine sine) on performance Any unwant-ed factor that would equally affect all modulationphases at a given frequency would then be eliminated inthe following equations

Finally we define the complex coefficient Cx as Cxfrac14(Pcosx P-cosx)2 i (Psinx P-sinx)2 In other words

Figure 1 Experimental design of Experiment 1 Examples of four presentation conditions (cosine sine cosine and sine) at onegiven frequency (4 Hz) during the easy visual search task Observers searched for a thorn among Ls (the letters were presented

randomly in four orientations 08 908 1808 and 2708 from upright) A trial started when subjects pressed the space bar on the

keyboard The stimuli appeared after a random delay (between 15 and 25 s) and stayed on the screen during 500 ms On each of

the stimuli a visual noise patch composed by random dots was displayed by transparency and its opacity fluctuated according to 1

of 40 conditions randomly intermixed 10 different frequencies (2ndash20 Hz in 2-Hz steps) and four phases (cosine sine cosine andsine) For all of these conditions there was always the same amount of stimulus information and visual noise only the temporal

ordering of the display frames varied A similar procedure was applied to a difficult visual search task (color-orientation conjunction

task)

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 3

Cx frac14

Z05

0

coseth2pxtTHORNAethtTHORNdtZ05

0

coseth2pxtTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

i

Z05

0

sineth2pxtTHORNAethtTHORNdtZ05

0

sineth2pxsTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

frac14

Z05

0

coseth2pxtTHORNAethtTHORN thorn coseth2pxtTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

i

Z05

0

sineth2pxtTHORNAethtTHORN thorn sineth2pxtTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

frac14Z05

0

coseth2pxtTHORNAethtTHORNdt i

Z05

0

sineth2pxtTHORNAethtTHORNdt

frac14Z05

0

AethtTHORNei2pxtdt

Fourier series analysis implies that the attentionalsampling function should be proportional to

AethtTHORNrsquoXlsquo

nfrac140

Cn=05e2piethn=05THORNt

In other words measuring the complex coefficients Cx

every 2 Hz (from 2 to 20 Hz) will give us an estimate ofthe attentional function (note that values greater than20 Hz are not considered here for practical purposesalthough of course they may be relevant for attention)

Experiment 1 Measuring attentional sampling dynamics

This experiment was intended to put into practice thetheory developed in the preceding section in order tovalidate our new paradigm Subjects were placed infront of a gray screen and were asked to keep their gazeon a fixation point at the center When the subjectspressed the space bar on the keyboard a fixed number

of stimuli (determined separately for each subject asexplained above) appeared at random but equallyspaced positions on a circle at 148 eccentricity after arandom delay between 15 and 25 s On each stimuluselement a dynamic visual noise was superimposed bytransparency consisting of a square filled with dots ofrandom luminance the square was 15 times largerthan the stimuli The dynamic visual noise was appliedwith a temporal modulation function following 1 of 40possible conditions (Figure 1) 10 different frequencies(2ndash20 Hz by steps of 2 Hz) and four different phases(sine cosinesine andcosine) randomly interleavedin different trials of the same blocks When themodulation value was maximal the noise was opaqueand the stimulus was invisible when the modulationwas minimal the noise was fully transparent and thestimulus was thus unaffected Note that for all 40conditions there was always the same amount ofstimulus information and of visual noise over thecourse of each trial (in other words only the temporalordering of the display frames differed betweenconditions not the contents of the frames)

We evaluated the performances of the subjects bycomputing d-primes Based on a complex decompositionanalysis we used these d-primes to assess the dynamicsof attentional sampling occurring during one visualsearch trial for the difficult and the easy tasks (Figure 2)As explained in the preceding section this methodconsists in first combining for each frequency the d-primes for sine andsine phase conditions and(separately) for cosine andcosine phase conditions andsecond combining the two resulting estimators to obtaina vector in the complex domain For each subject wethus obtained 10 complex vectors for the 10 differentfrequencies These vectors were considered as Fouriercoefficients defined by their length (ie oscillatoryamplitude) and pointing angle (ie oscillatory phase)The complex coefficients were then used to compute aninverse Fourier transform thereby estimating theattentional sampling function in the time domain foreach of the subjects For both tasks we then computedthe average estimated attentional function over allsubjects Finally we analyzed the amplitude spectra ofthese two estimated sampling functions We firstcomputed a fast Fourier transform (FFT) on the grand-average attentional functions (ie averaged over allsubjects) and looked at the obtained amplitude spec-trum We also calculated this amplitude spectrum foreach subject based on his or her individual attentionalfunctions and recomputed the average amplitude spec-trum For both analyses we evaluated the significance ofthe measured amplitude spectra by using a Monte Carloprocedure Surrogate data were created for each subjectunder the null hypothesis that hit rate and false alarmrate are independent of phase and frequency Thecomplex decomposition analysis was recomputed foreach surrogate (nfrac1410000) and the amplitude spectra of

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 4

surrogate attentional functions (either based on grand-average attentional functions or on individual functions)were used to estimate significance To achieve this the10000 surrogate amplitude spectra were ranked inascending order separately for each frequency The9501th 9901th 9991th and 10000th values were

considered as the respective limits of four differentconfidence intervals (95 99 999 and 9999)which are represented with different colors in thebackground of the four corresponding graphs Anexperimentally observed spectral amplitude value wasconsidered significantly different from the corresponding

Figure 2 Estimation of the attentional sampling function by a complex decomposition analysis For both visual search tasks for each

subject we measured the performance modulation induced by visual noise across the 10 different frequencies for the four phase

conditions (sine cosinesine andcosine) We then combined the performances across the four phases for each frequency in the

complex domain For each subject we thus obtained 10 complex vectors characterized by their angle (phase in the complex domain)

and their length (amplitude) These vectors were used as Fourier coefficients on which we finally applied an inverse fast Fourier

transform to estimate the dynamics of attentional sampling in the time domain during one visual search trial All data in this figure

are fictitious and for illustration purposes only

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 5

null distribution with p 005 if it exceeded the 95confidence threshold (and with p 001 above the 99confidence threshold and so on) To take into accountthe possible problem of multiple statistical comparisonsacross the 10 frequencies used (2ndash20 Hz) we adopted aconservative statistical threshold of p 0001 For both

tasks we also ran the same analysis discarding thesignals from stimulus onset (0ndash83ms) and stimulus offset(417ndash500 ms) This was aimed at verifying that anyperiodicity highlighted by the previous analysis wouldnot be due merely to an increased sensitivity to the onsetandor the offset of the stimuli

Experiment 2 Testing the validity of estimatedattentional functions

Based on the previous estimates of attentionalsampling dynamics for the difficult visual search (seeResults and Figure 6) we tested the functionalsignificance of the sampling function on subjectsrsquoperformances For two groups of subjects seven whohad also performed the difficult search task inExperiment 1 and seven new subjects we presented thesearch arrays (color-orientation conjunction) embed-ded in dynamic noise with signal-to-noise ratios (SNRs)following the grand-average estimated attentionalfunction (ie maximal signal when attention isexpected to be maximal this was called the lsquolsquotestrsquorsquocondition) or the opposite signal (ie maximal signalwhen attention is expected to be minimal this wascalled the lsquolsquocontrolrsquorsquo condition) Whereas in the testcondition subjects should perform the search taskefficiently in the control condition their performanceswould be expected to decrease because stimulusinformation no longer matches their natural samplingdynamics We equalized the histograms of SNRs acrossthe two conditions to ensure they had comparable totalsignal energy and high-pass filtered the attentionalfunction excluding the 2 Hz component to ensure thatour results would not be driven by the 2 Hz componentwhich could have reflected a nonperiodic samplingstrategy (eg increased sensitivity for stimulus onsetandor offset as found in the easy search estimatedfunction) The two presentation conditions (test vscontrol) were randomly interleaved during the exper-iment We looked at the performances of the subjects inthese two conditions by computing d-primes Wecompared test versus control conditions using one-tailed t tests according to the prediction that a searcharray presented with an SNR following the estimatedattentional function should lead to better searchperformance than the opposite signal

Results

Experiment 1

In a preliminary experiment on the same group ofsubjects (nfrac1414 for both tasks) we verified that subjectsused the appropriate search strategy for the two tasks

Figure 3 Attentional sampling function estimated for an easy

visual search trial Subjects (n frac14 14) performed an easy task

consisting in finding athorn among Ls d-primes were measured for

the 40 phase and frequency visual noise presentation

conditions Performances are represented in the top panel as a

modulation of d-prime with respect to the average computed

over all conditions (dashed baseline) On these performance

modulations we applied the complex decomposition analysis

(Figure 2) This analysis was performed for each individual

subject Then the individual estimated attentional sampling

functions were grand-averaged over all subjects (bottom panel)

The standard error of the mean is represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 6

that we intended to use in Experiment 1 finding a thornamong Ls or a conjunction of color (red or green) andorientation (308 or 3308 from upright) We presentedthe stimuli for unlimited durations with a set sizerandomly drawn between four and eight elements andcomputed the RT middot set size slopes As expected theslopes were near zero for the L versusthorn task 77 ms 679 ms per element t(13)frac14 31 p frac14 001 for targetpresent and 116 ms 6 67 ms per element t(13)frac14 55p 001 for target absentmdasht tests performed under thenull hypothesis that the slopes are equal to zero (iethe task involved minimal attention) Slopes werestrongly positive for the conjunction task 70 ms 6 77ms per element t(13) frac14 256 p 00001 for targetpresent and 1846 ms 6 274 ms per element t(13) frac14191 p 00001 for target absent (ie the taskinvolved significant attentional resources)

In the main experiment we estimated the attentionalsampling dynamics for these two visual search tasksTo do so we modulated stimulus information byapplying an oscillatory visual noise (cf Figure 1)according to 40 different conditions (randomly inter-leaved) 10 different frequencies (2ndash20 Hz 2-Hz steps)and four different phases (sine cosine sine orcosine) at each frequency In both tasks we measuredfor each condition (ie for a given phase andfrequency) the performance of the subjects in detectingthe target by computing d-primes Then we applied acomplex decomposition analysis to estimate the dy-namics of attentional sampling during one trial ofvisual search (cf Methods section and Figure 2) ateach frequency d-primes for the different phaseconditions were combined to obtain a complex vectorin the Fourier domain An inverse Fourier transformallowed us to estimate attentional dynamics in the timedomain (see Methods)

For the easy search the resulting grand-averageattentional function over 14 subjects (Figure 3)presented large fluctuations in d-prime To determinethe relative power and the exact frequency of theseeffects we computed an FFT to obtain the amplitudespectrum of the grand-average attentional function(Figure 4A) The first four frequencies (2 4 6 and 8Hz) were significantly greater than chance (p 103 aconservative statistical threshold due to multiplecomparisons across frequencies) In Figure 4B theamplitude spectrum was computed first on eachindividual sampling function and subsequently aver-aged In this case the spectrum presented an overalldecreasing shape significantly different from chance atfrequencies of 2 4 and 6 Hz The existence of similarlow-frequency components (2 4 6 Hz) in the amplitudespectra from both the individual and grand-averagesampling functions implies that these components notonly present increased amplitude for each subject butalso are strongly phase locked between subjects As wewill see these low-frequency components mostly reflectincreased sensitivity to the onset and offset of thestimulus sequence

The same analysis was performed to reveal theattention sampling function during the difficult searchtask (Figure 5) This function was characterized by amore restricted range of oscillatory frequencies a rapidoscillation appeared to be superimposed on a slowerfluctuation In the amplitude spectra of Figure 6 threespecific frequency components stand out one at 2 Hzone at 10 Hz and one at 18 to 20 Hz These peaks wereapparent in the amplitude spectrum of the grand-average function (Figure 6A) as well as on the averageamplitude spectrum of individual sampling functions(Figure 6B) As previously we can thus conclude thatthese peaks were due to both higher amplitude and

Figure 4 Amplitude spectra of the attention function estimated for the easy search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) at the four lowest frequencies 2 Hz 4 Hz 6 Hz and 8 Hz (B) Average amplitude spectrum of the

individual attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude

reaches significance ( p value 103) at the three lowest frequencies 2 Hz 4 Hz and 6 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 7

stronger phase locking between subjects at thesefrequencies In other words this 10-Hz oscillation at aparticular phase likely corresponds to a temporalsearch strategy that is common to most subjects for thisparticular task

To summarize Experiment 1 revealed that percep-tual sampling during an easy search (L vs thorn) engagedmostly low temporal frequency components whereas adifficult search (color-orientation conjunction) involvedthree distinct frequency peaks 2 Hz 10 Hz and 18 to20 Hz One might wonder whether the slow frequencieswith performance rising and falling on a time scale ofseveral hundred milliseconds could reflect a voluntaryattentional strategy andor the effect of onset and offsettransients To test the latter possibility we recomputedamplitude spectra on the central 333 ms of each 500-ms-long sampling functions (ie after truncating thefirst 83 ms after stimulus onset and the last 83 msbefore stimulus offset Figure 7) For the easy searchno significant frequency remained in the amplitudespectrum of the grand-average sampling function Inthe average amplitude spectrum of individual samplingfunctions the first three frequencies (3ndash9 Hz) weresignificantly greater than chance As explained beforethe absence of corresponding effects in the grand-average sampling function implies that these low-frequency components were not phase locked anymorebetween subjects In other words the previouslyobserved low-frequency effects during the easy searchtask (Figure 4) appeared to be due in large part to theonset and offset portions of the sampling functionsduring which all observers systematically displayedincreased sensitivity For the difficult search both theamplitude spectrum of the average sampling functionand the average amplitude spectrum of individualsampling functions displayed a significant peak at 9 HzTogether the results in Figure 6 and Figure 7 suggestthat the attentional sampling function during thisdifficult search task fluctuated periodically at 9 to 10Hz not just at stimulus onset and offset but also duringthe central portion of stimulus presentation Inaddition we can speculate that the 18 to 20 Hzcomponent observed in Figure 6 and to some extent inFigure 7 could correspond to a harmonic of this 9 to 10Hz periodicity In the next experiment we aimed todemonstrate the functional significance of the periodicsampling observed in the difficult search task

Experiment 2

The attention function measured in the difficultsearch task of Experiment 1 is supposed to representthe fluctuations in attention efficiency over time for anaverage search trial in an average observer If this istrue one should predict that search patterns system-

Figure 5 Attentional sampling function estimated for a difficult

visual search trial Subjects (n frac14 14) were asked to report the

presence or absence of a red grating oriented 308 from upright

among red gratings oriented 3308 from upright and green

gratings oriented 308 from upright (and vice versa) d-primes

were measured for the 40 phase and frequency visual noise

presentation conditions Performances are represented in the

top panel as a modulation of d-prime with respect to the

average computed over all conditions (dashed baseline) On

these performance modulations we applied the complex

decomposition analysis (Figure 2) This analysis was performed

for each individual subject Then the individual estimated

attentional sampling functions were grand-averaged over all

subjects (bottom panel) The standard error of the mean is

represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 8

Figure 6 Amplitude spectra of the attention function estimated for the difficult search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for four frequencies 2 Hz 10 Hz and 18ndash20 Hz (B) Average amplitude spectrum of the individual

attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for six frequencies 2 Hz 8ndash10 Hz 16ndash20 Hz

Figure 7 Control for potential onset-offset (nonperiodic) effects The analysis from Figures 3 to 6 was replicated with estimated

attentional functions truncated for the first 83 ms and the last 83 ms (leaving only the central 333 ms portion of the original 500-ms-

long functions) (A) For the easy search task no frequency reached significance in the amplitude spectrum of the grand-average

attentional function (middle note that the frequency resolution of this analysis is 3 Hz instead of 2 Hz because of the shortened

duration of the sampling function) In the average amplitude spectrum of individual attentional functions (right) significance ( p value

103) was reached for three frequencies 3 Hz 6 Hz and 9 Hz (B) For the difficult search task three frequencies reached

significance in the amplitude spectrum of the grand-average attentional function 3 Hz 6 Hz and 9 Hz In the average amplitude

spectrum of individual attentional functions significance ( p value 103) was reached for five frequencies 3 Hz 6 Hz 9 Hz 15 Hz

and 18 Hz with a peak frequency at 9 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 9

atically presented with an SNR following the estimatedgrand-average attentional function (test condition)should lead to better performance than with theopposite signal-to-noise modulation (control condi-tion) We directly tested this prediction in Experiment2 Notice that we high-pass filtered the attentionsampling function by discarding the 2 Hz componentthought to correspond to an onset-offset transienteffect and therefore not reflecting a true samplingperiodicity Overall the aim of Experiment 2 was totest whether rhythmic attention sampling significantlycontributes to search performance This experimentwas performed on two sets of subjects seven subjectsfrom the lsquolsquoinitial subjectsrsquorsquo group (ie subjects who alsoperformed the difficult search task in Experiment 1)and seven subjects from the lsquolsquonaive subjectsrsquorsquo group (iesubjects who had not been included in Experiment 1)

As previously we first verified that naive observersused an appropriate search strategy by computing theRT middot set size slopes As expected (and as in Experiment1) we obtained strongly positive slopes 889 ms 6 413ms per element t(6)frac14 25 p frac14 004 for target presentand 1495 ms 6 35 ms per element t(6) frac14 47 pfrac1400034 for target absent

In the main comparison for Experiment 2 thesubjectsrsquo performance (as measured by d-primes) wassignificantly higher when stimulus presentation fol-lowed the estimated sampling function than its opposite(Figure 8 mixed design two-way analysis of variancewithin-subject factor lsquolsquotestcontrolrsquorsquo F(1 12)frac14 82 pfrac140014) Moreover no significant difference was foundbetween subject groups (between-subject factor lsquolsquoinitialnaıversquorsquo F(1 12)frac14 09 p frac14 037) nor any significant

interaction between the two factors (lsquolsquotestcontrolrsquorsquo middotlsquolsquoinitialnaiversquorsquo F(1 12) frac14 06 pfrac14 0471)

In summary these results reveal that the grand-average attention sampling function estimated for thedifficult search task (or more precisely its frequencycomponents 2 Hz) is representative of the subjectsrsquosampling strategy in this specific task This is true notonly for the initial subjects indicating that the resultobtained with this new method in Experiment 1 isrobust and replicable but also for the naive subjectsmeaning that the estimated attention sampling functionfor this specific task can also apply to a more generalpopulation of human observers

Discussion

In this study we proposed a novel experimentalmethod based on periodic noise interference andFourier series analysis to characterize the dynamics ofperceptual sampling occurring during a difficult task(color-orientation conjunction) and an easy task (L vsthorn) This method produces for each task a time courserepresenting the sampling efficiency of attention overthe 500 ms of stimulus display Our major result is thepresence of a significant oscillation in this time coursefor the difficult task but not for the easy one A simplecontinuous model of attention sampling in whichperceptual efficiency is constant over time (exceptpossibly for random fluctuations) would have pre-dicted flat attentional sampling functions with notemporal fluctuation and therefore no peak in thecorresponding amplitude spectrum This is not what we

Figure 8 Testing the behavioral relevance of the estimated attentional function A difficult search array was presented with dynamic

noise fluctuations following two conditions lsquolsquotest conditionrsquorsquo with a signal-to-noise ratio (SNR) following the previously estimated

grand-average attentional function (Figure 5) and lsquolsquocontrol conditionrsquorsquo with an SNR following the opposite of the estimated grand-

average attentional function We predicted significantly higher performance in the former than the latter case In both cases the 2 Hz

component of the function was omitted (ie the function was high-pass filtered) so as to focus on genuine periodic effects This

experiment was performed on subjects who had previously participated in the first experiment the so-called lsquolsquoinitial subjectsrsquorsquo (nfrac147)

but also on a new set of subjects the so-called lsquolsquonaive subjectsrsquorsquo (nfrac14 7) A two-way analysis of variance (mixed design) revealed a

significant difference between control and test conditions F(1 12)frac14 82 pfrac14 0014 but no significant difference between naive and

initial subjects F(1 12) frac14 09 p frac14 037 nor any interaction between the two factors F(1 12) frac14 06 p frac14 0471

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 10

observed Both tasks presented significant peaks intheir amplitude spectrum (Figures 4 and 6) Assumingthat perceptual efficiency is optimal at one or possiblytwo specific moments of stimulus presentation as onewould expect for example due to greater stimulus-onsetandor -offset sensitivity should have predicted in-creasing andor decreasing attentional sampling func-tions and therefore a steadily decreasing amplitudespectrum with a maximum at the lowest frequency (2Hz) We did observe such a 2 Hz peak in both tasks andconfirmed that it was mainly due to stimulus-onset and-offset effects (Figure 7) However a major unexpectedresult was the presence of another significant localmaximum in the difficult task with a similar phase forall subjects (9ndash10 Hz Figure 6) which was absent forthe easy task (Figure 4) this peak can be explained onlyby assuming rhythmic perceptual sampling by atten-tion

The presence of a 9 to 10 Hz oscillation in the grand-average temporal function (Figure 6A) could in theorybe explained either by the existence of an equivalentoscillation (with a similar phase) at the level of eachindividual subject or by a chance combination ofsubject-specific peaks and troughs at different momentsthat would spuriously create an oscillation whenaveraged over subjects However our finding thatindividual subjects also present a 9 to 10 Hz peak intheir temporal functions (Figure 6B) clearly rules outthe latter explanation Rather it seems that this specifictemporal function with its 9 to 10 Hz oscillation at aparticular phase captures rather well the attentiondeployment of a lsquolsquotypicalrsquorsquo subject in this search task (asshown also in Experiment 2)

Such rhythmic sampling in a difficult task (color-orientation conjunction) appears naturally compatiblewith the classic sequential models of attention deploy-ment a periodic focusing of attention on different items(or subsets of items) within the search array until thetarget is found (Treisman amp Gelade 1980 Treisman ampSouther 1985 Wolfe 1994 Wolfe et al 1989)However we cannot fully rule out the alternativeparallel model of attention deployment in which allitems are sampled at the same time (Eckstein et al2000 Palmer et al 1993) Indeed it is conceivable thatall of the items in the difficult search were processedtogether within each attentional cycle even as theefficiency of this processing fluctuated periodically at9 to 10 Hz Of course this would represent asubstantial alteration of the original parallel model(Eckstein et al 2000 Palmer et al 1993) yet such arevision appears necessary to reconcile this model withour experimental findings Whether attention sampledthe entire search array or only a subset of elements theimportant conclusion of our study is that it did soperiodically rather than continuously (at least for thespecific difficult search task that we used)

Our novel paradigm appears suitable for revealingthe temporal dynamics of attention in at least somevisual search tasks The method is not infalliblehowever and it is worth reviewing its main limitationsand the possible ways it could be improved in futurestudies

First and foremost our paradigm relies on Fourierseries decomposition which is an intrinsically linearframework (see the Appendix for further description ofthe model) As such we must assume that the searchperformance on each trial is the product of an attentionsampling function and the superimposed noise functionand that this attentional sampling function does notsystematically vary when the noise function changes(for example when we apply sinusoidal noise atdifferent phases and frequencies) This assumption oflinearity is both a reasonable first approximation in ourendeavor to measure the dynamics of attention and anobvious potential limitation of the proposed techniqueIndeed it implies that many nonlinear attentionalstrategies would not be appropriately captured by thetechnique It is worth pointing out however that ourmethod was explicitly designed to prevent one commonform of nonlinearity that would have resulted from asteady-state masking by the sinusoidal noise (ieperception fluctuating in phase with the noise) or froma perceptual entrainment to the stimulus (ie percep-tion fluctuating in counterphase with the noise) Inthese two forms of entrainment one would expect ageneral decrease (or increase respectively) of perfor-mance at specific frequencies regardless of the phaseYet in our experiments at each frequency we did notmerely measure a response to two sinusoids separatedby 908 (cosine and sine as Fourier analysis wouldminimally require) but to all four cardinal sinusoids(ie cosine and its opposite sine and its opposite)Therefore any nonlinear perceptual entrainment orsteady-state masking that would have equally affectedall four noise sinusoids at a given frequency would haveresulted in a null attention modulation at thisfrequency To conclude this type of nonlinear con-found alone cannot explain our finding of a 10 Hzattentional modulation in the difficult search task

Another potential issue with the current version ofour paradigm is the possibility that increased percep-tual sensitivity at stimulus onset andor offset couldhave produced edge effects that would have concealedthe true temporal fluctuations of attention Indeedthere was no noise on the screen before or after our500-ms sampling window meaning that the onset andoffset stimulus information were potentially free offorward and backward masking respectively Thispotential confound led us to perform an additionalanalysis in which we removed the edges of our samplingwindow (Figure 7) Although this removal stillproduced a 10 Hz periodicity in the difficult search

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 11

taskmdashthereby confirming that this periodicity was notmerely an artifact induced by edge effectsmdashit also leftno systematic remaining temporal fluctuation in theeasy search task It is plausible however that suchtemporal fluctuations maybe even periodic oneswould have occurred in the easy search task if the onsetand offset transient information had not been directlyavailable to the observers Therefore in future studieswe propose to improve the paradigm by starting thesinusoidal noise modulation well before (eg 500 ms)and ending it well after the 500-ms critical samplingwindow during which the search array is actuallydisplayed This would essentially avoid edge effects byseparating the onset and offset transients from ourwindow of analysis

Finally the potential influence of eye movementsmust be considered Even though we instructed thesubjects not to move their eyes one might argue thatsome of the temporal fluctuations observed in oursampling functions could be the result of unwantedsaccadic andor micro-saccadic eye movements Thiswould imply that these eye movements are time lockedto stimulus onset and that they would entail differenttemporal patterns depending on the exact search taskSaccades andor microsaccades could not have directlyproduced the temporal pattern observed in the difficultsearch task however because they tend to occur atlower frequencies typically less than 5 Hz (Martinez-Conde Macknik amp Hubel 2004) It appears morelikely therefore that the 10 Hz periodicity obtainedin our difficult search task was a reflection of covertrather than overt attention sampling dynamics

Conclusions

To summarize using an innovative psychophysicalmethod we were able to argue that an easy search taskwas processed in an apparently continuous modewhereas a difficult search task was processed periodi-cally at a frequency of 9 to 10 Hz Of course ourconclusions are limited to the two search tasks tested inthis study and further work would be needed todetermine if they could be generalized to other easy anddifficult search tasks In addition a potential limitationof our study is that we sampled attentional dynamicsfrom only 2 to 20 Hz Values greater than 20 Hz (egin the gamma frequency range) were not consideredalthough they may be relevant for attention (BauerOostenveld Peeters amp Fries 2006 Gruber MullerKeil amp Elbert 1999 Jensen Kaiser amp Lachaux 2007Treue 2001) A number of recent studies havesuggested that attention samples sensory informationperiodically at frequencies between 7 and 13 Hz invarious other experimental situations even when only

one object is attended and thus no sequentialexploration is required (Busch amp Vanrullen 2010Vanrullen et al 2007 Vanrullen amp Dubois 2011) Thepresent demonstration of an intrinsic periodicity duringa visual search task adds significant weight to thisemerging view of periodic attention

Keywords visual search attention temporal dynam-ics periodicity Fourier analysis dynamic noise

Acknowledgments

This research was funded by a EURYI Awardto RV

Commercial relationships noneCorresponding author Laura DugueEmail lauraduguecercoups-tlsefrAddress CNRSndashCerco Toulouse France

References

Bauer M Oostenveld R Peeters M amp Fries P(2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduceslow-frequency activity in parieto-occipital areasJournal of Neuroscience 26 490ndash501

Busch N A amp Vanrullen R (2010) SpontaneousEEG oscillations reveal periodic sampling of visualattention Proceedings of the National Academy ofSciences USA 107 16048ndash16053

Eckstein M P Thomas J P Palmer J ampShimozaki S S (2000) A signal detection modelpredicts the effects of set size on visual searchaccuracy for feature conjunction triple conjunc-tion and disjunction displays Perception amp Psy-chophysics 62 425ndash451

Gobell J L Tseng C H amp Sperling G (2004) Thespatial distribution of visual attention VisionResearch 44 1273ndash1296

Gruber T Muller M M Keil A amp Elbert T(1999) Selective visual-spatial attention altersinduced gamma band responses in the human EEGClinical Neurophysiology 110 2074ndash2085

Jensen O Kaiser J amp Lachaux J P (2007) Humangamma-frequency oscillations associated with at-tention and memory Trends in Neuroscience 30317ndash324

Lu Z L amp Dosher BA (2008) Characterizingobservers using external noise and observer models

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 12

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

Cx frac14

Z05

0

coseth2pxtTHORNAethtTHORNdtZ05

0

coseth2pxtTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

i

Z05

0

sineth2pxtTHORNAethtTHORNdtZ05

0

sineth2pxsTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

frac14

Z05

0

coseth2pxtTHORNAethtTHORN thorn coseth2pxtTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

i

Z05

0

sineth2pxtTHORNAethtTHORN thorn sineth2pxtTHORNAethtTHORNdt

2

0BBBBBB

1CCCCCCA

frac14Z05

0

coseth2pxtTHORNAethtTHORNdt i

Z05

0

sineth2pxtTHORNAethtTHORNdt

frac14Z05

0

AethtTHORNei2pxtdt

Fourier series analysis implies that the attentionalsampling function should be proportional to

AethtTHORNrsquoXlsquo

nfrac140

Cn=05e2piethn=05THORNt

In other words measuring the complex coefficients Cx

every 2 Hz (from 2 to 20 Hz) will give us an estimate ofthe attentional function (note that values greater than20 Hz are not considered here for practical purposesalthough of course they may be relevant for attention)

Experiment 1 Measuring attentional sampling dynamics

This experiment was intended to put into practice thetheory developed in the preceding section in order tovalidate our new paradigm Subjects were placed infront of a gray screen and were asked to keep their gazeon a fixation point at the center When the subjectspressed the space bar on the keyboard a fixed number

of stimuli (determined separately for each subject asexplained above) appeared at random but equallyspaced positions on a circle at 148 eccentricity after arandom delay between 15 and 25 s On each stimuluselement a dynamic visual noise was superimposed bytransparency consisting of a square filled with dots ofrandom luminance the square was 15 times largerthan the stimuli The dynamic visual noise was appliedwith a temporal modulation function following 1 of 40possible conditions (Figure 1) 10 different frequencies(2ndash20 Hz by steps of 2 Hz) and four different phases(sine cosinesine andcosine) randomly interleavedin different trials of the same blocks When themodulation value was maximal the noise was opaqueand the stimulus was invisible when the modulationwas minimal the noise was fully transparent and thestimulus was thus unaffected Note that for all 40conditions there was always the same amount ofstimulus information and of visual noise over thecourse of each trial (in other words only the temporalordering of the display frames differed betweenconditions not the contents of the frames)

We evaluated the performances of the subjects bycomputing d-primes Based on a complex decompositionanalysis we used these d-primes to assess the dynamicsof attentional sampling occurring during one visualsearch trial for the difficult and the easy tasks (Figure 2)As explained in the preceding section this methodconsists in first combining for each frequency the d-primes for sine andsine phase conditions and(separately) for cosine andcosine phase conditions andsecond combining the two resulting estimators to obtaina vector in the complex domain For each subject wethus obtained 10 complex vectors for the 10 differentfrequencies These vectors were considered as Fouriercoefficients defined by their length (ie oscillatoryamplitude) and pointing angle (ie oscillatory phase)The complex coefficients were then used to compute aninverse Fourier transform thereby estimating theattentional sampling function in the time domain foreach of the subjects For both tasks we then computedthe average estimated attentional function over allsubjects Finally we analyzed the amplitude spectra ofthese two estimated sampling functions We firstcomputed a fast Fourier transform (FFT) on the grand-average attentional functions (ie averaged over allsubjects) and looked at the obtained amplitude spec-trum We also calculated this amplitude spectrum foreach subject based on his or her individual attentionalfunctions and recomputed the average amplitude spec-trum For both analyses we evaluated the significance ofthe measured amplitude spectra by using a Monte Carloprocedure Surrogate data were created for each subjectunder the null hypothesis that hit rate and false alarmrate are independent of phase and frequency Thecomplex decomposition analysis was recomputed foreach surrogate (nfrac1410000) and the amplitude spectra of

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 4

surrogate attentional functions (either based on grand-average attentional functions or on individual functions)were used to estimate significance To achieve this the10000 surrogate amplitude spectra were ranked inascending order separately for each frequency The9501th 9901th 9991th and 10000th values were

considered as the respective limits of four differentconfidence intervals (95 99 999 and 9999)which are represented with different colors in thebackground of the four corresponding graphs Anexperimentally observed spectral amplitude value wasconsidered significantly different from the corresponding

Figure 2 Estimation of the attentional sampling function by a complex decomposition analysis For both visual search tasks for each

subject we measured the performance modulation induced by visual noise across the 10 different frequencies for the four phase

conditions (sine cosinesine andcosine) We then combined the performances across the four phases for each frequency in the

complex domain For each subject we thus obtained 10 complex vectors characterized by their angle (phase in the complex domain)

and their length (amplitude) These vectors were used as Fourier coefficients on which we finally applied an inverse fast Fourier

transform to estimate the dynamics of attentional sampling in the time domain during one visual search trial All data in this figure

are fictitious and for illustration purposes only

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 5

null distribution with p 005 if it exceeded the 95confidence threshold (and with p 001 above the 99confidence threshold and so on) To take into accountthe possible problem of multiple statistical comparisonsacross the 10 frequencies used (2ndash20 Hz) we adopted aconservative statistical threshold of p 0001 For both

tasks we also ran the same analysis discarding thesignals from stimulus onset (0ndash83ms) and stimulus offset(417ndash500 ms) This was aimed at verifying that anyperiodicity highlighted by the previous analysis wouldnot be due merely to an increased sensitivity to the onsetandor the offset of the stimuli

Experiment 2 Testing the validity of estimatedattentional functions

Based on the previous estimates of attentionalsampling dynamics for the difficult visual search (seeResults and Figure 6) we tested the functionalsignificance of the sampling function on subjectsrsquoperformances For two groups of subjects seven whohad also performed the difficult search task inExperiment 1 and seven new subjects we presented thesearch arrays (color-orientation conjunction) embed-ded in dynamic noise with signal-to-noise ratios (SNRs)following the grand-average estimated attentionalfunction (ie maximal signal when attention isexpected to be maximal this was called the lsquolsquotestrsquorsquocondition) or the opposite signal (ie maximal signalwhen attention is expected to be minimal this wascalled the lsquolsquocontrolrsquorsquo condition) Whereas in the testcondition subjects should perform the search taskefficiently in the control condition their performanceswould be expected to decrease because stimulusinformation no longer matches their natural samplingdynamics We equalized the histograms of SNRs acrossthe two conditions to ensure they had comparable totalsignal energy and high-pass filtered the attentionalfunction excluding the 2 Hz component to ensure thatour results would not be driven by the 2 Hz componentwhich could have reflected a nonperiodic samplingstrategy (eg increased sensitivity for stimulus onsetandor offset as found in the easy search estimatedfunction) The two presentation conditions (test vscontrol) were randomly interleaved during the exper-iment We looked at the performances of the subjects inthese two conditions by computing d-primes Wecompared test versus control conditions using one-tailed t tests according to the prediction that a searcharray presented with an SNR following the estimatedattentional function should lead to better searchperformance than the opposite signal

Results

Experiment 1

In a preliminary experiment on the same group ofsubjects (nfrac1414 for both tasks) we verified that subjectsused the appropriate search strategy for the two tasks

Figure 3 Attentional sampling function estimated for an easy

visual search trial Subjects (n frac14 14) performed an easy task

consisting in finding athorn among Ls d-primes were measured for

the 40 phase and frequency visual noise presentation

conditions Performances are represented in the top panel as a

modulation of d-prime with respect to the average computed

over all conditions (dashed baseline) On these performance

modulations we applied the complex decomposition analysis

(Figure 2) This analysis was performed for each individual

subject Then the individual estimated attentional sampling

functions were grand-averaged over all subjects (bottom panel)

The standard error of the mean is represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 6

that we intended to use in Experiment 1 finding a thornamong Ls or a conjunction of color (red or green) andorientation (308 or 3308 from upright) We presentedthe stimuli for unlimited durations with a set sizerandomly drawn between four and eight elements andcomputed the RT middot set size slopes As expected theslopes were near zero for the L versusthorn task 77 ms 679 ms per element t(13)frac14 31 p frac14 001 for targetpresent and 116 ms 6 67 ms per element t(13)frac14 55p 001 for target absentmdasht tests performed under thenull hypothesis that the slopes are equal to zero (iethe task involved minimal attention) Slopes werestrongly positive for the conjunction task 70 ms 6 77ms per element t(13) frac14 256 p 00001 for targetpresent and 1846 ms 6 274 ms per element t(13) frac14191 p 00001 for target absent (ie the taskinvolved significant attentional resources)

In the main experiment we estimated the attentionalsampling dynamics for these two visual search tasksTo do so we modulated stimulus information byapplying an oscillatory visual noise (cf Figure 1)according to 40 different conditions (randomly inter-leaved) 10 different frequencies (2ndash20 Hz 2-Hz steps)and four different phases (sine cosine sine orcosine) at each frequency In both tasks we measuredfor each condition (ie for a given phase andfrequency) the performance of the subjects in detectingthe target by computing d-primes Then we applied acomplex decomposition analysis to estimate the dy-namics of attentional sampling during one trial ofvisual search (cf Methods section and Figure 2) ateach frequency d-primes for the different phaseconditions were combined to obtain a complex vectorin the Fourier domain An inverse Fourier transformallowed us to estimate attentional dynamics in the timedomain (see Methods)

For the easy search the resulting grand-averageattentional function over 14 subjects (Figure 3)presented large fluctuations in d-prime To determinethe relative power and the exact frequency of theseeffects we computed an FFT to obtain the amplitudespectrum of the grand-average attentional function(Figure 4A) The first four frequencies (2 4 6 and 8Hz) were significantly greater than chance (p 103 aconservative statistical threshold due to multiplecomparisons across frequencies) In Figure 4B theamplitude spectrum was computed first on eachindividual sampling function and subsequently aver-aged In this case the spectrum presented an overalldecreasing shape significantly different from chance atfrequencies of 2 4 and 6 Hz The existence of similarlow-frequency components (2 4 6 Hz) in the amplitudespectra from both the individual and grand-averagesampling functions implies that these components notonly present increased amplitude for each subject butalso are strongly phase locked between subjects As wewill see these low-frequency components mostly reflectincreased sensitivity to the onset and offset of thestimulus sequence

The same analysis was performed to reveal theattention sampling function during the difficult searchtask (Figure 5) This function was characterized by amore restricted range of oscillatory frequencies a rapidoscillation appeared to be superimposed on a slowerfluctuation In the amplitude spectra of Figure 6 threespecific frequency components stand out one at 2 Hzone at 10 Hz and one at 18 to 20 Hz These peaks wereapparent in the amplitude spectrum of the grand-average function (Figure 6A) as well as on the averageamplitude spectrum of individual sampling functions(Figure 6B) As previously we can thus conclude thatthese peaks were due to both higher amplitude and

Figure 4 Amplitude spectra of the attention function estimated for the easy search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) at the four lowest frequencies 2 Hz 4 Hz 6 Hz and 8 Hz (B) Average amplitude spectrum of the

individual attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude

reaches significance ( p value 103) at the three lowest frequencies 2 Hz 4 Hz and 6 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 7

stronger phase locking between subjects at thesefrequencies In other words this 10-Hz oscillation at aparticular phase likely corresponds to a temporalsearch strategy that is common to most subjects for thisparticular task

To summarize Experiment 1 revealed that percep-tual sampling during an easy search (L vs thorn) engagedmostly low temporal frequency components whereas adifficult search (color-orientation conjunction) involvedthree distinct frequency peaks 2 Hz 10 Hz and 18 to20 Hz One might wonder whether the slow frequencieswith performance rising and falling on a time scale ofseveral hundred milliseconds could reflect a voluntaryattentional strategy andor the effect of onset and offsettransients To test the latter possibility we recomputedamplitude spectra on the central 333 ms of each 500-ms-long sampling functions (ie after truncating thefirst 83 ms after stimulus onset and the last 83 msbefore stimulus offset Figure 7) For the easy searchno significant frequency remained in the amplitudespectrum of the grand-average sampling function Inthe average amplitude spectrum of individual samplingfunctions the first three frequencies (3ndash9 Hz) weresignificantly greater than chance As explained beforethe absence of corresponding effects in the grand-average sampling function implies that these low-frequency components were not phase locked anymorebetween subjects In other words the previouslyobserved low-frequency effects during the easy searchtask (Figure 4) appeared to be due in large part to theonset and offset portions of the sampling functionsduring which all observers systematically displayedincreased sensitivity For the difficult search both theamplitude spectrum of the average sampling functionand the average amplitude spectrum of individualsampling functions displayed a significant peak at 9 HzTogether the results in Figure 6 and Figure 7 suggestthat the attentional sampling function during thisdifficult search task fluctuated periodically at 9 to 10Hz not just at stimulus onset and offset but also duringthe central portion of stimulus presentation Inaddition we can speculate that the 18 to 20 Hzcomponent observed in Figure 6 and to some extent inFigure 7 could correspond to a harmonic of this 9 to 10Hz periodicity In the next experiment we aimed todemonstrate the functional significance of the periodicsampling observed in the difficult search task

Experiment 2

The attention function measured in the difficultsearch task of Experiment 1 is supposed to representthe fluctuations in attention efficiency over time for anaverage search trial in an average observer If this istrue one should predict that search patterns system-

Figure 5 Attentional sampling function estimated for a difficult

visual search trial Subjects (n frac14 14) were asked to report the

presence or absence of a red grating oriented 308 from upright

among red gratings oriented 3308 from upright and green

gratings oriented 308 from upright (and vice versa) d-primes

were measured for the 40 phase and frequency visual noise

presentation conditions Performances are represented in the

top panel as a modulation of d-prime with respect to the

average computed over all conditions (dashed baseline) On

these performance modulations we applied the complex

decomposition analysis (Figure 2) This analysis was performed

for each individual subject Then the individual estimated

attentional sampling functions were grand-averaged over all

subjects (bottom panel) The standard error of the mean is

represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 8

Figure 6 Amplitude spectra of the attention function estimated for the difficult search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for four frequencies 2 Hz 10 Hz and 18ndash20 Hz (B) Average amplitude spectrum of the individual

attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for six frequencies 2 Hz 8ndash10 Hz 16ndash20 Hz

Figure 7 Control for potential onset-offset (nonperiodic) effects The analysis from Figures 3 to 6 was replicated with estimated

attentional functions truncated for the first 83 ms and the last 83 ms (leaving only the central 333 ms portion of the original 500-ms-

long functions) (A) For the easy search task no frequency reached significance in the amplitude spectrum of the grand-average

attentional function (middle note that the frequency resolution of this analysis is 3 Hz instead of 2 Hz because of the shortened

duration of the sampling function) In the average amplitude spectrum of individual attentional functions (right) significance ( p value

103) was reached for three frequencies 3 Hz 6 Hz and 9 Hz (B) For the difficult search task three frequencies reached

significance in the amplitude spectrum of the grand-average attentional function 3 Hz 6 Hz and 9 Hz In the average amplitude

spectrum of individual attentional functions significance ( p value 103) was reached for five frequencies 3 Hz 6 Hz 9 Hz 15 Hz

and 18 Hz with a peak frequency at 9 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 9

atically presented with an SNR following the estimatedgrand-average attentional function (test condition)should lead to better performance than with theopposite signal-to-noise modulation (control condi-tion) We directly tested this prediction in Experiment2 Notice that we high-pass filtered the attentionsampling function by discarding the 2 Hz componentthought to correspond to an onset-offset transienteffect and therefore not reflecting a true samplingperiodicity Overall the aim of Experiment 2 was totest whether rhythmic attention sampling significantlycontributes to search performance This experimentwas performed on two sets of subjects seven subjectsfrom the lsquolsquoinitial subjectsrsquorsquo group (ie subjects who alsoperformed the difficult search task in Experiment 1)and seven subjects from the lsquolsquonaive subjectsrsquorsquo group (iesubjects who had not been included in Experiment 1)

As previously we first verified that naive observersused an appropriate search strategy by computing theRT middot set size slopes As expected (and as in Experiment1) we obtained strongly positive slopes 889 ms 6 413ms per element t(6)frac14 25 p frac14 004 for target presentand 1495 ms 6 35 ms per element t(6) frac14 47 pfrac1400034 for target absent

In the main comparison for Experiment 2 thesubjectsrsquo performance (as measured by d-primes) wassignificantly higher when stimulus presentation fol-lowed the estimated sampling function than its opposite(Figure 8 mixed design two-way analysis of variancewithin-subject factor lsquolsquotestcontrolrsquorsquo F(1 12)frac14 82 pfrac140014) Moreover no significant difference was foundbetween subject groups (between-subject factor lsquolsquoinitialnaıversquorsquo F(1 12)frac14 09 p frac14 037) nor any significant

interaction between the two factors (lsquolsquotestcontrolrsquorsquo middotlsquolsquoinitialnaiversquorsquo F(1 12) frac14 06 pfrac14 0471)

In summary these results reveal that the grand-average attention sampling function estimated for thedifficult search task (or more precisely its frequencycomponents 2 Hz) is representative of the subjectsrsquosampling strategy in this specific task This is true notonly for the initial subjects indicating that the resultobtained with this new method in Experiment 1 isrobust and replicable but also for the naive subjectsmeaning that the estimated attention sampling functionfor this specific task can also apply to a more generalpopulation of human observers

Discussion

In this study we proposed a novel experimentalmethod based on periodic noise interference andFourier series analysis to characterize the dynamics ofperceptual sampling occurring during a difficult task(color-orientation conjunction) and an easy task (L vsthorn) This method produces for each task a time courserepresenting the sampling efficiency of attention overthe 500 ms of stimulus display Our major result is thepresence of a significant oscillation in this time coursefor the difficult task but not for the easy one A simplecontinuous model of attention sampling in whichperceptual efficiency is constant over time (exceptpossibly for random fluctuations) would have pre-dicted flat attentional sampling functions with notemporal fluctuation and therefore no peak in thecorresponding amplitude spectrum This is not what we

Figure 8 Testing the behavioral relevance of the estimated attentional function A difficult search array was presented with dynamic

noise fluctuations following two conditions lsquolsquotest conditionrsquorsquo with a signal-to-noise ratio (SNR) following the previously estimated

grand-average attentional function (Figure 5) and lsquolsquocontrol conditionrsquorsquo with an SNR following the opposite of the estimated grand-

average attentional function We predicted significantly higher performance in the former than the latter case In both cases the 2 Hz

component of the function was omitted (ie the function was high-pass filtered) so as to focus on genuine periodic effects This

experiment was performed on subjects who had previously participated in the first experiment the so-called lsquolsquoinitial subjectsrsquorsquo (nfrac147)

but also on a new set of subjects the so-called lsquolsquonaive subjectsrsquorsquo (nfrac14 7) A two-way analysis of variance (mixed design) revealed a

significant difference between control and test conditions F(1 12)frac14 82 pfrac14 0014 but no significant difference between naive and

initial subjects F(1 12) frac14 09 p frac14 037 nor any interaction between the two factors F(1 12) frac14 06 p frac14 0471

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 10

observed Both tasks presented significant peaks intheir amplitude spectrum (Figures 4 and 6) Assumingthat perceptual efficiency is optimal at one or possiblytwo specific moments of stimulus presentation as onewould expect for example due to greater stimulus-onsetandor -offset sensitivity should have predicted in-creasing andor decreasing attentional sampling func-tions and therefore a steadily decreasing amplitudespectrum with a maximum at the lowest frequency (2Hz) We did observe such a 2 Hz peak in both tasks andconfirmed that it was mainly due to stimulus-onset and-offset effects (Figure 7) However a major unexpectedresult was the presence of another significant localmaximum in the difficult task with a similar phase forall subjects (9ndash10 Hz Figure 6) which was absent forthe easy task (Figure 4) this peak can be explained onlyby assuming rhythmic perceptual sampling by atten-tion

The presence of a 9 to 10 Hz oscillation in the grand-average temporal function (Figure 6A) could in theorybe explained either by the existence of an equivalentoscillation (with a similar phase) at the level of eachindividual subject or by a chance combination ofsubject-specific peaks and troughs at different momentsthat would spuriously create an oscillation whenaveraged over subjects However our finding thatindividual subjects also present a 9 to 10 Hz peak intheir temporal functions (Figure 6B) clearly rules outthe latter explanation Rather it seems that this specifictemporal function with its 9 to 10 Hz oscillation at aparticular phase captures rather well the attentiondeployment of a lsquolsquotypicalrsquorsquo subject in this search task (asshown also in Experiment 2)

Such rhythmic sampling in a difficult task (color-orientation conjunction) appears naturally compatiblewith the classic sequential models of attention deploy-ment a periodic focusing of attention on different items(or subsets of items) within the search array until thetarget is found (Treisman amp Gelade 1980 Treisman ampSouther 1985 Wolfe 1994 Wolfe et al 1989)However we cannot fully rule out the alternativeparallel model of attention deployment in which allitems are sampled at the same time (Eckstein et al2000 Palmer et al 1993) Indeed it is conceivable thatall of the items in the difficult search were processedtogether within each attentional cycle even as theefficiency of this processing fluctuated periodically at9 to 10 Hz Of course this would represent asubstantial alteration of the original parallel model(Eckstein et al 2000 Palmer et al 1993) yet such arevision appears necessary to reconcile this model withour experimental findings Whether attention sampledthe entire search array or only a subset of elements theimportant conclusion of our study is that it did soperiodically rather than continuously (at least for thespecific difficult search task that we used)

Our novel paradigm appears suitable for revealingthe temporal dynamics of attention in at least somevisual search tasks The method is not infalliblehowever and it is worth reviewing its main limitationsand the possible ways it could be improved in futurestudies

First and foremost our paradigm relies on Fourierseries decomposition which is an intrinsically linearframework (see the Appendix for further description ofthe model) As such we must assume that the searchperformance on each trial is the product of an attentionsampling function and the superimposed noise functionand that this attentional sampling function does notsystematically vary when the noise function changes(for example when we apply sinusoidal noise atdifferent phases and frequencies) This assumption oflinearity is both a reasonable first approximation in ourendeavor to measure the dynamics of attention and anobvious potential limitation of the proposed techniqueIndeed it implies that many nonlinear attentionalstrategies would not be appropriately captured by thetechnique It is worth pointing out however that ourmethod was explicitly designed to prevent one commonform of nonlinearity that would have resulted from asteady-state masking by the sinusoidal noise (ieperception fluctuating in phase with the noise) or froma perceptual entrainment to the stimulus (ie percep-tion fluctuating in counterphase with the noise) Inthese two forms of entrainment one would expect ageneral decrease (or increase respectively) of perfor-mance at specific frequencies regardless of the phaseYet in our experiments at each frequency we did notmerely measure a response to two sinusoids separatedby 908 (cosine and sine as Fourier analysis wouldminimally require) but to all four cardinal sinusoids(ie cosine and its opposite sine and its opposite)Therefore any nonlinear perceptual entrainment orsteady-state masking that would have equally affectedall four noise sinusoids at a given frequency would haveresulted in a null attention modulation at thisfrequency To conclude this type of nonlinear con-found alone cannot explain our finding of a 10 Hzattentional modulation in the difficult search task

Another potential issue with the current version ofour paradigm is the possibility that increased percep-tual sensitivity at stimulus onset andor offset couldhave produced edge effects that would have concealedthe true temporal fluctuations of attention Indeedthere was no noise on the screen before or after our500-ms sampling window meaning that the onset andoffset stimulus information were potentially free offorward and backward masking respectively Thispotential confound led us to perform an additionalanalysis in which we removed the edges of our samplingwindow (Figure 7) Although this removal stillproduced a 10 Hz periodicity in the difficult search

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 11

taskmdashthereby confirming that this periodicity was notmerely an artifact induced by edge effectsmdashit also leftno systematic remaining temporal fluctuation in theeasy search task It is plausible however that suchtemporal fluctuations maybe even periodic oneswould have occurred in the easy search task if the onsetand offset transient information had not been directlyavailable to the observers Therefore in future studieswe propose to improve the paradigm by starting thesinusoidal noise modulation well before (eg 500 ms)and ending it well after the 500-ms critical samplingwindow during which the search array is actuallydisplayed This would essentially avoid edge effects byseparating the onset and offset transients from ourwindow of analysis

Finally the potential influence of eye movementsmust be considered Even though we instructed thesubjects not to move their eyes one might argue thatsome of the temporal fluctuations observed in oursampling functions could be the result of unwantedsaccadic andor micro-saccadic eye movements Thiswould imply that these eye movements are time lockedto stimulus onset and that they would entail differenttemporal patterns depending on the exact search taskSaccades andor microsaccades could not have directlyproduced the temporal pattern observed in the difficultsearch task however because they tend to occur atlower frequencies typically less than 5 Hz (Martinez-Conde Macknik amp Hubel 2004) It appears morelikely therefore that the 10 Hz periodicity obtainedin our difficult search task was a reflection of covertrather than overt attention sampling dynamics

Conclusions

To summarize using an innovative psychophysicalmethod we were able to argue that an easy search taskwas processed in an apparently continuous modewhereas a difficult search task was processed periodi-cally at a frequency of 9 to 10 Hz Of course ourconclusions are limited to the two search tasks tested inthis study and further work would be needed todetermine if they could be generalized to other easy anddifficult search tasks In addition a potential limitationof our study is that we sampled attentional dynamicsfrom only 2 to 20 Hz Values greater than 20 Hz (egin the gamma frequency range) were not consideredalthough they may be relevant for attention (BauerOostenveld Peeters amp Fries 2006 Gruber MullerKeil amp Elbert 1999 Jensen Kaiser amp Lachaux 2007Treue 2001) A number of recent studies havesuggested that attention samples sensory informationperiodically at frequencies between 7 and 13 Hz invarious other experimental situations even when only

one object is attended and thus no sequentialexploration is required (Busch amp Vanrullen 2010Vanrullen et al 2007 Vanrullen amp Dubois 2011) Thepresent demonstration of an intrinsic periodicity duringa visual search task adds significant weight to thisemerging view of periodic attention

Keywords visual search attention temporal dynam-ics periodicity Fourier analysis dynamic noise

Acknowledgments

This research was funded by a EURYI Awardto RV

Commercial relationships noneCorresponding author Laura DugueEmail lauraduguecercoups-tlsefrAddress CNRSndashCerco Toulouse France

References

Bauer M Oostenveld R Peeters M amp Fries P(2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduceslow-frequency activity in parieto-occipital areasJournal of Neuroscience 26 490ndash501

Busch N A amp Vanrullen R (2010) SpontaneousEEG oscillations reveal periodic sampling of visualattention Proceedings of the National Academy ofSciences USA 107 16048ndash16053

Eckstein M P Thomas J P Palmer J ampShimozaki S S (2000) A signal detection modelpredicts the effects of set size on visual searchaccuracy for feature conjunction triple conjunc-tion and disjunction displays Perception amp Psy-chophysics 62 425ndash451

Gobell J L Tseng C H amp Sperling G (2004) Thespatial distribution of visual attention VisionResearch 44 1273ndash1296

Gruber T Muller M M Keil A amp Elbert T(1999) Selective visual-spatial attention altersinduced gamma band responses in the human EEGClinical Neurophysiology 110 2074ndash2085

Jensen O Kaiser J amp Lachaux J P (2007) Humangamma-frequency oscillations associated with at-tention and memory Trends in Neuroscience 30317ndash324

Lu Z L amp Dosher BA (2008) Characterizingobservers using external noise and observer models

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 12

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

surrogate attentional functions (either based on grand-average attentional functions or on individual functions)were used to estimate significance To achieve this the10000 surrogate amplitude spectra were ranked inascending order separately for each frequency The9501th 9901th 9991th and 10000th values were

considered as the respective limits of four differentconfidence intervals (95 99 999 and 9999)which are represented with different colors in thebackground of the four corresponding graphs Anexperimentally observed spectral amplitude value wasconsidered significantly different from the corresponding

Figure 2 Estimation of the attentional sampling function by a complex decomposition analysis For both visual search tasks for each

subject we measured the performance modulation induced by visual noise across the 10 different frequencies for the four phase

conditions (sine cosinesine andcosine) We then combined the performances across the four phases for each frequency in the

complex domain For each subject we thus obtained 10 complex vectors characterized by their angle (phase in the complex domain)

and their length (amplitude) These vectors were used as Fourier coefficients on which we finally applied an inverse fast Fourier

transform to estimate the dynamics of attentional sampling in the time domain during one visual search trial All data in this figure

are fictitious and for illustration purposes only

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 5

null distribution with p 005 if it exceeded the 95confidence threshold (and with p 001 above the 99confidence threshold and so on) To take into accountthe possible problem of multiple statistical comparisonsacross the 10 frequencies used (2ndash20 Hz) we adopted aconservative statistical threshold of p 0001 For both

tasks we also ran the same analysis discarding thesignals from stimulus onset (0ndash83ms) and stimulus offset(417ndash500 ms) This was aimed at verifying that anyperiodicity highlighted by the previous analysis wouldnot be due merely to an increased sensitivity to the onsetandor the offset of the stimuli

Experiment 2 Testing the validity of estimatedattentional functions

Based on the previous estimates of attentionalsampling dynamics for the difficult visual search (seeResults and Figure 6) we tested the functionalsignificance of the sampling function on subjectsrsquoperformances For two groups of subjects seven whohad also performed the difficult search task inExperiment 1 and seven new subjects we presented thesearch arrays (color-orientation conjunction) embed-ded in dynamic noise with signal-to-noise ratios (SNRs)following the grand-average estimated attentionalfunction (ie maximal signal when attention isexpected to be maximal this was called the lsquolsquotestrsquorsquocondition) or the opposite signal (ie maximal signalwhen attention is expected to be minimal this wascalled the lsquolsquocontrolrsquorsquo condition) Whereas in the testcondition subjects should perform the search taskefficiently in the control condition their performanceswould be expected to decrease because stimulusinformation no longer matches their natural samplingdynamics We equalized the histograms of SNRs acrossthe two conditions to ensure they had comparable totalsignal energy and high-pass filtered the attentionalfunction excluding the 2 Hz component to ensure thatour results would not be driven by the 2 Hz componentwhich could have reflected a nonperiodic samplingstrategy (eg increased sensitivity for stimulus onsetandor offset as found in the easy search estimatedfunction) The two presentation conditions (test vscontrol) were randomly interleaved during the exper-iment We looked at the performances of the subjects inthese two conditions by computing d-primes Wecompared test versus control conditions using one-tailed t tests according to the prediction that a searcharray presented with an SNR following the estimatedattentional function should lead to better searchperformance than the opposite signal

Results

Experiment 1

In a preliminary experiment on the same group ofsubjects (nfrac1414 for both tasks) we verified that subjectsused the appropriate search strategy for the two tasks

Figure 3 Attentional sampling function estimated for an easy

visual search trial Subjects (n frac14 14) performed an easy task

consisting in finding athorn among Ls d-primes were measured for

the 40 phase and frequency visual noise presentation

conditions Performances are represented in the top panel as a

modulation of d-prime with respect to the average computed

over all conditions (dashed baseline) On these performance

modulations we applied the complex decomposition analysis

(Figure 2) This analysis was performed for each individual

subject Then the individual estimated attentional sampling

functions were grand-averaged over all subjects (bottom panel)

The standard error of the mean is represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 6

that we intended to use in Experiment 1 finding a thornamong Ls or a conjunction of color (red or green) andorientation (308 or 3308 from upright) We presentedthe stimuli for unlimited durations with a set sizerandomly drawn between four and eight elements andcomputed the RT middot set size slopes As expected theslopes were near zero for the L versusthorn task 77 ms 679 ms per element t(13)frac14 31 p frac14 001 for targetpresent and 116 ms 6 67 ms per element t(13)frac14 55p 001 for target absentmdasht tests performed under thenull hypothesis that the slopes are equal to zero (iethe task involved minimal attention) Slopes werestrongly positive for the conjunction task 70 ms 6 77ms per element t(13) frac14 256 p 00001 for targetpresent and 1846 ms 6 274 ms per element t(13) frac14191 p 00001 for target absent (ie the taskinvolved significant attentional resources)

In the main experiment we estimated the attentionalsampling dynamics for these two visual search tasksTo do so we modulated stimulus information byapplying an oscillatory visual noise (cf Figure 1)according to 40 different conditions (randomly inter-leaved) 10 different frequencies (2ndash20 Hz 2-Hz steps)and four different phases (sine cosine sine orcosine) at each frequency In both tasks we measuredfor each condition (ie for a given phase andfrequency) the performance of the subjects in detectingthe target by computing d-primes Then we applied acomplex decomposition analysis to estimate the dy-namics of attentional sampling during one trial ofvisual search (cf Methods section and Figure 2) ateach frequency d-primes for the different phaseconditions were combined to obtain a complex vectorin the Fourier domain An inverse Fourier transformallowed us to estimate attentional dynamics in the timedomain (see Methods)

For the easy search the resulting grand-averageattentional function over 14 subjects (Figure 3)presented large fluctuations in d-prime To determinethe relative power and the exact frequency of theseeffects we computed an FFT to obtain the amplitudespectrum of the grand-average attentional function(Figure 4A) The first four frequencies (2 4 6 and 8Hz) were significantly greater than chance (p 103 aconservative statistical threshold due to multiplecomparisons across frequencies) In Figure 4B theamplitude spectrum was computed first on eachindividual sampling function and subsequently aver-aged In this case the spectrum presented an overalldecreasing shape significantly different from chance atfrequencies of 2 4 and 6 Hz The existence of similarlow-frequency components (2 4 6 Hz) in the amplitudespectra from both the individual and grand-averagesampling functions implies that these components notonly present increased amplitude for each subject butalso are strongly phase locked between subjects As wewill see these low-frequency components mostly reflectincreased sensitivity to the onset and offset of thestimulus sequence

The same analysis was performed to reveal theattention sampling function during the difficult searchtask (Figure 5) This function was characterized by amore restricted range of oscillatory frequencies a rapidoscillation appeared to be superimposed on a slowerfluctuation In the amplitude spectra of Figure 6 threespecific frequency components stand out one at 2 Hzone at 10 Hz and one at 18 to 20 Hz These peaks wereapparent in the amplitude spectrum of the grand-average function (Figure 6A) as well as on the averageamplitude spectrum of individual sampling functions(Figure 6B) As previously we can thus conclude thatthese peaks were due to both higher amplitude and

Figure 4 Amplitude spectra of the attention function estimated for the easy search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) at the four lowest frequencies 2 Hz 4 Hz 6 Hz and 8 Hz (B) Average amplitude spectrum of the

individual attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude

reaches significance ( p value 103) at the three lowest frequencies 2 Hz 4 Hz and 6 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 7

stronger phase locking between subjects at thesefrequencies In other words this 10-Hz oscillation at aparticular phase likely corresponds to a temporalsearch strategy that is common to most subjects for thisparticular task

To summarize Experiment 1 revealed that percep-tual sampling during an easy search (L vs thorn) engagedmostly low temporal frequency components whereas adifficult search (color-orientation conjunction) involvedthree distinct frequency peaks 2 Hz 10 Hz and 18 to20 Hz One might wonder whether the slow frequencieswith performance rising and falling on a time scale ofseveral hundred milliseconds could reflect a voluntaryattentional strategy andor the effect of onset and offsettransients To test the latter possibility we recomputedamplitude spectra on the central 333 ms of each 500-ms-long sampling functions (ie after truncating thefirst 83 ms after stimulus onset and the last 83 msbefore stimulus offset Figure 7) For the easy searchno significant frequency remained in the amplitudespectrum of the grand-average sampling function Inthe average amplitude spectrum of individual samplingfunctions the first three frequencies (3ndash9 Hz) weresignificantly greater than chance As explained beforethe absence of corresponding effects in the grand-average sampling function implies that these low-frequency components were not phase locked anymorebetween subjects In other words the previouslyobserved low-frequency effects during the easy searchtask (Figure 4) appeared to be due in large part to theonset and offset portions of the sampling functionsduring which all observers systematically displayedincreased sensitivity For the difficult search both theamplitude spectrum of the average sampling functionand the average amplitude spectrum of individualsampling functions displayed a significant peak at 9 HzTogether the results in Figure 6 and Figure 7 suggestthat the attentional sampling function during thisdifficult search task fluctuated periodically at 9 to 10Hz not just at stimulus onset and offset but also duringthe central portion of stimulus presentation Inaddition we can speculate that the 18 to 20 Hzcomponent observed in Figure 6 and to some extent inFigure 7 could correspond to a harmonic of this 9 to 10Hz periodicity In the next experiment we aimed todemonstrate the functional significance of the periodicsampling observed in the difficult search task

Experiment 2

The attention function measured in the difficultsearch task of Experiment 1 is supposed to representthe fluctuations in attention efficiency over time for anaverage search trial in an average observer If this istrue one should predict that search patterns system-

Figure 5 Attentional sampling function estimated for a difficult

visual search trial Subjects (n frac14 14) were asked to report the

presence or absence of a red grating oriented 308 from upright

among red gratings oriented 3308 from upright and green

gratings oriented 308 from upright (and vice versa) d-primes

were measured for the 40 phase and frequency visual noise

presentation conditions Performances are represented in the

top panel as a modulation of d-prime with respect to the

average computed over all conditions (dashed baseline) On

these performance modulations we applied the complex

decomposition analysis (Figure 2) This analysis was performed

for each individual subject Then the individual estimated

attentional sampling functions were grand-averaged over all

subjects (bottom panel) The standard error of the mean is

represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 8

Figure 6 Amplitude spectra of the attention function estimated for the difficult search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for four frequencies 2 Hz 10 Hz and 18ndash20 Hz (B) Average amplitude spectrum of the individual

attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for six frequencies 2 Hz 8ndash10 Hz 16ndash20 Hz

Figure 7 Control for potential onset-offset (nonperiodic) effects The analysis from Figures 3 to 6 was replicated with estimated

attentional functions truncated for the first 83 ms and the last 83 ms (leaving only the central 333 ms portion of the original 500-ms-

long functions) (A) For the easy search task no frequency reached significance in the amplitude spectrum of the grand-average

attentional function (middle note that the frequency resolution of this analysis is 3 Hz instead of 2 Hz because of the shortened

duration of the sampling function) In the average amplitude spectrum of individual attentional functions (right) significance ( p value

103) was reached for three frequencies 3 Hz 6 Hz and 9 Hz (B) For the difficult search task three frequencies reached

significance in the amplitude spectrum of the grand-average attentional function 3 Hz 6 Hz and 9 Hz In the average amplitude

spectrum of individual attentional functions significance ( p value 103) was reached for five frequencies 3 Hz 6 Hz 9 Hz 15 Hz

and 18 Hz with a peak frequency at 9 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 9

atically presented with an SNR following the estimatedgrand-average attentional function (test condition)should lead to better performance than with theopposite signal-to-noise modulation (control condi-tion) We directly tested this prediction in Experiment2 Notice that we high-pass filtered the attentionsampling function by discarding the 2 Hz componentthought to correspond to an onset-offset transienteffect and therefore not reflecting a true samplingperiodicity Overall the aim of Experiment 2 was totest whether rhythmic attention sampling significantlycontributes to search performance This experimentwas performed on two sets of subjects seven subjectsfrom the lsquolsquoinitial subjectsrsquorsquo group (ie subjects who alsoperformed the difficult search task in Experiment 1)and seven subjects from the lsquolsquonaive subjectsrsquorsquo group (iesubjects who had not been included in Experiment 1)

As previously we first verified that naive observersused an appropriate search strategy by computing theRT middot set size slopes As expected (and as in Experiment1) we obtained strongly positive slopes 889 ms 6 413ms per element t(6)frac14 25 p frac14 004 for target presentand 1495 ms 6 35 ms per element t(6) frac14 47 pfrac1400034 for target absent

In the main comparison for Experiment 2 thesubjectsrsquo performance (as measured by d-primes) wassignificantly higher when stimulus presentation fol-lowed the estimated sampling function than its opposite(Figure 8 mixed design two-way analysis of variancewithin-subject factor lsquolsquotestcontrolrsquorsquo F(1 12)frac14 82 pfrac140014) Moreover no significant difference was foundbetween subject groups (between-subject factor lsquolsquoinitialnaıversquorsquo F(1 12)frac14 09 p frac14 037) nor any significant

interaction between the two factors (lsquolsquotestcontrolrsquorsquo middotlsquolsquoinitialnaiversquorsquo F(1 12) frac14 06 pfrac14 0471)

In summary these results reveal that the grand-average attention sampling function estimated for thedifficult search task (or more precisely its frequencycomponents 2 Hz) is representative of the subjectsrsquosampling strategy in this specific task This is true notonly for the initial subjects indicating that the resultobtained with this new method in Experiment 1 isrobust and replicable but also for the naive subjectsmeaning that the estimated attention sampling functionfor this specific task can also apply to a more generalpopulation of human observers

Discussion

In this study we proposed a novel experimentalmethod based on periodic noise interference andFourier series analysis to characterize the dynamics ofperceptual sampling occurring during a difficult task(color-orientation conjunction) and an easy task (L vsthorn) This method produces for each task a time courserepresenting the sampling efficiency of attention overthe 500 ms of stimulus display Our major result is thepresence of a significant oscillation in this time coursefor the difficult task but not for the easy one A simplecontinuous model of attention sampling in whichperceptual efficiency is constant over time (exceptpossibly for random fluctuations) would have pre-dicted flat attentional sampling functions with notemporal fluctuation and therefore no peak in thecorresponding amplitude spectrum This is not what we

Figure 8 Testing the behavioral relevance of the estimated attentional function A difficult search array was presented with dynamic

noise fluctuations following two conditions lsquolsquotest conditionrsquorsquo with a signal-to-noise ratio (SNR) following the previously estimated

grand-average attentional function (Figure 5) and lsquolsquocontrol conditionrsquorsquo with an SNR following the opposite of the estimated grand-

average attentional function We predicted significantly higher performance in the former than the latter case In both cases the 2 Hz

component of the function was omitted (ie the function was high-pass filtered) so as to focus on genuine periodic effects This

experiment was performed on subjects who had previously participated in the first experiment the so-called lsquolsquoinitial subjectsrsquorsquo (nfrac147)

but also on a new set of subjects the so-called lsquolsquonaive subjectsrsquorsquo (nfrac14 7) A two-way analysis of variance (mixed design) revealed a

significant difference between control and test conditions F(1 12)frac14 82 pfrac14 0014 but no significant difference between naive and

initial subjects F(1 12) frac14 09 p frac14 037 nor any interaction between the two factors F(1 12) frac14 06 p frac14 0471

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 10

observed Both tasks presented significant peaks intheir amplitude spectrum (Figures 4 and 6) Assumingthat perceptual efficiency is optimal at one or possiblytwo specific moments of stimulus presentation as onewould expect for example due to greater stimulus-onsetandor -offset sensitivity should have predicted in-creasing andor decreasing attentional sampling func-tions and therefore a steadily decreasing amplitudespectrum with a maximum at the lowest frequency (2Hz) We did observe such a 2 Hz peak in both tasks andconfirmed that it was mainly due to stimulus-onset and-offset effects (Figure 7) However a major unexpectedresult was the presence of another significant localmaximum in the difficult task with a similar phase forall subjects (9ndash10 Hz Figure 6) which was absent forthe easy task (Figure 4) this peak can be explained onlyby assuming rhythmic perceptual sampling by atten-tion

The presence of a 9 to 10 Hz oscillation in the grand-average temporal function (Figure 6A) could in theorybe explained either by the existence of an equivalentoscillation (with a similar phase) at the level of eachindividual subject or by a chance combination ofsubject-specific peaks and troughs at different momentsthat would spuriously create an oscillation whenaveraged over subjects However our finding thatindividual subjects also present a 9 to 10 Hz peak intheir temporal functions (Figure 6B) clearly rules outthe latter explanation Rather it seems that this specifictemporal function with its 9 to 10 Hz oscillation at aparticular phase captures rather well the attentiondeployment of a lsquolsquotypicalrsquorsquo subject in this search task (asshown also in Experiment 2)

Such rhythmic sampling in a difficult task (color-orientation conjunction) appears naturally compatiblewith the classic sequential models of attention deploy-ment a periodic focusing of attention on different items(or subsets of items) within the search array until thetarget is found (Treisman amp Gelade 1980 Treisman ampSouther 1985 Wolfe 1994 Wolfe et al 1989)However we cannot fully rule out the alternativeparallel model of attention deployment in which allitems are sampled at the same time (Eckstein et al2000 Palmer et al 1993) Indeed it is conceivable thatall of the items in the difficult search were processedtogether within each attentional cycle even as theefficiency of this processing fluctuated periodically at9 to 10 Hz Of course this would represent asubstantial alteration of the original parallel model(Eckstein et al 2000 Palmer et al 1993) yet such arevision appears necessary to reconcile this model withour experimental findings Whether attention sampledthe entire search array or only a subset of elements theimportant conclusion of our study is that it did soperiodically rather than continuously (at least for thespecific difficult search task that we used)

Our novel paradigm appears suitable for revealingthe temporal dynamics of attention in at least somevisual search tasks The method is not infalliblehowever and it is worth reviewing its main limitationsand the possible ways it could be improved in futurestudies

First and foremost our paradigm relies on Fourierseries decomposition which is an intrinsically linearframework (see the Appendix for further description ofthe model) As such we must assume that the searchperformance on each trial is the product of an attentionsampling function and the superimposed noise functionand that this attentional sampling function does notsystematically vary when the noise function changes(for example when we apply sinusoidal noise atdifferent phases and frequencies) This assumption oflinearity is both a reasonable first approximation in ourendeavor to measure the dynamics of attention and anobvious potential limitation of the proposed techniqueIndeed it implies that many nonlinear attentionalstrategies would not be appropriately captured by thetechnique It is worth pointing out however that ourmethod was explicitly designed to prevent one commonform of nonlinearity that would have resulted from asteady-state masking by the sinusoidal noise (ieperception fluctuating in phase with the noise) or froma perceptual entrainment to the stimulus (ie percep-tion fluctuating in counterphase with the noise) Inthese two forms of entrainment one would expect ageneral decrease (or increase respectively) of perfor-mance at specific frequencies regardless of the phaseYet in our experiments at each frequency we did notmerely measure a response to two sinusoids separatedby 908 (cosine and sine as Fourier analysis wouldminimally require) but to all four cardinal sinusoids(ie cosine and its opposite sine and its opposite)Therefore any nonlinear perceptual entrainment orsteady-state masking that would have equally affectedall four noise sinusoids at a given frequency would haveresulted in a null attention modulation at thisfrequency To conclude this type of nonlinear con-found alone cannot explain our finding of a 10 Hzattentional modulation in the difficult search task

Another potential issue with the current version ofour paradigm is the possibility that increased percep-tual sensitivity at stimulus onset andor offset couldhave produced edge effects that would have concealedthe true temporal fluctuations of attention Indeedthere was no noise on the screen before or after our500-ms sampling window meaning that the onset andoffset stimulus information were potentially free offorward and backward masking respectively Thispotential confound led us to perform an additionalanalysis in which we removed the edges of our samplingwindow (Figure 7) Although this removal stillproduced a 10 Hz periodicity in the difficult search

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 11

taskmdashthereby confirming that this periodicity was notmerely an artifact induced by edge effectsmdashit also leftno systematic remaining temporal fluctuation in theeasy search task It is plausible however that suchtemporal fluctuations maybe even periodic oneswould have occurred in the easy search task if the onsetand offset transient information had not been directlyavailable to the observers Therefore in future studieswe propose to improve the paradigm by starting thesinusoidal noise modulation well before (eg 500 ms)and ending it well after the 500-ms critical samplingwindow during which the search array is actuallydisplayed This would essentially avoid edge effects byseparating the onset and offset transients from ourwindow of analysis

Finally the potential influence of eye movementsmust be considered Even though we instructed thesubjects not to move their eyes one might argue thatsome of the temporal fluctuations observed in oursampling functions could be the result of unwantedsaccadic andor micro-saccadic eye movements Thiswould imply that these eye movements are time lockedto stimulus onset and that they would entail differenttemporal patterns depending on the exact search taskSaccades andor microsaccades could not have directlyproduced the temporal pattern observed in the difficultsearch task however because they tend to occur atlower frequencies typically less than 5 Hz (Martinez-Conde Macknik amp Hubel 2004) It appears morelikely therefore that the 10 Hz periodicity obtainedin our difficult search task was a reflection of covertrather than overt attention sampling dynamics

Conclusions

To summarize using an innovative psychophysicalmethod we were able to argue that an easy search taskwas processed in an apparently continuous modewhereas a difficult search task was processed periodi-cally at a frequency of 9 to 10 Hz Of course ourconclusions are limited to the two search tasks tested inthis study and further work would be needed todetermine if they could be generalized to other easy anddifficult search tasks In addition a potential limitationof our study is that we sampled attentional dynamicsfrom only 2 to 20 Hz Values greater than 20 Hz (egin the gamma frequency range) were not consideredalthough they may be relevant for attention (BauerOostenveld Peeters amp Fries 2006 Gruber MullerKeil amp Elbert 1999 Jensen Kaiser amp Lachaux 2007Treue 2001) A number of recent studies havesuggested that attention samples sensory informationperiodically at frequencies between 7 and 13 Hz invarious other experimental situations even when only

one object is attended and thus no sequentialexploration is required (Busch amp Vanrullen 2010Vanrullen et al 2007 Vanrullen amp Dubois 2011) Thepresent demonstration of an intrinsic periodicity duringa visual search task adds significant weight to thisemerging view of periodic attention

Keywords visual search attention temporal dynam-ics periodicity Fourier analysis dynamic noise

Acknowledgments

This research was funded by a EURYI Awardto RV

Commercial relationships noneCorresponding author Laura DugueEmail lauraduguecercoups-tlsefrAddress CNRSndashCerco Toulouse France

References

Bauer M Oostenveld R Peeters M amp Fries P(2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduceslow-frequency activity in parieto-occipital areasJournal of Neuroscience 26 490ndash501

Busch N A amp Vanrullen R (2010) SpontaneousEEG oscillations reveal periodic sampling of visualattention Proceedings of the National Academy ofSciences USA 107 16048ndash16053

Eckstein M P Thomas J P Palmer J ampShimozaki S S (2000) A signal detection modelpredicts the effects of set size on visual searchaccuracy for feature conjunction triple conjunc-tion and disjunction displays Perception amp Psy-chophysics 62 425ndash451

Gobell J L Tseng C H amp Sperling G (2004) Thespatial distribution of visual attention VisionResearch 44 1273ndash1296

Gruber T Muller M M Keil A amp Elbert T(1999) Selective visual-spatial attention altersinduced gamma band responses in the human EEGClinical Neurophysiology 110 2074ndash2085

Jensen O Kaiser J amp Lachaux J P (2007) Humangamma-frequency oscillations associated with at-tention and memory Trends in Neuroscience 30317ndash324

Lu Z L amp Dosher BA (2008) Characterizingobservers using external noise and observer models

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 12

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

null distribution with p 005 if it exceeded the 95confidence threshold (and with p 001 above the 99confidence threshold and so on) To take into accountthe possible problem of multiple statistical comparisonsacross the 10 frequencies used (2ndash20 Hz) we adopted aconservative statistical threshold of p 0001 For both

tasks we also ran the same analysis discarding thesignals from stimulus onset (0ndash83ms) and stimulus offset(417ndash500 ms) This was aimed at verifying that anyperiodicity highlighted by the previous analysis wouldnot be due merely to an increased sensitivity to the onsetandor the offset of the stimuli

Experiment 2 Testing the validity of estimatedattentional functions

Based on the previous estimates of attentionalsampling dynamics for the difficult visual search (seeResults and Figure 6) we tested the functionalsignificance of the sampling function on subjectsrsquoperformances For two groups of subjects seven whohad also performed the difficult search task inExperiment 1 and seven new subjects we presented thesearch arrays (color-orientation conjunction) embed-ded in dynamic noise with signal-to-noise ratios (SNRs)following the grand-average estimated attentionalfunction (ie maximal signal when attention isexpected to be maximal this was called the lsquolsquotestrsquorsquocondition) or the opposite signal (ie maximal signalwhen attention is expected to be minimal this wascalled the lsquolsquocontrolrsquorsquo condition) Whereas in the testcondition subjects should perform the search taskefficiently in the control condition their performanceswould be expected to decrease because stimulusinformation no longer matches their natural samplingdynamics We equalized the histograms of SNRs acrossthe two conditions to ensure they had comparable totalsignal energy and high-pass filtered the attentionalfunction excluding the 2 Hz component to ensure thatour results would not be driven by the 2 Hz componentwhich could have reflected a nonperiodic samplingstrategy (eg increased sensitivity for stimulus onsetandor offset as found in the easy search estimatedfunction) The two presentation conditions (test vscontrol) were randomly interleaved during the exper-iment We looked at the performances of the subjects inthese two conditions by computing d-primes Wecompared test versus control conditions using one-tailed t tests according to the prediction that a searcharray presented with an SNR following the estimatedattentional function should lead to better searchperformance than the opposite signal

Results

Experiment 1

In a preliminary experiment on the same group ofsubjects (nfrac1414 for both tasks) we verified that subjectsused the appropriate search strategy for the two tasks

Figure 3 Attentional sampling function estimated for an easy

visual search trial Subjects (n frac14 14) performed an easy task

consisting in finding athorn among Ls d-primes were measured for

the 40 phase and frequency visual noise presentation

conditions Performances are represented in the top panel as a

modulation of d-prime with respect to the average computed

over all conditions (dashed baseline) On these performance

modulations we applied the complex decomposition analysis

(Figure 2) This analysis was performed for each individual

subject Then the individual estimated attentional sampling

functions were grand-averaged over all subjects (bottom panel)

The standard error of the mean is represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 6

that we intended to use in Experiment 1 finding a thornamong Ls or a conjunction of color (red or green) andorientation (308 or 3308 from upright) We presentedthe stimuli for unlimited durations with a set sizerandomly drawn between four and eight elements andcomputed the RT middot set size slopes As expected theslopes were near zero for the L versusthorn task 77 ms 679 ms per element t(13)frac14 31 p frac14 001 for targetpresent and 116 ms 6 67 ms per element t(13)frac14 55p 001 for target absentmdasht tests performed under thenull hypothesis that the slopes are equal to zero (iethe task involved minimal attention) Slopes werestrongly positive for the conjunction task 70 ms 6 77ms per element t(13) frac14 256 p 00001 for targetpresent and 1846 ms 6 274 ms per element t(13) frac14191 p 00001 for target absent (ie the taskinvolved significant attentional resources)

In the main experiment we estimated the attentionalsampling dynamics for these two visual search tasksTo do so we modulated stimulus information byapplying an oscillatory visual noise (cf Figure 1)according to 40 different conditions (randomly inter-leaved) 10 different frequencies (2ndash20 Hz 2-Hz steps)and four different phases (sine cosine sine orcosine) at each frequency In both tasks we measuredfor each condition (ie for a given phase andfrequency) the performance of the subjects in detectingthe target by computing d-primes Then we applied acomplex decomposition analysis to estimate the dy-namics of attentional sampling during one trial ofvisual search (cf Methods section and Figure 2) ateach frequency d-primes for the different phaseconditions were combined to obtain a complex vectorin the Fourier domain An inverse Fourier transformallowed us to estimate attentional dynamics in the timedomain (see Methods)

For the easy search the resulting grand-averageattentional function over 14 subjects (Figure 3)presented large fluctuations in d-prime To determinethe relative power and the exact frequency of theseeffects we computed an FFT to obtain the amplitudespectrum of the grand-average attentional function(Figure 4A) The first four frequencies (2 4 6 and 8Hz) were significantly greater than chance (p 103 aconservative statistical threshold due to multiplecomparisons across frequencies) In Figure 4B theamplitude spectrum was computed first on eachindividual sampling function and subsequently aver-aged In this case the spectrum presented an overalldecreasing shape significantly different from chance atfrequencies of 2 4 and 6 Hz The existence of similarlow-frequency components (2 4 6 Hz) in the amplitudespectra from both the individual and grand-averagesampling functions implies that these components notonly present increased amplitude for each subject butalso are strongly phase locked between subjects As wewill see these low-frequency components mostly reflectincreased sensitivity to the onset and offset of thestimulus sequence

The same analysis was performed to reveal theattention sampling function during the difficult searchtask (Figure 5) This function was characterized by amore restricted range of oscillatory frequencies a rapidoscillation appeared to be superimposed on a slowerfluctuation In the amplitude spectra of Figure 6 threespecific frequency components stand out one at 2 Hzone at 10 Hz and one at 18 to 20 Hz These peaks wereapparent in the amplitude spectrum of the grand-average function (Figure 6A) as well as on the averageamplitude spectrum of individual sampling functions(Figure 6B) As previously we can thus conclude thatthese peaks were due to both higher amplitude and

Figure 4 Amplitude spectra of the attention function estimated for the easy search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) at the four lowest frequencies 2 Hz 4 Hz 6 Hz and 8 Hz (B) Average amplitude spectrum of the

individual attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude

reaches significance ( p value 103) at the three lowest frequencies 2 Hz 4 Hz and 6 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 7

stronger phase locking between subjects at thesefrequencies In other words this 10-Hz oscillation at aparticular phase likely corresponds to a temporalsearch strategy that is common to most subjects for thisparticular task

To summarize Experiment 1 revealed that percep-tual sampling during an easy search (L vs thorn) engagedmostly low temporal frequency components whereas adifficult search (color-orientation conjunction) involvedthree distinct frequency peaks 2 Hz 10 Hz and 18 to20 Hz One might wonder whether the slow frequencieswith performance rising and falling on a time scale ofseveral hundred milliseconds could reflect a voluntaryattentional strategy andor the effect of onset and offsettransients To test the latter possibility we recomputedamplitude spectra on the central 333 ms of each 500-ms-long sampling functions (ie after truncating thefirst 83 ms after stimulus onset and the last 83 msbefore stimulus offset Figure 7) For the easy searchno significant frequency remained in the amplitudespectrum of the grand-average sampling function Inthe average amplitude spectrum of individual samplingfunctions the first three frequencies (3ndash9 Hz) weresignificantly greater than chance As explained beforethe absence of corresponding effects in the grand-average sampling function implies that these low-frequency components were not phase locked anymorebetween subjects In other words the previouslyobserved low-frequency effects during the easy searchtask (Figure 4) appeared to be due in large part to theonset and offset portions of the sampling functionsduring which all observers systematically displayedincreased sensitivity For the difficult search both theamplitude spectrum of the average sampling functionand the average amplitude spectrum of individualsampling functions displayed a significant peak at 9 HzTogether the results in Figure 6 and Figure 7 suggestthat the attentional sampling function during thisdifficult search task fluctuated periodically at 9 to 10Hz not just at stimulus onset and offset but also duringthe central portion of stimulus presentation Inaddition we can speculate that the 18 to 20 Hzcomponent observed in Figure 6 and to some extent inFigure 7 could correspond to a harmonic of this 9 to 10Hz periodicity In the next experiment we aimed todemonstrate the functional significance of the periodicsampling observed in the difficult search task

Experiment 2

The attention function measured in the difficultsearch task of Experiment 1 is supposed to representthe fluctuations in attention efficiency over time for anaverage search trial in an average observer If this istrue one should predict that search patterns system-

Figure 5 Attentional sampling function estimated for a difficult

visual search trial Subjects (n frac14 14) were asked to report the

presence or absence of a red grating oriented 308 from upright

among red gratings oriented 3308 from upright and green

gratings oriented 308 from upright (and vice versa) d-primes

were measured for the 40 phase and frequency visual noise

presentation conditions Performances are represented in the

top panel as a modulation of d-prime with respect to the

average computed over all conditions (dashed baseline) On

these performance modulations we applied the complex

decomposition analysis (Figure 2) This analysis was performed

for each individual subject Then the individual estimated

attentional sampling functions were grand-averaged over all

subjects (bottom panel) The standard error of the mean is

represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 8

Figure 6 Amplitude spectra of the attention function estimated for the difficult search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for four frequencies 2 Hz 10 Hz and 18ndash20 Hz (B) Average amplitude spectrum of the individual

attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for six frequencies 2 Hz 8ndash10 Hz 16ndash20 Hz

Figure 7 Control for potential onset-offset (nonperiodic) effects The analysis from Figures 3 to 6 was replicated with estimated

attentional functions truncated for the first 83 ms and the last 83 ms (leaving only the central 333 ms portion of the original 500-ms-

long functions) (A) For the easy search task no frequency reached significance in the amplitude spectrum of the grand-average

attentional function (middle note that the frequency resolution of this analysis is 3 Hz instead of 2 Hz because of the shortened

duration of the sampling function) In the average amplitude spectrum of individual attentional functions (right) significance ( p value

103) was reached for three frequencies 3 Hz 6 Hz and 9 Hz (B) For the difficult search task three frequencies reached

significance in the amplitude spectrum of the grand-average attentional function 3 Hz 6 Hz and 9 Hz In the average amplitude

spectrum of individual attentional functions significance ( p value 103) was reached for five frequencies 3 Hz 6 Hz 9 Hz 15 Hz

and 18 Hz with a peak frequency at 9 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 9

atically presented with an SNR following the estimatedgrand-average attentional function (test condition)should lead to better performance than with theopposite signal-to-noise modulation (control condi-tion) We directly tested this prediction in Experiment2 Notice that we high-pass filtered the attentionsampling function by discarding the 2 Hz componentthought to correspond to an onset-offset transienteffect and therefore not reflecting a true samplingperiodicity Overall the aim of Experiment 2 was totest whether rhythmic attention sampling significantlycontributes to search performance This experimentwas performed on two sets of subjects seven subjectsfrom the lsquolsquoinitial subjectsrsquorsquo group (ie subjects who alsoperformed the difficult search task in Experiment 1)and seven subjects from the lsquolsquonaive subjectsrsquorsquo group (iesubjects who had not been included in Experiment 1)

As previously we first verified that naive observersused an appropriate search strategy by computing theRT middot set size slopes As expected (and as in Experiment1) we obtained strongly positive slopes 889 ms 6 413ms per element t(6)frac14 25 p frac14 004 for target presentand 1495 ms 6 35 ms per element t(6) frac14 47 pfrac1400034 for target absent

In the main comparison for Experiment 2 thesubjectsrsquo performance (as measured by d-primes) wassignificantly higher when stimulus presentation fol-lowed the estimated sampling function than its opposite(Figure 8 mixed design two-way analysis of variancewithin-subject factor lsquolsquotestcontrolrsquorsquo F(1 12)frac14 82 pfrac140014) Moreover no significant difference was foundbetween subject groups (between-subject factor lsquolsquoinitialnaıversquorsquo F(1 12)frac14 09 p frac14 037) nor any significant

interaction between the two factors (lsquolsquotestcontrolrsquorsquo middotlsquolsquoinitialnaiversquorsquo F(1 12) frac14 06 pfrac14 0471)

In summary these results reveal that the grand-average attention sampling function estimated for thedifficult search task (or more precisely its frequencycomponents 2 Hz) is representative of the subjectsrsquosampling strategy in this specific task This is true notonly for the initial subjects indicating that the resultobtained with this new method in Experiment 1 isrobust and replicable but also for the naive subjectsmeaning that the estimated attention sampling functionfor this specific task can also apply to a more generalpopulation of human observers

Discussion

In this study we proposed a novel experimentalmethod based on periodic noise interference andFourier series analysis to characterize the dynamics ofperceptual sampling occurring during a difficult task(color-orientation conjunction) and an easy task (L vsthorn) This method produces for each task a time courserepresenting the sampling efficiency of attention overthe 500 ms of stimulus display Our major result is thepresence of a significant oscillation in this time coursefor the difficult task but not for the easy one A simplecontinuous model of attention sampling in whichperceptual efficiency is constant over time (exceptpossibly for random fluctuations) would have pre-dicted flat attentional sampling functions with notemporal fluctuation and therefore no peak in thecorresponding amplitude spectrum This is not what we

Figure 8 Testing the behavioral relevance of the estimated attentional function A difficult search array was presented with dynamic

noise fluctuations following two conditions lsquolsquotest conditionrsquorsquo with a signal-to-noise ratio (SNR) following the previously estimated

grand-average attentional function (Figure 5) and lsquolsquocontrol conditionrsquorsquo with an SNR following the opposite of the estimated grand-

average attentional function We predicted significantly higher performance in the former than the latter case In both cases the 2 Hz

component of the function was omitted (ie the function was high-pass filtered) so as to focus on genuine periodic effects This

experiment was performed on subjects who had previously participated in the first experiment the so-called lsquolsquoinitial subjectsrsquorsquo (nfrac147)

but also on a new set of subjects the so-called lsquolsquonaive subjectsrsquorsquo (nfrac14 7) A two-way analysis of variance (mixed design) revealed a

significant difference between control and test conditions F(1 12)frac14 82 pfrac14 0014 but no significant difference between naive and

initial subjects F(1 12) frac14 09 p frac14 037 nor any interaction between the two factors F(1 12) frac14 06 p frac14 0471

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 10

observed Both tasks presented significant peaks intheir amplitude spectrum (Figures 4 and 6) Assumingthat perceptual efficiency is optimal at one or possiblytwo specific moments of stimulus presentation as onewould expect for example due to greater stimulus-onsetandor -offset sensitivity should have predicted in-creasing andor decreasing attentional sampling func-tions and therefore a steadily decreasing amplitudespectrum with a maximum at the lowest frequency (2Hz) We did observe such a 2 Hz peak in both tasks andconfirmed that it was mainly due to stimulus-onset and-offset effects (Figure 7) However a major unexpectedresult was the presence of another significant localmaximum in the difficult task with a similar phase forall subjects (9ndash10 Hz Figure 6) which was absent forthe easy task (Figure 4) this peak can be explained onlyby assuming rhythmic perceptual sampling by atten-tion

The presence of a 9 to 10 Hz oscillation in the grand-average temporal function (Figure 6A) could in theorybe explained either by the existence of an equivalentoscillation (with a similar phase) at the level of eachindividual subject or by a chance combination ofsubject-specific peaks and troughs at different momentsthat would spuriously create an oscillation whenaveraged over subjects However our finding thatindividual subjects also present a 9 to 10 Hz peak intheir temporal functions (Figure 6B) clearly rules outthe latter explanation Rather it seems that this specifictemporal function with its 9 to 10 Hz oscillation at aparticular phase captures rather well the attentiondeployment of a lsquolsquotypicalrsquorsquo subject in this search task (asshown also in Experiment 2)

Such rhythmic sampling in a difficult task (color-orientation conjunction) appears naturally compatiblewith the classic sequential models of attention deploy-ment a periodic focusing of attention on different items(or subsets of items) within the search array until thetarget is found (Treisman amp Gelade 1980 Treisman ampSouther 1985 Wolfe 1994 Wolfe et al 1989)However we cannot fully rule out the alternativeparallel model of attention deployment in which allitems are sampled at the same time (Eckstein et al2000 Palmer et al 1993) Indeed it is conceivable thatall of the items in the difficult search were processedtogether within each attentional cycle even as theefficiency of this processing fluctuated periodically at9 to 10 Hz Of course this would represent asubstantial alteration of the original parallel model(Eckstein et al 2000 Palmer et al 1993) yet such arevision appears necessary to reconcile this model withour experimental findings Whether attention sampledthe entire search array or only a subset of elements theimportant conclusion of our study is that it did soperiodically rather than continuously (at least for thespecific difficult search task that we used)

Our novel paradigm appears suitable for revealingthe temporal dynamics of attention in at least somevisual search tasks The method is not infalliblehowever and it is worth reviewing its main limitationsand the possible ways it could be improved in futurestudies

First and foremost our paradigm relies on Fourierseries decomposition which is an intrinsically linearframework (see the Appendix for further description ofthe model) As such we must assume that the searchperformance on each trial is the product of an attentionsampling function and the superimposed noise functionand that this attentional sampling function does notsystematically vary when the noise function changes(for example when we apply sinusoidal noise atdifferent phases and frequencies) This assumption oflinearity is both a reasonable first approximation in ourendeavor to measure the dynamics of attention and anobvious potential limitation of the proposed techniqueIndeed it implies that many nonlinear attentionalstrategies would not be appropriately captured by thetechnique It is worth pointing out however that ourmethod was explicitly designed to prevent one commonform of nonlinearity that would have resulted from asteady-state masking by the sinusoidal noise (ieperception fluctuating in phase with the noise) or froma perceptual entrainment to the stimulus (ie percep-tion fluctuating in counterphase with the noise) Inthese two forms of entrainment one would expect ageneral decrease (or increase respectively) of perfor-mance at specific frequencies regardless of the phaseYet in our experiments at each frequency we did notmerely measure a response to two sinusoids separatedby 908 (cosine and sine as Fourier analysis wouldminimally require) but to all four cardinal sinusoids(ie cosine and its opposite sine and its opposite)Therefore any nonlinear perceptual entrainment orsteady-state masking that would have equally affectedall four noise sinusoids at a given frequency would haveresulted in a null attention modulation at thisfrequency To conclude this type of nonlinear con-found alone cannot explain our finding of a 10 Hzattentional modulation in the difficult search task

Another potential issue with the current version ofour paradigm is the possibility that increased percep-tual sensitivity at stimulus onset andor offset couldhave produced edge effects that would have concealedthe true temporal fluctuations of attention Indeedthere was no noise on the screen before or after our500-ms sampling window meaning that the onset andoffset stimulus information were potentially free offorward and backward masking respectively Thispotential confound led us to perform an additionalanalysis in which we removed the edges of our samplingwindow (Figure 7) Although this removal stillproduced a 10 Hz periodicity in the difficult search

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 11

taskmdashthereby confirming that this periodicity was notmerely an artifact induced by edge effectsmdashit also leftno systematic remaining temporal fluctuation in theeasy search task It is plausible however that suchtemporal fluctuations maybe even periodic oneswould have occurred in the easy search task if the onsetand offset transient information had not been directlyavailable to the observers Therefore in future studieswe propose to improve the paradigm by starting thesinusoidal noise modulation well before (eg 500 ms)and ending it well after the 500-ms critical samplingwindow during which the search array is actuallydisplayed This would essentially avoid edge effects byseparating the onset and offset transients from ourwindow of analysis

Finally the potential influence of eye movementsmust be considered Even though we instructed thesubjects not to move their eyes one might argue thatsome of the temporal fluctuations observed in oursampling functions could be the result of unwantedsaccadic andor micro-saccadic eye movements Thiswould imply that these eye movements are time lockedto stimulus onset and that they would entail differenttemporal patterns depending on the exact search taskSaccades andor microsaccades could not have directlyproduced the temporal pattern observed in the difficultsearch task however because they tend to occur atlower frequencies typically less than 5 Hz (Martinez-Conde Macknik amp Hubel 2004) It appears morelikely therefore that the 10 Hz periodicity obtainedin our difficult search task was a reflection of covertrather than overt attention sampling dynamics

Conclusions

To summarize using an innovative psychophysicalmethod we were able to argue that an easy search taskwas processed in an apparently continuous modewhereas a difficult search task was processed periodi-cally at a frequency of 9 to 10 Hz Of course ourconclusions are limited to the two search tasks tested inthis study and further work would be needed todetermine if they could be generalized to other easy anddifficult search tasks In addition a potential limitationof our study is that we sampled attentional dynamicsfrom only 2 to 20 Hz Values greater than 20 Hz (egin the gamma frequency range) were not consideredalthough they may be relevant for attention (BauerOostenveld Peeters amp Fries 2006 Gruber MullerKeil amp Elbert 1999 Jensen Kaiser amp Lachaux 2007Treue 2001) A number of recent studies havesuggested that attention samples sensory informationperiodically at frequencies between 7 and 13 Hz invarious other experimental situations even when only

one object is attended and thus no sequentialexploration is required (Busch amp Vanrullen 2010Vanrullen et al 2007 Vanrullen amp Dubois 2011) Thepresent demonstration of an intrinsic periodicity duringa visual search task adds significant weight to thisemerging view of periodic attention

Keywords visual search attention temporal dynam-ics periodicity Fourier analysis dynamic noise

Acknowledgments

This research was funded by a EURYI Awardto RV

Commercial relationships noneCorresponding author Laura DugueEmail lauraduguecercoups-tlsefrAddress CNRSndashCerco Toulouse France

References

Bauer M Oostenveld R Peeters M amp Fries P(2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduceslow-frequency activity in parieto-occipital areasJournal of Neuroscience 26 490ndash501

Busch N A amp Vanrullen R (2010) SpontaneousEEG oscillations reveal periodic sampling of visualattention Proceedings of the National Academy ofSciences USA 107 16048ndash16053

Eckstein M P Thomas J P Palmer J ampShimozaki S S (2000) A signal detection modelpredicts the effects of set size on visual searchaccuracy for feature conjunction triple conjunc-tion and disjunction displays Perception amp Psy-chophysics 62 425ndash451

Gobell J L Tseng C H amp Sperling G (2004) Thespatial distribution of visual attention VisionResearch 44 1273ndash1296

Gruber T Muller M M Keil A amp Elbert T(1999) Selective visual-spatial attention altersinduced gamma band responses in the human EEGClinical Neurophysiology 110 2074ndash2085

Jensen O Kaiser J amp Lachaux J P (2007) Humangamma-frequency oscillations associated with at-tention and memory Trends in Neuroscience 30317ndash324

Lu Z L amp Dosher BA (2008) Characterizingobservers using external noise and observer models

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 12

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

that we intended to use in Experiment 1 finding a thornamong Ls or a conjunction of color (red or green) andorientation (308 or 3308 from upright) We presentedthe stimuli for unlimited durations with a set sizerandomly drawn between four and eight elements andcomputed the RT middot set size slopes As expected theslopes were near zero for the L versusthorn task 77 ms 679 ms per element t(13)frac14 31 p frac14 001 for targetpresent and 116 ms 6 67 ms per element t(13)frac14 55p 001 for target absentmdasht tests performed under thenull hypothesis that the slopes are equal to zero (iethe task involved minimal attention) Slopes werestrongly positive for the conjunction task 70 ms 6 77ms per element t(13) frac14 256 p 00001 for targetpresent and 1846 ms 6 274 ms per element t(13) frac14191 p 00001 for target absent (ie the taskinvolved significant attentional resources)

In the main experiment we estimated the attentionalsampling dynamics for these two visual search tasksTo do so we modulated stimulus information byapplying an oscillatory visual noise (cf Figure 1)according to 40 different conditions (randomly inter-leaved) 10 different frequencies (2ndash20 Hz 2-Hz steps)and four different phases (sine cosine sine orcosine) at each frequency In both tasks we measuredfor each condition (ie for a given phase andfrequency) the performance of the subjects in detectingthe target by computing d-primes Then we applied acomplex decomposition analysis to estimate the dy-namics of attentional sampling during one trial ofvisual search (cf Methods section and Figure 2) ateach frequency d-primes for the different phaseconditions were combined to obtain a complex vectorin the Fourier domain An inverse Fourier transformallowed us to estimate attentional dynamics in the timedomain (see Methods)

For the easy search the resulting grand-averageattentional function over 14 subjects (Figure 3)presented large fluctuations in d-prime To determinethe relative power and the exact frequency of theseeffects we computed an FFT to obtain the amplitudespectrum of the grand-average attentional function(Figure 4A) The first four frequencies (2 4 6 and 8Hz) were significantly greater than chance (p 103 aconservative statistical threshold due to multiplecomparisons across frequencies) In Figure 4B theamplitude spectrum was computed first on eachindividual sampling function and subsequently aver-aged In this case the spectrum presented an overalldecreasing shape significantly different from chance atfrequencies of 2 4 and 6 Hz The existence of similarlow-frequency components (2 4 6 Hz) in the amplitudespectra from both the individual and grand-averagesampling functions implies that these components notonly present increased amplitude for each subject butalso are strongly phase locked between subjects As wewill see these low-frequency components mostly reflectincreased sensitivity to the onset and offset of thestimulus sequence

The same analysis was performed to reveal theattention sampling function during the difficult searchtask (Figure 5) This function was characterized by amore restricted range of oscillatory frequencies a rapidoscillation appeared to be superimposed on a slowerfluctuation In the amplitude spectra of Figure 6 threespecific frequency components stand out one at 2 Hzone at 10 Hz and one at 18 to 20 Hz These peaks wereapparent in the amplitude spectrum of the grand-average function (Figure 6A) as well as on the averageamplitude spectrum of individual sampling functions(Figure 6B) As previously we can thus conclude thatthese peaks were due to both higher amplitude and

Figure 4 Amplitude spectra of the attention function estimated for the easy search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) at the four lowest frequencies 2 Hz 4 Hz 6 Hz and 8 Hz (B) Average amplitude spectrum of the

individual attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude

reaches significance ( p value 103) at the three lowest frequencies 2 Hz 4 Hz and 6 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 7

stronger phase locking between subjects at thesefrequencies In other words this 10-Hz oscillation at aparticular phase likely corresponds to a temporalsearch strategy that is common to most subjects for thisparticular task

To summarize Experiment 1 revealed that percep-tual sampling during an easy search (L vs thorn) engagedmostly low temporal frequency components whereas adifficult search (color-orientation conjunction) involvedthree distinct frequency peaks 2 Hz 10 Hz and 18 to20 Hz One might wonder whether the slow frequencieswith performance rising and falling on a time scale ofseveral hundred milliseconds could reflect a voluntaryattentional strategy andor the effect of onset and offsettransients To test the latter possibility we recomputedamplitude spectra on the central 333 ms of each 500-ms-long sampling functions (ie after truncating thefirst 83 ms after stimulus onset and the last 83 msbefore stimulus offset Figure 7) For the easy searchno significant frequency remained in the amplitudespectrum of the grand-average sampling function Inthe average amplitude spectrum of individual samplingfunctions the first three frequencies (3ndash9 Hz) weresignificantly greater than chance As explained beforethe absence of corresponding effects in the grand-average sampling function implies that these low-frequency components were not phase locked anymorebetween subjects In other words the previouslyobserved low-frequency effects during the easy searchtask (Figure 4) appeared to be due in large part to theonset and offset portions of the sampling functionsduring which all observers systematically displayedincreased sensitivity For the difficult search both theamplitude spectrum of the average sampling functionand the average amplitude spectrum of individualsampling functions displayed a significant peak at 9 HzTogether the results in Figure 6 and Figure 7 suggestthat the attentional sampling function during thisdifficult search task fluctuated periodically at 9 to 10Hz not just at stimulus onset and offset but also duringthe central portion of stimulus presentation Inaddition we can speculate that the 18 to 20 Hzcomponent observed in Figure 6 and to some extent inFigure 7 could correspond to a harmonic of this 9 to 10Hz periodicity In the next experiment we aimed todemonstrate the functional significance of the periodicsampling observed in the difficult search task

Experiment 2

The attention function measured in the difficultsearch task of Experiment 1 is supposed to representthe fluctuations in attention efficiency over time for anaverage search trial in an average observer If this istrue one should predict that search patterns system-

Figure 5 Attentional sampling function estimated for a difficult

visual search trial Subjects (n frac14 14) were asked to report the

presence or absence of a red grating oriented 308 from upright

among red gratings oriented 3308 from upright and green

gratings oriented 308 from upright (and vice versa) d-primes

were measured for the 40 phase and frequency visual noise

presentation conditions Performances are represented in the

top panel as a modulation of d-prime with respect to the

average computed over all conditions (dashed baseline) On

these performance modulations we applied the complex

decomposition analysis (Figure 2) This analysis was performed

for each individual subject Then the individual estimated

attentional sampling functions were grand-averaged over all

subjects (bottom panel) The standard error of the mean is

represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 8

Figure 6 Amplitude spectra of the attention function estimated for the difficult search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for four frequencies 2 Hz 10 Hz and 18ndash20 Hz (B) Average amplitude spectrum of the individual

attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for six frequencies 2 Hz 8ndash10 Hz 16ndash20 Hz

Figure 7 Control for potential onset-offset (nonperiodic) effects The analysis from Figures 3 to 6 was replicated with estimated

attentional functions truncated for the first 83 ms and the last 83 ms (leaving only the central 333 ms portion of the original 500-ms-

long functions) (A) For the easy search task no frequency reached significance in the amplitude spectrum of the grand-average

attentional function (middle note that the frequency resolution of this analysis is 3 Hz instead of 2 Hz because of the shortened

duration of the sampling function) In the average amplitude spectrum of individual attentional functions (right) significance ( p value

103) was reached for three frequencies 3 Hz 6 Hz and 9 Hz (B) For the difficult search task three frequencies reached

significance in the amplitude spectrum of the grand-average attentional function 3 Hz 6 Hz and 9 Hz In the average amplitude

spectrum of individual attentional functions significance ( p value 103) was reached for five frequencies 3 Hz 6 Hz 9 Hz 15 Hz

and 18 Hz with a peak frequency at 9 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 9

atically presented with an SNR following the estimatedgrand-average attentional function (test condition)should lead to better performance than with theopposite signal-to-noise modulation (control condi-tion) We directly tested this prediction in Experiment2 Notice that we high-pass filtered the attentionsampling function by discarding the 2 Hz componentthought to correspond to an onset-offset transienteffect and therefore not reflecting a true samplingperiodicity Overall the aim of Experiment 2 was totest whether rhythmic attention sampling significantlycontributes to search performance This experimentwas performed on two sets of subjects seven subjectsfrom the lsquolsquoinitial subjectsrsquorsquo group (ie subjects who alsoperformed the difficult search task in Experiment 1)and seven subjects from the lsquolsquonaive subjectsrsquorsquo group (iesubjects who had not been included in Experiment 1)

As previously we first verified that naive observersused an appropriate search strategy by computing theRT middot set size slopes As expected (and as in Experiment1) we obtained strongly positive slopes 889 ms 6 413ms per element t(6)frac14 25 p frac14 004 for target presentand 1495 ms 6 35 ms per element t(6) frac14 47 pfrac1400034 for target absent

In the main comparison for Experiment 2 thesubjectsrsquo performance (as measured by d-primes) wassignificantly higher when stimulus presentation fol-lowed the estimated sampling function than its opposite(Figure 8 mixed design two-way analysis of variancewithin-subject factor lsquolsquotestcontrolrsquorsquo F(1 12)frac14 82 pfrac140014) Moreover no significant difference was foundbetween subject groups (between-subject factor lsquolsquoinitialnaıversquorsquo F(1 12)frac14 09 p frac14 037) nor any significant

interaction between the two factors (lsquolsquotestcontrolrsquorsquo middotlsquolsquoinitialnaiversquorsquo F(1 12) frac14 06 pfrac14 0471)

In summary these results reveal that the grand-average attention sampling function estimated for thedifficult search task (or more precisely its frequencycomponents 2 Hz) is representative of the subjectsrsquosampling strategy in this specific task This is true notonly for the initial subjects indicating that the resultobtained with this new method in Experiment 1 isrobust and replicable but also for the naive subjectsmeaning that the estimated attention sampling functionfor this specific task can also apply to a more generalpopulation of human observers

Discussion

In this study we proposed a novel experimentalmethod based on periodic noise interference andFourier series analysis to characterize the dynamics ofperceptual sampling occurring during a difficult task(color-orientation conjunction) and an easy task (L vsthorn) This method produces for each task a time courserepresenting the sampling efficiency of attention overthe 500 ms of stimulus display Our major result is thepresence of a significant oscillation in this time coursefor the difficult task but not for the easy one A simplecontinuous model of attention sampling in whichperceptual efficiency is constant over time (exceptpossibly for random fluctuations) would have pre-dicted flat attentional sampling functions with notemporal fluctuation and therefore no peak in thecorresponding amplitude spectrum This is not what we

Figure 8 Testing the behavioral relevance of the estimated attentional function A difficult search array was presented with dynamic

noise fluctuations following two conditions lsquolsquotest conditionrsquorsquo with a signal-to-noise ratio (SNR) following the previously estimated

grand-average attentional function (Figure 5) and lsquolsquocontrol conditionrsquorsquo with an SNR following the opposite of the estimated grand-

average attentional function We predicted significantly higher performance in the former than the latter case In both cases the 2 Hz

component of the function was omitted (ie the function was high-pass filtered) so as to focus on genuine periodic effects This

experiment was performed on subjects who had previously participated in the first experiment the so-called lsquolsquoinitial subjectsrsquorsquo (nfrac147)

but also on a new set of subjects the so-called lsquolsquonaive subjectsrsquorsquo (nfrac14 7) A two-way analysis of variance (mixed design) revealed a

significant difference between control and test conditions F(1 12)frac14 82 pfrac14 0014 but no significant difference between naive and

initial subjects F(1 12) frac14 09 p frac14 037 nor any interaction between the two factors F(1 12) frac14 06 p frac14 0471

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 10

observed Both tasks presented significant peaks intheir amplitude spectrum (Figures 4 and 6) Assumingthat perceptual efficiency is optimal at one or possiblytwo specific moments of stimulus presentation as onewould expect for example due to greater stimulus-onsetandor -offset sensitivity should have predicted in-creasing andor decreasing attentional sampling func-tions and therefore a steadily decreasing amplitudespectrum with a maximum at the lowest frequency (2Hz) We did observe such a 2 Hz peak in both tasks andconfirmed that it was mainly due to stimulus-onset and-offset effects (Figure 7) However a major unexpectedresult was the presence of another significant localmaximum in the difficult task with a similar phase forall subjects (9ndash10 Hz Figure 6) which was absent forthe easy task (Figure 4) this peak can be explained onlyby assuming rhythmic perceptual sampling by atten-tion

The presence of a 9 to 10 Hz oscillation in the grand-average temporal function (Figure 6A) could in theorybe explained either by the existence of an equivalentoscillation (with a similar phase) at the level of eachindividual subject or by a chance combination ofsubject-specific peaks and troughs at different momentsthat would spuriously create an oscillation whenaveraged over subjects However our finding thatindividual subjects also present a 9 to 10 Hz peak intheir temporal functions (Figure 6B) clearly rules outthe latter explanation Rather it seems that this specifictemporal function with its 9 to 10 Hz oscillation at aparticular phase captures rather well the attentiondeployment of a lsquolsquotypicalrsquorsquo subject in this search task (asshown also in Experiment 2)

Such rhythmic sampling in a difficult task (color-orientation conjunction) appears naturally compatiblewith the classic sequential models of attention deploy-ment a periodic focusing of attention on different items(or subsets of items) within the search array until thetarget is found (Treisman amp Gelade 1980 Treisman ampSouther 1985 Wolfe 1994 Wolfe et al 1989)However we cannot fully rule out the alternativeparallel model of attention deployment in which allitems are sampled at the same time (Eckstein et al2000 Palmer et al 1993) Indeed it is conceivable thatall of the items in the difficult search were processedtogether within each attentional cycle even as theefficiency of this processing fluctuated periodically at9 to 10 Hz Of course this would represent asubstantial alteration of the original parallel model(Eckstein et al 2000 Palmer et al 1993) yet such arevision appears necessary to reconcile this model withour experimental findings Whether attention sampledthe entire search array or only a subset of elements theimportant conclusion of our study is that it did soperiodically rather than continuously (at least for thespecific difficult search task that we used)

Our novel paradigm appears suitable for revealingthe temporal dynamics of attention in at least somevisual search tasks The method is not infalliblehowever and it is worth reviewing its main limitationsand the possible ways it could be improved in futurestudies

First and foremost our paradigm relies on Fourierseries decomposition which is an intrinsically linearframework (see the Appendix for further description ofthe model) As such we must assume that the searchperformance on each trial is the product of an attentionsampling function and the superimposed noise functionand that this attentional sampling function does notsystematically vary when the noise function changes(for example when we apply sinusoidal noise atdifferent phases and frequencies) This assumption oflinearity is both a reasonable first approximation in ourendeavor to measure the dynamics of attention and anobvious potential limitation of the proposed techniqueIndeed it implies that many nonlinear attentionalstrategies would not be appropriately captured by thetechnique It is worth pointing out however that ourmethod was explicitly designed to prevent one commonform of nonlinearity that would have resulted from asteady-state masking by the sinusoidal noise (ieperception fluctuating in phase with the noise) or froma perceptual entrainment to the stimulus (ie percep-tion fluctuating in counterphase with the noise) Inthese two forms of entrainment one would expect ageneral decrease (or increase respectively) of perfor-mance at specific frequencies regardless of the phaseYet in our experiments at each frequency we did notmerely measure a response to two sinusoids separatedby 908 (cosine and sine as Fourier analysis wouldminimally require) but to all four cardinal sinusoids(ie cosine and its opposite sine and its opposite)Therefore any nonlinear perceptual entrainment orsteady-state masking that would have equally affectedall four noise sinusoids at a given frequency would haveresulted in a null attention modulation at thisfrequency To conclude this type of nonlinear con-found alone cannot explain our finding of a 10 Hzattentional modulation in the difficult search task

Another potential issue with the current version ofour paradigm is the possibility that increased percep-tual sensitivity at stimulus onset andor offset couldhave produced edge effects that would have concealedthe true temporal fluctuations of attention Indeedthere was no noise on the screen before or after our500-ms sampling window meaning that the onset andoffset stimulus information were potentially free offorward and backward masking respectively Thispotential confound led us to perform an additionalanalysis in which we removed the edges of our samplingwindow (Figure 7) Although this removal stillproduced a 10 Hz periodicity in the difficult search

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 11

taskmdashthereby confirming that this periodicity was notmerely an artifact induced by edge effectsmdashit also leftno systematic remaining temporal fluctuation in theeasy search task It is plausible however that suchtemporal fluctuations maybe even periodic oneswould have occurred in the easy search task if the onsetand offset transient information had not been directlyavailable to the observers Therefore in future studieswe propose to improve the paradigm by starting thesinusoidal noise modulation well before (eg 500 ms)and ending it well after the 500-ms critical samplingwindow during which the search array is actuallydisplayed This would essentially avoid edge effects byseparating the onset and offset transients from ourwindow of analysis

Finally the potential influence of eye movementsmust be considered Even though we instructed thesubjects not to move their eyes one might argue thatsome of the temporal fluctuations observed in oursampling functions could be the result of unwantedsaccadic andor micro-saccadic eye movements Thiswould imply that these eye movements are time lockedto stimulus onset and that they would entail differenttemporal patterns depending on the exact search taskSaccades andor microsaccades could not have directlyproduced the temporal pattern observed in the difficultsearch task however because they tend to occur atlower frequencies typically less than 5 Hz (Martinez-Conde Macknik amp Hubel 2004) It appears morelikely therefore that the 10 Hz periodicity obtainedin our difficult search task was a reflection of covertrather than overt attention sampling dynamics

Conclusions

To summarize using an innovative psychophysicalmethod we were able to argue that an easy search taskwas processed in an apparently continuous modewhereas a difficult search task was processed periodi-cally at a frequency of 9 to 10 Hz Of course ourconclusions are limited to the two search tasks tested inthis study and further work would be needed todetermine if they could be generalized to other easy anddifficult search tasks In addition a potential limitationof our study is that we sampled attentional dynamicsfrom only 2 to 20 Hz Values greater than 20 Hz (egin the gamma frequency range) were not consideredalthough they may be relevant for attention (BauerOostenveld Peeters amp Fries 2006 Gruber MullerKeil amp Elbert 1999 Jensen Kaiser amp Lachaux 2007Treue 2001) A number of recent studies havesuggested that attention samples sensory informationperiodically at frequencies between 7 and 13 Hz invarious other experimental situations even when only

one object is attended and thus no sequentialexploration is required (Busch amp Vanrullen 2010Vanrullen et al 2007 Vanrullen amp Dubois 2011) Thepresent demonstration of an intrinsic periodicity duringa visual search task adds significant weight to thisemerging view of periodic attention

Keywords visual search attention temporal dynam-ics periodicity Fourier analysis dynamic noise

Acknowledgments

This research was funded by a EURYI Awardto RV

Commercial relationships noneCorresponding author Laura DugueEmail lauraduguecercoups-tlsefrAddress CNRSndashCerco Toulouse France

References

Bauer M Oostenveld R Peeters M amp Fries P(2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduceslow-frequency activity in parieto-occipital areasJournal of Neuroscience 26 490ndash501

Busch N A amp Vanrullen R (2010) SpontaneousEEG oscillations reveal periodic sampling of visualattention Proceedings of the National Academy ofSciences USA 107 16048ndash16053

Eckstein M P Thomas J P Palmer J ampShimozaki S S (2000) A signal detection modelpredicts the effects of set size on visual searchaccuracy for feature conjunction triple conjunc-tion and disjunction displays Perception amp Psy-chophysics 62 425ndash451

Gobell J L Tseng C H amp Sperling G (2004) Thespatial distribution of visual attention VisionResearch 44 1273ndash1296

Gruber T Muller M M Keil A amp Elbert T(1999) Selective visual-spatial attention altersinduced gamma band responses in the human EEGClinical Neurophysiology 110 2074ndash2085

Jensen O Kaiser J amp Lachaux J P (2007) Humangamma-frequency oscillations associated with at-tention and memory Trends in Neuroscience 30317ndash324

Lu Z L amp Dosher BA (2008) Characterizingobservers using external noise and observer models

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 12

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

stronger phase locking between subjects at thesefrequencies In other words this 10-Hz oscillation at aparticular phase likely corresponds to a temporalsearch strategy that is common to most subjects for thisparticular task

To summarize Experiment 1 revealed that percep-tual sampling during an easy search (L vs thorn) engagedmostly low temporal frequency components whereas adifficult search (color-orientation conjunction) involvedthree distinct frequency peaks 2 Hz 10 Hz and 18 to20 Hz One might wonder whether the slow frequencieswith performance rising and falling on a time scale ofseveral hundred milliseconds could reflect a voluntaryattentional strategy andor the effect of onset and offsettransients To test the latter possibility we recomputedamplitude spectra on the central 333 ms of each 500-ms-long sampling functions (ie after truncating thefirst 83 ms after stimulus onset and the last 83 msbefore stimulus offset Figure 7) For the easy searchno significant frequency remained in the amplitudespectrum of the grand-average sampling function Inthe average amplitude spectrum of individual samplingfunctions the first three frequencies (3ndash9 Hz) weresignificantly greater than chance As explained beforethe absence of corresponding effects in the grand-average sampling function implies that these low-frequency components were not phase locked anymorebetween subjects In other words the previouslyobserved low-frequency effects during the easy searchtask (Figure 4) appeared to be due in large part to theonset and offset portions of the sampling functionsduring which all observers systematically displayedincreased sensitivity For the difficult search both theamplitude spectrum of the average sampling functionand the average amplitude spectrum of individualsampling functions displayed a significant peak at 9 HzTogether the results in Figure 6 and Figure 7 suggestthat the attentional sampling function during thisdifficult search task fluctuated periodically at 9 to 10Hz not just at stimulus onset and offset but also duringthe central portion of stimulus presentation Inaddition we can speculate that the 18 to 20 Hzcomponent observed in Figure 6 and to some extent inFigure 7 could correspond to a harmonic of this 9 to 10Hz periodicity In the next experiment we aimed todemonstrate the functional significance of the periodicsampling observed in the difficult search task

Experiment 2

The attention function measured in the difficultsearch task of Experiment 1 is supposed to representthe fluctuations in attention efficiency over time for anaverage search trial in an average observer If this istrue one should predict that search patterns system-

Figure 5 Attentional sampling function estimated for a difficult

visual search trial Subjects (n frac14 14) were asked to report the

presence or absence of a red grating oriented 308 from upright

among red gratings oriented 3308 from upright and green

gratings oriented 308 from upright (and vice versa) d-primes

were measured for the 40 phase and frequency visual noise

presentation conditions Performances are represented in the

top panel as a modulation of d-prime with respect to the

average computed over all conditions (dashed baseline) On

these performance modulations we applied the complex

decomposition analysis (Figure 2) This analysis was performed

for each individual subject Then the individual estimated

attentional sampling functions were grand-averaged over all

subjects (bottom panel) The standard error of the mean is

represented in gray

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 8

Figure 6 Amplitude spectra of the attention function estimated for the difficult search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for four frequencies 2 Hz 10 Hz and 18ndash20 Hz (B) Average amplitude spectrum of the individual

attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for six frequencies 2 Hz 8ndash10 Hz 16ndash20 Hz

Figure 7 Control for potential onset-offset (nonperiodic) effects The analysis from Figures 3 to 6 was replicated with estimated

attentional functions truncated for the first 83 ms and the last 83 ms (leaving only the central 333 ms portion of the original 500-ms-

long functions) (A) For the easy search task no frequency reached significance in the amplitude spectrum of the grand-average

attentional function (middle note that the frequency resolution of this analysis is 3 Hz instead of 2 Hz because of the shortened

duration of the sampling function) In the average amplitude spectrum of individual attentional functions (right) significance ( p value

103) was reached for three frequencies 3 Hz 6 Hz and 9 Hz (B) For the difficult search task three frequencies reached

significance in the amplitude spectrum of the grand-average attentional function 3 Hz 6 Hz and 9 Hz In the average amplitude

spectrum of individual attentional functions significance ( p value 103) was reached for five frequencies 3 Hz 6 Hz 9 Hz 15 Hz

and 18 Hz with a peak frequency at 9 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 9

atically presented with an SNR following the estimatedgrand-average attentional function (test condition)should lead to better performance than with theopposite signal-to-noise modulation (control condi-tion) We directly tested this prediction in Experiment2 Notice that we high-pass filtered the attentionsampling function by discarding the 2 Hz componentthought to correspond to an onset-offset transienteffect and therefore not reflecting a true samplingperiodicity Overall the aim of Experiment 2 was totest whether rhythmic attention sampling significantlycontributes to search performance This experimentwas performed on two sets of subjects seven subjectsfrom the lsquolsquoinitial subjectsrsquorsquo group (ie subjects who alsoperformed the difficult search task in Experiment 1)and seven subjects from the lsquolsquonaive subjectsrsquorsquo group (iesubjects who had not been included in Experiment 1)

As previously we first verified that naive observersused an appropriate search strategy by computing theRT middot set size slopes As expected (and as in Experiment1) we obtained strongly positive slopes 889 ms 6 413ms per element t(6)frac14 25 p frac14 004 for target presentand 1495 ms 6 35 ms per element t(6) frac14 47 pfrac1400034 for target absent

In the main comparison for Experiment 2 thesubjectsrsquo performance (as measured by d-primes) wassignificantly higher when stimulus presentation fol-lowed the estimated sampling function than its opposite(Figure 8 mixed design two-way analysis of variancewithin-subject factor lsquolsquotestcontrolrsquorsquo F(1 12)frac14 82 pfrac140014) Moreover no significant difference was foundbetween subject groups (between-subject factor lsquolsquoinitialnaıversquorsquo F(1 12)frac14 09 p frac14 037) nor any significant

interaction between the two factors (lsquolsquotestcontrolrsquorsquo middotlsquolsquoinitialnaiversquorsquo F(1 12) frac14 06 pfrac14 0471)

In summary these results reveal that the grand-average attention sampling function estimated for thedifficult search task (or more precisely its frequencycomponents 2 Hz) is representative of the subjectsrsquosampling strategy in this specific task This is true notonly for the initial subjects indicating that the resultobtained with this new method in Experiment 1 isrobust and replicable but also for the naive subjectsmeaning that the estimated attention sampling functionfor this specific task can also apply to a more generalpopulation of human observers

Discussion

In this study we proposed a novel experimentalmethod based on periodic noise interference andFourier series analysis to characterize the dynamics ofperceptual sampling occurring during a difficult task(color-orientation conjunction) and an easy task (L vsthorn) This method produces for each task a time courserepresenting the sampling efficiency of attention overthe 500 ms of stimulus display Our major result is thepresence of a significant oscillation in this time coursefor the difficult task but not for the easy one A simplecontinuous model of attention sampling in whichperceptual efficiency is constant over time (exceptpossibly for random fluctuations) would have pre-dicted flat attentional sampling functions with notemporal fluctuation and therefore no peak in thecorresponding amplitude spectrum This is not what we

Figure 8 Testing the behavioral relevance of the estimated attentional function A difficult search array was presented with dynamic

noise fluctuations following two conditions lsquolsquotest conditionrsquorsquo with a signal-to-noise ratio (SNR) following the previously estimated

grand-average attentional function (Figure 5) and lsquolsquocontrol conditionrsquorsquo with an SNR following the opposite of the estimated grand-

average attentional function We predicted significantly higher performance in the former than the latter case In both cases the 2 Hz

component of the function was omitted (ie the function was high-pass filtered) so as to focus on genuine periodic effects This

experiment was performed on subjects who had previously participated in the first experiment the so-called lsquolsquoinitial subjectsrsquorsquo (nfrac147)

but also on a new set of subjects the so-called lsquolsquonaive subjectsrsquorsquo (nfrac14 7) A two-way analysis of variance (mixed design) revealed a

significant difference between control and test conditions F(1 12)frac14 82 pfrac14 0014 but no significant difference between naive and

initial subjects F(1 12) frac14 09 p frac14 037 nor any interaction between the two factors F(1 12) frac14 06 p frac14 0471

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 10

observed Both tasks presented significant peaks intheir amplitude spectrum (Figures 4 and 6) Assumingthat perceptual efficiency is optimal at one or possiblytwo specific moments of stimulus presentation as onewould expect for example due to greater stimulus-onsetandor -offset sensitivity should have predicted in-creasing andor decreasing attentional sampling func-tions and therefore a steadily decreasing amplitudespectrum with a maximum at the lowest frequency (2Hz) We did observe such a 2 Hz peak in both tasks andconfirmed that it was mainly due to stimulus-onset and-offset effects (Figure 7) However a major unexpectedresult was the presence of another significant localmaximum in the difficult task with a similar phase forall subjects (9ndash10 Hz Figure 6) which was absent forthe easy task (Figure 4) this peak can be explained onlyby assuming rhythmic perceptual sampling by atten-tion

The presence of a 9 to 10 Hz oscillation in the grand-average temporal function (Figure 6A) could in theorybe explained either by the existence of an equivalentoscillation (with a similar phase) at the level of eachindividual subject or by a chance combination ofsubject-specific peaks and troughs at different momentsthat would spuriously create an oscillation whenaveraged over subjects However our finding thatindividual subjects also present a 9 to 10 Hz peak intheir temporal functions (Figure 6B) clearly rules outthe latter explanation Rather it seems that this specifictemporal function with its 9 to 10 Hz oscillation at aparticular phase captures rather well the attentiondeployment of a lsquolsquotypicalrsquorsquo subject in this search task (asshown also in Experiment 2)

Such rhythmic sampling in a difficult task (color-orientation conjunction) appears naturally compatiblewith the classic sequential models of attention deploy-ment a periodic focusing of attention on different items(or subsets of items) within the search array until thetarget is found (Treisman amp Gelade 1980 Treisman ampSouther 1985 Wolfe 1994 Wolfe et al 1989)However we cannot fully rule out the alternativeparallel model of attention deployment in which allitems are sampled at the same time (Eckstein et al2000 Palmer et al 1993) Indeed it is conceivable thatall of the items in the difficult search were processedtogether within each attentional cycle even as theefficiency of this processing fluctuated periodically at9 to 10 Hz Of course this would represent asubstantial alteration of the original parallel model(Eckstein et al 2000 Palmer et al 1993) yet such arevision appears necessary to reconcile this model withour experimental findings Whether attention sampledthe entire search array or only a subset of elements theimportant conclusion of our study is that it did soperiodically rather than continuously (at least for thespecific difficult search task that we used)

Our novel paradigm appears suitable for revealingthe temporal dynamics of attention in at least somevisual search tasks The method is not infalliblehowever and it is worth reviewing its main limitationsand the possible ways it could be improved in futurestudies

First and foremost our paradigm relies on Fourierseries decomposition which is an intrinsically linearframework (see the Appendix for further description ofthe model) As such we must assume that the searchperformance on each trial is the product of an attentionsampling function and the superimposed noise functionand that this attentional sampling function does notsystematically vary when the noise function changes(for example when we apply sinusoidal noise atdifferent phases and frequencies) This assumption oflinearity is both a reasonable first approximation in ourendeavor to measure the dynamics of attention and anobvious potential limitation of the proposed techniqueIndeed it implies that many nonlinear attentionalstrategies would not be appropriately captured by thetechnique It is worth pointing out however that ourmethod was explicitly designed to prevent one commonform of nonlinearity that would have resulted from asteady-state masking by the sinusoidal noise (ieperception fluctuating in phase with the noise) or froma perceptual entrainment to the stimulus (ie percep-tion fluctuating in counterphase with the noise) Inthese two forms of entrainment one would expect ageneral decrease (or increase respectively) of perfor-mance at specific frequencies regardless of the phaseYet in our experiments at each frequency we did notmerely measure a response to two sinusoids separatedby 908 (cosine and sine as Fourier analysis wouldminimally require) but to all four cardinal sinusoids(ie cosine and its opposite sine and its opposite)Therefore any nonlinear perceptual entrainment orsteady-state masking that would have equally affectedall four noise sinusoids at a given frequency would haveresulted in a null attention modulation at thisfrequency To conclude this type of nonlinear con-found alone cannot explain our finding of a 10 Hzattentional modulation in the difficult search task

Another potential issue with the current version ofour paradigm is the possibility that increased percep-tual sensitivity at stimulus onset andor offset couldhave produced edge effects that would have concealedthe true temporal fluctuations of attention Indeedthere was no noise on the screen before or after our500-ms sampling window meaning that the onset andoffset stimulus information were potentially free offorward and backward masking respectively Thispotential confound led us to perform an additionalanalysis in which we removed the edges of our samplingwindow (Figure 7) Although this removal stillproduced a 10 Hz periodicity in the difficult search

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 11

taskmdashthereby confirming that this periodicity was notmerely an artifact induced by edge effectsmdashit also leftno systematic remaining temporal fluctuation in theeasy search task It is plausible however that suchtemporal fluctuations maybe even periodic oneswould have occurred in the easy search task if the onsetand offset transient information had not been directlyavailable to the observers Therefore in future studieswe propose to improve the paradigm by starting thesinusoidal noise modulation well before (eg 500 ms)and ending it well after the 500-ms critical samplingwindow during which the search array is actuallydisplayed This would essentially avoid edge effects byseparating the onset and offset transients from ourwindow of analysis

Finally the potential influence of eye movementsmust be considered Even though we instructed thesubjects not to move their eyes one might argue thatsome of the temporal fluctuations observed in oursampling functions could be the result of unwantedsaccadic andor micro-saccadic eye movements Thiswould imply that these eye movements are time lockedto stimulus onset and that they would entail differenttemporal patterns depending on the exact search taskSaccades andor microsaccades could not have directlyproduced the temporal pattern observed in the difficultsearch task however because they tend to occur atlower frequencies typically less than 5 Hz (Martinez-Conde Macknik amp Hubel 2004) It appears morelikely therefore that the 10 Hz periodicity obtainedin our difficult search task was a reflection of covertrather than overt attention sampling dynamics

Conclusions

To summarize using an innovative psychophysicalmethod we were able to argue that an easy search taskwas processed in an apparently continuous modewhereas a difficult search task was processed periodi-cally at a frequency of 9 to 10 Hz Of course ourconclusions are limited to the two search tasks tested inthis study and further work would be needed todetermine if they could be generalized to other easy anddifficult search tasks In addition a potential limitationof our study is that we sampled attentional dynamicsfrom only 2 to 20 Hz Values greater than 20 Hz (egin the gamma frequency range) were not consideredalthough they may be relevant for attention (BauerOostenveld Peeters amp Fries 2006 Gruber MullerKeil amp Elbert 1999 Jensen Kaiser amp Lachaux 2007Treue 2001) A number of recent studies havesuggested that attention samples sensory informationperiodically at frequencies between 7 and 13 Hz invarious other experimental situations even when only

one object is attended and thus no sequentialexploration is required (Busch amp Vanrullen 2010Vanrullen et al 2007 Vanrullen amp Dubois 2011) Thepresent demonstration of an intrinsic periodicity duringa visual search task adds significant weight to thisemerging view of periodic attention

Keywords visual search attention temporal dynam-ics periodicity Fourier analysis dynamic noise

Acknowledgments

This research was funded by a EURYI Awardto RV

Commercial relationships noneCorresponding author Laura DugueEmail lauraduguecercoups-tlsefrAddress CNRSndashCerco Toulouse France

References

Bauer M Oostenveld R Peeters M amp Fries P(2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduceslow-frequency activity in parieto-occipital areasJournal of Neuroscience 26 490ndash501

Busch N A amp Vanrullen R (2010) SpontaneousEEG oscillations reveal periodic sampling of visualattention Proceedings of the National Academy ofSciences USA 107 16048ndash16053

Eckstein M P Thomas J P Palmer J ampShimozaki S S (2000) A signal detection modelpredicts the effects of set size on visual searchaccuracy for feature conjunction triple conjunc-tion and disjunction displays Perception amp Psy-chophysics 62 425ndash451

Gobell J L Tseng C H amp Sperling G (2004) Thespatial distribution of visual attention VisionResearch 44 1273ndash1296

Gruber T Muller M M Keil A amp Elbert T(1999) Selective visual-spatial attention altersinduced gamma band responses in the human EEGClinical Neurophysiology 110 2074ndash2085

Jensen O Kaiser J amp Lachaux J P (2007) Humangamma-frequency oscillations associated with at-tention and memory Trends in Neuroscience 30317ndash324

Lu Z L amp Dosher BA (2008) Characterizingobservers using external noise and observer models

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 12

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

Figure 6 Amplitude spectra of the attention function estimated for the difficult search (A) Amplitude spectrum of the grand-average

attentional function Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for four frequencies 2 Hz 10 Hz and 18ndash20 Hz (B) Average amplitude spectrum of the individual

attentional functions Background p values were calculated using a Monte Carlo procedure (see Methods) The amplitude reaches

significance ( p value 103) for six frequencies 2 Hz 8ndash10 Hz 16ndash20 Hz

Figure 7 Control for potential onset-offset (nonperiodic) effects The analysis from Figures 3 to 6 was replicated with estimated

attentional functions truncated for the first 83 ms and the last 83 ms (leaving only the central 333 ms portion of the original 500-ms-

long functions) (A) For the easy search task no frequency reached significance in the amplitude spectrum of the grand-average

attentional function (middle note that the frequency resolution of this analysis is 3 Hz instead of 2 Hz because of the shortened

duration of the sampling function) In the average amplitude spectrum of individual attentional functions (right) significance ( p value

103) was reached for three frequencies 3 Hz 6 Hz and 9 Hz (B) For the difficult search task three frequencies reached

significance in the amplitude spectrum of the grand-average attentional function 3 Hz 6 Hz and 9 Hz In the average amplitude

spectrum of individual attentional functions significance ( p value 103) was reached for five frequencies 3 Hz 6 Hz 9 Hz 15 Hz

and 18 Hz with a peak frequency at 9 Hz

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 9

atically presented with an SNR following the estimatedgrand-average attentional function (test condition)should lead to better performance than with theopposite signal-to-noise modulation (control condi-tion) We directly tested this prediction in Experiment2 Notice that we high-pass filtered the attentionsampling function by discarding the 2 Hz componentthought to correspond to an onset-offset transienteffect and therefore not reflecting a true samplingperiodicity Overall the aim of Experiment 2 was totest whether rhythmic attention sampling significantlycontributes to search performance This experimentwas performed on two sets of subjects seven subjectsfrom the lsquolsquoinitial subjectsrsquorsquo group (ie subjects who alsoperformed the difficult search task in Experiment 1)and seven subjects from the lsquolsquonaive subjectsrsquorsquo group (iesubjects who had not been included in Experiment 1)

As previously we first verified that naive observersused an appropriate search strategy by computing theRT middot set size slopes As expected (and as in Experiment1) we obtained strongly positive slopes 889 ms 6 413ms per element t(6)frac14 25 p frac14 004 for target presentand 1495 ms 6 35 ms per element t(6) frac14 47 pfrac1400034 for target absent

In the main comparison for Experiment 2 thesubjectsrsquo performance (as measured by d-primes) wassignificantly higher when stimulus presentation fol-lowed the estimated sampling function than its opposite(Figure 8 mixed design two-way analysis of variancewithin-subject factor lsquolsquotestcontrolrsquorsquo F(1 12)frac14 82 pfrac140014) Moreover no significant difference was foundbetween subject groups (between-subject factor lsquolsquoinitialnaıversquorsquo F(1 12)frac14 09 p frac14 037) nor any significant

interaction between the two factors (lsquolsquotestcontrolrsquorsquo middotlsquolsquoinitialnaiversquorsquo F(1 12) frac14 06 pfrac14 0471)

In summary these results reveal that the grand-average attention sampling function estimated for thedifficult search task (or more precisely its frequencycomponents 2 Hz) is representative of the subjectsrsquosampling strategy in this specific task This is true notonly for the initial subjects indicating that the resultobtained with this new method in Experiment 1 isrobust and replicable but also for the naive subjectsmeaning that the estimated attention sampling functionfor this specific task can also apply to a more generalpopulation of human observers

Discussion

In this study we proposed a novel experimentalmethod based on periodic noise interference andFourier series analysis to characterize the dynamics ofperceptual sampling occurring during a difficult task(color-orientation conjunction) and an easy task (L vsthorn) This method produces for each task a time courserepresenting the sampling efficiency of attention overthe 500 ms of stimulus display Our major result is thepresence of a significant oscillation in this time coursefor the difficult task but not for the easy one A simplecontinuous model of attention sampling in whichperceptual efficiency is constant over time (exceptpossibly for random fluctuations) would have pre-dicted flat attentional sampling functions with notemporal fluctuation and therefore no peak in thecorresponding amplitude spectrum This is not what we

Figure 8 Testing the behavioral relevance of the estimated attentional function A difficult search array was presented with dynamic

noise fluctuations following two conditions lsquolsquotest conditionrsquorsquo with a signal-to-noise ratio (SNR) following the previously estimated

grand-average attentional function (Figure 5) and lsquolsquocontrol conditionrsquorsquo with an SNR following the opposite of the estimated grand-

average attentional function We predicted significantly higher performance in the former than the latter case In both cases the 2 Hz

component of the function was omitted (ie the function was high-pass filtered) so as to focus on genuine periodic effects This

experiment was performed on subjects who had previously participated in the first experiment the so-called lsquolsquoinitial subjectsrsquorsquo (nfrac147)

but also on a new set of subjects the so-called lsquolsquonaive subjectsrsquorsquo (nfrac14 7) A two-way analysis of variance (mixed design) revealed a

significant difference between control and test conditions F(1 12)frac14 82 pfrac14 0014 but no significant difference between naive and

initial subjects F(1 12) frac14 09 p frac14 037 nor any interaction between the two factors F(1 12) frac14 06 p frac14 0471

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 10

observed Both tasks presented significant peaks intheir amplitude spectrum (Figures 4 and 6) Assumingthat perceptual efficiency is optimal at one or possiblytwo specific moments of stimulus presentation as onewould expect for example due to greater stimulus-onsetandor -offset sensitivity should have predicted in-creasing andor decreasing attentional sampling func-tions and therefore a steadily decreasing amplitudespectrum with a maximum at the lowest frequency (2Hz) We did observe such a 2 Hz peak in both tasks andconfirmed that it was mainly due to stimulus-onset and-offset effects (Figure 7) However a major unexpectedresult was the presence of another significant localmaximum in the difficult task with a similar phase forall subjects (9ndash10 Hz Figure 6) which was absent forthe easy task (Figure 4) this peak can be explained onlyby assuming rhythmic perceptual sampling by atten-tion

The presence of a 9 to 10 Hz oscillation in the grand-average temporal function (Figure 6A) could in theorybe explained either by the existence of an equivalentoscillation (with a similar phase) at the level of eachindividual subject or by a chance combination ofsubject-specific peaks and troughs at different momentsthat would spuriously create an oscillation whenaveraged over subjects However our finding thatindividual subjects also present a 9 to 10 Hz peak intheir temporal functions (Figure 6B) clearly rules outthe latter explanation Rather it seems that this specifictemporal function with its 9 to 10 Hz oscillation at aparticular phase captures rather well the attentiondeployment of a lsquolsquotypicalrsquorsquo subject in this search task (asshown also in Experiment 2)

Such rhythmic sampling in a difficult task (color-orientation conjunction) appears naturally compatiblewith the classic sequential models of attention deploy-ment a periodic focusing of attention on different items(or subsets of items) within the search array until thetarget is found (Treisman amp Gelade 1980 Treisman ampSouther 1985 Wolfe 1994 Wolfe et al 1989)However we cannot fully rule out the alternativeparallel model of attention deployment in which allitems are sampled at the same time (Eckstein et al2000 Palmer et al 1993) Indeed it is conceivable thatall of the items in the difficult search were processedtogether within each attentional cycle even as theefficiency of this processing fluctuated periodically at9 to 10 Hz Of course this would represent asubstantial alteration of the original parallel model(Eckstein et al 2000 Palmer et al 1993) yet such arevision appears necessary to reconcile this model withour experimental findings Whether attention sampledthe entire search array or only a subset of elements theimportant conclusion of our study is that it did soperiodically rather than continuously (at least for thespecific difficult search task that we used)

Our novel paradigm appears suitable for revealingthe temporal dynamics of attention in at least somevisual search tasks The method is not infalliblehowever and it is worth reviewing its main limitationsand the possible ways it could be improved in futurestudies

First and foremost our paradigm relies on Fourierseries decomposition which is an intrinsically linearframework (see the Appendix for further description ofthe model) As such we must assume that the searchperformance on each trial is the product of an attentionsampling function and the superimposed noise functionand that this attentional sampling function does notsystematically vary when the noise function changes(for example when we apply sinusoidal noise atdifferent phases and frequencies) This assumption oflinearity is both a reasonable first approximation in ourendeavor to measure the dynamics of attention and anobvious potential limitation of the proposed techniqueIndeed it implies that many nonlinear attentionalstrategies would not be appropriately captured by thetechnique It is worth pointing out however that ourmethod was explicitly designed to prevent one commonform of nonlinearity that would have resulted from asteady-state masking by the sinusoidal noise (ieperception fluctuating in phase with the noise) or froma perceptual entrainment to the stimulus (ie percep-tion fluctuating in counterphase with the noise) Inthese two forms of entrainment one would expect ageneral decrease (or increase respectively) of perfor-mance at specific frequencies regardless of the phaseYet in our experiments at each frequency we did notmerely measure a response to two sinusoids separatedby 908 (cosine and sine as Fourier analysis wouldminimally require) but to all four cardinal sinusoids(ie cosine and its opposite sine and its opposite)Therefore any nonlinear perceptual entrainment orsteady-state masking that would have equally affectedall four noise sinusoids at a given frequency would haveresulted in a null attention modulation at thisfrequency To conclude this type of nonlinear con-found alone cannot explain our finding of a 10 Hzattentional modulation in the difficult search task

Another potential issue with the current version ofour paradigm is the possibility that increased percep-tual sensitivity at stimulus onset andor offset couldhave produced edge effects that would have concealedthe true temporal fluctuations of attention Indeedthere was no noise on the screen before or after our500-ms sampling window meaning that the onset andoffset stimulus information were potentially free offorward and backward masking respectively Thispotential confound led us to perform an additionalanalysis in which we removed the edges of our samplingwindow (Figure 7) Although this removal stillproduced a 10 Hz periodicity in the difficult search

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 11

taskmdashthereby confirming that this periodicity was notmerely an artifact induced by edge effectsmdashit also leftno systematic remaining temporal fluctuation in theeasy search task It is plausible however that suchtemporal fluctuations maybe even periodic oneswould have occurred in the easy search task if the onsetand offset transient information had not been directlyavailable to the observers Therefore in future studieswe propose to improve the paradigm by starting thesinusoidal noise modulation well before (eg 500 ms)and ending it well after the 500-ms critical samplingwindow during which the search array is actuallydisplayed This would essentially avoid edge effects byseparating the onset and offset transients from ourwindow of analysis

Finally the potential influence of eye movementsmust be considered Even though we instructed thesubjects not to move their eyes one might argue thatsome of the temporal fluctuations observed in oursampling functions could be the result of unwantedsaccadic andor micro-saccadic eye movements Thiswould imply that these eye movements are time lockedto stimulus onset and that they would entail differenttemporal patterns depending on the exact search taskSaccades andor microsaccades could not have directlyproduced the temporal pattern observed in the difficultsearch task however because they tend to occur atlower frequencies typically less than 5 Hz (Martinez-Conde Macknik amp Hubel 2004) It appears morelikely therefore that the 10 Hz periodicity obtainedin our difficult search task was a reflection of covertrather than overt attention sampling dynamics

Conclusions

To summarize using an innovative psychophysicalmethod we were able to argue that an easy search taskwas processed in an apparently continuous modewhereas a difficult search task was processed periodi-cally at a frequency of 9 to 10 Hz Of course ourconclusions are limited to the two search tasks tested inthis study and further work would be needed todetermine if they could be generalized to other easy anddifficult search tasks In addition a potential limitationof our study is that we sampled attentional dynamicsfrom only 2 to 20 Hz Values greater than 20 Hz (egin the gamma frequency range) were not consideredalthough they may be relevant for attention (BauerOostenveld Peeters amp Fries 2006 Gruber MullerKeil amp Elbert 1999 Jensen Kaiser amp Lachaux 2007Treue 2001) A number of recent studies havesuggested that attention samples sensory informationperiodically at frequencies between 7 and 13 Hz invarious other experimental situations even when only

one object is attended and thus no sequentialexploration is required (Busch amp Vanrullen 2010Vanrullen et al 2007 Vanrullen amp Dubois 2011) Thepresent demonstration of an intrinsic periodicity duringa visual search task adds significant weight to thisemerging view of periodic attention

Keywords visual search attention temporal dynam-ics periodicity Fourier analysis dynamic noise

Acknowledgments

This research was funded by a EURYI Awardto RV

Commercial relationships noneCorresponding author Laura DugueEmail lauraduguecercoups-tlsefrAddress CNRSndashCerco Toulouse France

References

Bauer M Oostenveld R Peeters M amp Fries P(2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduceslow-frequency activity in parieto-occipital areasJournal of Neuroscience 26 490ndash501

Busch N A amp Vanrullen R (2010) SpontaneousEEG oscillations reveal periodic sampling of visualattention Proceedings of the National Academy ofSciences USA 107 16048ndash16053

Eckstein M P Thomas J P Palmer J ampShimozaki S S (2000) A signal detection modelpredicts the effects of set size on visual searchaccuracy for feature conjunction triple conjunc-tion and disjunction displays Perception amp Psy-chophysics 62 425ndash451

Gobell J L Tseng C H amp Sperling G (2004) Thespatial distribution of visual attention VisionResearch 44 1273ndash1296

Gruber T Muller M M Keil A amp Elbert T(1999) Selective visual-spatial attention altersinduced gamma band responses in the human EEGClinical Neurophysiology 110 2074ndash2085

Jensen O Kaiser J amp Lachaux J P (2007) Humangamma-frequency oscillations associated with at-tention and memory Trends in Neuroscience 30317ndash324

Lu Z L amp Dosher BA (2008) Characterizingobservers using external noise and observer models

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 12

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

atically presented with an SNR following the estimatedgrand-average attentional function (test condition)should lead to better performance than with theopposite signal-to-noise modulation (control condi-tion) We directly tested this prediction in Experiment2 Notice that we high-pass filtered the attentionsampling function by discarding the 2 Hz componentthought to correspond to an onset-offset transienteffect and therefore not reflecting a true samplingperiodicity Overall the aim of Experiment 2 was totest whether rhythmic attention sampling significantlycontributes to search performance This experimentwas performed on two sets of subjects seven subjectsfrom the lsquolsquoinitial subjectsrsquorsquo group (ie subjects who alsoperformed the difficult search task in Experiment 1)and seven subjects from the lsquolsquonaive subjectsrsquorsquo group (iesubjects who had not been included in Experiment 1)

As previously we first verified that naive observersused an appropriate search strategy by computing theRT middot set size slopes As expected (and as in Experiment1) we obtained strongly positive slopes 889 ms 6 413ms per element t(6)frac14 25 p frac14 004 for target presentand 1495 ms 6 35 ms per element t(6) frac14 47 pfrac1400034 for target absent

In the main comparison for Experiment 2 thesubjectsrsquo performance (as measured by d-primes) wassignificantly higher when stimulus presentation fol-lowed the estimated sampling function than its opposite(Figure 8 mixed design two-way analysis of variancewithin-subject factor lsquolsquotestcontrolrsquorsquo F(1 12)frac14 82 pfrac140014) Moreover no significant difference was foundbetween subject groups (between-subject factor lsquolsquoinitialnaıversquorsquo F(1 12)frac14 09 p frac14 037) nor any significant

interaction between the two factors (lsquolsquotestcontrolrsquorsquo middotlsquolsquoinitialnaiversquorsquo F(1 12) frac14 06 pfrac14 0471)

In summary these results reveal that the grand-average attention sampling function estimated for thedifficult search task (or more precisely its frequencycomponents 2 Hz) is representative of the subjectsrsquosampling strategy in this specific task This is true notonly for the initial subjects indicating that the resultobtained with this new method in Experiment 1 isrobust and replicable but also for the naive subjectsmeaning that the estimated attention sampling functionfor this specific task can also apply to a more generalpopulation of human observers

Discussion

In this study we proposed a novel experimentalmethod based on periodic noise interference andFourier series analysis to characterize the dynamics ofperceptual sampling occurring during a difficult task(color-orientation conjunction) and an easy task (L vsthorn) This method produces for each task a time courserepresenting the sampling efficiency of attention overthe 500 ms of stimulus display Our major result is thepresence of a significant oscillation in this time coursefor the difficult task but not for the easy one A simplecontinuous model of attention sampling in whichperceptual efficiency is constant over time (exceptpossibly for random fluctuations) would have pre-dicted flat attentional sampling functions with notemporal fluctuation and therefore no peak in thecorresponding amplitude spectrum This is not what we

Figure 8 Testing the behavioral relevance of the estimated attentional function A difficult search array was presented with dynamic

noise fluctuations following two conditions lsquolsquotest conditionrsquorsquo with a signal-to-noise ratio (SNR) following the previously estimated

grand-average attentional function (Figure 5) and lsquolsquocontrol conditionrsquorsquo with an SNR following the opposite of the estimated grand-

average attentional function We predicted significantly higher performance in the former than the latter case In both cases the 2 Hz

component of the function was omitted (ie the function was high-pass filtered) so as to focus on genuine periodic effects This

experiment was performed on subjects who had previously participated in the first experiment the so-called lsquolsquoinitial subjectsrsquorsquo (nfrac147)

but also on a new set of subjects the so-called lsquolsquonaive subjectsrsquorsquo (nfrac14 7) A two-way analysis of variance (mixed design) revealed a

significant difference between control and test conditions F(1 12)frac14 82 pfrac14 0014 but no significant difference between naive and

initial subjects F(1 12) frac14 09 p frac14 037 nor any interaction between the two factors F(1 12) frac14 06 p frac14 0471

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 10

observed Both tasks presented significant peaks intheir amplitude spectrum (Figures 4 and 6) Assumingthat perceptual efficiency is optimal at one or possiblytwo specific moments of stimulus presentation as onewould expect for example due to greater stimulus-onsetandor -offset sensitivity should have predicted in-creasing andor decreasing attentional sampling func-tions and therefore a steadily decreasing amplitudespectrum with a maximum at the lowest frequency (2Hz) We did observe such a 2 Hz peak in both tasks andconfirmed that it was mainly due to stimulus-onset and-offset effects (Figure 7) However a major unexpectedresult was the presence of another significant localmaximum in the difficult task with a similar phase forall subjects (9ndash10 Hz Figure 6) which was absent forthe easy task (Figure 4) this peak can be explained onlyby assuming rhythmic perceptual sampling by atten-tion

The presence of a 9 to 10 Hz oscillation in the grand-average temporal function (Figure 6A) could in theorybe explained either by the existence of an equivalentoscillation (with a similar phase) at the level of eachindividual subject or by a chance combination ofsubject-specific peaks and troughs at different momentsthat would spuriously create an oscillation whenaveraged over subjects However our finding thatindividual subjects also present a 9 to 10 Hz peak intheir temporal functions (Figure 6B) clearly rules outthe latter explanation Rather it seems that this specifictemporal function with its 9 to 10 Hz oscillation at aparticular phase captures rather well the attentiondeployment of a lsquolsquotypicalrsquorsquo subject in this search task (asshown also in Experiment 2)

Such rhythmic sampling in a difficult task (color-orientation conjunction) appears naturally compatiblewith the classic sequential models of attention deploy-ment a periodic focusing of attention on different items(or subsets of items) within the search array until thetarget is found (Treisman amp Gelade 1980 Treisman ampSouther 1985 Wolfe 1994 Wolfe et al 1989)However we cannot fully rule out the alternativeparallel model of attention deployment in which allitems are sampled at the same time (Eckstein et al2000 Palmer et al 1993) Indeed it is conceivable thatall of the items in the difficult search were processedtogether within each attentional cycle even as theefficiency of this processing fluctuated periodically at9 to 10 Hz Of course this would represent asubstantial alteration of the original parallel model(Eckstein et al 2000 Palmer et al 1993) yet such arevision appears necessary to reconcile this model withour experimental findings Whether attention sampledthe entire search array or only a subset of elements theimportant conclusion of our study is that it did soperiodically rather than continuously (at least for thespecific difficult search task that we used)

Our novel paradigm appears suitable for revealingthe temporal dynamics of attention in at least somevisual search tasks The method is not infalliblehowever and it is worth reviewing its main limitationsand the possible ways it could be improved in futurestudies

First and foremost our paradigm relies on Fourierseries decomposition which is an intrinsically linearframework (see the Appendix for further description ofthe model) As such we must assume that the searchperformance on each trial is the product of an attentionsampling function and the superimposed noise functionand that this attentional sampling function does notsystematically vary when the noise function changes(for example when we apply sinusoidal noise atdifferent phases and frequencies) This assumption oflinearity is both a reasonable first approximation in ourendeavor to measure the dynamics of attention and anobvious potential limitation of the proposed techniqueIndeed it implies that many nonlinear attentionalstrategies would not be appropriately captured by thetechnique It is worth pointing out however that ourmethod was explicitly designed to prevent one commonform of nonlinearity that would have resulted from asteady-state masking by the sinusoidal noise (ieperception fluctuating in phase with the noise) or froma perceptual entrainment to the stimulus (ie percep-tion fluctuating in counterphase with the noise) Inthese two forms of entrainment one would expect ageneral decrease (or increase respectively) of perfor-mance at specific frequencies regardless of the phaseYet in our experiments at each frequency we did notmerely measure a response to two sinusoids separatedby 908 (cosine and sine as Fourier analysis wouldminimally require) but to all four cardinal sinusoids(ie cosine and its opposite sine and its opposite)Therefore any nonlinear perceptual entrainment orsteady-state masking that would have equally affectedall four noise sinusoids at a given frequency would haveresulted in a null attention modulation at thisfrequency To conclude this type of nonlinear con-found alone cannot explain our finding of a 10 Hzattentional modulation in the difficult search task

Another potential issue with the current version ofour paradigm is the possibility that increased percep-tual sensitivity at stimulus onset andor offset couldhave produced edge effects that would have concealedthe true temporal fluctuations of attention Indeedthere was no noise on the screen before or after our500-ms sampling window meaning that the onset andoffset stimulus information were potentially free offorward and backward masking respectively Thispotential confound led us to perform an additionalanalysis in which we removed the edges of our samplingwindow (Figure 7) Although this removal stillproduced a 10 Hz periodicity in the difficult search

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 11

taskmdashthereby confirming that this periodicity was notmerely an artifact induced by edge effectsmdashit also leftno systematic remaining temporal fluctuation in theeasy search task It is plausible however that suchtemporal fluctuations maybe even periodic oneswould have occurred in the easy search task if the onsetand offset transient information had not been directlyavailable to the observers Therefore in future studieswe propose to improve the paradigm by starting thesinusoidal noise modulation well before (eg 500 ms)and ending it well after the 500-ms critical samplingwindow during which the search array is actuallydisplayed This would essentially avoid edge effects byseparating the onset and offset transients from ourwindow of analysis

Finally the potential influence of eye movementsmust be considered Even though we instructed thesubjects not to move their eyes one might argue thatsome of the temporal fluctuations observed in oursampling functions could be the result of unwantedsaccadic andor micro-saccadic eye movements Thiswould imply that these eye movements are time lockedto stimulus onset and that they would entail differenttemporal patterns depending on the exact search taskSaccades andor microsaccades could not have directlyproduced the temporal pattern observed in the difficultsearch task however because they tend to occur atlower frequencies typically less than 5 Hz (Martinez-Conde Macknik amp Hubel 2004) It appears morelikely therefore that the 10 Hz periodicity obtainedin our difficult search task was a reflection of covertrather than overt attention sampling dynamics

Conclusions

To summarize using an innovative psychophysicalmethod we were able to argue that an easy search taskwas processed in an apparently continuous modewhereas a difficult search task was processed periodi-cally at a frequency of 9 to 10 Hz Of course ourconclusions are limited to the two search tasks tested inthis study and further work would be needed todetermine if they could be generalized to other easy anddifficult search tasks In addition a potential limitationof our study is that we sampled attentional dynamicsfrom only 2 to 20 Hz Values greater than 20 Hz (egin the gamma frequency range) were not consideredalthough they may be relevant for attention (BauerOostenveld Peeters amp Fries 2006 Gruber MullerKeil amp Elbert 1999 Jensen Kaiser amp Lachaux 2007Treue 2001) A number of recent studies havesuggested that attention samples sensory informationperiodically at frequencies between 7 and 13 Hz invarious other experimental situations even when only

one object is attended and thus no sequentialexploration is required (Busch amp Vanrullen 2010Vanrullen et al 2007 Vanrullen amp Dubois 2011) Thepresent demonstration of an intrinsic periodicity duringa visual search task adds significant weight to thisemerging view of periodic attention

Keywords visual search attention temporal dynam-ics periodicity Fourier analysis dynamic noise

Acknowledgments

This research was funded by a EURYI Awardto RV

Commercial relationships noneCorresponding author Laura DugueEmail lauraduguecercoups-tlsefrAddress CNRSndashCerco Toulouse France

References

Bauer M Oostenveld R Peeters M amp Fries P(2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduceslow-frequency activity in parieto-occipital areasJournal of Neuroscience 26 490ndash501

Busch N A amp Vanrullen R (2010) SpontaneousEEG oscillations reveal periodic sampling of visualattention Proceedings of the National Academy ofSciences USA 107 16048ndash16053

Eckstein M P Thomas J P Palmer J ampShimozaki S S (2000) A signal detection modelpredicts the effects of set size on visual searchaccuracy for feature conjunction triple conjunc-tion and disjunction displays Perception amp Psy-chophysics 62 425ndash451

Gobell J L Tseng C H amp Sperling G (2004) Thespatial distribution of visual attention VisionResearch 44 1273ndash1296

Gruber T Muller M M Keil A amp Elbert T(1999) Selective visual-spatial attention altersinduced gamma band responses in the human EEGClinical Neurophysiology 110 2074ndash2085

Jensen O Kaiser J amp Lachaux J P (2007) Humangamma-frequency oscillations associated with at-tention and memory Trends in Neuroscience 30317ndash324

Lu Z L amp Dosher BA (2008) Characterizingobservers using external noise and observer models

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 12

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

observed Both tasks presented significant peaks intheir amplitude spectrum (Figures 4 and 6) Assumingthat perceptual efficiency is optimal at one or possiblytwo specific moments of stimulus presentation as onewould expect for example due to greater stimulus-onsetandor -offset sensitivity should have predicted in-creasing andor decreasing attentional sampling func-tions and therefore a steadily decreasing amplitudespectrum with a maximum at the lowest frequency (2Hz) We did observe such a 2 Hz peak in both tasks andconfirmed that it was mainly due to stimulus-onset and-offset effects (Figure 7) However a major unexpectedresult was the presence of another significant localmaximum in the difficult task with a similar phase forall subjects (9ndash10 Hz Figure 6) which was absent forthe easy task (Figure 4) this peak can be explained onlyby assuming rhythmic perceptual sampling by atten-tion

The presence of a 9 to 10 Hz oscillation in the grand-average temporal function (Figure 6A) could in theorybe explained either by the existence of an equivalentoscillation (with a similar phase) at the level of eachindividual subject or by a chance combination ofsubject-specific peaks and troughs at different momentsthat would spuriously create an oscillation whenaveraged over subjects However our finding thatindividual subjects also present a 9 to 10 Hz peak intheir temporal functions (Figure 6B) clearly rules outthe latter explanation Rather it seems that this specifictemporal function with its 9 to 10 Hz oscillation at aparticular phase captures rather well the attentiondeployment of a lsquolsquotypicalrsquorsquo subject in this search task (asshown also in Experiment 2)

Such rhythmic sampling in a difficult task (color-orientation conjunction) appears naturally compatiblewith the classic sequential models of attention deploy-ment a periodic focusing of attention on different items(or subsets of items) within the search array until thetarget is found (Treisman amp Gelade 1980 Treisman ampSouther 1985 Wolfe 1994 Wolfe et al 1989)However we cannot fully rule out the alternativeparallel model of attention deployment in which allitems are sampled at the same time (Eckstein et al2000 Palmer et al 1993) Indeed it is conceivable thatall of the items in the difficult search were processedtogether within each attentional cycle even as theefficiency of this processing fluctuated periodically at9 to 10 Hz Of course this would represent asubstantial alteration of the original parallel model(Eckstein et al 2000 Palmer et al 1993) yet such arevision appears necessary to reconcile this model withour experimental findings Whether attention sampledthe entire search array or only a subset of elements theimportant conclusion of our study is that it did soperiodically rather than continuously (at least for thespecific difficult search task that we used)

Our novel paradigm appears suitable for revealingthe temporal dynamics of attention in at least somevisual search tasks The method is not infalliblehowever and it is worth reviewing its main limitationsand the possible ways it could be improved in futurestudies

First and foremost our paradigm relies on Fourierseries decomposition which is an intrinsically linearframework (see the Appendix for further description ofthe model) As such we must assume that the searchperformance on each trial is the product of an attentionsampling function and the superimposed noise functionand that this attentional sampling function does notsystematically vary when the noise function changes(for example when we apply sinusoidal noise atdifferent phases and frequencies) This assumption oflinearity is both a reasonable first approximation in ourendeavor to measure the dynamics of attention and anobvious potential limitation of the proposed techniqueIndeed it implies that many nonlinear attentionalstrategies would not be appropriately captured by thetechnique It is worth pointing out however that ourmethod was explicitly designed to prevent one commonform of nonlinearity that would have resulted from asteady-state masking by the sinusoidal noise (ieperception fluctuating in phase with the noise) or froma perceptual entrainment to the stimulus (ie percep-tion fluctuating in counterphase with the noise) Inthese two forms of entrainment one would expect ageneral decrease (or increase respectively) of perfor-mance at specific frequencies regardless of the phaseYet in our experiments at each frequency we did notmerely measure a response to two sinusoids separatedby 908 (cosine and sine as Fourier analysis wouldminimally require) but to all four cardinal sinusoids(ie cosine and its opposite sine and its opposite)Therefore any nonlinear perceptual entrainment orsteady-state masking that would have equally affectedall four noise sinusoids at a given frequency would haveresulted in a null attention modulation at thisfrequency To conclude this type of nonlinear con-found alone cannot explain our finding of a 10 Hzattentional modulation in the difficult search task

Another potential issue with the current version ofour paradigm is the possibility that increased percep-tual sensitivity at stimulus onset andor offset couldhave produced edge effects that would have concealedthe true temporal fluctuations of attention Indeedthere was no noise on the screen before or after our500-ms sampling window meaning that the onset andoffset stimulus information were potentially free offorward and backward masking respectively Thispotential confound led us to perform an additionalanalysis in which we removed the edges of our samplingwindow (Figure 7) Although this removal stillproduced a 10 Hz periodicity in the difficult search

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 11

taskmdashthereby confirming that this periodicity was notmerely an artifact induced by edge effectsmdashit also leftno systematic remaining temporal fluctuation in theeasy search task It is plausible however that suchtemporal fluctuations maybe even periodic oneswould have occurred in the easy search task if the onsetand offset transient information had not been directlyavailable to the observers Therefore in future studieswe propose to improve the paradigm by starting thesinusoidal noise modulation well before (eg 500 ms)and ending it well after the 500-ms critical samplingwindow during which the search array is actuallydisplayed This would essentially avoid edge effects byseparating the onset and offset transients from ourwindow of analysis

Finally the potential influence of eye movementsmust be considered Even though we instructed thesubjects not to move their eyes one might argue thatsome of the temporal fluctuations observed in oursampling functions could be the result of unwantedsaccadic andor micro-saccadic eye movements Thiswould imply that these eye movements are time lockedto stimulus onset and that they would entail differenttemporal patterns depending on the exact search taskSaccades andor microsaccades could not have directlyproduced the temporal pattern observed in the difficultsearch task however because they tend to occur atlower frequencies typically less than 5 Hz (Martinez-Conde Macknik amp Hubel 2004) It appears morelikely therefore that the 10 Hz periodicity obtainedin our difficult search task was a reflection of covertrather than overt attention sampling dynamics

Conclusions

To summarize using an innovative psychophysicalmethod we were able to argue that an easy search taskwas processed in an apparently continuous modewhereas a difficult search task was processed periodi-cally at a frequency of 9 to 10 Hz Of course ourconclusions are limited to the two search tasks tested inthis study and further work would be needed todetermine if they could be generalized to other easy anddifficult search tasks In addition a potential limitationof our study is that we sampled attentional dynamicsfrom only 2 to 20 Hz Values greater than 20 Hz (egin the gamma frequency range) were not consideredalthough they may be relevant for attention (BauerOostenveld Peeters amp Fries 2006 Gruber MullerKeil amp Elbert 1999 Jensen Kaiser amp Lachaux 2007Treue 2001) A number of recent studies havesuggested that attention samples sensory informationperiodically at frequencies between 7 and 13 Hz invarious other experimental situations even when only

one object is attended and thus no sequentialexploration is required (Busch amp Vanrullen 2010Vanrullen et al 2007 Vanrullen amp Dubois 2011) Thepresent demonstration of an intrinsic periodicity duringa visual search task adds significant weight to thisemerging view of periodic attention

Keywords visual search attention temporal dynam-ics periodicity Fourier analysis dynamic noise

Acknowledgments

This research was funded by a EURYI Awardto RV

Commercial relationships noneCorresponding author Laura DugueEmail lauraduguecercoups-tlsefrAddress CNRSndashCerco Toulouse France

References

Bauer M Oostenveld R Peeters M amp Fries P(2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduceslow-frequency activity in parieto-occipital areasJournal of Neuroscience 26 490ndash501

Busch N A amp Vanrullen R (2010) SpontaneousEEG oscillations reveal periodic sampling of visualattention Proceedings of the National Academy ofSciences USA 107 16048ndash16053

Eckstein M P Thomas J P Palmer J ampShimozaki S S (2000) A signal detection modelpredicts the effects of set size on visual searchaccuracy for feature conjunction triple conjunc-tion and disjunction displays Perception amp Psy-chophysics 62 425ndash451

Gobell J L Tseng C H amp Sperling G (2004) Thespatial distribution of visual attention VisionResearch 44 1273ndash1296

Gruber T Muller M M Keil A amp Elbert T(1999) Selective visual-spatial attention altersinduced gamma band responses in the human EEGClinical Neurophysiology 110 2074ndash2085

Jensen O Kaiser J amp Lachaux J P (2007) Humangamma-frequency oscillations associated with at-tention and memory Trends in Neuroscience 30317ndash324

Lu Z L amp Dosher BA (2008) Characterizingobservers using external noise and observer models

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 12

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

taskmdashthereby confirming that this periodicity was notmerely an artifact induced by edge effectsmdashit also leftno systematic remaining temporal fluctuation in theeasy search task It is plausible however that suchtemporal fluctuations maybe even periodic oneswould have occurred in the easy search task if the onsetand offset transient information had not been directlyavailable to the observers Therefore in future studieswe propose to improve the paradigm by starting thesinusoidal noise modulation well before (eg 500 ms)and ending it well after the 500-ms critical samplingwindow during which the search array is actuallydisplayed This would essentially avoid edge effects byseparating the onset and offset transients from ourwindow of analysis

Finally the potential influence of eye movementsmust be considered Even though we instructed thesubjects not to move their eyes one might argue thatsome of the temporal fluctuations observed in oursampling functions could be the result of unwantedsaccadic andor micro-saccadic eye movements Thiswould imply that these eye movements are time lockedto stimulus onset and that they would entail differenttemporal patterns depending on the exact search taskSaccades andor microsaccades could not have directlyproduced the temporal pattern observed in the difficultsearch task however because they tend to occur atlower frequencies typically less than 5 Hz (Martinez-Conde Macknik amp Hubel 2004) It appears morelikely therefore that the 10 Hz periodicity obtainedin our difficult search task was a reflection of covertrather than overt attention sampling dynamics

Conclusions

To summarize using an innovative psychophysicalmethod we were able to argue that an easy search taskwas processed in an apparently continuous modewhereas a difficult search task was processed periodi-cally at a frequency of 9 to 10 Hz Of course ourconclusions are limited to the two search tasks tested inthis study and further work would be needed todetermine if they could be generalized to other easy anddifficult search tasks In addition a potential limitationof our study is that we sampled attentional dynamicsfrom only 2 to 20 Hz Values greater than 20 Hz (egin the gamma frequency range) were not consideredalthough they may be relevant for attention (BauerOostenveld Peeters amp Fries 2006 Gruber MullerKeil amp Elbert 1999 Jensen Kaiser amp Lachaux 2007Treue 2001) A number of recent studies havesuggested that attention samples sensory informationperiodically at frequencies between 7 and 13 Hz invarious other experimental situations even when only

one object is attended and thus no sequentialexploration is required (Busch amp Vanrullen 2010Vanrullen et al 2007 Vanrullen amp Dubois 2011) Thepresent demonstration of an intrinsic periodicity duringa visual search task adds significant weight to thisemerging view of periodic attention

Keywords visual search attention temporal dynam-ics periodicity Fourier analysis dynamic noise

Acknowledgments

This research was funded by a EURYI Awardto RV

Commercial relationships noneCorresponding author Laura DugueEmail lauraduguecercoups-tlsefrAddress CNRSndashCerco Toulouse France

References

Bauer M Oostenveld R Peeters M amp Fries P(2006) Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduceslow-frequency activity in parieto-occipital areasJournal of Neuroscience 26 490ndash501

Busch N A amp Vanrullen R (2010) SpontaneousEEG oscillations reveal periodic sampling of visualattention Proceedings of the National Academy ofSciences USA 107 16048ndash16053

Eckstein M P Thomas J P Palmer J ampShimozaki S S (2000) A signal detection modelpredicts the effects of set size on visual searchaccuracy for feature conjunction triple conjunc-tion and disjunction displays Perception amp Psy-chophysics 62 425ndash451

Gobell J L Tseng C H amp Sperling G (2004) Thespatial distribution of visual attention VisionResearch 44 1273ndash1296

Gruber T Muller M M Keil A amp Elbert T(1999) Selective visual-spatial attention altersinduced gamma band responses in the human EEGClinical Neurophysiology 110 2074ndash2085

Jensen O Kaiser J amp Lachaux J P (2007) Humangamma-frequency oscillations associated with at-tention and memory Trends in Neuroscience 30317ndash324

Lu Z L amp Dosher BA (2008) Characterizingobservers using external noise and observer models

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 12

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

assessing internal representations with externalnoise Psychological Review 115 44ndash82

Martinez-Conde S Macknik S L amp Hubel D H(2004) The role of fixational eye movements invisual perception Nature Reviews Neuroscience 5229ndash240

Palmer J Ames C T amp Lindsey D T (1993)Measuring the effect of attention on simple visualsearch Journal of Experimental Psychology Hu-man Perception and Performance 19 108ndash130

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1) 7 105ndash120 httpwwwjournalofvisionorgcontent217 doi101167217 [PubMed] [Article]

Stroud J M (1956) The fine structure of psycholog-ical time In H Quastler (Ed) Information theoryin psychology Problems and methods (pp 174ndash207)New York The Free Press

Treisman A amp Gelade G (1980) A feature-integration theory of attention Cognitive Psychol-ogy 12 97ndash136

Treisman A amp Souther J (1985) Search asymmetryA diagnostic for preattentive processing of separa-ble features Journal of Experimental PsychologyGeneral 114 285ndash310

Treue S (2001) Neural correlates of attention inprimate visual cortex Trends in Neuroscience 24295ndash300

Vanrullen R Carlson T amp Cavanagh P (2007) Theblinking spotlight of attention Proceedings of theNational Academy of Sciences USA 104 19204ndash19209

Vanrullen R amp Dubois J (2011) The psychophysicsof brain rhythms Frontiers in Psychology 2 203

VanRullen R amp Koch C (2003) Is perceptiondiscrete or continuous Trends in Cognitive Science7 207ndash213

Vanrullen R Reddy L amp Koch C (2005)Attention-driven discrete sampling of motion per-ception Proceedings of the National Academy ofSciences USA 102 5291ndash5296

Wolfe J M (1994) Visual search in continuousnaturalistic stimuli Vision Research 34 1187ndash1195

Wolfe J M (1998) Visual search A review In HPashler (Ed) Attention (pp 13ndash73) LondonUniversity College London Press

Wolfe J M Cave K R amp Franzel S L (1989)Guided search An alternative to the featureintegration model for visual search Journal ofExperimental Psychology Human Perception andPerformance 15 419ndash433

Appendix

In this article we postulated that perception could beapproximated (potentially with an additive or multi-plicative constant) by a linear combination of stimulusinformation and attentional sampling that is

P frac14Z05

0

SethtTHORNAethtTHORNdt eth1THORN

Further we argued that the variable P in Equation 1could be approximated by the subjectsrsquo behavioralperformance as measured experimentally by d0 valuesWith these assumptions we could then reconstruct theattentional sampling function A(t) using Fourier seriesanalysis as a combination of sinusoidal functionsweighed by the d0 values recorded experimentally forvarious frequencies and phases of stimulation (see theExperimental Procedure section in the main text)

This reasoning however does not consider the fullsequence of perceptual processes affecting the stimulusdisplay (including the external noise imposed in ourparadigm) nor the decision mechanism (potentiallysubjected to internal noise) by which the perceptualoutcome is turned into a yesno motor response oneach trial Here we demonstrate within a full percep-tual and decisional model (inspired by the visual noise-masking literature Lu and Dosher 2008 Solomon2002) that d0 is indeed a valid approximation of thevariable P in Equation 1

Our observer model is depicted schematically inSupplementary Figure S1 It assumes that a time-varying signal pattern signal(t) combined with a time-varying external noise pattern noise(t) is sampled intime according to an attentional sampling functionA(t) The resulting pattern is then compared with atarget template w via a template-matching operationThe final yesno decision process returns yes (targetdetected) whenever the template match subjected toGaussian-distributed internal noise g exceeds a prede-fined criterion c that is

wtpatternthorn g c eth2THORNThe template-matching and decision processes inEquation 2 are identical to those described by Solomon(2002 equation 2 p 106) In summary the observerdepicted in Supplementary Figure S1 will respond yeswhen

wt

Z 05

0

AethtTHORN signalethtTHORN thorn noiseethtTHORNfrac12 dtthorn g c eth3THORN

and will respond no otherwiseLet us now apply this observer model to the

conditions of our Experiment 1 On every trial i of our

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 13

experiment the display consists in a time-invariantstimulus pattern signali and a time-invariant externalnoise pattern noisei both of them amplitude-modulat-ed in time by a sinusoidal function of the samefrequency xi but opposite phase (let us call ui the phaseapplied to the signal) Within the context of ourexperiment Equation 3 can therefore be rewritten as

wt

ZAethtTHORN signali

1thorn coseth2pxitthorn uiTHORN2

thorn noisei

1 coseth2pxitthorn uiTHORN2

dtthorn gi c

eth4THORNBecause the signali and noisei patterns are time-invariant the equation becomes

wt signali thorn noisei

2

Z05

0

AethtTHORNdt

thorn wt signali noisei

2

Z 05

0

AethtTHORNcoseth2pxitthorn uiTHORNdt

thorn gi c eth5THORN

We further simplify this equation by assuming that the(positive) attentional sampling function A(t) is scaled

such that its integral over the display interval is equal to1 thus

wt signali thorn noisei

2

thorn wt signali noisei

2

Z 05

0

AethtTHORNcoseth2pxitthorn uiTHORNdt

thorn gi c eth6THORNThe remaining integral in Equation 6 directly corre-sponds to the perceptual variable P used in the maintext (more precisely Pcosx Psinx P-cosx and P-sinx areobtained by setting the phase ui to 0p2 p and p2respectively) This variable P fluctuates between1 and1 When P frac14 1 reflecting an ideal match between theattentional sampling function and the sinusoidalstimulation for that trial the external noise terms inEquation 6 are canceled and the response dependssolely on the input signal pattern On the other handwhen Pfrac141 implying that the attentional samplingfunction oscillates in phase opposition with thesinusoidal stimulation for that trial the signal terms inEquation 6 disappear and the response solely dependson the noise

As explained above it behooves us to demonstratethat d0 is a valid approximation for this variable P Forevery possible value of P between1 and 1 (by steps of001) we simulated the model observerrsquos response for5000 trials (half of them with a target and halfwithout) using the following parameters The internalnoise gi was taken from a Gaussian distribution withmean of 0 and standard deviation of 1 (this value waschosen to produce a range of d0 values between 0 and2 compatible with those observed experimentally) Weassumed that the signal pattern was identical to thetarget template on target-present trials (that is

Figure S1 Observer model The time-varying input display

(combining stimulus and externally imposed noise) is sampled

by an attentional sampling function The resulting pattern is

matched with a target template and a noisy decision process

compares the amount of match with an internal criterion

Figure S2 d0 of the observer model for various values of the

variable P (corresponding to the integral in Equation 6) The

existence of a near-linear relationship validates our choice of

using d0 as an experimental approximation of P

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 14

wtsignali frac14 1) and was orthogonal to the targettemplate on target-absent trials (that is wtsignali frac14 0)The noise pattern was also assumed to be orthogonalon average to the target template but with trial-by-trialdeviations in either direction that is the dot productwtnoisei was taken from a Gaussian distribution with amean of 0 and standard deviation of 05 (this valueimplies that a fully random noise pattern will matchthe target template better than the target itself on

228 of trialsmdasha rather liberal estimate) Finally thecriterion c was chosen (based on the observed falsealarm rates during the experiment) to be c frac14 05Supplementary Figure S2 illustrates the d0 values ofthis model observer obtained for various values of thevariable P Based on this near-linear relationship wecan conclude that in our study d0 was indeed anappropriate experimental approximation of the vari-able P

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 15

• Introduction
• Methods
• e01
• e02
• e03
• f01
• e04
• e05
• f02
• Results
• f03
• f04
• f05
• f06
• f07
• Discussion
• f08
• Conclusions
• Bauer1
• Busch1
• Eckstein1
• Gobell1
• Gruber1
• Jensen1
• Lu1
• MartinezConde1
• Palmer1
• Solomon1
• Stroud1
• Treisman1
• Treisman2
• Treue1
• Vanrullen1
• Vanrullen2
• VanRullen1
• Vanrullen3
• Wolfe1
• Wolfe2
• Wolfe3
• Appendix
• e06
• e07
• e08
• e09
• e10
• e11
• s01
• s02

wtsignali frac14 1) and was orthogonal to the targettemplate on target-absent trials (that is wtsignali frac14 0)The noise pattern was also assumed to be orthogonalon average to the target template but with trial-by-trialdeviations in either direction that is the dot productwtnoisei was taken from a Gaussian distribution with amean of 0 and standard deviation of 05 (this valueimplies that a fully random noise pattern will matchthe target template better than the target itself on

228 of trialsmdasha rather liberal estimate) Finally thecriterion c was chosen (based on the observed falsealarm rates during the experiment) to be c frac14 05Supplementary Figure S2 illustrates the d0 values ofthis model observer obtained for various values of thevariable P Based on this near-linear relationship wecan conclude that in our study d0 was indeed anappropriate experimental approximation of the vari-able P

Journal of Vision (2014) 14(2)11 1ndash15 Dugue amp VanRullen 15

• Introduction
• Methods
• e01
• e02
• e03
• f01
• e04
• e05
• f02
• Results
• f03
• f04
• f05
• f06
• f07
• Discussion
• f08
• Conclusions
• Bauer1
• Busch1
• Eckstein1
• Gobell1
• Gruber1
• Jensen1
• Lu1
• MartinezConde1
• Palmer1
• Solomon1
• Stroud1
• Treisman1
• Treisman2
• Treue1
• Vanrullen1
• Vanrullen2
• VanRullen1
• Vanrullen3
• Wolfe1
• Wolfe2
• Wolfe3
• Appendix
• e06
• e07
• e08
• e09
• e10
• e11
• s01
• s02