Competition in Visual Working Memory by Stephen M - T-Space

Competition in Visual Working Memory

by

Stephen M. Emrich

A thesis submitted in conformity with the requirements

for the degree of Doctor of Philosophy

Department of Psychology

University of Toronto

© Copyright by Stephen M. Emrich 2011

ii

Competition in Visual Working Memory

Stephen M. Emrich

Doctor of Philosophy

Department of Psychology

University of Toronto

2011

Abstract

The processing of information within the visual system is limited by several cognitive and neural

bottlenecks. One critical bottleneck occurs in visual working memory (VWM), as the amount of

information that can be maintained on-line is limited to three to four items. While numerous

theories have addressed this limited capacity of VWM, it is unclear how processing bottlenecks

in the initial selection and perception of visual information affect the number or precision of

representations that can be maintained in VWM. The purpose of this dissertation was to examine

whether early competition for resources within the visual system limits the number or precision

of representation that can be maintained in VWM. To establish whether competitive interactions

affect VWM, Chapters 1 – 4 tested whether performance on VWM tasks was related to the

distance between memory items. The results of these experiments reveal that when objects are

presented close together in space, VWM performance is impaired relative to when those same

objects are presented further apart. Using a three-component model of continuous responses in a

recall task, Chapters 3 – 4 demonstrated that the distance between objects primarily affects the

precision of responses, and increases the number of non-target errors. Chapter 5 extended these

iii

findings to distractors, demonstrating that multiple distractors affect the precision and accuracy

of VWM responses. Chapters 6 – 7 tested how attentional selection can bias memory

representations, revealing that objects that are given high attentional priority were reported with

greater precision. Finally, Chapters 8 and 9 examined bias-signals as a potential source of

individual differences in VWM performance, revealing that high-performers have more precise

representations of sub-capacity representations than low-performers. Together, these results

reveal that VWM performance is limited by competition for representation within the visual

system, and that attention plays a critical role in resolving competition and consequently,

determining the contents of VWM.

iv

Acknowledgements

I am incredibly grateful to the support and guidance of my primary advisor and mentor,

Susanne Ferber. Over the last eight years she has been a constant source of encouragement, and I

greatly appreciate everything she has done to help foster my growth as a researcher. I am also

indebted to Jay Pratt for always providing words of encouragement and wisdom. Many thanks as

well to Adam Anderson for always helping me see the bigger picture.

I have been fortunate to be surrounded by an incredible group of collaborators and

colleagues that have not only challenged and inspired me, but have also made research much

more enjoyable. I am most indebted to Maha Adamo and Naseem Al-Aidroos for years of

discussions that have shaped my ideas and my research in incalculable ways. Thank you also to

Ben Amsel, Doug Garrett, Taylor Schmitz, Bobby Stojanoski, Josh Susskind, Greg West, and

Kristin Wilson for years of enjoyable exchanges I hope will continue. I also owe a great deal to

Carson Pun who has always been around helping make much of this work possible. Thank you to

a great group of undergraduate research assistants who have helped me in the last 6 years: Justin

Ruppel, Felix Lee, Paul Lu, and Steve Thomas. A special thanks to Stefan Bostan who

contributed to all aspects of the research reported in Chapters 4 to 9 and Kuhan

Puvanenthiranathan, who helped collect data for Chapter 2.

I am also incredibly indebted to Paul Bays at University College – London for sharing his

analysis methods with me, and to Matthew Hilimire at Georigia Tech for helpful discussions

about the Ptc.

Finally, I would also like to thank my entire family for their support, encouragement, and

understanding.

Most of all, I would like to thank my wife. You gave me both the inspiration and support

to see this through. Thank you.

v

Table of Contents

Abstract ......................................................................................................................................................... ii

Acknowledgements ...................................................................................................................................... iv

Table of Contents .......................................................................................................................................... v

List of Figures .............................................................................................................................................. vii

Introduction .................................................................................................................................................. 1

Chapter 1: Spatial Competition Affects VWM Performance ...................................................................... 18

Chapter 2: Competition Affects VWM for Complex More than Simple Objects ........................................ 25

Method ................................................................................................................................................... 26

Results and Discussion ............................................................................................................................ 27

Chapter 3: Competitive Interactions Increase Recall Errors ....................................................................... 31

Method ................................................................................................................................................... 33


Chapter 4: Competitive Interactions Affect the Precision of VWM Representations ................................ 41

Method ................................................................................................................................................... 41


Chapter 5: Multiple Distractors Affect the Precision and Number of VWM Representations ................... 47

Method ................................................................................................................................................... 48


Chapter 6: Bottom‐Up Attentional Capture Biases VWM Responses ........................................................ 59

Method ................................................................................................................................................... 60


Chapter 7: Colour and Location Cues Bias VWM Responses Independently ............................................. 66

Method ................................................................................................................................................... 67


Chapter 8: VWM Capacity is Correlated with Response Precision ............................................................. 74

Method ................................................................................................................................................... 76


Chapter 9: Ptc Predicts VWM Capacity ....................................................................................................... 80

Method ................................................................................................................................................... 81

vi


General Discussion ...................................................................................................................................... 93

References ................................................................................................................................................ 109

vii

List of Figures Figure 1 ‐ Example of the continuous‐response method of measuring and modeling VWM performance. (Left

Panel) During the probe, the observer’s goal is to select the colour of the colour wheel that

corresponds to the probed (highlighted) item. Note that this paradigm is the one that is used in

Chapter 3. (Right Panel) Responses can be modeled according to a number of factors. Target responses

are assumed to be normally distributed around the target value (T). The amount of spread in

responses (error) provides a measure of precision (1/SD). In a two‐factor model, guesses are modeled

as a uniform distribution across all responses. In a three‐factor model, non‐target responses are

considered to be normally distributed around non‐target values (N), with the same concentration as

the target responses. Adopted from Bays, Catalo & Hussain (2009). ........................................................ 6

Figure 2 ‐ Stimuli and procedure used in the experiment in Chapter 1. (A) Examples of the low‐ and high‐

competition displays. Stimuli are presented within a smaller area in the high‐competition condition. (B)

Example of the sequence of events of the change‐detection task employed in the experiment in

Chapter 1. The observer’s goal is to detect changes to the target stimuli. ............................................ 21

Figure 3 ‐ Average accuracy observed in the remember‐all condition in the experiment in Chapter 1. Accuracy was

higher overall in the low spatial‐competition condition. Error bars denote within‐subject 95%

confidence intervals (CIs). ........................................................................................................................ 23

Figure 4 ‐ Average accuracy as a function of filtering and spatial competition condition presented in Chapter 1,

collapsed across set size. Accuracy was higher overall in the low‐competition condition, as well as in

the remember‐all condition. Error bars denote within‐subject 95% confidence intervals. .................... 24

Figure 5 ‐ Examples of complex stimuli used in Chapter 2. For the simple stimuli condition, shaded cubes were

replaced with coloured squares. .............................................................................................................. 27

Figure 6 ‐ Accuracy for simple and complex objects, averaged across competition condition. Error bars denote 95%

within‐subject CIs. .................................................................................................................................... 28

Figure 7 ‐ Accuracy for simple and complex objects under low and high competition, averaged across set size. Error

bars denote within‐subject 95% CIs. ........................................................................................................ 29

Figure 8 ‐ Examples of the low‐ and high‐competition conditions used in the experiment in Chapter 3. .................. 34

Figure 9 ‐ Measures of relative precision (top left) and mean maximum likelihood estimate values for the three

factors used to model responses in Chapter 3. Error bars denote within‐subject SEM. (Top Left) As in

previous experiments, relative precision decreased according to a power‐law function. The fit lines

represent the power‐law fit for both low and high competitions. (Top Right) The proportion of target

responses decreases as a function of the number of sample items, with no difference between high‐

and low‐competition conditions. (Bottom Left) The proportion of non‐target responses increases with

set size, and is greater in the high‐competition than in the low‐competition condition. (Bottom Right)

The proportion of guesses increases with set size, and is unaffected by the distance between objects.

................................................................................................................................................................. 37

viii

Figure 10 ‐ Example schematics of the stimulus configurations in Chapter 4. Stimuli are not drawn to scale. .......... 41

Figure 11 – Precision, PT, PNT, and PG obtained for each separation condition in the experiment in Chapter 4, plotted

by the distance between two items. Dashed lines correspond to linear contrasts. Error bars denote

within‐subject SEM. ................................................................................................................................. 44

Figure 12 – Example of Targets Only and Targets + Distractors condition used in experiment 5. Circles were used as

targets for half of the participants. The set size 4 condition is shown for both conditions..................... 49

Figure 13 ‐ Results of experiment in Chapter 5. In the targets + distractors condition, set size indicates the total

number of items (50% distractors). Error bars denote within‐subject SEM. ........................................... 51

Figure 14 – Correlation between VWM capacity (K) and the change in precision when the display contains two

distractors and two targets, relative to when only two targets are present. .......................................... 54

Figure 15 ‐ Schematic of the experiment used in Chapter 6. The locations of the two onset and two no‐onset

memory items are highlighted by solid and dotted lines, respectively. .................................................. 60

Figure 16 ‐ The precision of responses (top left), and the proportion of target, non‐target, and random responses

observed in the experiment in Chapter 6. The x‐axis indicates the number of onset targets. Two

additional no‐onset targets were presented in each condition. Error bars indicate within‐subject SEM.

................................................................................................................................................................. 62

Figure 17 ‐ Stimulus and trial schematics for the experiment in Chapter 7. (Top Panel) The four cues created by

colour and spatial cues. For colour matching trials (Col+), the cue colour matches the colour of the

probed sample item. In the location matching trials (Loc+), the cue is presented at the location of the

target sample item. Thus, both colour and location matches are defined with respect to the probe,

rather than the sample. The dotted line marks the location of the eventual target probe (not present

during the task). (Bottom Panel) A schematic of a Col‐Loc+ cue trial. ..................................................... 67

Figure 18 ‐ The precision of responses (top left), and the proportion of target, non‐target, and random responses

as a function of cue validity and cue type. Error bars indicate within‐subject SEM. The solid horizontal

line indicates neutral cue performance, with the dotted lines indicating +/‐ within‐subject SEM. ........ 69

Figure 19 – Correlation between VWM capacity and overall response precision for a single object. ........................ 77

Figure 20 ‐ Schematic of trials performed in experiment in Chapter 9. In the filter condition, participants were

instructed to only try to remember only the circles or the squares. In the remember‐all task,

participants were instructed to remember all four items on the cued side. For display purposes, the

stimuli on the uncued side are displayed in grey. .................................................................................... 83

Figure 21 – ERP results observed in the experiment presented in Chapter 9. (Top Panel) ERP waveforms for the

channel pairs of interest. The dotted lines indicate the windows used for calculating each of the

averages. (Bottom Panel) Average waveforms for each of the ERPs, calculated at the peak channels

(see text). Error bars denote within‐subject SEM. ................................................................................... 87

ix

Figure 22 ‐ Relationship between memory capacity and Ptc amplitude. (Left Panel) ERP waveforms for high‐ and

low‐capacity participants in the remember‐all condition. (Right Panel) The change in Ptc amplitude

between high‐ and low‐competition conditions is significantly correlated with VWM capacity. ........... 88

Figure 23 ‐ A schematic of a biased‐competition model of VWM. Blue regions identify local competition for

representation. Green regions identify those where global competition for VWM resources may occur.

Red regions identify potential sources of top‐down feedback that serve as bias signals for target

selection. Solid arrows indicate recurrent interactions between local and global competition. Dashed

arrows indicate possible sources of top‐down selection mechanisms. ................................................... 99

1

Introduction

In order to perform many everyday tasks, it is necessary to maintain and manipulate

information that is no longer present in the environment. The cognitive process of maintaining

information online is known as working memory; it enables us to hold information in mind after

it has been removed from view. As with all cognitive processes, however, working memory is

limited by information processing bottlenecks in the human nervous system (Marois & Ivanoff,

2005). As a result of these processing bottlenecks, there are limits on the amount of information

that can be simultaneously maintained in working memory.

In the visual domain, studies have typically demonstrated that individuals can

successfully retain roughly three or four items in working memory over short delays, with

performance decreasing thereafter (Luck & Vogel, 1997). Thus, for the last decade, the

predominant model has suggested that visual working memory (VWM) has a “capacity” limit –

i.e., that the number of items that can be stored online in VWM is limited to three or four items.

Currently, however, there are debates as to whether in fact VWM has a limited number of

“slots”, or whether VWM draws upon a limited pool of resources that restricts the precision with

which items can be represented (Bays & Husain, 2008). Accordingly, processing bottlenecks

may not limit the number of items that can be maintained in VWM, but may instead limit the

precision of those representations that can be stored (Bays & Husain, 2008). Whether it is the

number or precision of VWM representations that are limited, the neurobiological mechanisms

underlying these limits have remained elusive. That is, although it is clear that there are

representational limitations in VWM, it is currently unknown what the exact factors are that limit

the number or precision of those representations.

In the present thesis, I report a series of studies that investigate how factors that influence

the initial selection and representation of visual objects may limit VWM performance. In other

words, these studies will examine whether processing bottlenecks prior to the stage of

maintenance limit the number or precision of representations accessible to VWM. Specifically,

using behavioural and event-related potential (ERP) experiments, I address the question of

whether the capacity or precision of VWM representations is limited by competitive interactions.

By examining competition as a neurobiological factor limiting information processing in VWM,

these studies attempt to elucidate the origins of representational limits in working memory.

2

The experiments presented in this thesis reveal three main findings about the nature of

VWM: First, inter-item competition affects VWM performance; specifically, increasing

competitive interactions by presenting items closer together in space reduces the precision of

VWM responses, without affecting the number of items that can be stored. Second, competition

affects the ability to correctly recall the location of target information, leading to an increase in

the proportion of non-target response errors. Third, competition for representation in VWM can

be biased via bottom-up or top-down selection mechanisms. Before outlining the motivation for

the current thesis in more detail, I will first provide an overview of VWM, as well as describe the

techniques used for assessing VWM that are employed in the present thesis.

Working Memory – An Overview

The experimental study of different memory systems has a long and rich history in

psychology. For the purposes of this thesis, it is important to distinguish between iconic or

sensory memory on the one hand and working memory on the other. Iconic or sensory memory

is the shortest form of informational persistence in the nervous system. Information in this

memory system is typically considered to be an “icon” of the original sensory stimulus that

extends the sensory percept of the stimulus for very brief durations (~100 – 150 ms) and is

sensitive to transients such as spatial displacement and masking (Irwin & Thomas, 2008). As

such, iconic memory can be considered a high-capacity but low-stability buffer system for visual

information.

Short-term memory (STM) can be distinguished from iconic memory in a number of

ways. First, STM has a duration that can be an order of magnitude longer than sensory memory,

lasting around four seconds or longer (Zhang & Luck, 2009). Second, in contrast to iconic

memory, STM is relatively insensitive to exposure duration and visual transients (Irwin &

Thomas, 2008). Finally, and most important for the purposes of this thesis, STM has a limited

capacity for high-precision information which is in stark contrast to the virtually limitless

amount of information that is represented in sensory memory (Sperling, 1960). Thus, STM

cannot be seen as temporally-extended sensory memory, even though similar physiological

mechanisms may underlie both processes (i.e., sustained neural activity). Instead, STM

represents neurobiological and cognitive processes distinct from iconic memory that serve to

maintain a small amount of high-fidelity information that can be used across saccades and the

physical displacement of the visual scene.

3

The term working memory (WM) largely replaced the term short-term memory (STM)

following the publication of Baddeley and Hitch’s (1974) multiple component model of working

memory. The term working memory was intended to emphasize that STM is often involved in

the active manipulation of information in service of other cognitive tasks or operations

(Baddeley & Hitch, 1974). On theoretical grounds, the model proposed a central executive that

controls and oversees the active manipulation of information in the slave systems that are

specific to the sensory modality of that information (i.e., visual and auditory).

Measuring Capacity and Precision in Visual Working Memory

The focus of the present thesis is the system responsible for the maintenance of visual

information after it has been removed from view. Visual working memory (VWM) or visual

short-term memory (VSTM1) has been most widely studied using the one-shot change-detection

task developed by Phillips (1974) and first studied extensively by Luck and Vogel (1997). In the

change-detection task, participants are presented with simultaneous simple visual stimuli. These

objects are typically simple coloured shapes, although more complex object and symbolic stimuli

have been used (Alvarez & Cavanagh, 2004). This sample display is presented only briefly

(typically 100 – 500 ms), and is followed by a delay interval lasting from several hundred

milliseconds to several seconds. The participant’s task is to remember as many of the items as

possible over the delay period. Following the delay, a change-detection probe is presented, cuing

participants to make a forced-choice discrimination as to whether the items in the probe display

contain a change in some dimension relative to the original sample display.

Using the change-detection task, individual VWM capacity can be estimated using a

formula originally developed by Pashler (1988) and modified by Cowan (2001). Using this

formula it is possible to estimate the average number of items successfully remembered for any

given set-size. That is, given a set size of S, capacity, K, is estimated as:

* ( 1)K S H CR

where H and CR correspond to the proportion of hits and correct rejections. Thus, this formula

yields an estimate of the average number of items successfully remembered (K) for any given set

size (S) while accounting for guessing (CR – 1). Applying this formula across a range of

experiments, it has been found that the average number of items that can be successfully

1 Although the terms VSTM and VWM are sometimes used to distinguish between the perceptual and executive aspects of visual memory, for the purposes of this thesis these terms are interchangeable.

4

remembered is typically between three and four, independent of set size (Cowan, 2001). That is,

even though overall change-detection accuracy decreases above a set size of three to four items,

applying the Pashler-Cowan formula indicates that an average of roughly three or four items is

correctly reported, indicating that VWM capacity plateaus around three items and remains

relatively constant. Between individuals, however, VWM capacity varies greatly, from as low as

one to as high as six items (Fukuda, Awh, & Vogel, 2010).

The use of K-estimates for measuring memory capacity across a range of set sizes has

helped to elucidate the neural mechanisms that mediate VWM. For example, by examining K-

estimates across a range of set sizes, functional MRI (fMRI) studies have identified VWM-

related activity in the bilateral intraparietal sulcus (IPS), lateral occipital (LO) and anterior

cingulate (AC) regions (Todd & Marois, 2004; Xu & Chun, 2006). That is, activity in these

regions increases with increasing set size, and reaches an asymptote around three or four items

(i.e., the capacity of VWM). Furthermore, individual differences in peak IPS activity during the

delay period correspond closely to individual K-estimates (Todd & Marois, 2005), suggesting

that this region plays a critical role in the active storage of information in VWM.

Although working memory performance and capacity can be measured using other tasks

(e.g., visual search: Alvarez & Cavanagh, 2004; Emrich, Ruppel, Al-Aidroos, Pratt, & Ferber,

2008; Emrich, Al-Aidroos, Pratt, & Ferber, 2009), the one-shot change-detection paradigm has a

number of advantages over these alternatives. First, the change-detection task provides a simple

yet elegant means to incrementally vary the load placed on VWM resources. That is, by

manipulating the number of items presented in the memory sample, it is possible to measure WM

performance across a range of information loads, while holding the task demands constant. In a

seminal study, Luck and Vogel (1997) demonstrated that individuals could successfully maintain

about three or four simple coloured objects in VWM; above four items, however, overall

accuracy decreased as a function of the number of items presented. This effect of decreasing

change-detection performance for memory displays containing more than three or four items was

found to be consistent across a range of different stimuli, sample times, and other manipulations

(Vogel, Woodman, & Luck, 2001). Thus, these experiments provided strong evidence that the

capacity of VWM is limited to roughly three or four items.

A second advantage of the one-shot change-detection paradigm is that it is possible to

elucidate multiple aspects of VWM encoding and maintenance independent of changes in task

5

demands. For example, in one study (Awh, Barton, & Vogel, 2007), the sample items were either

simple or complex objects, and the changes to the probe display could be either within-category

or between-category changes. While overall capacity estimates were reduced with complex

objects, performance for complex objects was similar to that of simple objects when the change

between the sample and test displays required a between-category comparison. Thus, using a

simple modification of the change-detection technique, this study demonstrated that it may be

possible to store a similar number of simple and complex objects in VWM, but that complex

objects are stored at a lower resolution.

Third, the one-shot change-detection paradigm is advantageous over other paradigms in

that it emphasizes the perceptual aspects of STM rather than the executive control aspects of

working memory. That is, the change-detection task can be performed without needing to

perform cognitive manipulations to the information stored in VWM. Although the ability to

manipulate and use information that is stored in WM is clearly important for the performance of

many real-world tasks, these functions are largely independent of the capacity of information

stored in VWM (i.e., executive control functions are downstream of memory storage; Postle,

Berger, & D'Esposito, 1999). Thus, by minimizing the retrieval and manipulation demands, any

observed differences in capacity (across individuals or stimulus conditions) can be largely

attributed to differences in encoding or maintenance, rather than differences in executive

function.

Although the change-detection paradigm offers many advantages, one significant draw-

back is that change detection is a high-threshold approach to studying VWM capacity. That is,

individuals are assumed to have a perfect representation of items that are correctly reported, and

no representation of items which are not correctly reported. Typically, approaches based on

signal-detection theory can better account for psychophysical performance than high-threshold

approaches. Recently, several studies have applied a signal-detection approach to measuring

VWM performance by using a simple variant of typical change-detection tasks. For example, in

an approach first used by Wilken and Ma (2004) and recently adopted by number of recent

studies (Bays, Catalao, & Husain, 2009; Fougnie, Asplund, & Marois, 2010; Johnson, Spencer,

Luck, & Schöner, 2009; Zhang & Luck, 2008, 2009), observers indicate the identity of the

probed item by selecting its colour from a colour wheel (Figure 1). Rather than assessing a high-

threshold binary response (i.e., whether an item is or is not in memory), performance in this type

6

of task is measured by calculating the amount of error in each response relative to the target

value.

The principal advantage of using a continuous-response method over the one-shot

change-detection task is that responses can be analyzed to reveal multiple characteristics of

performance. The primary measure of interest observed in a continuous-response task is the trial-

to-trial variability in the responses, or precision, which is measured as the inverse of the standard

deviation (SD) of error of all responses. Studies examining the precision of VWM

representations have revealed that precision decreases as a function of the number of to-be-

remembered items (Bays & Husain, 2008; Wilken & Ma, 2004). As such, these findings have

provided evidence against the theory that VWM has a limited number of “slots”, and instead

suggest that representing a large number of items in VWM comes at the expense of precision.

That is, as the number of items in the display increases, the precision of responses decreases,

indicating that there is not a discrete shift in VWM performance related to the number of items

Figure 1 ‐ Example of the continuous‐response method of measuring and modeling VWM performance. (Left Panel) During the probe, the observer’s goal is to select the colour of the colour wheel that corresponds to the probed (highlighted) item. Note that this paradigm is the one that is used in Chapter 3. (Right Panel) Responses can be modeled according to a number of factors. Target responses are assumed to be normally distributed around the target value (T). The amount of spread in responses (error) provides a measure of precision (1/SD). In a two‐factor model, guesses are modeled as a uniform distribution across all responses. In a three‐factor model, non‐target responses are considered to be normally distributed around non‐target values (N), with the same concentration as the target responses. Adopted from Bays, Catalo & Hussain (2009).

7

that can be stored. Thus, these findings suggest that VWM is a flexible pool of resources, rather

than a system with a fixed-resolution capacity.

In addition to measuring the precision of VWM responses, it is possible to utilize

probabilistic mixture models to account for multiple aspects of performance. In an influential

study, Zhang and Luck (2008) used a two-factor mixture model to describe both the error of

responses, as well as the probability that items were correctly recalled from memory. This model

assumed that responses should fall into two categories: if an item is remembered, it should be

recalled with some amount of error around the target response; if, however, the target item is not

successfully maintained in VWM, then the responses should be random with respect to the target

colour. Thus, this model assumes that all responses should be a mixture of a normal distribution

with some amount of error (the trials on which the target was correctly recalled) and a uniform

distribution (the trials on which the target was forgotten). Using this method, the authors

demonstrated that while the probability of correctly recalling an item from memory (Pm) changes

from loads of three to six items, the amount of error (SD) in those responses does not. This

changes, however, for smaller set sizes. The amount of error (or its inverse precision) of target

responses does change from one to three items, along with a small decrease in Pm, leading the

authors to suggest that because no changes in SD are observed beyond three items, VWM can

maintain a limited number of discrete fixed-resolution representations. According to this

account, it is also possible to average memory resources across all items when the set size is

below VWM capacity, which is why precision for a small number of items is greater than that for

set sizes at or above VWM capacity (but see Bays & Husain, 2008).

The two component (Pm and precision) mixture-model has proven to be an important

tool for probing the nature of VWM representations providing insights that are unavailable from

a change-detection task, or even from examining precision alone. For example, this two-

component model has been used to identify that the probability of forgetting an item increases

with longer retention intervals, while the amount of error in those representations remains

unchanged (Zhang & Luck, 2009).Furthermore, the number of features (e.g., colour and

orientation) that need to be maintained has significant effects on the precision of responses,

while having no effect on the number of objects that can be maintained (Fougnie et al., 2010).

Despite its success, a recent study has demonstrated that an additional component may be

required for fully understanding the pattern of responses observed in continuous-response VWM

8

tasks (Bays et al., 2009). According to this study, responses that are assumed to be random

guesses in a two-component model (because they fall within a seemingly uniform distribution

relative to the target) are in fact responses in which the observer mistakenly reports the colour of

one of the non-target (uncued) items. That is, these responses are in fact not guesses, but are

instead correctly-recalled items that are reported at the wrong location (possibly indicating an

error in spatial working memory). Thus, the three-component mixture model (in which PNT

represents the probability of misremembering the target location) provides a method for

elucidating aspects of VWM performance that are unavailable with other methods, and provides

a more accurate description of performance than a two-factor model; however, the precise

constraints and limitations on representations in VWM remain unclear.

In summary, both the change-detection paradigm and continuous-response

psychophysical approaches have advantages for measuring VWM performance. Although studies

using either of these approaches have provided a mixture of evidence in favour of both fixed-

capacity “slot”-models and resource models, it has been consistently demonstrated that the

amount of information that can be stored in VWM is limited. Thus, the aim of the remainder of

this thesis is to attempt to elucidate the nature of this information-processing limit.

Evidence for Early Representation Limits in VWM

Few studies have examined whether the processing bottleneck of VWM capacity emerges

at the level of initial representation and encoding. That is, it is typically assumed that the

processing bottleneck in VWM emerges from a limitation in the available resources or “slots” to

maintain objects in memory, rather than in the initial selection, representation, and encoding of

visual information. This assertion follows from the finding that increases in the duration of the

sample stimulus beyond 500 ms tends not to have an effect on VWM capacity (Vogel et al.,

2001, 2006).The primary aim of the present thesis is to examine how early representational

bottlenecks affect VWM performance. Specifically, I will address whether competition for

processing resources in the visual system can limit the number or resolution of information

stored in VWM.

There are three lines of evidence that indicate that bottlenecks in the initial stage of

perceiving and encoding visual objects may limit the number or precision of VWM

representations. First, numerous findings indicate that VWM depends on the same neural

resources that mediate the perception of visual stimuli. Both neuroimaging and primate

9

neurophysiology studies have demonstrated convincingly that the maintenance of visual

information depends on the same brain regions that mediate the perception of that information

(Fuster & Alexander, 1971; Fuster & Jervey, 1981, 1982; Nakamura & Kubota, 1995; Miyashita

& Chang, 1988). For example, the maintenance of faces in VWM depends on activity in the

face-sensitive fusiform face area (FFA; Druzgal & D'Esposito, 2003; Postle, Druzgal, &

D'Esposito, 2003), and memory for gratings can be decoded from activity in early visual cortex

(Harrison & Tong, 2009; Serences, Ester, Vogel, & Awh, 2009).

The finding that VWM may depend on the activation of the same brain regions that are

involved in perception is consistent with the proposal by Postle (2006) that working memory is

an “emergent” property of information processing in the brain. According to this account,

working memory does not comprise a set of specialized systems, but instead is a property that

exists across sensory-, representation-, and action-related systems in the brain. Specifically,

Postle argues that working memory functions arise by recruiting these specialized perceptual and

motor systems via controlled, directed attention. Consistent with this proposal, recent studies

have called into question the role of the IPS as a specialized VWM system. That is, the

recruitment of the IPS in VWM tasks may reflect other aspects of object processing (Mitchell &

Cusack, 2008) or attention (Magen, Emmanouil, McMains, Kastner, & Treisman, 2009) that are

not specific to VWM maintenance.

Based on the evidence that VWM requires sustained activation of the same brain regions

that are involved in the initial perception and representation of those features, it follows that

factors that affect non-working memory functions (i.e., sensation, representation, and action)

may also impact the ability to encode the same information into working memory. Put another

way, if working memory does not comprise a set of specialized systems, then working memory

may not be subject to a set of specialized factors limiting its capacity (e.g., number of “slots” in

the IPS) or the resolution of its representations. Instead, the number or precision of VWM

representations may be limited by the same neurobiological factors that affect initial processing

stages, and are therefore upstream of VWM maintenance. Furthermore, even if VWM does

depend on the activation of a specialized set of regions (e.g., IPS), the number or precision of

items that can be maintained in memory should be limited by the earliest processing bottlenecks.

Therefore, if there are limits to the number of sufficiently high-resolution representations that

10

can be created by the regions that mediate the perception of visual information, this bottleneck

will place severe restrictions on the amount of information successfully maintained in VWM.

The second line of evidence that bottlenecks limiting VWM representations may emerge

prior to the maintenance stage is that VWM performance is affected by the complexity of the

comparison between stimuli. For example, using a wide range of stimuli – including coloured

squares, cubes, line drawings, letters, and Chinese characters – a study by Alvarez and Cavanagh

(2004) demonstrated that change-detection performance decreased as the complexity of the

stimuli increased. Importantly, complexity was measured by examining the search slope for

those same stimuli in a visual search task. Across object complexity, the search slope was almost

perfectly correlated with change-detection performance, indicating that mechanisms that govern

visual search efficiency affect the number of items that can be successfully stored in VWM. The

obvious conclusion that can be drawn from this finding (and the one that has been the focus of

subsequent studies) is that VWM capacity may be limited by the complexity of the items that are

being stored.

An additional conclusion from the Alvarez and Cavanagh study, however, is that the

effect of complexity on VWM storage is not arbitrary, but instead may be related to the difficulty

of encoding those stimuli and/or making comparisons between them. That is, the search rate for

a given stimulus reflects, among other things, the amount of time required to perceive and

encode those items. Consequently, the strong correlation between VWM capacity and search rate

suggests that the observed differences in VWM performance for different stimuli may reflect

differences in the difficulty of encoding and comparison. Put another way, it is possible that

fewer complex items can be stored in VWM not because of storage limitations for complex

items, but because complex items are more difficult to perceive, creating a processing bottleneck

that is upstream of VWM maintenance.

A third line of research that suggests that bottlenecks of VWM performance occur prior

to encoding comes from the observation that VWM performance is affected by the number of

competing representations in the visual scene. In other words, as the number of to-be-

remembered items increases the probability of correctly recalling any item decreases (Luck &

Vogel, 1997). While this finding has been taken as evidence that the number of items that can be

stored in VWM is limited, it may also be accounted for by considering the role of competition

for representation within limited perceptual resources.

11

Evidence for Competitive Interactions in VWM

Recent studies by Bays and colleagues (Bays et al., 2009; Bays & Husain, 2008)

demonstrated that the precision with which objects and locations are recalled from memory

showed the largest decrease between set sizes one and two. As mentioned above, precision is

measured as the inverse of the standard deviation of the response function around the target

value. Thus, as the number of items increased, there was more variation in the proportion of

responses, indicating a noisier representation. Modeling this data revealed that precision

decreased according to a power law function, indicating that the decrease in precision may be

directly related to a proportional decrease in available resources.

Other studies have similarly demonstrated that the amount of noise in a VWM

representation increases as a function of the number of presented items (Wilken & Ma, 2004),

even when the number of items is less than the “capacity” of VWM (Zhang & Luck, 2008).

These results are ambiguous, however, as they could be taken as evidence for either a limitation

in the ability to store information in VWM, or a limitation in perceptual/representational

resources. Given the well-documented finding that perceptual representations are impoverished

for multiple objects relative to a single object (Duncan, 1984, 1980), it is likely that the

presentation of multiple objects results in increased competition for limited resources prior to the

maintenance stage of VWM. Put another way, as the number of presented objects increases, the

number of items competing for limited perceptual resources increases, potentially resulting in

noisier VWM representations.

In addition to the effects of load on the precision of VWM responses, previous studies

have demonstrated that VWM performance decreases in the presence of irrelevant distractors

(Vogel, McCollough, & Machizawa, 2005; McNab & Klingberg, 2008; Fukuda & Vogel, 2009).

That is, these studies demonstrated that high-capacity subjects are better able to exclude

irrelevant distractors from VWM, as well as resist the capture of attention by irrelevant

distractors, relative to low-capacity subjects. Importantly, the low-capacity subjects typically

suffer from a filtering-cost, in which performance is impaired in the presence of distractors

relative to when no distractors are present. These findings suggest that irrelevant distractors

compete for limited VWM resources, particularly in low-capacity subjects. Although this finding

has been taken as evidence of a poor selection mechanism (Fukuda & Vogel, 2009), these

findings could similarly provide evidence that competition between items in the visual scene

12

affects the ability to encode and maintain information in VWM. In other words, the presentation

of irrelevant distractors increases the number of objects competing for representation by limited

perceptual resources, as well as memory resources. Consequently, the observed decreases in

performance in the presence of distractors may be the result of competition for early perceptual

representation. Although this competition may ultimately be overcome via selection mechanisms

(Fukuda & Vogel, 2009), these findings suggest that competition for resources may play an

important role in limiting VWM performance.

In summary, the evidence presented above suggests strongly that there are early

representation limits in the visual system that limit the amount of information that can be

accurately encoded, maintained and recalled in VWM. Moreover, numerous findings

demonstrate that VWM performance is affected by the number of competing representations.

This competition can come in the form of competition between targets (target-target

competition), or competition between targets and distractors (target-distractor competition). In

both cases, the addition of items to the visual display negatively impacts VWM performance. In

other words, every item in the visual scene competes for perceptual representations, as well as

for representation in VWM (Duncan & Humphreys, 1989). Consequently, understanding the

precise effects of competition on the ability to encode and maintain information in VWM may be

critical to understanding the nature of VWM limits. Therefore, the aim of the present thesis is to

examine what effect competition for representation has on the number or precision of

representations that can be stored in VWM. This question is largely motivated by the framework

of the biased-competition model of attention (Desimone & Duncan, 1995).

Mechanisms of Competition in the Visual System

Cells in the visual system are tuned to respond to a preferred stimulus in a particular

region of space. The region of space over which a cell will respond (the receptive field) is

smallest at the level of V1, and becomes larger at higher areas of the extra-striate cortex.

Competition arises if two objects fall within the same receptive field of a cell, as both objects

compete for that neuron’s response (Desimone & Duncan, 1995). Consequently, competing

objects interact in a mutually suppressive way, leading to suppressed responses for a stimulus

relative to when no competition exists between objects (Kastner, De Weerd, Desimone, &

Ungerleider, 1998; Kastner et al., 2001). Importantly, the effects of competition can be mitigated

via selective attention.

13

It is important to distinguish competition within a receptive field from spatial summation.

For competition to occur, two objects must fall within a single receptive field, thereby competing

for that cell’s response. In contrast, spatial summation occurs when the inputs to a cell are

averaged, thereby altering the response and receptive field properties of a given cell (Movshon,

Thompson, & Tolhurst, 1978). That is, information that is outside of the classical receptive field

can have an effect on how a neuron responds to a given stimulus, relative to when that stimulus

appears alone. Furthermore, these responses are typically facilitatory, leading to reduced

detection thresholds for simple visual features (Kapadia, Ito, Gilbert, & Westheimer, 1995;

Kapadia, Westheimer, & Gilbert, 1999). Thus, while competition occurs within the receptive

field of a particular neuron, spatial summation occurs by summing the inputs to a particular cell.

Although these processes are somewhat distinct, spatial summation may serve to mitigate the

effects of competition, as attention may alter the weight of particular inputs to facilitate the

processing of one or another stimulus (Ghose & Maunsell, 2008).

Behavioural impairments as a result of competition are well documented. For example,

reporting one feature of two objects is much more difficult than reporting two features of the

same object (Duncan, 1984). Importantly, psychophysical studies have also demonstrated that

attending to one object reduces perceptual processing of nearby objects (Mounts, 2000; Bahcall

& Kowler, 1999), an effect referred to as localized attentional inhibition (LAI; McCarley,

Mounts, & Kramer, 2004). Furthermore, these studies have suggested that competition has its

greatest impact on performance when objects need to be individuated, as opposed to just

detecting target features (McCarley & Mounts, 2007).

In many ways, the effects of competition and LAI are similar to those of crowding.

Crowding refers to the impaired discrimination in the presence of nearby contours, and is a form

of inhibitory interaction similar to that of competition (Levi, 2008). Crowding has been observed

for a wide range of stimuli, including words (Toet & Levi, 1992), orientation discrimination

(Andriessen & Bouma, 1976), and face recognition (Louie, Bressler, & Whitney, 2007).

Although debated, crowding may only occur in peripheral vision (Toet & Levi, 1992), and the

distance over which crowding can occur depends on stimulus eccentricity (Levi, 2008). Although

the mechanisms of crowding are not fully known, some authors have suggested that crowding

depends on mechanisms similar to that of competition (Nandy & Tjan, 2007). There may be

some differences between crowding and competition, however. For example, Beck and Kastner

14

(2007), demonstrated that there is less suppressive interaction for identical stimuli than for

stimuli that differ in colour and orientation; in contrast, crowding is much stronger when flankers

are identical to targets (Levi, 2008). Thus, although crowding, LAI, and competition may not

reflect identical processes, they are likely all dependent on similar mechanisms of suppressive

interactions (i.e., target processing is affected by nearby objects).

Although competition has a negative impact on numerous perceptual tasks, several

studies have demonstrated that competition in the nervous system can be resolved via top-down

and bottom-up bias signals (Desimone & Duncan, 1995; Duncan & Humphreys, 1989). These

signals have the dual benefit of (a) increasing neural responses to the attended item and (b)

decreasing neural responses to unattended items, thereby acting as a filtering mechanism

(Kastner & Ungerleider, 2001). There may, however, be an upper-limit on the amount of

competition that can be resolved at any moment. For example, although abrupt onsets capture

attention, leading to prioritization of newly presented stimuli, there may be capacity limits on the

number of items that can be prioritized. Interestingly, this capacity limit may be similar to that of

VWM (Yantis & Johnson, 1990).

Previous authors have suggested that the objects that win out under competition will gain

access to further mnemonic processing (Bundesen, 1990; Duncan & Humphreys, 1989; Kastner

& Ungerleider, 2001). Consistent with this suggestion, a recent study has provided evidence that

competition between items may limit VWM performance. Specifically, Ihssen and colleagues

(Ihssen, Linden, & Shapiro, 2010) demonstrated that change-detection performance was greater

for two sequentially presented arrays compared to simultaneously presented arrays. Thus, this

study suggests that competitive interactions between simultaneously presented stimuli limited the

number or resolution of items that could be stored in VWM. As such, I will argue in this thesis

that the representations maintained in VWM are limited by these early processing bottlenecks

owing to competition between items in the display.

As mentioned above, the recent models of VWM capacity suggest that the precision of

VWM representations decreases according to a power law as a function of the number of to-be-

remembered items (Bays et al., 2009; Bays & Husain, 2008). Although this decrease could be

viewed as a simple decrease in the distribution of a common resource pool, it is possible that the

decrease could be understood as an increase in the amount of competition between items (Bays

et al., 2009). That is, as the number of items in the display increases, there is proportionally more

15

competition for representation within a finite number of visual fields. In support of this

interpretation, a recent study demonstrated how spatial competition could account for similar

capacity limitations in object tracking (Franconeri, Jonathan, & Scimeca, 2010). Although this

data set was interpreted within the framework of centre-surround activation-inhibition

mechanisms, a competition account in which objects interact in a mutually suppressive way

would explain the data equally well.

In summary, competition is an important neurobiological mechanism that affects the

neural and behavioural evaluation of visual information. As such, competitive interactions

between items in a visual scene may limit the number or precision of the representations that can

be maintained in memory. Importantly, these effects are likely to occur at the early stages of

representation or encoding, and thus, are likely to be independent of any storage “capacity” limit.

Therefore, the aim of the present thesis is to determine whether competition between objects

affects VWM performance. If competition is an important determinant of VWM performance, a

number of predictions can be made. First, VWM performance should vary as a function of the

amount of competition between items. When competition is greatest (i.e., when many items fall

within the same receptive fields), VWM performance should be most impaired. Second, VWM

performance should decrease as a function of the number of items in the display, independent of

any storage “capacity.” Although the observation that performance decreases as a function of the

number of items has been borne out in past experiments (Barton, Ester, & Awh, 2009; Bays &

Husain, 2008), this hypothesis has not been tested directly with respects to a competition

framework. Third, if the number or precision of VWM representations are affected by

competition, then biasing responses in favour of some items should increase the probability and

precision of recalling those items, while similarly reducing performance for items which are not

given priority.

Although biased-competition is a theory of visual attention, it is important to emphasize

that the present research is not simply aimed at examining interactions between VWM and

attention. Instead, the studies presented in this thesis are aimed at exploring the physiological

mechanism of competition as a potential limiting factor in VWM performance. Attention will

ultimately play an important role in a competition-account of VWM, as attention can serve as the

mechanism for resolving competitive interactions and for determining the contents of VWM

(Desimone & Duncan, 1995; Kastner & Ungerleider, 2001). Competition, however, is considered

16

in this framework to be a neurobiological mechanism that affects all aspects of visual cognition,

including the encoding and maintenance of information in VWM.

Thesis Overview

The purpose of the present thesis is to examine whether VWM performance is mediated

by known information-processing limitations of the visual system, namely, competitive

interactions. That is, in order for information to be successfully encoded, maintained, and

retrieved from VWM, that information must first overcome early processing bottlenecks (i.e.,

those that occur prior to VWM maintenance). Thus, competition for early representation may

limit the number or precision of representations that can be maintained in VWM.

The studies presented in this thesis address two broad questions. First, how is VWM

performance affected by competition? I will address the mechanisms of VWM performance at

the group level (i.e., why performance is affected by factors such as complexity and number of

items). To examine how VWM performance is affected by competition, the studies of Chapters 1

to 5 focus on the effect of spatial separation on VWM performance, as competition between

stimuli is increased as the spatial separation between two objects decreases. Using a variant of

the one-shot change-detection paradigm, the effect of competition on VWM performance is

established for target-target and target-distractor separation (Chapter 1), and for simple and

complex objects (Chapter 2). These experiments demonstrate that increasing competition for

representation decreases change-detection performance, consistent with the competition

hypothesis. Using a continuous-response method, Chapters 3 to 5 examine the effect of

competition on the number and precision of representations in VWM, as well as on the number

and types of memory errors. Performance is examined on multiple objects relative to a single

item (Chapter 3), and for a larger number of items and locations (Chapter 4). Overall, the results

reveal that increasing competition affects the precision of VWM responses. Specifically, the

precision of responses decreased under high competition, resulting in an increase in the number

of non-target errors. The subsequent experiment (Chapter 5) demonstrates that distractors and

targets have similar effects on VWM performance, although the effect of distractors is mitigated

relative to the effects of targets.

The second question to be addressed in this thesis is how is competition for

representation overcome? Specifically, Chapters 6 through 9 test the hypothesis that competition

can be overcome via bias signals (selective attention). Using a continuous-response method, the

17

effects of bias via bottom-up sensory signals (Chapter 6) and spatial and feature cues (Chapter 7)

on the precision and number of VWM representations are examined. These studies demonstrate

that both bottom-up and top-down selection can bias the contents of VWM and mitigate the

effects of competition on responses. Also, individual-differences in perceptual representations

mediate individual-differences in VWM performance (Chapter 8), as VWM capacity correlates

significantly with the precision of a single VWM representation. Finally, the neural signals

associated with competition are examined in an ERP experiment (Chapter 9) to test whether this

activity is related to individual differences in VWM capacity. Together, the findings presented in

this thesis reveal that competition for representation significantly affects VWM performance.

Specifically, competition for neural representation affects the precision of responses, and this

competition results in an increase in the proportion of trials on which the target location is mis-

remembered. Furthermore, these studies indicate that all objects compete for representation

within capacity-limited VWM storage, but the contents of VWM can be biased by spatial or

object-based selection mechanisms. In addition, the ability to bias selection (thereby resolving

competition) is a significant predictor of VWM capacity. Together, these findings have

significant implications for our understanding of the nature of processing limitations in visual

cognition.

18

Chapter 1: Spatial Competition Affects VWM Performance

As described in the Introduction, if multiple objects are presented in the visual display,

they interact in a mutually suppressive way and compete for neural representation (Desimone &

Duncan, 1995; Kastner et al., 1998; Kastner et al., 2001). Furthermore, the effects of these

competitive interactions are spatially graded. That is, because the size of receptive fields in the

visual stream increases from less than 2˚ in primary visual cortex to 4-6˚ in V4 (Kastner et al.,

2001), two objects that fall within 2˚ should, on average, compete for representation in both V1

through V4, but when separated by several degrees should only compete for representation

within areas V4 and beyond. The behavioral consequences of this change in receptive field size

has been demonstrated in the form of localized attentional interference (LAI), whereby the

processing of multiple targets is most impaired when both items are contained within a very

small region of space, with performance increasing as a function of the distance between items

(Bahcall & Kowler, 1999; McCarley & Mounts, 2007, 2008; McCarley et al., 2004; Mounts,

2000; Mounts & Gavett, 2004; Mounts, McCarley, & Terech, 2007). Thus, as the distance

between targets decreases, the competition for neural representation increases, resulting in

greater impairments in the identification and representation of those stimuli.

If competition also affects the ability to encode or maintain information in VWM, similar

decreases in performance should be observed when the distance between memory stimuli is

reduced. Consequently, the present study aimed to test whether competition for neural

representation (defined as the distance between objects) has a similar effect on VWM

performance as is observed during LAI. In other words, if the distance between objects affects an

observer’s ability to attend to and perceive multiple visual objects, do these competitive

interactions similarly affect the ability to encode and maintain information in VWM?

In addition to the effects of spatial separation, there are other ways through which VWM

performance may be affected by competition between items. For example, competition for

representation should increase as a function of the number of competing elements in the visual

display. In other words, each target item added to the display reflects an additional item that is

competing for neural representation. This suggests that if competition affects VWM

performance, then performance should decrease as a function of the number of items in the

display. Evidence in support of this competition account comes from the finding that both the

19

probability of recalling an item and the precision of those representations decreases even from

one to two items (Bays & Husain, 2008; Wilken & Ma, 2004; Zhang & Luck, 2008). These

findings support the proposal that competitive interactions between stimuli in the visual field

may affect the number and precision of items that are maintained in VWM. Moreover, these

effects occur with even only two items, suggesting that competition affects VWM independent of

any capacity limitations. Thus, competition may not necessarily be restricted to local effects, but

may extend globally (across the entire visual field).

In addition to competition increasing as a function of the number of targets, evidence

suggests that task-irrelevant distractors compete for representation in VWM. Specifically, neural

measures of VWM storage increase in the presence of distractors (McNab & Klingberg, 2008;

Vogel et al., 2005), suggesting that all items in the visual scene compete for access to capacity-

limited VWM, regardless of whether or not they are relevant for the task. In other words, as more

items are added to the display (whether targets or distractors), VWM performance suffers,

indicating that this increased competition plays a critical role in creating or maintaining

representations in VWM.

In summary, the goal of the first experiment is to test whether competition for

representation affects VWM performance. Competition between items in a change-detection task

was manipulated either spatially (i.e., by decreasing the distance between items) or by

manipulating the degree of competition for capacity-limited resources (i.e., by varying the

number of elements in the display). If spatial competition is a limiting factor of VWM, then

change-detection performance should be directly related to the distance between sample items;

performance should decline when target items are close together and are competing for

representation within a greater number of receptive fields. If this spatial competition affects the

ability to create high-resolution representations of target items at early stages of visual

processing, then manipulating the distance between objects should affect change-detection

accuracy at set sizes that are both above and below the capacity of VWM. In other words, items

should compete for representation in early visual processing independent of capacity limitations

in VWM. Increasing the number of targets or distractors in the display should, however, increase

the competition for representation by capacity-limited resources, in addition to increasing the

amount of spatial competition between items. Therefore, if competition for representation affects

VWM performance, change-detection accuracy should decrease in the presence of additional

20

targets and distractors, independent of whether the number of target items is below or above

VWM capacity.

Method

Participants. Fifteen (1 male, 13 right-handed, ages: 18 – 27 years; M = 20.7) individuals

participated in this experiment. Participants were recruited from undergraduate psychology

courses at the University of Toronto and were compensated with partial course-credit or $10/hr.

All procedures were approved by the University of Toronto Research Ethics Board.

Stimuli. Participants were seated in front of the test monitor at a distance of

approximately 57 cm. Stimuli were presented using Presentation (Neurobehavioral Systems), and

displayed on a 19-inch CRT-monitor using a refresh rate of 60-Hz. The memory stimuli

extended 0.8˚ x 0.8˚ of visual angle for both circles and squares and could appear in 9 potential

colours, selected randomly without re-sampling on each trial: black (RGB: 0, 0, 0), white (255,

255, 255), light blue (0, 162, 232), dark blue (63, 72, 204), green (0, 146, 63), magenta (255, 0,

128), purple (144, 30, 120), yellow (255, 245, 0) and red (218, 37, 29).

Procedure. The task consisted of two trial types, which were presented in alternating

blocks. The participant’s task in both types of trials was to remember 2, 4, or 6 coloured memory

stimuli. Target stimuli were either squares or circles, assigned randomly to each individual. All

results are presented averaged across target types. In the filter condition, target items (e.g., 4

squares) were presented along with 2 additional distractor items (e.g., 2 circles). Participants

were instructed to remember the target items while ignoring the distractor items. In the

remember-all condition, target items were both circles and squares. That is, in the remember-all

condition, 2 of the items from the filter condition were removed from each set size, resulting in

the same number of target stimuli. In the filter condition, the number of circles and squares on

each trial were randomly assigned in pairs.

At the start of each trial, a black fixation cross was presented at the centre of the display

for 750 ms to prepare the subject for the upcoming memory sample. Immediately afterwards, the

sample array was presented for 150 ms, during which the fixation was removed from the display

to eliminate any competitive effects of the fixation. Although encoding time is known to have

important effects on perceptual and memory processes (e.g., Vogel, Woodman & Luck, 2006),

the amount of encoding time was kept short throughout this thesis to minimize the effect of eye

movements, as well as to remain consistent with the majority of previous studies. Eye position

21

was not monitored in any of the tasks. Following the sample display, a 1,000 ms delay period

was presented. After the delay, the probe display was presented until a response was made. The

probe and sample displays contained the same number of stimuli in the same spatial

configuration, with no changes to the circles:squares ratio. Participants were instructed to

indicate with a keyboard response whether any of the target stimuli changed colour between the

initial sample and probe displays (50% of trials). In the filter condition, changes could occur only

to one of the target shapes (e.g., squares). In the remember-all condition, changes were randomly

assigned to one of either the circle or square items. The next trial (beginning with the fixation

cross) was presented immediately after each response.

Critically for the purposes of the present experiment, the distance between the stimuli

was manipulated in two conditions. In the high-competition condition, stimuli were presented

centered on 9 locations on an invisible 2˚ x 2˚grid. Thus, two stimuli could be separated by 0.2˚ –

1.2˚ and were contained within 1.8˚ – 2.8˚ of visual angle. In the low-competition condition, the

grid was extended by 6˚, resulting in 2.2˚ - 5.2˚ separation between two stimuli, which were

contained within 3.8˚ - 6.8˚ of visual angle (Figure 2).

Figure 2 ‐ Stimuli and procedure used in the experiment in Chapter 1. (A) Examples of the low‐ and high‐competition displays. Stimuli are presented within a smaller area in the high‐competition condition. (B) Example of the sequence of events of the change‐detection task employed in the experiment in Chapter 1. The observer’s goal is to detect changes to the target stimuli.

Participants performed 60 trials of each instruction (filter and remember all) x

competition (high and low competition) x set size (2, 4, and 6) cell, for a total of 720 trials.

Instructions for each set of instructions (filter and remember all) were provided at the beginning

22

of each block of 60 trials. Participants were instructed to take a short break between blocks. To

familiarize participants with the procedure, and to obtain a baseline measure of VWM-capacity,

participants performed 50-trials each of a 2, 4 and 6 item change-detection task prior to

beginning the experimental task. The stimuli used were seven square stimuli (excluding magenta

and dark blue) that were 0.65˚ x 0.65˚, and were presented around the fixation on an invisible 4˚

x 4˚ grid. Capacity (K) was estimated for each task using the Pashler-Cowan formula described

above. K-estimates were all between 2 and 6 items, with an average capacity of 3.1. All subjects

performed both tasks above chance, and thus none of the participants were excluded from

analysis.

Results and Discussion

To establish whether inter-item competition affects change-detection performance,

accuracy (% hits - % false alarms) was first analyzed for the remember-all condition using a 2

(high vs. low competition) x 3 (set size) repeated-measures analysis of variance (ANOVA).

Consistent with previous studies, accuracy decreased as the number of to-be-remembered items

increased, F(2,28) = 104.3, MSE = .017, p < .001. Importantly, accuracy was also impaired under

the high-competition condition, relative to the low-competition condition, F(1,14) = 6.6, MSE =

.008, p = .02. Thus, decreasing the inter-item distance between targets had a detrimental effect on

VWM performance (Figure 3). Furthermore, no significant interaction between set-size and

competition was observed, F(1,14) = .21, MSE = .009, p = .81, indicating that the effect of

competition on change-detection performance was consistent across all set sizes.

To examine whether non-target items similarly compete for representation, a full 2 (filter

vs. remember-all) x 2 (high vs. low competition) x 3 (set size) repeated measures ANOVA was

performed. Consistent with the above results, main effects of set size and spatial competition

were observed, F(2,28) = 181,7, MSE = .02, p < .001 and F(1,14) = 14.2, MSE = .007, p = .002,

respectively. The main effect of filtering condition was also significant, F(1,14) = 26, MSE =

.008, p < .001 (Figure 3). Importantly, none of the interactions were significant, all Fs < 1.3, ps >

.28, indicating that spatial competition has a similar effect on change-detection performance

regardless of the number of targets or the presence of distractors.

The results of the present experiment reveal two important findings. First, accuracy

decreased as the number of stimuli competing for representation was increased. That is,

increasing the number of sample items, or presenting the target items along with distractors, both

23

significantly reduced change-detection accuracy. Second, change-detection performance is

affected by the distance between sample items: when sample items are presented close together

in space, change-detection performance is impaired relative to when those items are presented

further apart. This was true for all set sizes, indicating that spatial competition has an important

effect on VWM accuracy, independent of capacity. These findings are consistent with

behavioural evidence that suppressive interactions have a spatially graded effect on the selection

and identification of multiple targets (Mounts, 2000; Mounts & Gavett, 2004). In other words, as

memory items are presented over a smaller region of space, they compete for representation by

more neurons (i.e., those with small receptive fields). Thus, this finding provides strong support

for the hypothesis that competition for representation affects VWM processes.

This novel finding that spatial competition affects VWM independent of capacity

limitations suggests that the typically observed effects of set size on change-detection

performance may also be attributed, at least in part, to the effects of competition. That is, as more

items are added to the display, each item competes for representation within a finite number of

visual fields. Thus, the competition account may provide an alternative explanation to the finding

that the fidelity of VWM representations decrease as a function of set size (Barton et al., 2009;

Bays & Husain, 2008); rather than competing for a limited pool of memory resources, each

Figure 3 ‐ Average accuracy observed in the remember‐all condition in the experiment in Chapter 1. Accuracy was higher overall in the low spatial‐competition condition. Error bars denote within‐subject 95% confidence intervals (CIs).

24

additional item competes for neural representation at the earlier stages of perception or encoding.

This increased spatial competition would therefore likely result in a more noisy and less precise

representation maintained downstream in capacity-limited VWM.

To summarize, the absence of any observed interactions of spatial competition with set

size or filtering condition indicates that items in the visual scene compete for representation

irrespective of the number of items in the display. Importantly, because these effects are

observed with even two items, it precludes any possibility that decreases in performance are the

result of constraints placed on storage capacity limits. These results are consistent with findings

that multiple objects compete for early perceptual representation (Desimone & Duncan, 1995;

Kastner et al., 2001), in addition to competing for capacity-limited VWM resources. In other

words, spatial competition for representation likely occurs upstream of VWM maintenance, and

may affect the ability to encode high-resolution representations. Consequently, the next chapter

will address whether competition similarly affects VWM performance for simple and complex

objects.

Figure 4 ‐ Average accuracy as a function of filtering and spatial competition condition presented in Chapter 1, collapsed across set size. Accuracy was higher overall in the low‐competition condition, as well as in the remember‐all condition. Error bars denote within‐subject 95% confidence intervals.

25

Chapter 2: Competition Affects VWM for Complex More than Simple

Objects

The results of Chapter 1 indicate that reducing the distance between items (thereby

increasing competition for representation) impairs change-detection accuracy. These results

support the hypothesis that spatial competition is an important limiting factor in the formation of

visual representations, including those that are encoded and maintained in VWM. That is, not

only does the distance between objects affect the ability to attend to and identify objects (e.g.,

Duncan, 1984; Mounts, 2000; Mounts & Gavett, 2004), it also affects the ability to create and

maintain representations in VWM. Importantly, the absence of any interaction with set size

indicates that these effects are likely not the result of capacity limitations in VWM storage;

instead, spatial competition likely affects the earliest stages of representation (i.e., encoding or

consolidation). That is, competition is likely to affect the resolution of VWM representations,

rather than imposing capacity constraints on the number of items that can be stored.

If spatial competition affects VWM performance at the stage of initially perceiving and

identifying object stimuli, rather than at the stage of maintenance, then objects that are more

difficult to represent and perceive should be more affected by manipulations that increase

competition (i.e., the distance between objects). For example, as mentioned in the Introduction, a

study by Alvarez and Cavanagh (2004) demonstrated that the number of items that could be

stored in VWM was largely predicted by the time required to represent, encode, and compare

those objects in a visual search task. As such, it is possible that the typical finding of lower

VWM capacity estimates for more complex items may not be the result of storage limitations,

but instead may be due to the nature of these complex items: complex items are more difficult to

perceive, resulting in less precise representations maintained downstream in VWM (Awh et al.,

2007). Consequently, the goal of Chapter 2 is to examine the effects of spatial separation on

simple and complex objects. If VWM performance for complex objects is impaired because they

are more difficult to initially perceive and identify, then increasing the amount of competition

between sample items should have a greater effect on change-detection performance for complex

objects than for simple objects.

Why should the distance between objects affect complex objects more than simple

objects? If complex objects are comprised of a conjunction of features, then it is unlikely that

26

these objects can be represented by neurons in early visual cortex alone, as early visual cortex

codes for only simple features (i.e., colour, line orientation, etc). Instead, in order to represent

more complex stimuli, simple features are pooled in regions of higher visual cortex. Thus, the

earliest regions at which these complex items could be represented as bound objects would be

further along the visual pathway where neurons have larger receptive fields. Consequently,

decreasing the spatial separation between complex objects should have a detrimental effect on

performance, since these would be competing for representation in the entirety of the neurons

that could code for them. In contrast, simple objects could be uniquely represented by neurons

with smaller receptive fields in earlier stages of visual processing. Thus, simple objects should be

less affected by reducing the distance between objects, as they would likely be able to maintain

some unique representation in areas of early visual cortex (where there are smaller receptive

fields).

In addition, if complex objects are comprised of shared features, then they will compete

for the representation of those neurons that code for those features. This means that competition

should be even more difficult to resolve for these complex objects with shared features than for

simple objects comprised of unique features. Thus, when complex objects are presented close

together they will be subject to spatial competition (competition for representation within

receptive fields) as well as feature-based competition. In contrast, simple objects, when

presented close together, should only exhibit the effects of spatial competition, resulting in a less

dramatic impairment. Therefore, the second experiment examines whether reducing the space

between sample items has a greater effect on complex objects than on simple objects.

Method

Participants. Fifteen (2 male, 14 right-handed, ages: 18 – 36 years; M = 21.3) individuals

participated in the experiment. Participants were recruited from undergraduate psychology

courses at the University of Toronto and were compensated with partial course credit or $10/hr.

Stimuli and Procedure. The procedure was similar to that of Chapter 1. Following a 750

ms central fixation, a memory sample of 2, 4 or 6 items was presented for 150 ms, and

participants were instructed to remember as many of the items as possible over a 1,000 ms delay.

After the delay, the probe display was presented, and participants indicated whether any of the

items in the probe display differed from those presented in the initial sample display (50% of

trials). The probe display contained the same number of objects as the sample display.

27

In alternating blocks of 50 trials, participants were presented with simple (coloured

squares) or complex (shaded cubes) stimuli (Figure 5). The simple stimuli consisted of the same

set of coloured squares used in Chapter 1. Cube stimuli were composed of three sides, one each

of black, grey and white. As with the simple stimuli, a total of 9 different cube stimuli were used.

To make the cube details easier to identify, the size of the stimuli were increased to

extend a maximum of 1.2˚ x 1.2˚ visual angle. As in the previous experiment, the distance

between sample stimuli was manipulated. In the high-competition condition, stimuli were

presented centered on an invisible 9 x 9 grid extending 2.8˚ x 2.8˚. Thus, the centre-to-centre

distance between objects was 0.2˚ - 1.6˚, and two objects could be contained within a total area

of 2.6˚ – 4.0˚. In the low-competition condition, the size of the grid was extended to 8.4˚ x 8.4˚,

with stimuli separated by 3.0˚ - 7.2˚ and contained within an area of 5.4˚ – 9.6˚.

Participants performed 60 trials of each set size (2, 4, and 6) x object complexity (simple

and complex) x competition (low and high) cell, for a total of 720 trials. As with the experiment

in Chapter 1, participants first performed 150 trials of practice using simple stimuli. K-estimates

ranged from 2.7 – 5 items (M = 3.6).


Analyzing accuracy (% hits - % false alarms) for the simple stimuli, repeated measures

ANOVA revealed a significant main effect of set size, F(2,28) = 86.48, MSE = .012, p < .001,

along with a moderately significant effect of competition, F(1,14) = 3.83, MSE = .004, p = .07.

No significant interaction was observed, F(2,28) = 2.1, MSE = .006, p = .15. These results

largely confirm the effects of Chapter 1, with a slightly diminished effect of competition (see

discussion below).

Figure 5 ‐ Examples of complex stimuli used in Chapter 2. For the simple stimuli condition, shaded cubes were replaced with coloured squares.

28

Examining accuracy for the full comparison of the 2 (object complexity) x 2 (spatial

competition) x 3 (set size) ANOVA, significant main effects of complexity, F(1,14) = 336.7,

MSE = .02, p < .001, spatial competition, F(1,14) = 15, MSE = .009, p = .002, and set size,

F(2,28) = 328.1, MSE = .007, p < .001, were observed. A significant interaction between object

complexity and set size was observed, F(2,28) = 7.77, MSE = .01, p = .002 (Figure 6).

Importantly, a moderately significant interaction between complexity and spatial competition

was also detected, F(1,14) = 3.23, MSE = .009, p = .09. Thus, collapsing across set sizes to

compare the simple effects, reducing the distance between objects had a significant effect on

complex objects, t(14) = 3.24, SEM = .024, p = .006, while the effect of the spatial competition

manipulation on the simple stimuli was much smaller, t(14) = 2, MSE = .014, p = .07 (Figure 7).

No other significant interactions were observed, Fs > 1.2, ps > .26.

The results of the present experiment reveal two important findings. First, the results

confirm the effects observed in Chapter 1, demonstrating that reducing the distance between

objects impairs change-detection performance. This observation is extended in the present

experiment to complex as well as simple visual objects. Second, this effect was found to be

larger for the complex stimuli (shaded cubes) than for the simple stimuli (coloured squares). As

outlined above, this effect is predicted by a competition account, as complex objects should

compete for the representation of shared features, and can only be fully represented by neurons

with larger receptive fields (i.e., those in higher visual cortex). This additional level of

competition likely accounts for the observed interaction between competition and complexity.

Figure 6 ‐ Accuracy for simple and complex objects, averaged across competition condition. Error bars denote 95% within‐subject CIs.

29

Although the effect of spatial competition on simple objects was mitigated when

compared with the results of the first experiment, this difference is likely the result of the

changes to the size of the stimuli and the area over which stimuli were presented. That is,

increasing the size of the stimuli resulting in the high-competition configuration to be presented

over a larger area than that of Experiment 1. Thus, since objects could only compete for

representation in regions with larger receptive fields (those large enough to encompass both

objects), this increase in sample area likely resulted in overall less competition relative to that of

the first experiment.

Previous studies have suggested that fewer complex objects can be stored in VWM.

Consistent with this account, we observed lower overall change-detection performance for

complex objects than for simple objects, as well as a significant interaction between set size and

object complexity. While it is possible that these findings support the proposal that fewer

complex objects can be maintained in VWM, the absence of any interaction between spatial

competition and set size argues against this account. That is, increasing the competition between

sample items had a greater effect on complex objects than on simple objects, regardless of

whether the number of simple or complex objects was below or above the capacity of VWM.

This suggests that the effect of spatial competition on change-detection accuracy was

independent of the number of simple or complex objects that can be stored in VWM. Instead,

these results support the hypothesis that competition affects VWM performance upstream of

VWM maintenance, altering the ability to establish high-fidelity visual representations.

Figure 7 ‐ Accuracy for simple and complex objects under low and high competition, averaged across set size. Error bars denote within‐subject 95% CIs.

30

In summary, the results of the experiments in Chapters 1 and 2 indicate that increasing

the competition between items by reducing the distance between them significantly impairs

VWM, as revealed by a reduction in change-detection accuracy. Furthermore, the competition

hypothesis could also account for the effects of set size (Chapters 1 and 2), distractors (Chapter

1) and stimulus complexity (Chapter 2) on change-detection performance. That is, all of these

manipulations increase the degree of competition for representation within the visual system.

Consequently, these findings suggest that competition for representation may play a critical role

in determining the contents of VWM. Importantly, the findings presented in Chapters 1 and 2

strongly suggest that competition affects the front-end of VWM performance by limiting

information at the perceptual, encoding stages, rather than imposing capacity limitations that

affect the number of items that can maintained in VWM. Consequently, Chapters 3 – 5 use a

psychophysical approach to determine the precise effect of spatial competition on the number

and precision of VWM representations.

31

Chapter 3: Competitive Interactions Increase Recall Errors

The results of Chapters 1 and 2 demonstrate that change-detection performance is

impaired when objects are presented over a small region of space, and are therefore competing

for representation in a larger number of receptive fields. These effects parallel the behavioral

impairments observed in the selection and perception of multiple visual stimuli (Bahcall &

Kowler, 1999; Duncan, 1984; Mounts, 2000), suggesting that spatial competition plays a critical

role in many aspects of visual cognition.

The results of Chapters 1 and 2 suggest that competition affects change-detection

performance independent of capacity limitations, indicating that it likely affects performance

upstream of VWM maintenance during initial perception or encoding, before information is

consolidated into VWM. This hypothesis is consistent with findings that spatial competition

affects performance at the level of early perceptual and representational stages, as even the

identification of simple objects is impaired when objects compete for representation (Bahcall &

Kowler, 1999; Duncan, 1980, 1984; Mounts, 2000; Mounts & Gavett, 2004). Thus, these results

strongly suggest that competition affects the precision of VWM representations, rather than the

number of items that can be maintained. Consequently, the goal of the present experiment is to

use a psychophysical approach to assess the precise effect of competition on the number and

precision of representations stored in VWM.

Precision is a measure of error defined as the inverse of the standard deviation of error of

all responses (Bays & Husain, 2008). Thus, since low-resolution representations are likely to be

reported with more error than high-resolution representations, precision can be used as a proxy

for measuring resolution. Importantly, multiple studies have demonstrated using a recall task that

the precision of VWM responses decreases as a function of the number of to-be-remembered

items (Bays & Husain, 2008; Wilken & Ma, 2004). Although this finding may indicate that

precision is affected by competition, it is unclear whether this competition is spatial in nature

(i.e., objects compete for representation with receptive fields) or whether it is related only to

competition for cognitive resources (i.e., memory capacity).

Examining precision alone, however, does not reveal the number of items that are

correctly recalled during the task. That is, examining trial-to-trial variability alone makes no

assumptions about how an observer might respond if they forgot the target and simply guessed.

Accordingly, Zhang and Luck (2008) used a two-factor mixture model to describe both the error

32

of responses, as well as the probability that items were correctly recalled from memory. In this

study, participants had to select the colour of the target item from a colour wheel. The analysis

used by Zhang and Luck to model the responses in this task assumed that if the target item is not

successfully maintained in VWM, then the responses should be random with respect to the target

colour. Thus, this model holds that all responses should be a mixture of a normal distribution

with some amount of error (the trials on which the target was correctly recalled) and a uniform

distribution (the trials on which the target was forgotten). This model was updated by Bays and

colleagues (Bays et al., 2009), who demonstrated that some of the responses that are assumed to

be random guesses in a two-component model (because they fall within a seemingly uniform

distribution relative to the target) are in fact responses in which the observer mistakenly reports

the colour of one of the non-target (uncued) items (Figure 1). That is, these responses are not

“guesses,” but are instead correctly-recalled items that are reported at the wrong location. These

non-target responses, therefore, indicate that the reported item was in fact contained in memory,

but possibly reflect errors in spatial working memory (misremembering the target location).

Thus, the three-component model not only provides a second measure of error that may be

important to understanding the entirety of VWM performance, but it also provides a more

accurate or complete picture of the number of items correctly recalled.

Accordingly, the goal of the present experiment was to use the three-component model

described by Bays and colleagues (Bays et al., 2009) to examine the effect of competition on the

precision of VWM representations, as well as on the number of items correctly and incorrectly

recalled. Building on the findings of Chapters 1 and 2, competition was manipulated by varying

the distance between sample items in a continuous-response task similar to that of previous

studies (Bays et al., 2009; Wilken & Ma, 2004; Zhang & Luck, 2008). If competition does affect

the perception and identification of high-resolution representations that are to be encoded and

maintained in VWM, then competition should affect only the precision of those representations,

rather than the number of items correctly recalled. If instead competition imposes constraints on

the storage of items in capacity-limited VWM, then increasing competition between objects

should affect the number of items correctly recalled, as well as potentially decrease the precision

of those representations.

33

Method

Participants. Eighteen undergraduates (4 male, 17 right-handed, ages: 19 – 36; M = 22)

from the University of Toronto participated in this experiment for partial course credit. All

participants had normal or corrected-to-normal vision, and had self-reported normal colour

vision.

Apparatus. Participants were seated in front of a 19-inch CRT monitor at a distance of

approximately 57 cm. The screen resolution was 1280 x 960 and the refresh-rate was 60 Hz.

Stimuli were generated and presented using Matlab and the Psychophysics Toolbox 3 extensions

(Brainard, 1997; Kleiner et al., 2007).

Stimuli and Procedure. Each trial began with the presentation of 1 – 3 coloured squares

for 100 ms, presented around a central black fixation cross 1.2˚ x 1.2˚. Following a 1,000 ms

blank delay containing only the fixation, the probe display was presented. During the probe, the

locations of the sample items were indicated by black placeholder boxes. One of the sample

locations was identified as the target location by displaying a placeholder with a line width of 8

pixels. The remaining non-target locations were indicated with a line width of 4 pixels. In

addition to the memory placeholders, a colour-wheel containing all possible sample colours was

presented surrounding the test area (Figure 1).

Participants were instructed to remember as many of the sample items as possible over

the delay period, and to indicate the colour of the target square by clicking on the location on the

colour-wheel that most closely matched the colour of the cued item. During instructions,

emphasis was placed on accuracy, and participants were instructed to make multiple responses if

the selected colour did not match closely enough to the remembered colour of the target item.

Each time a colour was selected, the placeholder at the target location was replaced with a

coloured square that matched the shade of the participant’s response. The trial ended when the

subject pressed the keyboard spacebar, and the final selected colour was taken as the

participant’s response for that trial. Participants were also instructed to guess a random colour if

they could not accurately recall the colour of the target item, and to actively avoid selecting the

colour of a non-target item if they knew that it was not the colour of the probed location. Each

trial was followed by a 500 ms ITI, with only the fixation cross displayed.

Memory stimuli were selected from an array of 252 colours, each of which corresponded

to an equally-spaced point on the HSV (hue – saturation –value) colour wheel. Saturation and

34

value (brightness) were held constant at the maximum value. Thus, the intensity and brightness

of the stimuli remained largely constant regardless of particular stimulus hue. For the probe

display, the colour-wheel contained all 252 colours arranged on an annulus with an outer radius

of 8.3˚ and an inner radius of 6˚. The colours of all target and sample stimuli were selected

randomly, with the constraint that any two sample colours had to be separated by a minimum of

20 values on the colour wheel (28.5˚).

Sample squares were 1.2˚ x 1.2˚, and were presented at one of 18 locations on an

invisible circle with a radius of 5.4˚. To examine the effects of competition, the distance

between sample stimuli was manipulated. In the high-competition condition, two stimuli were

located at adjacent locations, while in the low-competition condition, two empty locations

occupied the locations between two stimuli. Thus, for a set size of two items, high-competition

stimuli were separated by a centre-to-centre distance of 1.9˚, and were contained within 3.1˚,

while low-competition stimuli were separated by a distance of 5.4˚, and were contained within

6.6˚. For the set size 3 condition, the same configuration of stimuli was used, such that each

display contained two stimuli in a high-competition configuration, with the third item located in

a low-competition configuration relative to one of the other stimuli (Figure 8). Thus, the total

area over which three items were presented was 6.6˚, as in the Load 2 condition. When two items

were presented, the location target square was assigned randomly. For set size three, the target

item in the high-competition was randomly sampled from either of the two high-competition

items.

Participants performed a total of 80 trials of each condition in blocks of 50 trials. Prior to

beginning the experimental task, participants performed 50 trials each of a 2-, 4- and 6-item

change-detection task to establish baseline VWM capacity and to become familiarized with the

Figure 8 ‐ Examples of the low‐ and high‐competition conditions used in the experiment in Chapter 3.

35

general procedure. K-estimates ranged between 1.7 – 5.0 (M = 3.2). This practice task was

followed by instructions for the experimental task, and a practice of 5 – 15 trials. The length of

the entire experiment was approximately 1 hour.

Analysis. The analysis was performed using the method of Bays and colleagues (Bays et

al., 2009). For each trial, the angular distance between the selected colour value and the target

colour value was calculated to obtain a measure of error. For each participant and condition,

precision was calculated as the inverse of the SD of responses, as in Bays in Husain (2008). The

SD was adjusted for a circular response distribution, as well as corrected for guessing as in Bays

et al. (2009).

In addition, a probabilistic model of responses was calculated according to the method of

Bays and colleagues (Bays et al., 2009). The model assumes a mixture of three components that

are each fit to the distribution of responses for each participant and condition: target responses

(PT), which are normally distributed around the target value using a circular analogue of the

Gaussian distribution (the Von Mises distribution); non-target responses (PNT), which are drawn

from a normal distribution of responses around the non-target value with the same SD as the

target responses; and random guesses (Pg) which have a uniform distribution across all potential

values. This method differs from the method of Zhang & Luck (2008), in that the proportion of

responses which are assumed to be guesses will be smaller, as some of these responses will be

extracted by PNT. Values for each parameter were obtained separately for each participant and

condition using maximum likelihood estimation according to the method of Bays et al. (2009).

Hypothesis tests for experimental manipulations were performed using repeated-

measures analysis of variance (ANOVA) and t-tests for the maximum likelihood estimates of

each of the measures obtained from each subject and condition. Effects of set size were

examined across loads 1 – 3, while the effects of spatial competition (low vs. high) were

considered only for loads 2 and 3.

Parameter estimates for each observer were examined for outliers prior to analysis. To

exclude the possibility that participants adopted a strategy of attending to only the low- or high-

competition items in the load 3 configuration, differences in accuracy between the two types of

probes were examined for outliers. Two individuals demonstrated a greater than 20% change in

PT between the two conditions, and were therefore eliminated from analysis. The SD of the

difference in accuracy for the remaining 16 participants was 6%.

36


The aim of the present experiment is to examine the effect of competition on the

precision of all responses, as well as on the proportion and error of target and non-target

responses. For example, if spatial competition affects the fidelity of VWM representations, the

distance between items should have a significant effect on the precision of responses.

Overall precision. To determine the effect of competition on the precision of VWM

responses, precision was first examined using the method of Bays and colleagues (Bays et al.,

2009; Bays & Husain, 2008) by calculating the reciprocal of the SD of error of all responses.

Averaging across competition conditions, precision of responses decreased as the number of

memory items increased, F(2,30) = 63.7, MSE = .28, p < .001, including between one and two

items, t(15) = 6.4, SEM = .217, p < .001. In addition, the precision of responses relative to the

proportion of resources available per item was examined by dividing the precision of each

condition by the precision obtained for a single item. Using this method revealed that precision

decreased according to a power-law function (Figure 9 – top left). This effect was observed for

both low- and high-competition conditions. Thus, these results confirm the findings of Bays and

colleagues (Bays et al., 2009; Bays & Husain, 2008) suggesting that as the number of to-be-

remembered items increases, the precision of any single memory representation decreases

proportionally according to a power law.

Examining the effect of competition revealed that while precision was reduced by

increasing the number of items from two to three, F(1,15) = 62.04, MSE = .13, p < .001,

reducing the distance between memory items had no observable effect on precision, F(1,15) =

.514, MSE = .071, p = .48. Furthermore, no interaction was observed between set size and

competition, F(1,15) = .366, MSE = .10, p = .554. Thus, contrary to the hypothesis that

competition affects the precision of representations, the fidelity of VWM was unaffected by the

distance between competing objects.

Target precision (three-component model). To examine the precision of target responses

alone, the SD of the normal distribution obtained from the three-component mixture model was

examined (Zhang & Luck, 2008). That is, this measure reflects the precision of only those items

that are correctly reported (PT). As with the precision of all responses, a significant effect of set

size was found when collapsing across competition condition, F(2,30) = 22.6, MSE = .002, p <

.001. Thus, each sample item added to the display increased the error of target responses,

37

including between one and two items, t(15) = -7.38, SEM = .009, p < .001. Neither the effect of

competition, F(1,15) = 1.45, MSE = .003, p = .25, nor the interaction with set size, F(1,15), MSE

= .003, p = .37, were significant. Thus, the precision of target representations was affected by the

number of sample items, but not by the distance between items.

Figure 9 ‐ Measures of relative precision (top left) and mean maximum likelihood estimate values for the three factors used to model responses in Chapter 3. Error bars denote within‐subject SEM. (Top Left) As in previous experiments, relative precision decreased according to a power‐law function. The fit lines represent the power‐law fit for both low and high competitions. (Top Right) The proportion of target responses decreases as a function of the number of sample items, with no difference between high‐ and low‐competition conditions. (Bottom Left) The proportion of non‐target responses increases with set size, and is greater in the high‐competition than in the low‐competition condition. (Bottom Right) The proportion of guesses increases with set size, and is unaffected by the distance between objects.

38

Proportion of target responses (three-component model). To determine whether

competition affects whether sample items are correctly recalled, the maximum likelihood

estimates of the proportion of target responses (PT) from the mixture model were examined.

Collapsing across competition conditions, a significant effect of set size on PT was observed,

F(2,30) = 27.5, MSE = .004, p < .001. As with precision, this effect was found to be significant

between one (M = .97) and two (M = .92) items, t(15) = 3.11, SEM = .015, p = .007, indicating

that increasing the number of items in the display has a consistent effect on VWM performance,

even when the number of sample items is well within the four-item “capacity” of VWM (Figure

9 – Top Right). Examining the effects of competition for loads two and three revealed a

significant main effect of set size, F(1,15) = 39.5, MSE = .005, p < .001, but no effect of spatial

competition, F(1,15) = .41, MSE = .002, p = .53, and no interaction, F(1,15) = .96, MSE = .001,

p = .34. Thus, while increasing the number of items in the display reduces the probability of any

item being correctly recalled, the distance between objects had no effect on target accuracy.

Proportion of non-target responses (three-component model). An important additional

question is whether spatial competition between items affects the ability to correctly recall the

location of each item. Thus, using the three-component model of Bays and colleagues (Bays et

al., 2009), the effect of competition on the number of non-targets reported (PNT) was examined.

Confirming previous findings, PNT increased as the number of to-be-remembered items

increased, F(1,15) = 12.2, MSE = .001, p = .003. Importantly, the distance between items (i.e.,

competition) also had a significant effect on PNT, F(1,15) = 8.4, MSE = .002, p = .011. Thus,

more non-target responses were made under high spatial competition (M = .05) relative to when

the sample objects were located further apart (M = .019). In addition, a significant interaction

between set size and competition was observed, F(1,15) = 5.8, MSE = .001, p = .029. That is, the

effect of competition was stronger at the larger set size (Figure 9 – Bottom Left).

Proportion of guesses (three-component model). The proportion of guesses (PG) also

increased with set size, F(2,30) = 13.1, MSE = .004, p < .001, collapsed across competition

condition. In contrast, neither the effect of competition, F(1,15) = 3.1, MSE = .003, p = .098, nor

the interaction were significant, F(1,15) = .816, MSE = .003, p = .38 (Figure 9 – Bottom Right).

Thus, although competition had a significant effect on the number of non-target responses,

competition between items had no effect on the proportion of guesses.

39

Overall, the results of this experiment confirm the results of Chapters 1 and 2 by

demonstrating that competitive interactions affect VWM performance, and extend this finding

beyond those available through high-threshold approaches. The results are mixed, however,

neither confirming nor refuting the hypothesis that competition affects the resolution of VWM

representations, rather than the number of items stored. As spatial competition had no effect on

the proportion of target responses (PT), these results suggest that competition acts independently

of VWM maintenance. That is, the number of items that were correctly reported was unaffected

by changes in competitive interactions, indicating that the distance between objects has no effect

on the ability to maintain or recall information from VWM. Competition also had no observable

affect on the precision of target responses (either the precision of target responses, or overall

precision), indicating that the items that were stored in memory were stored with a similar

resolution irrespective of the amount of spatial competition between items.

Although both the number and precision of targets reported was unaffected by the

competition manipulation, more non-target responses (PNT) were made under high-competition

compared to low-competition. Thus, these results are consistent with the effects observed in

Chapters 1 and 2, confirming that competitive interactions do affect VWM performance. It is

unclear, however, why spatial competition affected the PNT without affecting target accuracy or

precision. One possibility is that the ability to correctly recall the locations of each sample item

was affected by competition. That is, Xu and Chun (2009) have argued that there are two

important processes necessary to both VWM and object recognition: each item must first be

individuated, followed by a second stage during which the features of those objects must be

enhanced and encoded. Accordingly, when two items compete for neural representation within a

receptive field, it is possible that it is more difficult to individuate each item and subsequently

bind the correct features to the right location. Therefore, if competition affects the ability to

perform these processes of individuation and feature binding, this may impair the ability to recall

which item was presented at the target location, leading to an increase in the number of non-

target responses.

Another possible explanation for why spatial competition affected only the proportion of

non-target responses may be that mutually-suppressive interactions between high-competition

items may decrease the strength (or gain) of the memory signal, thereby increasing the amount of

decision errors. That is, when competition is low, the gain on the memory representation may be

40

higher, reducing the decision threshold for reporting the low competition item. When spatial

competition is high, however, the signal corresponding to the memory representation may be

lower. In other words, low-competition items may be easy to remember as well as easy to reject

when another item is forgotten, since the strength of the representation is high. On the other

hand, while high-competition items may be remembered just as easily, they may be more

difficult to reject under conditions of uncertainty, resulting in a greater number of incorrect

responses towards those items.

In summary, the results of the present experiment neither confirm nor refute the

hypothesis that spatial competition affects the resolution of VWM representations, rather than

the number of items stored, since both precision and PT were unaffected by the distance between

items. The finding that non-target responses ( PNT) were affected by competition, particularly at

larger set sizes, does confirm that competition affects VWM performance. One possibility is that

the tested distances between items was not large enough to observe effects of competition on the

number or precision of memory representations. Furthermore, it is possible that below the

“capacity” of VWM, the available remaining resources can mitigate the effects of competition on

the resolution of those representations. Accordingly, in Chapter 4, the effect of competition was

tested using a set size of four items that were presented over a larger range of inter-item

distances.

41

Chapter 4: Competitive Interactions Affect the Precision of VWM

Representations

The results of Chapter 3 indicate that competition increases the number of non-target

responses (errors) in a recall task, but that the proportion of target responses, as well as the

precision of target responses, remained unaffected. It is possible, however, that competition was

not tested over a large enough area to observe effects on PT or precision. That is, previous studies

have demonstrated that behavioural effects of competition are spatially graded, as the ability to

select and identify multiple objects improves as the distance between objects increases (Bahcall

& Kowler, 1999; Mounts, 2000; Mounts & Gavett, 2004); consequently, increasing the distance

between objects even further may reduce the effects of competition, resulting in stronger

differences. Furthermore, the effects of competition on PNT observed in Chapter 3 were larger at

set size 3 than at set size 2, indicating that competition may have a greater effect on responses at

larger set sizes. Accordingly, the goal of Chapter 4 is to examine the effects of competition

between four objects presented over a larger range of inter-item distances.

Method

Participants. Fourteen University of Toronto undergraduates (1 male, 13 right-handed,

ages: 18 – 23 years; M = 20.0) participated in this experiment for partial course credit.

Figure 10 ‐ Example schematics of the stimulus configurations in Chapter 4. Stimuli are not drawn to scale.

42

Stimuli and Procedure. The procedure was identical to that of Chapter 3, with the

following exceptions. On each trial, participants were presented with four coloured sample

stimuli for 150 ms, randomly presented in one of four conditions. Thus, the set size in this task

was held constant. Critically, the distance between objects was manipulated, as the stimuli were

presented with increasing separation between items (Separation 1 – 4). That is, all four sample

items were presented at adjacent locations in Separation 1, and with 1 – 3 empty locations

between items in Separations 2 – 4, respectively (Figure 10). The linear distances between items

and total area are presented in Table 1. As in the experiment in Chapter 3, the goal for

participants was to remember as many of the items as possible over a 1,000 ms delay period, and

to select from a colour-wheel the colour of the probed item. Responses were analyzed using the

same method as in the previous chapter, calculating the precision of all responses (1/SD), as well

as obtaining maximum likelihood estimates for each of the three factors in the model described

by Bays and colleagues (Bays et al., 2009). The colours and locations of each of the sample

patters were assigned randomly, with the constraint that any two sample colours had to be

separated by a minimum of 20 values on the colour wheel (28.5˚). Participants performed 50

trails each of a 2-, 4-, and 6-item change-detection task prior to beginning the experiment. K-

estimates in this task ranged between 1.6 and 4.8 items (M = 3.4). All participants performed

above chance and were included in analysis.

Table 1. Distances between stimuli for Experiment in Chapter 4.

Centre-to-Centre Distance Total Area

Minimum Maximum Minimum Maximum

Separation 1 1.9˚ 7.5˚ 3.1˚ 8.7˚

Separation 2 3.7˚ 10.6˚ 4.9˚ 11.8˚

Separation 3 5.4˚ 10.8˚ 6.6˚ 12˚

Separation 4 7.5˚ 10.8˚ 8.7˚ 12˚

43


As in Chapter 3, the precision of all responses were examined, as were the maximum

likelihood estimates for each of the values in the three-factor model. No effect of spatial

separation on any measure of precision was observed in Chapter 3, possibly because spatial

separation between stimuli was insufficient.

Overall precision. Contrary to the results of Chapter 3, a repeated measures ANOVA

revealed a significant effect of separation on the precision of all responses, F(3,33) = 2.9, MSE =

.04, p = .05 (Figure 11). This effect was largely due to the improvements observed at Separation

4, although the linear contrast was nearly significant, F(1,11) = 4.7, p = .053. Thus, increasing

the distance between items significantly reduced the amount of error in responses, indicating a

higher-fidelity representation in VWM.

Target precision (three-component model). Examining the precision of these target

representations revealed that there was no effect of competition on the SD of target responses,

F(3,33) = .94, SEM = .02, p = .43. Thus, although competition affected the precision of all

responses, targets were reported with similar precision independent of the amount of spatial

competition between them. This contradiction is discussed in detail below.

Proportion of target responses (three-component model). Examining the mean values

obtained from the three-factor model revealed that increasing the separation between items had

no effect on PT, F(3,33) = .46, MSE = .01, p = .71. That is, consistent with the effects of Chapter

3, spatial competition had no effect on the number of correctly reported targets.

Proportion of non-target responses (three-component model). The distance between

memory items did have a significant effect on the number of non-target items reported, F(3,33) =

3.42, MSE = .006, p = .028. That is, consistent with the effects presented in Chapter 3, more non-

target errors were made when competition was greater. Furthermore, the effect of separation on

PNT demonstrated a significant linear contrast, F(1,11) = 8.09, p = .016. Thus, these results

extend the findings of Chapter 3 by demonstrating that the effect of competition on the number

of incorrectly reported non-targets is linearly proportional to the distance between items.

Proportion of guesses (three-component model). Finally, examining the effect of

competition on PG indicated that the proportion of guesses remained unaffected by the separation

between memory items, F(3,33) = .4, MSE = .02, p = .76, just as in Chapter 3. Thus, although the

44

number of non-target responses was greater under high-competition, the number of random

responses remained unchanged.

Figure 11 – Precision, PT, PNT, and PG obtained for each separation condition in the experiment in Chapter 4, plotted by the distance between two items. Dashed lines correspond to linear contrasts. Error bars denote within‐subject SEM.

The results of the current experiment extend the findings of Chapter 3 by providing

further evidence that VWM performance is affected by competition. The results demonstrate that

while the distance between items had a significant effect on the number of non-target items

reported, the degree of competition had no effect on the number of correctly reported targets,

even when presented over a larger area. Unlike the results of Chapter 3, however, competition

did have a significant effect on the precision of responses. The finding that competition affected

the precision of responses without affecting the number of targets correctly recalled is consistent

45

with the proposal that competition affects performance independently of capacity limitations in

VWM storage. That is, competition for representation appears to affect aspects of visual

processing that are independent of the number of items that can be stored in, and accurately

recalled from, VWM.

Interestingly, although spatial competition did have a significant effect on the precision

of all responses (1/SD of all responses), the distance between objects had no effect on the SD of

the normal distribution of reported targets obtained in the mixture model. That is, when targets

were reported, they were reported with a similar amount of precision, regardless of the amount of

competition between items. How does competition affect overall precision without affecting the

resolution of target representations? Because precision represents the amount of error in all

responses (including those that are incorrect), precision may reflect many aspects of performance

that are unrelated to the ability to correctly and accurately report target items. Specifically, there

appears to be a significant relationship between precision and the proportion of non-target

responses (PNT), as these properties demonstrate similar but opposing effects (Figure 11). Thus, a

decrease in overall precision may reflect the increased tendency to report non-target items under

high competition. Accordingly, if competition biases participants to make a non-target response,

rather than respond at random, this change in response pattern may account for the decrease in

precision.

Importantly, the effect of spatial competition on PNT and precision was found to be

linearly related to the distance between objects. This provides evidence that the effects of

competition on precision and PNT are likely to occur upstream of VWM maintenance, impacting

performance at the initial perception and representation of the to-be-remembered objects. That is,

as objects are presented over a smaller area, they compete for representation within the smaller

receptive fields of early visual cortex. This increased competition results in a proportional

decrease in the ability to identify, encode, and maintain visual representations, consistent with

the effects of LAI (Bahcall & Kowler, 1999; McCarley & Mounts, 2007; Mounts, 2000; Mounts

& Gavett, 2004). Thus, overall the results suggest that competition affects the precision, but not

the number of items stored in VWM. While the resolution of targets remains relatively constant,

spatial competition likely affects the perceptual processing of sample items that leads to in an

increase in the number of non-target responses.

46

So far, the experiments presented in Chapters 3 and 4 examined the effects of competing

target information on memory performance. In the next chapter, I will examine the effects of

competing distractors on the number and precision of VWM representations.

47

Chapter 5: Multiple Distractors Affect the Precision and Number of

VWM Representations

In Chapters 3 and 4, the effect of competition on the precision and number of VWM

representations was examined. In addition to demonstrating that increased spatial competition

reduces precision and increases response errors, the results confirm the findings of Bays and

colleagues (Bays et al., 2009; Bays & Husain, 2008) that adding items to the memory array

reduces precision according to a power-law function. Furthermore, the proportion of target

responses and non-target errors are also affected by the total number of items in the display.

Thus, the number of to-be-remembered items has a critical effect on all aspects of memory

performance.

The finding that memory performance decreases as a function of the number of items has

previously been interpreted in a number of ways. While some argue that this change in

performance is related to a reduction or the averaging memory resources allocated to each item

(Bays et al., 2009; Zhang & Luck, 2008), others have suggested that this effect indicates a

reduction in the mnemonic resolution (the quality of the representations) of the objects stored in

memory (Barton et al., 2009). A third possibility may be that performance decreases at larger set

sizes because more items are competing for representation within a finite number of receptive

fields. Thus, adding more items to the display may increase the number of competitive

interactions, resulting in greater impairments in VWM performance. This account may also be

compatible with a resolution account, in which the increased competition between additional

items results in less precise representations.

One way to examine whether memory resources or mnemonic resolution are affected by

the total number of items may be to examine whether distractors presented during the sample

display have the same effect as targets. That is, if additional targets affect only the proportion of

memory resources allocated to each item, then distractors should have virtually no effect on the

precision and number of responses, since very few memory resources should be allocated to

them. If, however, additional targets increase the number of competitive interactions, and

therefore reduce the resolution of memory representations, then distractors may have a similar

effect on the precision and types of responses as targets. In other words, if targets and distractors

compete equally for perceptual representation, but not for memory representation, then

48

distractors should affect the front-end of VWM (perception and encoding) more so than the

back-end of VWM (maintenance and recall). Accordingly, the goal of the present experiment is

to examine the effects of distractors on the number and precision of VWM representations.

Method

Participants. Twelve University of Toronto undergraduate (1 male, 10 right-handed,

ages: 19 – 22 years; M = 20.2) participated in this experiment for partial course credit.

Participants had normal or corrected-to-normal vision.

Stimuli and Procedure. The procedure was similar to that of Chapter 3, with the

following exceptions. The goal of the task was to remember and report the colours of target

items presented during the sample display, while ignoring the distractor items. Target and

distractor stimuli could be either circles or squares, and participants were alternately assigned to

one of the two target types (Figure 12). Target and distractor stimuli remained constant

throughout the task. That is, the target for one subject could be squares on every trial, with

circles as the distractors.

During the sample display, observers were presented with 1, 2 or 4 items for 150 ms,

along with a central fixation (presented throughout). For set size 1, a single target item was

presented. For set sizes 2 and 4, sample items were presented randomly in one of two conditions:

in the targets only condition, all the items presented were targets (e.g., squares); in the targets +

distractors condition, half of the items presented were targets (e.g., squares) while the other half

were distractors (e.g., circles). Both the locations and the colours of target and distractor stimuli

were assigned randomly. Following the 1,000 ms delay, the locations of all stimuli (targets and

distractors) were highlighted by black boxes, and the probed item, corresponding to the location

of one of the target stimuli, was highlighted by a thick black line. Thus, the placeholders

indicated the locations of targets and distractors, but not their identities. Only target items could

be probed as the target. Participants were instructed to select the colour from the colour-wheel

that most-closely matched that of the probed target item. The first colour selected was taken as

the participant’s response and ended the trial. A 500 ms ITI was presented between trials.

Participants performed 70 trials of each of the 5 set size x filtering task cells, for a total of 350

trials. The task was performed in blocks of 50 trials, with a self-paced break between trials. As in

the previous experiments, participants first performed 50 trials each of a 2-, 4-, and 6-item

49

change-detection task to obtain a measure of working memory capacity (K). Capacity was

estimated using the formula described in the Introduction. K-estimates were between 1.9 and 4.3

with an average capacity of 2.9 items.

Analysis. Analysis was performed using the same method used in Chapter 3. Precision

was calculated as the inverse of the SD of all responses. In addition, maximum likelihood

estimates were obtained for the three factors of PT, PNT and PG. The values obtained from the

maximum likelihood estimates were analysed using a 2 (filtering condition) x 3 (set size)

repeated-measures ANOVA. The set size was defined as the total number of items (targets or

distractors), where the set size 1 condition was used as the baseline performance for both

conditions. The exception to this was the analysis performed on the proportion of non-target

responses (PNT) as the set-size of 1 cannot contain any non-target responses. Accordingly, only

the set sizes of 2 and 4 were analyzed for this variable. Importantly, all of the colour values were

entered in the analysis as potential non-target values. That is, the analysis was performed

assuming the distractor colours could have an equivalent effect on responses as other non-target

values, but that no effects on responses would be observed if participants successfully ignored

these distractors. Responses were analyzed for outliers, and all subjects were included in the final

analysis.

Figure 12 – Example of Targets Only and Targets + Distractors condition used in experiment 5. Circles were used as targets for half of the participants. The set size 4 condition is shown for both conditions.

50


The goal of the present experiment is to examine whether distractors have a similar effect

as targets on the precision and number of responses in a VWM task. If distractors only compete

for representation with targets in the initial (perceptual) stages of representation, then the

addition of distractors should affect the precision of responses, but without affecting the number

of target items (PT) correctly recalled. In other words, if individuals are successfully able to

ignore the distractor items, distractors should have very little effect on the number of target

responses. If, however, distractors compete with targets for VWM resources, then the addition of

distractors should affect the proportion of target responses, in addition to affecting precision. If

individuals can partially-ignore the distractor items, however, then the effect a distractor has on

PT may be mitigated relative to the effect of adding the same number of targets. Moreover, if

distractors do compete for memory resources, as opposed to just affecting the ability to encode

targets, then the number of non-target responses (PNT) should similarly be affected by the

presence of distractors.

Overall precision. Consistent with the results of Chapter 3, repeated-measure ANOVA

revealed that the precision of responses decreased as the set size increased, F(2,22) = 28.191,

MSE = .139, p < .001. The effect of filtering condition was also significant, F(1,11) = 56.145,

MSE = .432, p < .001, as was the interaction, F(2,22) = 10.043, MSE = .139, p = .001. That is,

the effect of set size on precision was mitigated in the targets + distractors condition relative to

the targets only condition (Figure 13). Planned paired t-tests revealed that while the addition of

one distractor to a display containing only one target had no effect on precision, t(11) = .319,

SEM = .1836, p = .756, precision for two targets was significantly reduced compared to a single

target, t(11) = 6.089, SEM = .16804, p < .001.That is, adding one distractor had no effect on

precision, but adding a second target to the display did. Adding two distractors to a display of

two targets did affect precision, however, as precision was reduced relative to the presentation of

two targets alone t(11) = 4.429, SEM = .15982, p = .001. The precision for two distractors and

two targets was greater than the precision for four targets, t(11) = -3.91, SEM = .1107, p = .002,

indicating that while distractors did affect precision, they did not have the same effect as targets.

Overall, these effects demonstrate that targets and distractors have different effects on precision.

Specifically, the effect of distractors on precision is mitigated in comparison to the effect of

adding the same number of targets.

51

Target precision (three-component model). Examining the effect of set size and filtering

condition on the SD of target responses revealed that, in addition to the significant effect of set

size, F(1,11) = 6.427, MSE = .004, p = .028, a significant effect of filtering condition was

observed, F(2,22) = 3.657, MSE = .003, p = .043. Thus, the SD of responses was greater in the

targets only condition than in the targets + distractors condition, indicating that targets were

encoded with less error when some of the items were distractors. The interaction between set size

and filtering condition was not significant, F(2,22) = 2.47, MSE = .003, p = .108, indicating that

while the targets were reported with less error when some of the items were distractors,

additional targets and distractors increased the SD of responses at a similar rate. In other words,

adding distractors to the display could affect the resolution of target items stored in memory,

although this effect was mitigated relative to the effect of adding targets.

Figure 13 ‐ Results of experiment in Chapter 5. In the targets + distractors condition, set size indicates the total number of items (50% distractors). Error bars denote within‐subject SEM.

52

Proportion of target responses (three-component model). Examining the effects of set

size and filtering condition on PT revealed that the proportion of target responses decreased as a

function of set size, F(2,22) = 24.836, MSE = .006, p < .001. In addition, a significant effect of

filtering condition was observed, F(1,11) = 41.153, MSE = .018, p < .001, as PT was greater in

the targets + distractors condition than in the target-only condition. Thus, adding distractors to

the display had a smaller effect on the proportion of target responses relative to adding the same

number of target items. The interaction between set size and filtering condition was also

significant, F(2,22) = 17.427, MSE = .003, p < .001. Planned comparisons revealed that while the

presentation of two targets significantly reduced PT relative to a single target, t(11) = 2.857, SEM

= .024, p = .016, the addition of one distractor had no effect, t(11) = 1.088, SEM = .028, p = .3.

The addition of two distractors along with two targets did, however, have a significant effect on

PT relative to presentation of when two targets presented alone, t(11) = 3.519, SEM = .03936, p =

.005; however, this effect was significantly smaller than those observed when presenting the

same number of targets, t(11) = -5.162, SEM = .03659, p < .001. Thus, while the addition of

distractors reduced the proportion of target responses, this effect overall was mitigated relative to

the addition of targets, particularly when only a single distractor was presented along with one

target item.

Proportion of non-target responses (three-component model). To examine changes in the

PNT, only those conditions with more than one item were examined. Increases in the total set size

had a significant effect on the proportion of non-target responses, F(1,11) = 9.066, MSE = .003,

p = .012, indicating that as the number of items presented increased, observers made more errors

reporting a non-target colour. The effect of filtering condition was also significant, F(1,11) =

26.103, MSE = .005, p < .001, indicating that PNT was larger when only targets were presented.

The interaction between set size and filtering condition was not significant, F(1,11) = .055, MSE

= .005, p = .82. That is, although PNT was lower overall in the targets + distractors condition, the

rate of non-target responses increased at the same rate as the targets only condition when more

items were added to the display. Planned one-sample t-tests revealed that, while the proportion of

non-target responses was significantly greater than zero when two targets were presented, t(11) =

3.099, SEM = .00209, p = .01, the addition of a single distractor had no effect on PNT, t(11) =

1.483, SEM = .00209, p = .17. Thus, although the presentation of two targets increases the

number of non-target responses, participants are able to successfully ignore one distractor. The

53

addition of more than one distractor, however, did increase the proportion of non-target

responses, as planned comparisons revealed a significant increase in PNT when two targets and

two distractors were presented, relative to when only two targets were presented, t(11) = -2.971,

SEM = .01990, p = .013. Furthermore, the proportion of PNT when four target items were

presented was statistically indistinguishable from when the four items consisted of an equal

number of targets and distractors, t(11) = 1.521, SEM = .03392, p = .156. Thus, when the total

number of memory items is large, targets and distractors have a similar effect on the number of

incorrect responses.

Proportion of guesses (three-component model). Increasing the set size of the sample

array had a significant effect on the proportion of guesses, F(1,11) = 8.575, MSE = .007, p =

.014, as did the filtering condition, F(2,22) = 21.968, MSE = .013, p < .001. Thus, the proportion

of trials on which participants selected a random colour was greater when only targets were

presented than in the targets + distractors condition. The interaction was also significant, F(2,22)

= .6734, MSE = .005, p = .005. Planned comparisons revealed that neither two targets nor one

target and one distractor increased the proportion of guesses relative to only a single target, t(11)

= -1.827, SEM = .032, p = .095, and t(11) = -.985, SEM = .03126, p = .346. Importantly, the

proportion of guesses was greater for two targets and two distractors than for two targets alone,

t(11) = -2.263, MSE = .0351, p = .045, but less than the effect of four targets, t(11) = 2.376, SEM

= .01309, p = .037. These results indicate that while the effect of targets on “forgetting” was

greater than the effect of distractors, the addition of distractors to the display still increased the

proportion of trials on which participants made random responses.

Filtering efficiency and memory capacity. In previous experiments, the effect of

distractors on VWM performance has been attributed to the degree to which participants can

exclude those irrelevant distractors from VWM. Individuals with low-VWM capacity have been

shown to encode more irrelevant distractors than high-VWM capacity individuals (McNab &

Klingberg, 2008; Vogel et al., 2005). Specifically, to examine whether non-target items are

encoded in memory along with target items, these studies have used ERPs and fMRI to examine

the effects of distractors on the neural measures of VWM encoding and maintenance (McNab &

Klingberg, 2008; Vogel et al., 2005). These studies have demonstrated that distractors increase

the neural activity associated with VWM maintenance, suggesting that irrelevant items are

encoded in memory. These effects, however, appear to be larger for low-capacity subjects than

54

for high-capacity subjects, indicating that to some extent, VWM capacity may be related to

individual differences in the selective control of attention (Fukuda & Vogel, 2009). Thus, it is

possible that high-capacity individuals may be less affected by the presence of distractors in the

present task, leading to more precise VWM representations. Accordingly, peak VWM-capacity

obtained from the change-detection task performed at the beginning of the session was correlated

with the difference in performance observed between the condition with only two targets, and the

targets + distractors condition with the same number of targets. This analysis revealed that VWM

capacity was significantly correlated with the decrease in precision that occurred when two

distractors were added to the display, r = .582, p = .047 (Figure 14). Interestingly, this

relationship demonstrated a positive correlation, such that the addition of distractors affected the

precision of all responses more for high-capacity individuals than for that of low-capacity

individuals. This difference was not due to floor effects for low-capacity individuals, as all

participants demonstrated a 15% or greater decrease in precision. The addition of distractors,

however, was not related to the proportion of target responses, r = -.045, p = .889, or to the

proportion of non-target responses, r = .388, p = .213.

The observed relationship between VWM capacity and precision seems to contradict

previous findings that high-capacity subjects encode fewer irrelevant distractors than low-

capacity subjects (McNab & Klingberg, 2008; Vogel et al., 2005). That is, contrary to previous

Figure 14 – Correlation between VWM capacity (K) and the change in precision when the display contains two distractors and two targets, relative to when only two targets are present.

55

findings, the addition of distractors negatively affected the precision of responses more for high-

capacity subjects than for low-capacity subjects. Since PT and PNT were unrelated to capacity,

however, the change in precision likely does not reflect a greater number of non-target items

stored in memory for high-capacity subjects than for low-capacity subjects. Interestingly, in the

two targets + two distractors condition, overall precision tended to be higher for high-capacity

individuals, r = .539, p = .071. Thus, even though the presence of distractors had a greater effect

on precision for high-capacity individuals, they still had more precise responses than low-

capacity subjects. One possible explanation for this discrepancy is that the process of filtering

distractors may take cognitive resources away from encoding target information. In other words,

even though high-capacity subjects start out with more precise representations, it may be

possible that by dedicating more resources to filtering distractors, high-capacity individuals may

suffer a greater reduction in response precision. Future research is needed to explore this

possibility.

Overall, the results of this experiment demonstrated that while distractors can have an

effect on all aspects of VWM performance, this is not the case for a single distractor. That is,

adding a single distractor to the display had no effect on the precision of VWM responses.

Likewise, a single distractor had no effect on the proportion of target or non-target responses, or

on the proportion of random guesses. Thus, a single target was reported with as much accuracy

and precision whether it was presented with or without a distractor. These effects are in stark

contrast to the observation that two targets significantly reduce response precision and PT relative

to a single target, while the proportion of non-target responses and guesses increase. These

findings indicate that observers are able to successfully exclude a single distractor, almost as

though that item were not even present.

In contrast, the addition of two distractors to the display had a significant and consistent

effect on memory responses. That is, the addition of two distractors significantly reduced

precision, as well as the proportion of target responses, relative to when two targets were

presented alone. Importantly, the proportion of non-target responses also increased relative to the

presence of two targets, indicating that participants occasionally reported the colour of a

distractor item when two distractors were presented. The effect of these distractors, however,

was consistently reduced relative to the effects of the same set size of targets only, indicating that

targets and distractors did not have an equivalent effect on performance. Interestingly, the

56

performance observed with two targets and two distractors was consistently similar to the

extrapolated performance for three targets in the targets only condition. This effect is consistent

with the observation that participants are successfully able to exclude one distractor in the targets

+ distractors condition. In other words, these results suggest that, independent of the total set size

or the number of distractors, observers were able to consistently prevent one distractor from

affecting VWM performance; the effect of adding additional distractors, however, is equivalent

to the effect of adding a single target.

Alternatively, it is possible that each distractor is only partially inhibited, resulting in a

mitigated effect of distractors on VWM performance relative to targets. These effects may be

small enough for a single distractor that performance is indistinguishable from performance

without distractors, but may accumulate with the addition of more distractors.

What might be the mechanism underlying the partial inhibition of distractors? One

possibility is that participants are successfully able to bias neural responses to selectively

enhance responses to targets while minimizing responses to distractors. That is, because the

shape of the targets and distractors are known, top-down selective attention could be used to

enhance the responses of the features of those targets, while simultaneously reducing the

responses to the features of the distractor. This type of top-down biasing of attention is one of the

central tenants of the biased-competition model of attention (Desimone & Duncan, 1995;

Kastner & Ungerleider, 2001), and has been shown to be an effective means of reducing the

effects of competition within the visual system (Chelazzi, Duncan, Miller, & Desimone, 1998;

Chelazzi, Miller, Duncan, & Desimone, 1993, 2001).

Although the top-down biasing of attention may eliminate the effects of competition from

a single object, this may be more difficult when more items are added, as the non-target item can

no longer be distinguished by a unique feature. That is, Duncan and Humphreys (1989)

demonstrated that visual search time increases as non-targets become more heterogeneous. That

is, similar non-targets can be grouped together and excluded from the search; however, if the

non-targets are dissimilar to each other, then they will need to be individually matched against

the template of the search target and rejected. Thus, in the present task, when only a single

distractor is presented along with a target, both the colour and shape of the target and distractor

are different from the target, resulting in a relatively efficient selection of target features. In

contrast, when more targets and distractors are added, the dissimilarity between distractors (as a

57

result of having different colours) will prevent these items from being uniformly rejected,

resulting in an increased effect of distractor interference.

As mentioned above, a number of studies have demonstrated that the neural measures of

VWM encoding and maintenance increase in the presence of distractors (McNab & Klingberg,

2008; Vogel et al., 2005). Importantly, one study demonstrated that preparatory activity in

response to the trial instructions preceded the presentation of the memory display. Activity in the

middle frontal gyrus, as well as the basal ganglia, differed depending on whether the instructions

were to remember all of the presented items, or to filter some pre-defined target. Furthermore,

activity in the basal ganglia was strongly predictive of both memory capacity, and filtering

efficiency. Thus, activity in these regions may correspond to the types of bias signals required to

mitigate the effects of competing distractors on VWM performance.

Finally, the finding that distractors affect PT and PG in addition to PNT indicates that not

only do distractors compete for perceptual representation, but they also compete for

representation in capacity-limited VWM. This is consistent with previous findings which

demonstrate that distractors increase the neural activity associated with the maintenance of

information in VWM (McNab & Klingberg, 2008; Vogel et al., 2005). These effects, however,

are in contrast to the effects observed in Chapters 3 and 4, which suggested that increases in

competition affect the precision of VWM responses, as well as the proportion of non-target

errors, but not the proportion of correctly-recalled targets. This suggests a distinction between

two different effects of competition: increasing spatial competition between items (by decreasing

distance) affects the resolution of encoded information, without affecting the amount of

information stored in VWM; in contrast, adding items to the display (targets or distractors)

affects both the resolution of memory representations as well as the proportion of representations

maintained in VWM. Thus, each item in the visual scene obligatorily competes for VWM

resources, consistent with previous proposals (Duncan & Humphreys, 1989; Kastner &

Ungerleider, 2001). In other words, adding items to a display has the potential to affect both the

front-end of VWM (i.e., selection and encoding) as well as the back end of VWM (maintenance

and retrieval); in contrast, the distance between those items can affect the ability to form high-

precision representations (front-end of VWM), while having no effect on the number of items

maintained or recalled (back-end of VWM).

58

Overall, the results of this study provide further evidence that competition for

representation affects the ability to accurately encode, maintain, and report items in VWM. The

next two experiments will examine whether bottom-up or top-down attentional mechanisms can

bias the contents of VWM and reduce the effects of competition.

59

Chapter 6: Bottom‐Up Attentional Capture Biases VWM Responses

The results of the previous experiment demonstrate that top-down selection mechanisms

can play an important role in determining the contents of VWM. That is, individuals could

selectively ignore a single distractor, although this selection mechanism breaks down in the

presence of multiple competing distractors. This is consistent with one of the central tenants of

the biased-competition model. That is, neural responses can be biased to respond to some stimuli

more than others, thereby eliminating some of the mutually suppressive effects of competition.

Specifically, attention serves to bias competition towards highly-salient or task-relevant items.

Consequently, the goal of the experiments in Chapters 6 and 7 is to examine the effects of

different kinds of attentional bias on VWM performance.

The attentional system can largely be divided into two separate mechanisms: bottom-up

attention, which is driven by the salience of the stimuli in the environment, and top-down

attention, which is driven by the goals and objectives of the observer (Posner & Cohen, 1984).

According to competition models of attention, these two methods of attention serve to resolve

competition within the visual system, resulting in enhanced perceptual processing of attended

information (Desimone & Duncan, 1995; Kastner & Ungerleider, 2001). For example, previous

studies have demonstrated that items that appear with abrupt onsets automatically capture

attention and receive prioritized processing (Yantis & Johnson, 1990), indicating that attention is

captured by bottom-up stimulus signals. In contrast, top-down attention can be voluntarily

directed to particular locations in space, or to particular target features, depending on the task

goals (Folk, Remington, & Johnston, 1992). Importantly, bottom-up and top-down attention are

subserved by distinct neural regions within the parietal lobes (Posner, Walker, Friedrich, &

Rafal, 1984; Shulman, d'Avossa, Tansy, & Corbetta, 2002). The goal of the present experiment

is to examine the effects of bottom-up attention on VWM responses. Chapter 7 will examine the

effects of top-down attention.

In addition to enhancing perceptual processes, numerous authors have suggested that

attended information receives priority for entry into capacity-limited VWM resources

(Bundesen, 1990; Duncan & Humphreys, 1989). This question was tested in a study by Schmidt

and colleagues (Schmidt, Vogel, Woodman, & Luck, 2002). In this study, they demonstrated that

change-detection performance was better for items that were presented at a cued location than it

was for items presented at uncued locations. This effect was observed both for spatial cues

60

(cueing the location of a potential target), as well as for abrupt onsets. Thus, this study

demonstrates that both top-down (spatial) and bottom-up (onset) cues capture attention, and that

the attended information gets prioritized entry into VWM. The question remains, however, how

bottom-up prioritization affects the precision and number of VWM representations. It is possible

that attentional prioritization can serve to bias responses, thereby mitigating the effects of

competition (e.g., by decreasing the number of non-target errors). Consequently, the goal of the

present experiment is to examine the effects of abrupt onsets on the number and precision of

VWM representations.

Method

Participants. Twelve University of Toronto undergraduate students (6 male, 11 right-

handed, ages: 18 – 22 years; M = 19.9) participated in this experiment for partial course credit.

Stimuli and Procedure. The procedure was similar to that of Chapters 3 – 5, with the

following exceptions. Each trial began with the presentation of four bright grey squares (RGB =

250, 250, 250) on a light grey background (RGB = 128, 128, 128). The squares were presented at

random locations on an invisible circle around the central black fixation cross (Figure 15). These

gray squares were the same size as the sample memory stimuli, and served as placeholders for

the no-onset memory items. After 1,000 ms, two of the grey squares were removed from the

display. At the same time, four or six coloured sample memory stimuli were presented for 100

ms. Importantly, two of the sample squares were presented at the locations of the two remaining

placeholders (no-onset items). The other two or four sample items were presented at locations

that did not previously contain placeholders (onset items). Thus, these stimuli were presented

with a larger transient than the stimuli presented at the no-onset placeholders. The colours of the

Figure 15 ‐ Schematic of the experiment used in Chapter 6. The locations of the two onset and two no‐onset memory items are highlighted by solid and dotted lines, respectively.

61

sample squares were randomly selected from the available colour-space. Following a 1,000 ms

delay, the probe screen appeared, and subjects were cued to report the colour of one of the

sample items. The probe could appear with equal probability at the location of an onset or no-

onset stimulus. The probe location was randomly selected on each trial.

Participants performed 70 trials in each of the target type (onset vs. no-onset) x set size

(2-onset vs. 4-onset) conditions. Responses were analyzed for precision, PT, PNT, and PG,

according to the method described above. To identify individuals who adopted a strategy of

attending to only onset or no-onset items, the proportion of non-target responses were analyzed

for outliers (indicating incorrect responses). The results of two subjects were excluded for

demonstrating a bias towards reporting non-onset targets in one condition (M = .66); the SD of

PNT for the remaining 10 participants was .10. No outliers were observed in any other condition.


Overall precision. Results were analyzed using a 2 (set size) x 2 (target type) repeated-

measures ANOVA. Increasing the number of onset targets had a significant effect on the

precision of responses, F(1,9) = 27.701, MSE = .035, p = .001. That is, precision decreased as the

total set size increased, consistent with previous results. The type of target (onset vs. no onset)

also had a significant effect on precision, F(1,9) = 5.945, MSE = .011, p = .037, indicating that

responses were more precise for onset targets than for no-onset targets. The interaction was also

significant, F(1,9) = 5.512, MSE = .005, p = .043, as paired-comparisons revealed that precision

was higher for onset targets in the 2-onsets condition, t(9) = 3.166, SEM = .04168, p = .011, but

not in the 4-onsets condition, t(9) = .733, SEM = .03729, p = .482. That is, while precision for

two onset targets was greater than that of the non-onset targets, presenting four onset targets

mitigated any benefit of onsets on the precision of responses (Figure 16).

Target precision (three-component model). The SD of target responses was unaffected by

any of the manipulations, Fs <.6, ps > .45, indicating that targets were reported with a similar

amount of error, regardless of whether they appeared with or without onsets. In other words, the

fidelity of items that were encoded and maintained in memory was similar for high-priority

(exogenously selected) and low-priority (unselected) items.

Proportion of target responses (three-component model). Examining the effect of the

number of onset targets on the proportion of target responses revealed a significant effect of set

size, F(1,9) = 12.662, MSE = .029, p = .006, as PT decreased as the number of targets increased.

62

Although a significant effect of target type (onset vs. no-onset) was also observed F(1,9) =

9.597, MSE = .009, p = .013, paired-comparisons revealed that the effect was only marginally

significant for the 4-onsets condition, t(9) = 2.006, SEM = .06374, p = .076, and not significant

for the 2-onsets condition, t(9) = 1.362, SEM = .04281, p = .206. The interaction between set size

and target type was not significant, F F(1,9) = .592, MSE = .020, p = .462, Thus, while

participants were better overall at reporting onset targets, these effects tended to be small.

Proportion of non-target responses (three-component model). An important question is

whether the bottom-up capture of attention by onset targets can bias responses towards those

items, leading to an increase in incorrectly reported onset items. The number of onset targets had

Figure 16 ‐ The precision of responses (top left), and the proportion of target, non‐target, and random responses observed in the experiment in Chapter 6. The x‐axis indicates the number of onset targets. Two additional no‐onset targets were presented in each condition. Error bars indicate within‐subject SEM.

63

a significant effect on PNT, F(1,9) = 16.453, MSE = .008, p = .003, indicating that more errors

were made as set size increased. Importantly, however, the type of target had a significant effect

on the proportion of non-target responses, F(1,9) = 10.522, MSE = .004, p = .01. That is, PNT was

higher when no-onset items were probed than when onset targets were probed. In other words,

when no-onset items were probed, the number of non-target responses increased, reflecting an

increased bias to report those items that appeared with transients, even when they were not

targets. Furthermore, although the interaction between the number of onset targets and target

type was not significant, F(1,9) = 1.963, MSE = .011, p = .18, the proportion of non-target

responses was noticeably larger for no-onset targets when four onset targets were presented, t(9)

= -2.421, SEM = -.2191, p = .039. In contrast, the type of target probe had no effect on the

proportion of non-target responses when only two onset targets were presented, t(9) = -.651,

SEM = -.08695, p = .531. Thus, errors in reporting onset and no-onset items occurred roughly

equally when only two onsets appeared, but responses were biased in favour of the onset items

when the number of onset targets increased to four. This suggests that only a limited number of

representations could be formed, and that when the set size was large, onset items were

prioritized over the no-onset targets. This finding is discussed in more detail below.

Proportion of guesses (three-component model). The proportion of random responses

increased marginally when four onset targets were presented instead of two onset targets, F(1,9)

= 4.967, MSE = .06, p = .053, while the type of target probed had no effect on the proportion of

guesses, F(1,9) = .426, MSE = .007, p = .53. The interaction was not significant, F(1,9) = .043,

MSE = .034, p = .84.

Overall, the results of the present experiment confirm the results of Schmidt and

colleagues (Schmidt et al., 2002), indicating that items that are presented with abrupt onsets are

more likely to be reported, and reported with more accuracy, than those that appear without

abrupt onsets. This finding indicates that the contents of attentional selection are given priority

for entry into VWM. Furthermore, the results of this experiment extend the current knowledge

by showing that PNT was greater for the no-onset targets, indicating that the non-targets that

appeared with onsets were incorrectly reported more often than the non-targets that appeared

without abrupt onsets. This finding is consistent with a biased-competition model of VWM, as it

suggests that attention biases responses in favour of those high-priority objects, leading to an

increase in non-target responses for those objects over those that are given less priority. In other

64

words, items that were presented with onsets captured attention, and as a result were reported far

more frequently than those that appeared without onsets. Moreover, those items that appeared

with onsets were reported more frequently even when they were not targets, indicating that this

kind of attentional bias has significant implications for how items are encoded, maintained, and

recalled from VWM. Specifically, the increase in non-target responses indicates that in addition

to affecting front-end processes by determining whether an item gets into capacity limited

VWM, strong attentional biases also affect back-end processes by increasing response bias

towards those items.

Interestingly, the effect of onsets on the precision and number of responses was largely

dependent on the number of items that appeared with onsets relative to the total number of item

presented. That is, when there were only four items presented (two with onsets), no differences

between onset- and no-onset targets were observed in either the proportion of target responses, or

in the proportion of non-target responses. The precision of responses, however, was higher for

the two-onset targets than for the targets that appeared without onsets. Thus, when the total

number of items was within the capacity of VWM (i.e., around four), the allocation of attention

largely had no effect on whether an item was correctly recalled. Instead, items that were attended

were encoded with better precision, suggesting that the effect of attention was to increase the

fidelity of the perceptual and mnemonic representations in favour of the selected objects. In other

words, when the total number of items was small, attentional selection seems to have improved

the resolution of the selected objects, without affecting the number of items encoded or

maintained in VWM.

In contrast, when four targets were presented with onsets, in addition to two no-onset

items, significant differences in the proportion of target and non-target responses were observed.

That is, by biasing attention towards the four onset targets, all onset items were recalled correctly

more often than those that appeared without onsets. Furthermore, the onset non-targets were also

incorrectly reported more often, suggesting a strong response bias to report these items over

those that did not capture attention. These effects, however, were independent of the precision of

responses. Thus, when the number of onset targets was large, the contents of VWM were biased

in favour of those items, at the expense of the lower-priority no-onset targets.

Together, these results suggest that the effect of attentional selection on VWM

representations depends on the number of items in a visual scene. When the total number of

65

items is within the capacity of VWM, both low- and high-priority stimuli are encoded in VWM.

Those items that are the focus of attention do gain some processing advantage, however, as they

are encoded with greater precision than those that are given less priority. In contrast, when the

number of items exceeds the capacity of VWM, low-priority objects are often excluded from

VWM entirely. Only those items that are the focus of attention will be encoded and maintained

in VWM, and with similar precision. Thus, these findings support the proposals made by many

authors that items that are selected by attention win out in the competition for limited processing

resources, enabling those items to be encoded into VWM for further processing (Bundesen,

1990; Duncan & Humphreys, 1989; Kastner & Ungerleider, 2001). This experiment adds to this

literature by demonstrating that the focus of attention also has an effect on the precision of sub-

capacity VWM representations.

In the next chapter, I will examine the effects of top-down attentional cues on the number

and precision of VWM representations.

66

Chapter 7: Colour and Location Cues Bias VWM Responses

Independently

The results of the experiment in Chapter 6 demonstrate that when the number of items is

small, the precision of VWM representations is biased by the automatic capture of bottom-up

attention. The aim of the present experiment is to examine the effect of top-down spatial and

colour cues on biasing the number and precision of VWM representations.

As mentioned above, previous studies have demonstrated that spatial cues can bias the

contents of VWM (Schmidt et al., 2002). That is, cueing observers to the location of a potential

target improves performance at the cued location, while reducing accuracy at the uncued

locations. Furthermore, Zhang and Luck (2008) demonstrated that the probability of reporting a

spatially-cued target was improved relative to the uncued locations. Because the experiment by

Zhang and Luck only examined responses using a two-factor model, it is unclear what effect

these cues have on the proportion of non-target responses. That is, the results of the experiment

presented in Chapter 6 indicate that attentional priority can increase the likelihood of reporting

the selected item, even when that item is not the target. Thus, one of the goals of the present

experiment is to examine the effects of spatial cues on the proportion of non-target responses.

In addition, a central tenet of the biased-competition model of attention is the idea that

responses can be biased in favour of the features of desired objects, in addition to locations

(Chelazzi et al., 1998; Chelazzi et al., 1993, 2001). That is, when performing a search for a

target, holding a template of the target in memory biases neural responses in favour of objects

that match the desired target. Thus, an important question is whether the contents of VWM can

be similarly biased by feature-based attentional effects. Previous studies have suggested that cues

can facilitate the uploading of target information into VWM, but only when both the location and

the colour of the cue match the target information (Adamo, 2010; Al-Aidroos, 2010). Thus, the

second aim of the present experiment is to examine the effect of cueing a target-matching or non-

matching colour on the VWM accuracy, and compare and contrast those effects with those

observed by spatial cues. That is, if top-down biases are critical for VWM encoding and

maintenance, then cuing a participant to the location or feature of an upcoming item may help

facilitate these top-down biases. In other words, knowing the colour or location of an upcoming

target should facilitate encoding processes for those items, resulting in more robust and precise

representations for the cued items. In contrast, cueing the incorrect location or colour should

67

disrupt these encoding processes, resulting in reduced VWM accuracy. Accordingly, the present

experiment uses a cueing paradigm that manipulates both the colour and location of cues.

Method

Participants. Fifteen University of Toronto undergraduate students (6 male, 11 right-

handed, ages: 18 – 22 years; M = 19.9) participated in this experiment for partial course credit.

Stimuli and procedure. The procedure was similar to that of Chapters 3 – 6, with the

following exceptions. On every trial, participants were presented with four sample memory

stimuli, presented on an invisible circle around a central fixation point. Prior to the sample

display, participants were presented with one of six different cue displays (Figure 17). The

majority of these cue displays contained one coloured box presented at the location of where one

Figure 17 ‐ Stimulus and trial schematics for the experiment in Chapter 7. (Top Panel) The four cues created by colour and spatial cues. For colour matching trials (Col+), the cue colour matches the colour of the probed sample item. In the location matching trials (Loc+), the cue is presented at the location of the target sample item. Thus, both colour and location matches are defined with respect to the probe, rather than the sample. The dotted line marks the location of the eventual target probe (not present during the task). (Bottom Panel) A schematic of a Col‐Loc+ cue trial.

68

of the sample stimuli would appear. The colour and location of the cue was manipulated in

relationship with the colour and location of the probed target: The colour of the cue could either

match the colour of the probed target (col+) or one of the un-probed sample items (col-). The

location of the cue was presented either at the location of the probed target (loc+) or at the

locations of one of the other un-probed sample items (loc-). Thus, four different conditions were

created by crossing the features of colour and location. In addition, neutral trials were also

presented, in which all four of the target locations were cued with colours that matched each of

the sample items at the same locations. Finally, to offset the predictability of the col+ loc+

conjunction, an additional control condition was also presented using a cue that matched the

colour of the cued location, with the probe appearing at a non-matching location (control-). Thus,

this condition was similar to the col-loc- trials, with the exception that the cue matched the

colour of the cued non-target. Cues lasted 100 ms and were 1.8˚ x 1.8˚ with a line width of 0.15˚.

The sample was presented for 100 ms immediately following the cue, and the delay lasted for

1,000 ms. Sample locations and colours were selected randomly, as was the location of the

probe.

The number of trials in each condition was: 60 trials of col+ loc+; 70 trials each of col+

loc- and col- loc+; and 80 trials of each of the other conditions. Thus, neither the cues (colour or

location) nor the conjunctions were predictive of the target probe. Participants were instructed

that the cue was not predictive of the target location or colour, and were instructed to ignore the

initial cue display. Responses were analyzed for measures of precision, PT, PNT, and PG. Mean

values for each of these measures were analyzed for outliers (> 3 SDs). None of the participants

exceeded these criteria, so all subjects were included in analysis.


To determine the effect of cues on the precision and number of VWM representations,

responses were analyzed using a 2 (colour validity) x 2 (location validity) repeated-measures

ANOVA. In addition, measures of performance in each of these conditions were compared with

neutral cues, to determine whether the cue manipulations significantly improved or impaired

performance. Performance in the control condition was similar to that of the col-loc- condition,

and is not discussed.

Overall precision. The validity of the colour cue had a significant effect on response

precision, F(1,13) = 5.159, MSE = .048, p = .041, as precision was consistently higher when the

69

cue matched the colour of the probed target than when the colour did not match the probed item

(Figure 18). In addition, the location of the cue had a significant effect on precision, F(1,13) =

11.66, MSE = .067, p = .005, as precision was better when cue was presented at the target

location than at a non-target location. These effects were largely additive, as the interaction did

not reach significance, F(1,13) = 4.063, MSE = .019, p = .065. Thus, precision was highest when

a colour-matching cue was presented at the target location, and lowest when neither dimension

matched the probed item. Planned comparisons confirmed this effect: thus, the col+ loc+ cue

significantly improved precision over neutral cues, t(13) = 3.349, SEM = .09258, p = .005, and

the col- loc- cue marginally impaired performance, t(13) = -1.877, SEM = .0315, p = .083,

although this effect was not significant. The presence of only a single matching feature, however,

had no effect on precision relative to neutral cues, ts < 1.49, ps > .16. Therefore, although both

the colour and location of the cue affected the precision of VWM responses, the presence of only

one matching feature was not sufficient to affect performance.

Figure 18 ‐ The precision of responses (top left), and the proportion of target, non‐target, and random responses as a function of cue validity and cue type. Error bars indicate within‐subject SEM. The solid horizontal line indicates neutral cue performance, with the dotted lines indicating +/‐ within‐subject SEM.

70

Target precision (three-component model). Neither the colour of the cue, F(1,13) = .913,

MSE = .01, p = .357, nor the cue location, F(1,13) = .38, MSE = .005, p = .584, significantly

affected the amount of error (SD) in target responses. A significant interaction was observed,

however, F(1,13) = 5.56, MSE = .004, p = .035. That is, the col+ cues reduced error in target

responses when presented at the target location (loc+), but increased error when presented at a

non-target location (loc-). None of the effects of cueing colour or location were significantly

different from neutral cues, ts < 1.4, ps > .21, however, indicating that the effects of cues on

target error were small.

Proportion of target responses (three-component model). Examining the effect of cuing

condition on the proportion of target responses revealed a moderately-significant effect of cue

colour, F(1,13) = 4.126, MSE = .017, p = .063. Moreover, the location of the cue had a

significant effect on PT, F(1,13) = 15.239, MSE = .016, p = .002. That is, the proportion of target

responses was larger when the cue was presented at the location of the probed item, than at a

non-target location. The interaction between colour and location cues was not significant,

F(1,13) = .458, MSE = .008, p = .51. Comparing the effects of the cues with neutral cues

revealed that the col+ loc+ cue had a significant effect on performance relative to a neutral cue,

t(13) = 3.364, SEM = .04724, p = .005, while performance in the other conditions was not

significantly different from the neutral condition, ts < 1.6, ps > .13. Thus, the effects of colour

and locations cues did not significantly improve the probability that a memory item was recalled

unless the cue matched both the location and colour of the probed target.

Proportion of non-target responses (three-component model). The type of colour cue did

not have a significant effect on PNT, as non-target responses were lower when the cue matched

the colour of the probed target, F(1,13) = 6.56, MSE = .009, p = .0924. Moreover, the location of

the cue did not have a significant effect on PNT, F(1,13) = 1.797, MSE = .011, p = .203.

Furthermore, the interaction between colour and location cues was not significant, F = .428, MSE

= .007, p = .524. Although none of the main effects were significant, planned paired comparisons

revealed that the proportion of non-target responses was reduced in both col+ conditions relative

to neutral cues, ts < -2.8, ps < .02, and not significantly different from the neutral cues in the col-

conditions, ts < 1, ps > .35. Thus, when the colour of the cue matched the colour of the probed

items, the number of non-target responses was reduced relative to neutral cues. Interestingly,

71

these effects are distinct from the effects of the cues on target responses. The different effects of

the cues on target and non-target responses are discussed in more detail below.

Proportion of guesses (three-component model). Examining the proportion of random

responses revealed that the colour of the cue had no effect on PG, F(1,13) = .015, MSE = .031, p

= .905. The location of the cue did, however, have a significant effect on the proportion of

guesses, F(1,13) = 7.910, MSE = .016, p = .015, as significantly more guesses were made in the

col- conditions. No interaction was observed, F(1,13) = .9, MSE = .015, p = .36. Paired-

comparisons revealed, however, that none of the cueing conditions significantly increased or

reduced the proportion of guesses relative to the neutral cues, ts < 1.49, ps > .163.

Overall, the results of this experiment demonstrate that the colour and location of cues

have significant effects on the number and precision of memory items reported, without any

strong indication that these cues interact. Cues that matched the location of the probed item

significantly improved the probability that the target was reported correctly, relative to an

invalidly-cued location, but had little effect on the proportion of non-target responses. That is,

targets were reported more accurately when participants were cued to the target location, but the

number of errors did not increase when cued to an invalid location. In contrast, the colour of the

cue did not have an effect on the proportion of target responses, but a valid colour cue

significantly reduced the number of non-target responses relative to a neutral cue. In other words,

cueing participants to the colour of the target reduced the likelihood that a non-target colour was

reported. This reduction in non-target responses occurred even though target performance did not

always improve when the cue matched the colour of the target (i.e., in the col+ loc- condition).

Although the effects of colour and location cues were somewhat independent, the conjunction of

colour-matching and location-matching cues benefited performance the most, indicating that the

benefits (as well as the impairments) associated with these different cues are additive.

Overall, the greatest benefits of performance tended to occur in the col+ loc+ condition,

when the cue matched both the color and location of the target. This suggests that the effects of

the two types of cues can combine to facilitate VWM encoding and maintenance. Although it

could be argued that the addition of a col+ loc+ cue extends the encoding time for target

information, the effect of these cues exceeds that of the neutral cues which also match both the

colour and location of the targets. While the neutral cues also match the targets on both features,

however, they do so for all four sample items, rather than biasing encoding towards a single

72

item. Thus, the effect of cueing on VWM performance appears to be restricted to conditions in

which the cue helps facilitate top-down processing of target information (i.e., when only a single

cue is present).

The different effects observed for colour and location cues suggests that these features

potentially operate on different aspects of the visual system, which may in turn affect different

aspects of VWM performance. A number of authors have suggested that accurate VWM

performance depends on the activation of two distinct systems. In particular, Xu and Chun

(2009) have argued that one system selects and individuates a limited number of visual objects

from the scene, while a second system enhances and encodes the features of those selected

objects. These processes likely depend on different neural substrates, with object individuation

depending on the inferior IPS, while feature enhancement and encoding depends on the superior

IPS and LOC. Thus, one possibility is that location and colour cues operate on these distinct

systems necessary for VWM. That is, location cues may serve to bias the selection and

individuation of a particular object. Consequently, by cueing attention to a particular location,

cued objects may be more likely to be encoded and subsequently recalled, leading to an increase

in PT. Non-target responses may not be affected by these location cues, however, as invalidly-

cued objects may similarly benefit from the effects of selection, enabling these objects to be

easily rejected when other locations are probed. In contrast, colour cues may serve to facilitate

the second system, by biasing the encoding and enhancement of those colour features. If colour

cues don’t facilitate the selection and individuation of any of the sample items, colour cues alone

may not affect the likelihood that a particular target is recalled; instead, these cues may help to

bind the cued features to selected objects more accurately, resulting in a reduction in the number

of colours reported incorrectly at the probed location.

Independent of the effects of colour and location cues on the two systems proposed by

Xu and Chun (2009), it is clear that these cues serve to improve aspects of VWM performance.

Thus, cueing the location or colour of a sample item likely serves to bias neural responses in

favour of that cued item, while minimizing the interference of competing representations. That

is, just as spatial cues or search templates can bias neural responses in favour of a target in a

perceptual task (e.g., visual search), top-down attentional selection may similarly bias the

selection and encoding of those targets into VWM. In other words, mechanisms that mitigate

neural competition for representation in the perceptual system similarly increase VWM accuracy

73

and reduce memory errors for those items. Those items that are selected as targets (either by

spatial or featural selection) are given enhanced perceptual processing, resulting in more stable

and higher-fidelity VWM representations. Thus, biasing responses in favour of a particular item

may play a critical role in determining whether that item is successfully encoded, maintained and

recalled from VWM.

Overall, the results of the experiments in Chapters 6 and 7 demonstrate that both bottom-

up and top-down biasing of attention plays in important role in prioritizing mnemonic as well as

perceptual processing. Interestingly, Yantis and Johnson (1990) demonstrated that the number of

items that could be prioritized by abrupt onsets was limited to roughly four items. This limit

corresponds closely to the roughly four-item capacity limit of VWM (Cowan, 2001; Luck &

Vogel, 1997). Furthermore, the neural substrates that mediate VWM largely overlap with those

that mediate attentional selection (Corbetta, Kincade, Ollinger, McAvoy, & Shulman, 2000;

Corbetta & Shulman, 2002; Mitchell & Cusack, 2008; Shulman et al., 2002; Todd & Marois,

2004). Consequently, given that the selection of information appears to play such an important

role in determining the contents of VWM, this raises the possibility that the capacity limit of

VWM is related to the ability to select and prioritize information, enabling the efficient encoding

of information into VWM. This question will be examined in more detail in Chapters 8 and 9.

74

Chapter 8: VWM Capacity is Correlated with Response Precision

The results of the previous two chapters demonstrate that objects compete for capacity-

limited storage, with priority given to items that are the focus of attention. Attentional biases,

therefore, may act as a gateway for encoding and updating of information in VWM. While the

previous experiments have examined VWM performance at the group level, the aim of the

experiments in Chapters 8 and 9 is to examine VWM performance at the individual level. That

is, these experiments will attempt to address how resolving competition through bias signals may

contribute to individual differences in VWM performance2.

Although theories abound as to why the number of representations that can be stored in

VWM is limited (Cowan, 2001), it remains unknown precisely why VWM performance (or

capacity) is limited, or why it differs dramatically between individuals. A number of experiments

have provided evidence that VWM capacity may be related to attentional mechanisms; namely,

the ability to select and encode target information, while excluding irrelevant distractors, has

been shown to be strongly correlated with VWM capacity (Vogel et al., 2005). Furthermore, the

origin of this ability may reside in activation in the frontal cortex and the basal ganglia (McNab

& Klingberg, 2008); that is, these regions may send top-down signals to posterior parietal cortex

to exert attentional control over which environmental stimuli should have access to VWM

resources. Moreover, it has been demonstrated that high-capacity subjects are better able to resist

the capture of selective attention, suggesting that VWM capacity is related to the ability to

control attention (Fukuda et al., 2010).

Although the capture of attention by irrelevant information can explain why low-capacity

subjects encode more irrelevant items in VWM than high-capacity subjects, this mechanism does

not explain why high-capacity subjects are better able to encode relevant information, even in

the absence of distractors. In other words, the ability of high-capacity subjects to store more

items in VWM cannot be explained only by a gateway mechanism in which irrelevant

information is excluded; instead, a mechanism explaining differences in VWM ability must also

be able to explain differences in the ability to select and encode relevant information.

2 Independent of storage limitations, there are dramatic differences in VWM performance between individuals that must be explained by either resource or slot‐based theories. Although the previous experiments do not address the issue of resource vs. slot‐based models directly, they are largely supportive of a system in which the number of high‐precision representations that can be maintained is limited (i.e., a slot model). Accordingly, for the purposes of this chapter, VWM is assumed to have a limited capacity. The issue of how the results of this thesis relate to resource or slot‐based models will be discussed in detail in the General Discussion.

75

As mentioned above, Xu and Chun (2009) have described a two-stage mechanism of

VWM and perception: in the first stage, a limited number of objects are selected and

individuated; in the second stage, the features of those selected objects must be enhanced and

encoded in VWM in order for a high-precision representation to be formed. The former process

appears to be served primarily by the inferior IPS, while the latter process is mediated by the

superior IPS, as well as the LOC. Thus, one possibility is that individual differences in VWM

ability may reflect differences related to these two processes. In other words, these stages may

determine the ability to select and encode relevant information into VWM, and may therefore

mediate individual differences in VWM abilities.

In the first stage proposed by Xu and Chun (2009), a limited number of objects is

selected from the visual scene for further visual processing. Thus, individuals who are able to

select more items may have a higher VWM capacity than those who are able to select fewer

items. This process may in fact be similar to the control mechanism required for selecting

relevant information while ignoring distractors (Fukuda & Vogel, 2009). Because this selection

operates independently of feature processing, this stage is likely to have little effect on the

fidelity of mnemonic representations. In other words, if VWM capacity is related to the ability to

select a large number of items, then high-capacity individuals will be able to store a greater

number of items in VWM, but these representations may not necessarily have greater precision

than those maintained by low-capacity individuals.

In the second stage described by Xu and Chun (2009), the features of selected objects

must be enhanced and encoded in order for them to gain access to higher cognitive processes

such as object recognition and VWM. This process of enhancing features may represent a type of

top-down bias signal, resolving the suppressive-interactions between competing representations.

By enhancing the features of selected objects, these objects become high-fidelity VWM

representations, whereas objects which are poorly enhanced may be poorly remembered or

forgotten entirely. Thus, if VWM capacity depends on this second stage, high-capacity subjects

may be better able to enhance the features of selected targets, resulting in more precise

representations. Furthermore, this ability to enhance target features should be independent of

storage limitations in VWM; thus, high-capacity individuals should have more precise mnemonic

representations even when the number of items is below the capacity of VWM. For example, in

Chapter 5 it was demonstrated that high-capacity individuals had more precise representations

76

than low-capacity individuals. It is unclear, however, whether this effect is related to the filtering

requirements when distractors are presented along with targets.

Accordingly, the goal of the present experiment is to examine whether the precision of

VWM representations is predictive of individual differences in VWM performance. If

differences in VWM-capacity are limited only by the ability to select and individuate objects,

then only the number of targets correctly reported, and not the precision of those representations,

should be related to VWM capacity. If, however, VWM capacity is related to the ability to

enhance relevant features of selected objects, than the precision of VWM representations (even

those below capacity) should be predictive of individual differences in VWM capacity.

Method

To examine the relationship between VWM-capacity and precision, further analysis was

performed on data from Chapter 3 (Experiment A). In addition, 24 subjects (4 male, 22 right-

handed, ages: 19 – 23 M = 20) who participated in the experiments in Chapters 4 and 5

performed an additional task (Experiment B). The procedure was similar to that of Chapters 3

and 4, with the following exceptions: Participants were presented with one coloured square for

50, 150, 300, or 500 ms. The colour and location of the sample items were selected randomly.

After the 1,000 ms delay, participants reported the colour of the sample item as accurately as

possible by clicking the location on the colour wheel. As in the experiment in Chapter 3,

participants could adjust responses and ended the trial by pressing the space bar. 60 trials of each

ISI length were presented in a random order.

To identify the relationship between VWM capacity and memory performance, precision

and target accuracy (PT) in each of the set size conditions (Experiment A) and ISIs (Experiment

B) were correlated with the maximum K-estimates obtained from the change-detection task

(loads 2, 4 and 6) performed at the beginning of the session.


In order to examine whether VWM-capacity is related to the ability to enhance features

and form high-fidelity representations, the precision of responses from set sizes 1 – 3 (collapsed

across competition conditions) were correlated with VWM capacity (K). Of critical interest is

whether VWM capacity correlates with the precision of remembering one or two objects. That is,

if capacity is related to the ability to enhance and encode features, then this relationship should

77

exist independent of differences in the control of selective attention, as well as differences in

storage ability. In other words, if VWM performance is related to the ability to enhance target

features, high-capacity individuals should be able to form more precise representations even

when the number of items maintained is small (i.e., within the capacity of both low- and high-

capacity subjects). Supporting this hypothesis, VWM capacity was significantly correlated with

the precision of responses at set size one, r(16) = .50, p = .046 (Figure 19), and marginally

significant at set size two, r = .469, p = .067. The correlation between VWM capacity and

precision was not-significant at set size three, r = .341, p = .197. In other words, high-capacity

subjects have more precise representations for a small number of items relative to low-capacity

subjects; however, when multiple objects are presented, response precision is unrelated to VWM

capacity. Importantly, the probability of reporting the target (PT) was uncorrelated with memory

capacity at set size one, r = .20, p = .49. Thus, even though high- and low-capacity subjects are

similarly accurate at reporting the identity of one target, high-capacity subjects do so with much

greater precision.

These results support the hypothesis that individual differences in VWM capacity are

related to the ability to enhance and encode target features. Importantly, this effect is strongest

when only a single item is presented. That is, high-capacity subjects are better able to create

high-fidelity representations of below-capacity representations, indicating that this effect is

unrelated to storage limitations. Furthermore, because this relationship exists for only a single

object, it also argues against an attentional selection or gating mechanism. Consequently, the

Figure 19 – Correlation between VWM capacity and overall response precision for a single object.

78

relationship between VWM capacity and precision suggests that VWM may depend on processes

that mediate the formation of high-resolution representations, namely the enhancement and

encoding of target features.

Although the relationship between VWM capacity and precision of responses for a single

object suggests that these effects are independent of storage limits, it is possible that the

relationship reflects differences in the ability to maintain precise representations over extended

periods of time. That is, it is possible that low-capacity subjects start off with high-precision

representations that degrade by the end of the 1,000 ms delay. Accordingly, to examine this

relationship independent of maintenance demands, 24 subjects performed an additional task to

examine precision for a single object at ISIs that were within (50 – 300 ms) the range of iconic

memory, as well as for longer durations that depend on VWM (500 ms). Although capacity was

not significantly correlated with precision at 150, 300, or 500 ms, rs < .27, ps > .21, capacity was

significantly correlated at the 50 ms ISI, r = .46, p = .022. Thus, the precision of representations

formed within the first 50 ms of perception are a significant predictor of VWM capacity.

Overall, these results support the hypothesis that the enhancement and encoding of object

features (via top-down feedback) may mediate individual differences in VWM performance.

That is, at both 50 ms and 1,000 ms delays, VWM capacity was significantly correlated with the

precision of responses for a single object. That these effects were observed for only one object

and as early as 50 ms suggest that this relationship cannot be attributed to differences in VWM

storage limitations. That is, although all subjects could correctly recall a single object for 50 or

1,000 ms, high-capacity individuals could report targets with much greater precision than low-

capacity individuals. These results suggest that processes that affect the formation of

representations as early as 50 ms are directly related to VWM capacity.

Although the relationship between VWM capacity and precision was not significant

across all ISIs, a repeated measures ANOVA revealed there was a significant effect of ISI on

precision, F(3,69) = 7.708, MSE = .341, p < .001, as precision increased linearly as a function of

ISI, F(1,23) = 16.852, MSE = .394, p < .001. Importantly, these effects were observed without

ISI significantly affecting the probability of reporting the target (PT), F(3,69) = 2.118, MSE =

.001, p = .106. That is, the precision of responses significantly increased at longer ISIs, without

any changes in PT. This finding is consistent with evidence that it takes time for representations

to consolidate or “vulcanize” in VWM (Vogel et al., 2006), although the effect of consolidation

79

on precision observed here occurs over a longer period of time than that observed on change-

detection performance in previous studies. Consequently, the process of consolidation may

suggest an explanation for why no relationship between capacity and precision was observed at

intermediate ISIs. Precision may be established very early on by early perceptual processes; this

may be followed by a second stage that occurs gradually over time, in which the precision of

representations are enhanced and vulcanized via top-down feedback. In other words, high-

capacity subjects may be able to first select and individuate objects (within the 150 ms sample)

better than low-capacity subjects, consistent with an attentional control mechanism; second,

high-capacity subjects may be better able to enhance features of selected targets. The result is a

greater number of high-precision representations stored in VWM.

Overall, the results of the present experiment suggest that enhancing the features of

targets may play as important a role in determining VWM capacity as the initial selection of

target items (Fukuda & Vogel, 2009). Although object selection and feature enhancement may

appear at first blush to be different processes, in reality, they may represent different aspects of

top-down control (i.e., bias signals). Without cognitive intervention, competing representations

interact in a mutually suppressive way. Some objects may receive attentional priority via the

bottom-up capture of attention, but these objects may be task-irrelevant. Thus, without top-down

control of the visual system, the contents of perception, VWM, and ultimately consciousness,

would be driven primarily by the physical strength of the incoming visual stimuli. By applying

top-down bias signals, however, the observer derives more control over what information gains

access to higher-processing. In other words, by biasing responses in favour of a particular

location, or by matching incoming stimuli against target templates, these top-down bias signals

operate as gating mechanisms controlling what information gains access to further processing.

The stronger and more accurate these top-down bias signals, the better the control over access to

higher cognition further downstream.

In the next section, I will examine potential ERP markers of top-down bias and their

relationship to VWM capacity.

80

Chapter 9: Ptc Predicts VWM Capacity

The results of the previous experiments suggest that top-down attentional biases play an

important role in determining the contents of VWM. Furthermore, VWM capacity seems to be

driven by individual differences in the ability to select and enhance target features among

competing items. Consequently, the goal of the present chapter is to find neural markers of

individual differences in VWM performance that are predictive of resolving competition.

Potential neural correlates associated with resolving competition have been identified in a

series of recent studies by Hilimire and colleagues (Hilimire, Mounts, Parks, & Corballis, 2009,

2010). In one experiment (Hilimire et al., 2009), participants were told to search for a target

letter (“T”) that could appear in either green or orange. This target was flanked by a distractor

(“L”) that could be one, three, or five locations away from the target. Thus, competition was

greatest at the smallest target-distractor separation. The authors reported effects related to this

competition manipulation in two ERPs: they found that the amplitude of the N2pc increased as

target-distractor separation increased; in addition, they also reported a novel ERP that

demonstrated decreasing amplitude with increasing target-distractor separation. This latter

component, which occurred around 300 ms, was named the Ptc for positive temporal

contralateral.

While the N2pc is thought to reflect processes related to selecting targets from distractors

(Eimer, 1996; Luck & Hillyard, 1994a, 1994b; Woodman, Arita, & Luck, 2009; Hopf et al.,

2000; Hopf, Boelmans, Schoenfeld, Luck, & Heinze, 2004), the change in amplitude in both the

N2pc and Ptc as a function of the distance between objects suggests that both components reflect

the amount of competition between multiple visual items. Importantly, in a control experiment,

the amplitude of the Ptc was drastically reduced when participants were only required to indicate

on which side of the display the target appeared. This finding, in combination with the lateral-

temporal topography of the Ptc, led the authors to suggest that the Ptc reflects the bias signal

required to resolve competition between targets and distractors. Thus, it is possible that this

activity reflects the processes associated with LOC activity that enables target features to be

enhanced and encoded for further cognitive processing (Xu, 2007; Xu & Chun, 2006).

Specifically, because the onset of the Ptc occurs prior to the onset of the ERP associated with

VWM maintenance (the CDA), the Ptc likely reflects processes upstream of VWM maintenance

81

(e.g., feature selection or encoding). Furthermore, given that the N2pc is related to spatial

selection, it is possible that this activity reflects the individuation and selection of targets that

will subsequently be enhanced and encoded in memory. In other words, the N2pc and Ptc may

reflect, respectively, the two stages necessary for items to be successfully encoded into VWM

(Xu & Chun, 2009).

The goal of the present experiment is to examine whether the Ptc is evident during a

VWM task, and whether either the Ptc or N2pc are related to individual differences in VWM

capacity. If either the Ptc or the N2pc are related to the selection and encoding of target

information into VWM, then this activity should be related to VWM performance even when the

memory task contains only targets. In other words, if N2pc and Ptc are evident only in the

presence of distractors, this would suggest these components reflect the filtering of irrelevant

information, rather than the active selection and encoding of target information. Consequently,

the current task will use a variant of a VWM task in which both targets and distractors are

presented, and compare this activity to a condition in which only targets are present.

Furthermore, if VWM performance depends on the biased selection and encoding of target

information, then this activity should be greater when the selection and encoding of target items

is more difficult (i.e., under high spatial competition). Thus, in the current experiment, the

distance between items (targets and distractors) will be manipulated to examine the effect of

competition on these ERPs.

Method

Participants. Twenty-two adults, including the author, from the University of Toronto

community participated in this experiment. Four participants were excluded from analyses (see

below), resulting in a total of 18 participants (7 male, 16 right-handed, ages: 19 – 29 years; M =

22.2). Participants received partial course credit and/or were provided $15/hour for participating.

All participants were right-handed and had normal or corrected-to-normal vision. The procedures

were approved by the University of Toronto Research Ethics Board.

Stimuli and Procedure. The task was displayed on a 20-inch CRT monitor using a

resolution of 1600 x 1200 and a 60-Hz refresh rate. Stimuli were presented on a grey background

(RGB = 128, 128, 128). Participants were seated roughly 57-cm from the display, and instructed

to maintain central fixation during all trials.

82

The experiment consistent of two conditions performed in alternating blocks. Each trial

began with the presentation of a central fixation cross and arrow cue, presented for 200 ms. The

arrow cue indicated the side of the screen that the participant was to attend, but without moving

their eyes away from the fixation. After a 600 – 800 ms ISI, selected randomly, four coloured

stimuli (two circles; two squares) were presented on the cued side of the display for 100 ms. In

the filter condition, participants were instructed to remember only the colours of the target

stimuli (circles or squares) while ignoring the two non-target stimuli. In the remember-all

condition, participants were instructed to remember the colours of all four stimuli (Figure 20).

Participants were randomly assigned either circle or square targets, and all results are described

collapsed across target type. The colours of the four shapes were randomly assigned, with no

single colour being repeated on the target side. Four stimuli were also presented on the

unattended side, and the colours, shapes, and locations of these stimuli were randomly selected.

Circle and square stimuli subtended 1˚ x 1˚. The nine colours used for the sample stimuli were:

black (RGB = 31; 26; 23); blue (0; 124; 195); green (0; 146; 163); purple (144; 30; 120); red

(218; 37; 29); white (255; 255; 255); and yellow (255; 245; 0);

Sample stimuli were presented at one of nine locations on an invisible semi-circle on

each half of the display. Each semi-circle had a radius of 6˚ and was centred around the fixation.

To investigate the effect of spatial competition on the Ptc and filtering performance, the distance

between the stimuli on the cued side was manipulated. Thus, targets and distractors (e.g., circles

and squares) were presented in pairs. In the high-competition condition, circles and squares were

presented in adjacent locations, while the pairs of stimuli were separated by four to six locations.

In the low-competition condition, circles and squares were separated by three locations, and these

pairs were separated by one or two locations. Thus, in the high-competition condition, each

circle-square pairing was separated by 2.08˚ (centre-to-centre) while the pairs of stimuli were

separated by at least 7.71 – 10.39˚; in the low-competition condition, each circle-square pairing

was separated by 6˚ (centre-to-centre), while the pairs of stimuli were separated by a minimum

of 2.08 – 4.16˚. Importantly, in the low-competition configuration, the two closest stimuli were

both either circles or squares. Thus, although stimuli could appear next to each other in the low-

competition condition, these two items were always both either targets or distractors in the

filtering condition. The configurations of sample stimuli were assigned randomly on each trial.

83

The memory sample was followed by a 917 ms delay, after which a probe display was

presented for 1,000 ms. The probe contained one stimulus presented at the location of one of the

target stimuli on the cued side. On half of the trials, the probe stimulus matched the colour of the

object presented at the probed location during the sample display (match trials), while on the

other 50% of trials, the probe stimulus was presented in a colour that was not present in the

original sample display. The shape of the probe stimulus never changed from the sample array.

In the filter condition, the probed item was always one of the target shapes (circles or squares).

In the remember-all trials, the probed item was selected randomly from any of the four stimuli.

After each trial, participants were given the opportunity to blink, and the next trial was initiated

with a space bar response. Participants performed 96 trials of each condition, on each side, for a

total of 768 trials. Each participant practiced 10 – 30 trials before starting the experiment, with

feedback. No feedback was given during the experiment.

Figure 20 ‐ Schematic of trials performed in experiment in Chapter 9. In the filter condition, participants were instructed to only try to remember only the circles or the squares. In the remember‐all task, participants were instructed to remember all four items on the cued side. For display purposes, the stimuli on the uncued side are displayed in grey.

84

Before the experiment, participants performed 50 trials each of a 2-, 4- and 6- item

change-detection task, using a similar design as the one used in Chapters 1 – 8. Estimates of

VWM capacity (K-estimates) were obtained using this task.

Electrophysiological recording and analysis. The electroencephalogram (EEG) was

obtained from 64 active Ag/AgCl electrodes (Biosemi ActiveTwo system), digitally recorded at

512 Hz, mounted on an elastic cap using the international 10/20 system. Digital file conversion

was performed used PolyRex software, and analysis was performed in MATLAB using the

EEGlab and ERPlab extensions. All electrodes were referenced off-line to the average of the left

and right mastoids. The horizontal electrooculogram (HEOG), recorded as the difference in

activity between electrodes placed lateral to the external canthi, was used to measure horizontal

eye movements. The vertical electrooculogram (VEOG), used to detect eye blinks, was recorded

from electrodes mounted beneath the left and right eyes and referenced to the frontal electrodes

directly above the eyes. EEG and EOG signals were digitally down-sampled to 250 Hz. Prior to

averaging and analysis, the HEOG and VEOG were band-pass filtered at 0.1 – 30 Hz, applied

offline. The EEG was low-pass filtered 50-Hz for statistical analysis, and a 30-Hz low-pass filter

was applied to average ERPs for display purposes.

Trials that were contaminated with eye blinks (> 80 V VEOG) were excluded from

analysis. The HEOG was analyzed for horizontal eye-movements, and trials that contained step-

functions greater than 30 V were excluded from analysis. Four subjects with trial rejection rates

greater than 20% in any condition were excluded from analysis. The remaining subjects had a

mean rejection rate of 4.26%, with an average maximum rejection rate of 7.4%.

The ERPs were computed by averaging the EEG form 200 ms prior to the onset of the

sample display and ending 900 ms post stimulus onset. The ERPs were baseline-corrected to the

200 ms window prior to the onset of sample array. Analysis of ERPs was restricted to the two

posterior electrode pairs used by Hilimire et al. (2009), P7/P8 and PO3/PO4. We also included

the electrode pairs PO7/P08, as the Ptc was observed to be largest at these channels. Ipsilateral

and contralateral waveforms were computed for each condition, defined relative to the attended

(cued) side. Analysis was performed on difference waves computed by subtracting ipsilateral

from contralateral responses. For the ERP analysis, averages were calculated for three time

windows: the N2pc, from 190 – 240 ms; the Ptc, from 290 – 340 ms; and the CDA, from 400 –

800 ms.

85


Behavioural Analysis. Prior to the experiment, a measure of VWM-capacity was obtained

using a standard change-detection task. Maximum K-estimates were obtained using the formula

developed by Pashler (1988) and revised by Cowan (2001). All K-estimates fell between 1 and 5

(M = 3.2), consistent with previous studies. A median-split was performed on the maximum

values to obtain high- and low-capacity groups.

Performance on both the filter and remember-all trials was analyzed by calculating K-

estimates for each condition, weighted by the number of target items, and analyzed using a 2

(filter vs. remember-all) x 2 (high- vs. low-competition) repeated-measures ANOVA. The

analysis revealed a main-effect of filtering condition, F(1,17) = 56.588, MSE = .165, p < .001.

That is, significantly more items were recalled in the remember-all condition, where all four

items were targets. No effect of spatial competition condition was observed, F(1,17) = .435, MSE

= .029, p = .518. The interaction between filtering condition and competition was also not

significant, F(1,17) = .001, MSE = .025, p = .979. Thus, the configuration of stimuli in the high-

and low-competition conditions had no effect on the number of items recalled.

In previous studies it was found that low-capacity subjects were more affected by

filtering demands than high-capacity subjects. That is, the difference in change-detection

performance between the filtering condition and the no-filter condition with the same number of

targets was significantly correlated with VWM capacity (McNab & Klingberg, 2008; Vogel et

al., 2005). To examine this effect, the filtering cost was calculated as the difference between K-

estimates obtained in the load-2 condition of the change-detection task performed before the

experiment and the K-estimates obtained in the filter condition (averaged across competition

conditions). Consistent with previous findings, the presence of distractors affected low-capacity

subjects more than high-capacity individuals, as revealed by a significant correlation between

VWM capacity and filtering cost, r = -.53, p = .023.

Electrophysiological Analysis. Activity in posterior channels contralateral to the attended

search array was compared to the activity in ipsilateral channels for each condition. The resulting

difference-waves revealed three ERPs of interest: the negative N2pc, with an onset around 190

ms; the positive Ptc, immediately following the N2pc and beginning around 290 ms; and the

negative, sustained CDA, occurring immediately after the offset of the Ptc.

86

N2pc. First, the amplitude of the N2pc was compared for the three electrode pairs over

the 50 ms window around the peak (200 – 250 ms). The analysis revealed a significant effect of

channel location, F(2,34) = 10.497, MSE = .473, p < .001 as the N2pc was larger over PO3/PO4

than over PO7/PO8, t(17) = -3.307, SEM = .13562, p = .004, as well as P7/P8, t(17) = -4.224,

SEM = .10911, p = .001. Consequently, the subsequent analyses were restricted to the PO3/PO4

pair.

Examining the amplitude of the N2pc using a 2 (filter vs. remember-all) x 2 (high vs. low

competition) repeated-measures ANOVA revealed a significant effect of filtering condition,

F(1,17) = 5.948, MSE = .231, p = .026. That is, the amplitude of the N2pc was greater when

distractors had to be ignored than when all the items were to be selected, consistent with

previous studies. Furthermore, N2pc amplitude was significantly greater under the high-

competition condition, F(1,17) = 19.735, MSE = .521, p < .001. No significant interaction was

observed, F(1,17) = .861, MSE = .260, p = .37. Thus, the amplitude of the N2pc increases when

the difficulty of selecting targets is increased (i.e., under high competition), particularly when

targets must be separated from distractors (Figure 21).

Ptc. Using repeated-measures ANOVA, the mean amplitude of the positive Ptc from 290

– 340 ms was compared across the three channel sets. The analysis revealed a significant effect

of channel location F(2,34) = 4.048, MSE = .662, p = .026, as the amplitude of the Ptc was larger

over P7/P8 than over PO3/PO4, t(17) = -2.991, SEM = .1138, p = .008. The Ptc was also

marginally larger over PO7/PO8 than over PO3/PO4, t(17) = -2.005, SEM = .16335, p = .061,

and was not significantly different between PO7/PO8 and P7/P8, t(17) = -.103, SEM = .25018, p

= .92. Thus, the Ptc demonstrated a more anterior and lateral distribution compared to that of the

N2pc, consistent with previous studies. Accordingly, analysis was restricted to PO7/PO8 and

P7/P8 pairs.

Averaging across the electrode pairs, repeated-measures ANOVA revealed a moderately

significant effect of filtering condition, F(1,17) = 4.395, MSE = .794, p = .051. In contrast to the

N2pc, however, the amplitude of the Ptc was larger in the remember-all condition than in the

filtering condition. This suggests that while the N2pc plays an important role in the filtering of

distractor information, the Ptc may play a larger role in the selection or enhancement of target

information. That is, the N2pc may reflect distractor-processing, whereas the Ptc may reflect the

processing of target information. Although no significant effect of spatial competition was

87

observed for the average Ptc amplitude, F(1,17) = 1.745, MSE = .346, p = .204, analysis of

PO7/PO8 revealed a moderately significant effect of competition, F(1,17) = 1.689, MSE = .529,

p = .092. Thus, consistent with previous studies, the amplitude of the Ptc tended to be larger

under high-competition. The interaction between filtering and competition conditions was not

significant, F(1,17) = .170, MSE = .391, p = .391.

Figure 21 – ERP results observed in the experiment presented in Chapter 9. (Top Panel) ERP waveforms for the channel pairs of interest. The dotted lines indicate the windows used for calculating each of the averages. (Bottom Panel) Average waveforms for each of the ERPs, calculated at the peak channels (see text). Error bars denote within‐subject SEM.

88

CDA. Examining the amplitude of the CDA using a 400 ms window revealed no

significant effect of channel location, F(2,34) = 1.080, MSE = .633, p = .35. Accordingly, further

analysis was performed over an average of all three channel pairs. The effect of filtering

condition on CDA amplitude was not significant, F(1,17) = .169, MSE = .281, p = .69. That is,

the amplitude of the CDA was not larger in the remember-all condition, despite the presence of

two additional targets. Examining memory-capacity as a between-subjects factor revealed that

this effect was not driven primarily by low-capacity subjects, F(1,16) = .000, MSE = .298, p =

.99. That is, neither high-capacity nor low-capacity subjects demonstrated a significantly greater

CDA in the remember-all condition than in the filter condition, suggesting a similar number of

items were encoded into VWM in both conditions. Interestingly, a significant main effect of

competition was observed, F(1,17) = 19.606, MSE = .132, p < .001. Furthermore, no significant

interaction was found, F(1,17) < .001, MSE = .286, p = .99. That is, the amplitude of the CDA

was greater in the high-competition condition, regardless of whether some of the items were to

be ignored. This suggests that under high-spatial competition, individuals had a more difficult

time excluding distractors from VWM, consistent with the effects observed in earlier chapters.

Overall, the results of this study are largely consistent with the findings of Hilimire et al.

(2009, 2010), as the Ptc was found to be distinct from the N2pc, both in terms of its spatial

topography and its sensitivity to target and distractor information. Consistent with previous

Figure 22 ‐ Relationship between memory capacity and Ptc amplitude. (Left Panel) ERP waveforms for high‐ and low‐capacity participants in the remember‐all condition. (Right Panel) The change in Ptc amplitude between high‐ and low‐competition conditions is significantly correlated with VWM capacity.

89

studies, both the N2pc and the Ptc were observed to be greater when items were presented close

together in space (i.e., under high-spatial competition), although this effect was not as strong for

the Ptc. These findings extend the findings of Hilimire et al., however, as the Ptc was greater

when all the items presented were targets. This is in contrast to the N2pc which is typically larger

when targets have to be selected among distractors. Thus, while the N2pc may play an important

role in both target selection and distractor filtering, the Ptc may primarily reflect the

enhancement or recognition of target features. Importantly, these two steps correspond to the

two-stage model of Xu and Chun (2009). That is, while the N2pc may reflect the initial selection

and individuation of a limited number of targets, the Ptc may reflect the subsequent enhancement

and encoding of target information.

If the N2pc and the Ptc reflect processes critical to forming high-resolution

representations and encoding them in VWM, it is possible that this activity is related to VWM

capacity. In particular, the results of Chapter 8 suggest that the ability to enhance object features

and create high-resolution representations is predictive of VWM capacity. Consequently, if

VWM depends on the ability to select and enhance target features, then VWM capacity should

be related to Ptc activity, but not necessarily to the amplitude of the N2pc, as the Ptc appears to

reflect target-related processing. To test this question, the increase in N2pc and Ptc amplitude

from the low- to high-competition condition was compared with VWM capacity. For the N2pc,

the change in amplitude between high- and low-competition conditions was unrelated to VWM

capacity, regardless of whether the goal was to filter distractors, r = .01, p = .97, or to remember

all the targets, r = .329, p = .18. Across both filtering and remember-all conditions, the change in

amplitude of the Ptc was also not significantly correlated with VWM capacity when averaged

over PO7/PO8 and P7/P8, rs < .34, ps > .18. Examining PO7/PO8 only, however, revealed a

significant correlation between the change in Ptc amplitude in the remember-all condition and

VWM capacity, r = .47, p .049. Thus, whereas low-capacity subjects exhibited little difference in

the amplitude of the Ptc between high- and low-competition targets, high-capacity subjects

showed a much larger Ptc response when targets were presented in close proximity (Figure 22).

The results of this experiment provide further support for the model described by Xu and

Chun, and provide potential ERP correlates of selection and encoding stages. That is, when

presented with a visual scene, there are two stages that must occur for items to gain access to

higher cognitive processes such as working memory and object recognition. First, a limited

90

number of targets must be selected and individuated. This may be particularly true in the

presence of distractors, otherwise task-irrelevant information may be processed at the expense of

relevant targets. Given that the N2pc is larger in the presence of distractors, and occurs prior to

the onset of both the Ptc and the CDA, it is likely that the N2pc may reflect this process. That is,

the N2pc may reflect the process of selecting and individuating targets to enable further

processing of those items. This is consistent with findings that the amplitude of the N2pc is

larger when the number of target items increases. Thus, when more items need to be selected, or

when targets are more difficult to individuate (e.g., under high-spatial competition), the

amplitude of the N2pc increases, consistent with a selection and individuation mechanism.

In the second stage of VWM and perception proposed by Xu and Chun (2009), the

features of the selected objects are enhanced and encoded. Thus, the Ptc likely reflects these

feature enhancement and encoding processes, as it is not evident when targets do not need to be

identified. Furthermore, the Ptc occurs immediately after the N2pc, but before the CDA,

suggesting that it may play a critical role in encoding stages after targets have already been

selected. Critically, the amplitude of the Ptc was not greater when some of the items were

distractors; in contrast, the Ptc was largest when all four items were targets, suggesting that this

activity reflects processes related to target processing, rather than those related to distractor

filtering.

In addition, the finding that changes in Ptc amplitude are correlated with VWM capacity

provides further support that the Ptc reflects target enhancement and encoding processes. That is,

if VWM depends on top-down bias signals to enhance and encode target features for further

cognitive processing, then those individuals who are better able to enhance and encode target

features should perform better on VWM-dependent tasks. Thus, the finding that the Ptc

amplitude changes between high and low-competition conditions is consistent with this account:

When targets compete for representation, high-capacity individuals are better able to enhance

and encode relevant features, enabling more items to be maintained in and recalled from VWM.

In other words, high-capacity individuals are better able to bias the selection of competing

representations. This account is consistent with the results of Chapter 8, in which VWM capacity

was correlated with the precision of VWM responses. In other words, top down selection,

enhancement and encoding of target features leads to more precise VWM representations, and

therefore, better VWM performance.

91

Although these results provide further support for the biased-competition account of

VWM, there are a few discrepancies from previous findings that are worth mentioning. First,

unlike in previous studies, the effect of target-distractor or target-target separation on the

amplitude of the Ptc was not significant at P7/P8, and only marginally significant at PO7/PO8.

There are a number of possible explanations for this discrepancy. The configuration in the

present design used four items rather than two. Consequently, the overall amount of competition

between all the items in the display may have mitigated the effects of the Ptc. In fact,

behavioural performance in the low-competition condition was no worse than in the high-

competition condition, suggesting that the current configuration may not have optimized the

competitive interactions between items. Alternatively, the difference in effects between the

current and previous studies may be methodological, as previous studies have applied high-pass

filters during data analysis, which can increase the amplitude of a positive ERP that is situated

between two negative ERPs (i.e., the N2pc and the CDA). Thus, future studies are required to

fully disentangle the electrophysiological nature of the Ptc. The finding that competition

marginally increased the Ptc amplitude does, however, confirm that general finding that the Ptc

is sensitive to the distance between items.

In addition, although previous studies have demonstrated differences in CDA amplitude

between filtering and remember-all conditions, particularly for high-capacity subjects, no such

difference was observed here. In other words, even though the remember-all condition contained

twice as many targets compared to the filtering condition, the amplitude of maintenance-related

activity was similar in both conditions. Although the absence of a difference between these

conditions is somewhat surprising, the finding that the CDA was larger under high competition

compared to the low-competition condition may reveal a potential explanation. That is,

maintenance-related activity was greater when items were presented in a high-competition

configuration, regardless of whether some of those items were distractors. This suggests that

competition for representation may have a greater effect on whether or not irrelevant items are

encoded in memory. Thus, even high-capacity subjects may have failed to exclude the distractors

from VWM, resulting in an increase in CDA amplitude. In other words, when competition

between targets and distractors was high, distractors may have gained obligatory access to VWM

resources. Thus, this seemingly contradictory finding provides further support that competition

for representation is a critical aspect of VWM.

92

Finally, it is unclear why the distance between objects had a significant effect on the

amplitude of the CDA in the present study when a similar manipulation had no effect on the

CDA in a previous study (McCollough, Machizawa, & Vogel, 2007). The difference between

these findings may be due to differences in stimuli parameters, as in the present study the stimuli

were presented in pairs with a constant distance from fixation, whereas stimuli in the previous

study were randomly located throughout the display in pairs of two or four. In particular, if the

effect of distance on VWM performance depends on competition for representation within a

receptive field, then an object’s distance from fixation, as well as the distance between objects,

could significantly alter the extent of competition.

In summary, the results of this present chapter demonstrate two important findings: first,

the Ptc is present during a VWM task, particularly when only targets are presented, indicating

that this ERP reflects the processing of target information; second, changes in Ptc amplitude are

correlated with VWM capacity, indicating that this ERP likely reflects the enhancement and

encoding of target features necessary for VWM. Thus, this experiment reveals the Ptc as a likely

neural correlate of top-down target processing necessary for resolving competition and

establishing high-precision mnemonic representations.

93

General Discussion

The purpose of the present thesis was to examine the effects of competition on VWM

performance. Specifically, this thesis aimed to determine how the number and precision of

VWM representations is affected by competitive interactions, and how competition for

representation can be overcome by selection biases. To demonstrate that competitive interactions

affect VWM performance, Chapters 1 – 4 tested whether performance on VWM tasks was

affected by the distance between memory items. The results of these experiments revealed that

when objects are presented close together in space, VWM performance is impaired relative to

when those same objects are presented further apart. In other words, spatial competition for

representation within the visual system affects the ability to establish and maintain information

on-line in working memory. The effects of competition on VWM were observed for simple and

complex objects, as well as for both targets and distractors. Using a three-component model of

continuous responses in a recall task, chapters 3 – 4 demonstrated that while increasing the

number of items (targets or distractors) affects the probability of correctly recalling an item (PT),

decreasing the distance between objects primarily affects the precision of responses (1/SD), as

well as increasing the number of incorrectly-recalled non-targets (PNT). Chapter 5 extended

these findings to distractors, demonstrating that multiple distractors affect the precision and

accuracy of VWM responses. To demonstrate how competition for representation in VWM is

overcome via selection bias, Chapters 6 – 7 examined VWM performance for high- and low-

priority objects. Objects given high attentional priority were typically reported with greater

precision and accuracy than those given less priority. Paradoxically, high-priority objects (i.e.,

those presented with onsets) were also sometimes reported incorrectly more often than lower-

priority items indicating a strong bias towards these objects. Finally, Chapters 8 and 9 examined

bias-signals as a potential source of individual differences in VWM performance, revealing that

high-performers have more precise representations of sub-capacity representations than low-

performers. The neural correlate of this bias could be the Ptc, occurring around 300 ms over

lateral occipital and temporal cortex. Overall, the results of this thesis support the conclusion that

VWM performance is limited by competition for representation within the visual system. Before

outlining a model of how competition affects VWM performance, I will first address how

competition affects the various measures of performance observed in the present experiments.

94

Precision, Resolution, and Accuracy in VWM

The experiments presented in Chapters 3 – 8 used the method of recall established by

Wilken and Ma (2004), and the data were analyzed using the method of Bays, Catallo and

Husain (Bays et al., 2009). These studies were aimed at determining whether competition affects

the number or the precision of representations in VWM. Thus, these studies examined both the

precision of overall responses (1/SD), as well as the proportion of responses to targets (PT), non-

targets (PNT), and random responses (PG). As in previous studies, increasing the set size had a

significant effect on all aspects of performance. Within a set size, however, increasing the

competition between items (by decreasing the distance between objects) had either no effect

(Chapter 3) or a small effect (Chapter 4) on the precision of responses. In both cases, however,

competitive interactions between objects significantly increased the number of non-target

responses. Importantly, the extent of spatial competition had no effect on the number of target

responses, or on the SD of target responses. Thus, while the number of items stored and correctly

recalled was unaffected by within-set size changes in spatial competition, overall accuracy was

affected, as revealed by an increase in the number of non-target errors. These results suggest that

while competition can affect VWM performance (as was observed in the change-detection task

in Chapters 1 and 2), changes in within-set-size competition largely have no effect on either the

number of items stored, or on the fidelity of those correctly-recalled items.

How does competition affect both change-detection performance and response precision,

without affecting the number or accuracy of recalled items? One possibility is that the number of

non-target responses reflects low-resolution memory for objects that are not the focus of

selection biases (see below). For example, Sligte and colleagues (Sligte, Scholte, & Lamme,

2008; Sligte, Vandenbroucke, Scholte, & Lamme, 2011) have suggested that there are three

forms of visual memory: a high-capacity and fleeting iconic memory; a low-capacity but stable

VSTM; and an intermediate memory, which they termed fragile VSTM. This intermediate

memory store is thought to contain a larger number of low-resolution representations than the

number of high-resolution items stored by VWM that can be probed through the use of

attentional cues that appear before a change-detection probe. Thus, when subjects are cued to a

particular location, participants can report more items than if no cue were presented, indicating a

larger capacity memory than is available during a typical change-detection task (Griffin &

Nobre, 2003; Makovski, Sussman, & Jiang, 2008; Sligte et al., 2008; Sligte et al., 2011).

95

According to this account, if an item is not the focus of attention, and therefore, not a

high-precision representation recalled in VWM, it may instead be a low-precision representation

stored by fragile-VSTM. This was revealed in one study by demonstrating that a smaller

percentage of sample items can be recalled (i.e., identified from memory) than the percentage of

changes detected (Sligte et al., 2011). In other words, a large number of low-resolution items can

be used to perform detection tasks, while a smaller number of those can actually be recalled,

suggesting that change-detection performance depends on the presence of additional low-

resolution memory representations.

The possibility that there exist low-resolution representations that are outside the focus of

attention could potentially explain the totality of results observed in these studies. If an item is

recalled correctly, it is likely a high-resolution representation becomes the focus of selective

attention, and therefore, stored within capacity-limited VWM. When the probed target is

forgotten, it is easy to reject the high-resolution items stored in VWM because, as the identities

and locations of those objects are known. Low-resolution representations, however, may affect

performance by biasing responses towards these non-targets, particularly under a high-

competition configuration. That is, when target items are forgotten, responses are biased towards

the low-resolution, low-fidelity representations. As the competition between items increases, the

threshold for reporting these low-resolution items decreases, resulting in an increase in

incorrectly-reported items.

The possibility that low-resolution representations affect performance on the types of

responses in a recall task has important implications. In the model described by Bays and

colleagues and used in the present design (Bays et al., 2009), the concentration (distribution) of

non-target responses is assumed to be equivalent to that of correctly reported targets. If,

however, non-target responses reflect the contribution of lower-resolution representations than of

those that are correctly recalled, these responses should encompass a wider distribution. Thus,

the proportion of non-target responses may in fact be greater than that assumed by current

models. This question should be addressed in future studies.

Another possible explanation for the present findings is that competitive interactions may

increase the difficulty of binding the features of objects to particular locations. For example,

studies have demonstrated that performing a change-detection task becomes increasingly

difficult as the number of original sample items is removed from the probe display (Nasr,

96

Moeeny, & Esteky, 2008). That is, as the context of the original sample display becomes less

intact, accuracy decreases and reaction times increase. This may be in part because VWM for

object features depends on configural relationships (Jiang, Chun, & Olson, 2004; Jiang, Olson, &

Chun, 2000). Consequently, items that compete for representation may be more susceptible to

errors in spatial memory, as the competing objects increase the difficulty of identifying which

feature belongs to which object. While these types of errors may reduce overall change-detection

accuracy (e.g., errors in spatial memory may be misidentified as changes, or may prevent the

accurate identification of changes), spatial errors may be exhibited primarily as non-target

responses in a recall task (i.e., identifying the correct colour at the incorrect location). That is,

when target items are outside the focus of attention, non-target items may have an increased

weight under high competition, resulting in a non-target response, rather than a random guess.

Thus, errors in spatial memory may affect non-target responses without simultaneously affecting

the probability or precision of target responses.

Overall, although the precision of all responses is proportional to the number of items

presented, as demonstrated by Bays and colleagues, precision does not always accurately reflect

VWM performance (Bays et al., 2009; Bays & Husain, 2008). In the present series of

experiments, decreases in precision were frequently observed in response to increases in

competition. These changes in precision, however, were often not accompanied by changes in

the probability of recalling a target, or in the amount of error in these target responses. Instead,

losses in precision were most frequently associated with increases in response errors (in the form

of non-target responses). Furthermore, it is unclear whether measures of precision also reflect

contributions from intermediate (fragile) forms of memory, or how overall precision relates to

resolution. Thus, while precision may serve as a catch-all measure of VWM performance that

describes overall performance, it may not be a useful measure for describing the number or

quality of representations stored in VWM. Instead, the three-factor model likely serves as a

better method of describing the totality of VWM responses.

A Biased-Competition Model of VWM

From the moment information reaches visual cortex, objects in the visual scene compete

for representation. In the initial stages of cortical processing, the most prevalent mode of

competition is local competition. That is, mutually-suppressive interactions occur between

objects, resulting in mitigated neural responses relative to when objects are presented in isolation

97

(Kastner & Ungerleider, 2001). Because this competition is localized (i.e., spatially specific),

competition for representation increases as processing ascends the visual pathways. That is, as

information moves from V1 to V4 and beyond, the receptive field size of neurons increase,

resulting in competition for representation over a larger region of space (Kastner et al., 2001).

Furthermore, competition for representation at early levels of visual processing is proportional to

the distance between objects: as objects compete for representation for a greater number of

receptive fields, the effects of competition increase.

It is worth mentioning that because local competition is happening at the level of the

receptive field, effects of competition on VWM performance should be greater when all objects

are presented within a single hemifield. Thus, further support for the competition account comes

from previous studies that have demonstrated that change-detection performance is greater when

items are presented bilaterally, rather than within only a single hemifield (Umemoto, Drew,

Ester, & Awh, 2010). One may argue, therefore, that the effects of distance on memory

performance in the present studies may be the result of a greater number of bilateral trials in the

low-competition conditions (when items were spread further apart). While competition may have

a greater effect on unilateral stimuli, however, they should also exist for bilaterally presented

stimuli, as previous studies have established that competition occurs both within- and between-

hemifields (Geng et al., 2006; Mathôt, Hickey, & Theeuwes, 2010).

The most significant way of increasing competition for representation in early visual

cortex is by increasing the number of objects in the scene, thereby increasing the number of

competitive interactions. That is, in the present thesis (Chapter 3) as well as in previous studies

(Bays et al., 2009; Zhang & Luck, 2008), the largest decreases in resolution (or the precision of

responses) appears to occur between one and two items (i.e., when competitive interactions are

introduced). Further increases in the number of presented objects continue to affect the precision

of responses, suggesting further decreases in the resolution of perceptual representations. Thus,

as the number of items (and therefore, the number of competitive interactions) increases, the

resolution of objects appears to decrease proportionally. Importantly, these effects appear to

occur independently of any capacity limits of VWM, suggesting that competition likely affects

the resolution of representations upstream of VWM maintenance. That is, competition affects the

precision of representations below the capacity of VWM, suggesting that these effects are

independent of limitations on VWM storage. Furthermore, adding distractors to the display also

98

appears to have a significant effect on the resolution of representations, although the suppressive

interactions of distractors may be somewhat mitigated by selection biases (Chapter 5; see below).

Overall, however, the results of these and other experiments suggest an inverse relationship

between competition and resolution: as competition for neural representation increases, the

resolution of the resulting representations decrease.

In addition to local competitive interactions, the results of the current experiments

suggest that objects compete globally for capacity-limited cognitive resources. This competition

likely occurs in two stages, as proposed by Xu and Chun (Xu & Chun, 2009). First, a limited

number of objects will be selected or individuated. Because the number of objects that can be

selected is limited by neural resources, objects compete for this selection. Second, the features of

some of the selected objects are enhanced and encoded, enabling those objects to be fully

processed (e.g., identified or stored in VWM). The first stage of capacity-limited processing

likely occurs in the iIPS, and the second stage originates predominately in the sIPS and LOC (Xu

& Chun, 2006). Importantly, while the activation of iIPS is largely independent of information

load, activity in the sIPS and LOC appears to be highly sensitive to the complexity of objects

(Xu, 2007; Xu & Chun, 2006). Thus, it is in this second stage that the resolution information

resolved in early visual cortex is extracted.

The final resolution of these representations is ultimately still limited by competitive

interactions in early visual cortex. In others words, local competition in early perceptual areas

largely sets the resolution of representations, at least with limited exposure times. Activation of

the IPS and LOC likely serves to stabilize (or vulcanize) representations, making them more

resistant to interference from eye movements, or changes in the display. These two stages,

therefore, can likely account for differences in VWM performance that result from different

probe types (Sligte et al., 2008): low-resolution, fragile representations may be extracted for

numerous objects, while further constraints limit the number of high-resolution, stable

(vulcanized) representations that can be formed. Importantly, both of these stages are capacity

limited, either in terms of the number of items that can be represented within a receptive field, or

in terms of the number of cognitive resources to represent those items. Therefore, items compete

for representation at both stages, and those items that win-out gain access to further cognitive

processing that facilitate such processes as object recognition and VWM.

99

Figure 23 ‐ A schematic of a biased‐competition model of VWM. Blue regions identify local competition for representation. Green regions identify those where global competition for VWM resources may occur. Red regions identify potential sources of top‐down feedback that serve as bias signals for target selection. Solid arrows indicate recurrent interactions between local and global competition. Dashed arrows indicate possible sources of top‐down selection mechanisms.

Although all objects in the scene compete for global representation (i.e., representation

by capacity-limited cognitive resources), they do not compete for representation equally.

Numerous selection biases can alter the relative weights of particular objects, increasing the

likelihood that these objects gain access to VWM. These selection biases can largely be

100

considered attentional effects, and are consistent with the mechanisms for resolving competition

within the attentional system (Desimone & Duncan, 1995). The means of biasing selection can

largely be divided into the two forms of selective attention: bottom-up, and top-down.

Bottom-up mechanisms of selection are largely automatic processes that do not depend

on cognition or task demands. Primarily, stimuli that have strong physical salience and stand out

from the background will be given higher priority than those objects that have weaker physical

signal. Thus, those objects that appear with abrupt onsets (Chapter 6) are prioritized over those

that appear without onsets, as onsets are typically predictive of new information. If the contents

of VWM were determined solely by the bottom-up, stimulus-driven capture of attention, VWM

would be of limited use, as the contents of memory would often not support the individual’s

current goals. Fortunately, in addition to bottom-up biases, top-down feedback can also affect

visual processing and the control of selection. Top down selection can occur either by spatial or

object (feature) selection, although these processes may interact (Chapter 7). The mechanism by

which top-down selection may affect early perceptual processes is by altering the responses of

neurons in early visual cortex, thereby enhancing the processing of relevant (cued) information,

and mitigating the suppressive effects of distractors (Desimone & Duncan, 1995).

The biasing of spatial location has been shown to improve performance on numerous

discrimination tasks, and furthermore, has been shown to improve VWM performance (Schmidt

et al., 2002). In the experiment presented in Chapter 7, cuing a particular location with a

peripheral cue increased the probability that the target at the cued location was recalled correctly.

This is consistent with previous findings that central cues can increase the probability of

correctly reporting the target (Zhang & Luck, 2008). Thus, top-down spatial cues appear to bias

the selection of the cued target, thereby reducing the effects of competing distractors, and

supports stronger encoding and consolidation of the information at the cued location. In fact,

aside from the number of sample items, cueing the target was the only manipulation in the series

of experiments presented in this thesis to have a significant effect on the SD of target responses.

Therefore, biasing selection via spatial attention may be one of the primary means of both

reducing local competition (i.e., competition for receptive fields in early visual cortex), as well as

biasing the contents of selection for further cognitive processing by capacity-limited resources

(i.e., global competition).

101

In addition to spatial bias, top-down selection can bias responses towards object features.

This may be particularly important when the location of a target is unknown (Desimone &

Duncan, 1995). In order for object selection to occur, the features of the desired target must be

held on-line in WM to bias competition in favour of the target objects. In one series of studies

(Chelazzi et al., 1998; Chelazzi et al., 1993, 2001), monkeys were presented with a target

stimulus at the beginning of a visual search trial. Responses to the target stimulus were sustained

in inferior temporal (IT) cortex, indicating that these object features had become activated and

held on-line. When the search was presented, cells responded to both the target stimulus and the

distractor. Very quickly, however, responses to the distractor object become suppressed, while

the target responses remain high. Thus, top-down selection can bias responses in favour of

particular targets, thereby mitigating the effects of competition.

Just as spatial selection is an effective way of biasing the contents of VWM, object

selection may also serve to determine which items gain access to capacity-limited resources. For

example, in tasks in which targets and distractors can be distinguished by colour and shape,

preparatory activity was exhibited in bilateral middle frontal gyrus (MFG) and the basal ganglia

in response to the instructions to remember or ignore the critical distractors (McNab &

Klingberg, 2008). Activation in these regions may therefore reflect the top-down bias signal to

select and ignore particular items. Although it is unclear whether the selection of targets and the

filtering of distractors depends on the same type of preparatory activity, it is likely the case that if

features of target items are known, this information could be used to bias the selection and

encoding of those items into VWM. Furthermore, because search templates require that target

information be stored online in VWM (Chelazzi et al., 1998; Chelazzi et al., 1993, 2001), storing

target features may be a much more effective way of selecting and encoding that information

than biases that depend on suppressing distractor information, as holding target information in

VWM already primes responses in favour of the target (Soto, Heinke, Humphreys, & Blanco,

2005; Soto, Humphreys, & Rotshtein, 2007).

As mentioned above, Xu and Chun (2009) suggested that a two-stage process is

necessary for object recognition and VWM. While the first stage involves object individuation

and selection (i.e., spatial selection), the second stage involves the selection and enhancement of

target features (i.e., object selection). Thus, it is possible that these two processes reflect similar

mechanisms to those that are involved in the a priori selection of target features. That is, after the

102

onset of competing stimuli, it may be necessary to bias responses related to the location of

objects (spatial bias) as well as bias the responses to each of the relevant target features (object

bias). These post-hoc selection biases may be particularly important with short stimulus

durations, as well as with large sample sizes. As with other bottom-up or top-down biases,

selecting and enhancing relevant target locations and features may mitigate the effects of

competitive interactions, thereby enabling these items to form high-precision, VWM

representations. Consequently, the ability to select and enhance relative information may be an

important determinate of VWM processes. For example, although encoding time was kept brief

(< 200 ms) throughout this thesis, it is possible that longer encoding times may enable bias

signals to operate more effectively, resulting in the consolidation of more robust and stable

representations (Vogel et al., 2006). The data presented in Chapters 8 and 9 suggest that

processes related to the selection and enhancement of target features may in fact be related to

VWM capacity. That is, individuals who are better able to enhance the features of target

representations are able to establish higher-precision representations, enabling the storage and

recollection of a greater number of visual objects (Chapter 8). Moreover, the N2pc and the Ptc

may represent processes related to the selection of targets and the biasing of object features,

respectively (Chapter 9). Thus, these ERP components may represent the analogue of processes

demonstrated by the iIPS the sIPS/LOC during VWM tasks.

The results of the present experiments are also consistent with the idea that VWM has a

fixed capacity for high-resolution representations, although a greater number of lower-resolution

items may also be stored. That is, even with the biasing of attention, only a limited number of

items can be successfully reported (Chapter 7). The question remains, however, why VWM

capacity is limited at all. One possibility is that VWM capacity represents the upper limits of the

selection bias within the visual system. Thus, items that are not the focus of selective attentional

biases would be excluded from VWM, but potentially stored as lower-resolution, fragile

representations (Sligte et al., 2008; Sligte et al., 2011). In support of this account, previous

studies have demonstrated that the number of items that can be prioritized through abrupt onsets

is limited to about four items (Yantis & Johnson, 1990). That is, observers are more accurate at

reporting items that appear with onsets than those that appear without onsets, but this effect is

limited to four items. Consequently, if only a limited number of items can be prioritized by

attention, this bottleneck would limit the number of high-precision representations that could be

103

maintained downstream in VWM. This account could further explain why the SD of target

responses is largely unaffected by set size manipulations above four items, but is affected by

below-capacity manipulations: as items are added to the display, competitive interactions reduce

the precision and resolution with which items can be perceived and encoded in VWM; however,

beyond the capacity of VWM, increasing the number of items may not affect the precision of

target responses, as attentional biases would mitigate the competitive effects of the additional

(unselected) items. The (four) selected targets would continue to compete for representation,

resulting in representations that are no more or less precise than would be observed for the four

stand-alone targets. In other words, attentional biases may fix the precision of target responses,

as well as the number of targets that could be stored.

As stated in the Introduction, the present thesis was largely motivated by the biased-

competition model of attention first proposed by Desimone and Duncan (1995). As such, the

model of VWM outlined in here bears several similarities to the biased competition model of

attention. In fact, the biased-competition model of attention was largely a model describing how

information was first selected for processing, with the implicit or explicit assumption that

selected information would be available for subsequent cognitive processing, including (or

especially) VWM (Duncan & Humphreys, 1989; Kastner & Ungerleider, 2001). Thus, in many

ways, a biased-competition account of VWM is not distinct from a model of attention, but

merely an extension of it.

Relationship to Other Models of VWM

As stated in the Introduction, most current models of VWM assume that limits in VWM

performance are related to limits in storage ability, rather than earlier processing bottlenecks.

The results of this current thesis suggest, however, that competition for representation may play a

critical role in both the number of mnemonic representations that can be formed, as well as in the

fidelity of those representations. Consequently, in the final section I will explore how the current

findings relate to current models of VWM, as well as identify potential implications for these

theories.

Currently, the predominant debate in the literature surrounding the nature of information

processing in VWM relates to whether VWM capacity represents a limited number of fixed-

resolution slots (e.g., Cowan & Rouder, 2009; Fukuda et al., 2010), or whether VWM represents

a limited pool of resources that can be flexibly allocated to items in the visual scene (e.g., Bays

104

& Husain, 2009; Huang, 2010). While the experiments in the present thesis were not designed

specifically to address the debate of fixed-slot vs. flexible-resource models, the results may

provide some insight into this debate. Specifically, the results of the present thesis are largely

consistent with theories of VWM in which the fidelity and resolution of VWM is affected by the

amount of object information contained within target items, but in which there is an upper-limit

on the number of items that can be stored (Alvarez & Cavanagh, 2004). That is, the number of

items that can be stored is affected by both the number of items in the display (Chapters 1 – 3),

as well as the complexity of those objects (Chapter 2); however, the results do not support a pure

resource theory, as the fidelity items stored in memory was not affected by increases in set size

beyond 4 items (Chapter 6). In other words, the results of Chapter 6 are consistent with the

findings of Zhang and Luck (2008) that there is a fixed limit on the resolution of the items

correctly recalled from VWM, suggesting an upper limit in the number of items that can be

stored.

In order to account for the decreases in resolution for representations below the capacity

of VWM (e.g., from one to three items), Zhang and Luck (2008) proposed that memory

resources could be averaged for those items below the capacity of VWM. Thus, according to this

account, resources can never be completely allocated to only one representation, but are instead

averaged across many representations (up to the number of available slots). Thus, this

slots+averaging model provides an alternative to a model in which resources can be allocated

preferentially to a single item. This model assumes, however, that resolution is determined

purely by storage resources.

The results of the present thesis may provide an alternative account for why the number

of items that can be stored is fixed, while the resolution of those representations depends on the

set size. Specifically, it is possible that the precision of a single representation is greater than the

precision for three representations not because memory resources are averaged across multiple

items, but because competitive interactions between multiple items degrade the precision of

those representations. That is, the resolution of mnemonic representations may be determined by

competitive interactions upstream of VWM maintenance; the more items competing for

representation, the lower the precision of those representations. Selection biases, however, may

serve to minimize the effects of those items which are outside of the selection limit, resulting in a

plateau in mnemonic resolution.

105

There are three advantages of the competition account over a slots+averaging model.

First, a competition account does not require strict constraints on cognitive operations (i.e.,

averaging of resources, without pure resource flexibility). Instead, this account is parsimonious

with the known behavioural and neurobiological effects of competition on perceptual

representations (Desimone & Duncan, 1995). Second, a competition account may be more

parsimonious with the neural mechanisms of VWM than a slots+averaging model. That is, if

memory resources can be allocated over many items, it is unclear why neural measures of VWM

increase with memory load (Todd & Marois, 2004; Vogel & Machizawa, 2004; Xu & Chun,

2006). If memory resources could be averaged over multiple items, one might expect neural

measures of VWM maintenance to plateau at one or two items, as all available memory

resources could be allocated to these items. Instead, a competition account could explain a loss

of precision with increasing set size while still maintaining a one-to-one relationship between set

size and VWM-related activity. Third, a competition account may also explain how attentional

cues can affect the likelihood of recalling a cued item, while having little-to-no effect on the

precision of those representations (Zhang & Luck, 2008): that is, while attentional cues may bias

the encoding of cued items, the precision of the cued items and uncued items are largely

determined by competition for perceptual, rather than mnemonic, resources. In other words,

selection biases can mitigate the effects of competition, but they cannot eliminate them entirely.

Thus, a competition account could be considered a slots+resource model in which memory

capacity represents a fixed number of slots, while precision is determined by a common pool of

perceptual, rather than mnemonic, resources.

As mentioned above, the present results may also share some similarities with a recent

model proposed by Sligte and colleagues (Sligte et al., 2011). That is, under high competition the

number of non-target errors increase even when the proportion and precision of target responses

is unaffected. This suggests that VWM performance is perhaps affected by additional

representations (i.e., those that are affected by competition). This is consistent with the proposal

by Sligte and colleagues (Sligte et al., 2011) that in addition to a very limited-capacity, high-

resolution and stable VSTM, there is also a lower-resolution, fragile form of VSTM. This

“fragile-VSTM” can store a much larger number of representations than are available to stable-

VSTM, and can therefore affect task performance (e.g., change-detection). In other words,

targets that are the focus of attention may be high-precision representations that are stored in

106

stable-VSTM and correctly recalled; in contrast, those that are outside the focus of attention may

be stored as lower-precision fragile-VSTM representations that are reported with greater error.

These representations may therefore account for the increase in the number of errors under high

competition observed here.

Although it is not fully clear from the present results whether non-target responses reflect

fragile-VSTM representations, it is possible that the model proposed by Sligte and colleagues,

along with a competition account, could account for some of the evidence in support of resource

models of VWM. Specifically, Bays and colleagues (Bays et al., 2009; Bays & Husain, 2008)

have demonstrated convincingly that the precision of responses in memory tasks decrease as a

function of the number of items in the visual scene. These findings suggest that VWM represents

a limited pool of resources that are distributed among all items in the display, but that resources

can be allocated preferentially to particular items (e.g., those that are the targets of planned

saccades). If fragile-VSTM representations can affect performance, however, this could

potentially account for the results of Bays and colleagues, while still supporting a limited-

capacity and high-fidelity VWM. That is, some of the low-precision responses at large set sizes

observed in the studies by Bays could be attributed to the influence of a high-capacity, low-

resolution memory system. Thus, when the number of items is large, the responses are made up

both of low-capacity VWM responses, and higher-capacity fragile-VSTM responses. The

precision of representations stored by both the high- and low-capacity systems may be

determined primarily by the competition for representation in early visual processing. Thus,

competition for perceptual representation could be the limiting resource determining the

precision of responses, as opposed to the limited resource of a single memory system. In other

words, because objects compete for representation and the initial selection and encoding stages,

the precision of responses could appear to follow a continuous exponential function, even though

responses are determined by two different memory systems. Independent of the precise

mnemonic mechanism, a competition account may provide an alternative account for the

systematic decrease in precision that occurs with increasing set size: as the number of competing

representations increase, there is a proportional decrease in the precision of representations

maintained downstream.

As mentioned in the previous chapters, the results of the present thesis are also consistent

with the model of VWM and perception proposed by Xu and Chun (2009). Specifically, the

107

results support the idea that VWM represents a fixed capacity limit, but that the capacity and

precision of VWM may be limited in part by the total amount of information contained within

objects (i.e., fewer high-precision complex items can be maintained in VWM). Furthermore, one

experiment demonstrated that the superior IPS stores only the number of relevant features, not

integrated objects (Xu, 2007). This finding may provide support for the competition account,

since it provides further evidence that the limited memory capacity for complex objects may not

necessarily be the result of storage limitations for complex, bound objects. Instead, the lower

capacity for complex objects may be a consequence of the increased difficulty of representing

complex objects in perceptual regions, and the additional resources necessary for encoding and

resolving the individual features. For example, activation of the superior IPS could represent a

bias signal necessary for selecting and maintaining activation of multiple features in perceptual

regions. Features that are more difficult to perceive and resolve may require greater top-down

selection bias, resulting in a reduced VWM capacity.

One outstanding question is how the activation of brain regions that subserve VWM

encoding and maintenance are affected by competition. That is, it is unclear what effect, if any,

competition has on the activation of the inferior and superior IPS, or the LOC. An interesting

next step in exploring the competition model of VWM would be to examine whether greater

activity is observed in these regions when sample items are presented under high- and low-

competition configurations. Given the distinct patterns of VWM-related activity exhibited by the

iIPS and the sIPS/LOC, any differences in activation under high- and low-competition could

elucidate whether spatial competition primarily affects the selection and individuation of objects

(iIPS) or whether competition affects the encoding and enhancement of target features

(sIPS/LOC). The results of Chapter 9 suggest, however, that competition may affect object

individuation more than feature encoding. That is, both the N2pc and Ptc were affected by the

distance between objects, although this effect was only marginally significant for the Ptc. Thus,

if these ERPs correspond respectively to the processes subserved by the iIPS and sIPS/LOC, this

would indicate that competition may have a greater effect on the selection and individuation of

target objects. Although further research is required to relate these ERPs to the processes

identified using fMRI, this study and others (Adamo, 2010) reveal ERPs as a viable method for

elucidating the distinct neural processes that mediate VWM. It also remains unclear whether

competition between object features has a similar effect on memory performance, although a

108

number of studies have begun to investigate this question (Bays, Wu, & Husain, 2010; Lin &

Luck, 2009).

Conclusions

In summary, based on the results of this thesis a biased-competition model of VWM has

been proposed. According to this model, the fidelity of representations encoded in VWM is

limited by competitive interactions that occur upstream of VWM maintenance. As such, this

model provides a potential neurobiological mechanism for understanding the capacity and

fidelity limitations in VWM: As more items are added to the display, the number of competitive

interactions increases, resulting in less precise VWM representations. Furthermore, this model

appeals to the attentional literature to address the gating-mechanisms of capacity-limited

memory resources. Specifically, bottom-up and top-down selection biases can serve to limit the

effects of competition, and to bias the contents of VWM. Moreover, this model attempts to

explain between-individual differences in VWM capacity by proposing that the capacity and

fidelity of VWM representations may depend on top-down selection mechanisms. In other

words, capacity may not be related to differences in maintenance capacity, but rather to

differences in attentional or executive control. Therefore, this model derived from the results of

this thesis provides a novel framework for studying and interpreting information processing in

visual cognition.

109

References

Adamo, M. (2010). The nature of working memory: Separate, flexible resources for location- vs.

feature-based representations in visual short-term memory. (Unpublished doctoral

dissertation). University of Toronto, Toronto, Ontario, Canada.

Al-Aidroos, N. (2010). On the Reflexive Prioritisation of

Locations in Visual Space (Unpublished doctoral dissertation). University of Toronto,

Toronto, Ontario, Canada.

Alvarez, G. A., & Cavanagh, P. (2004). The capacity of visual short-term memory is set both by

visual information load and by number of objects. Psychological Science, 15(2), 106-111.

Andriessen, J. J., & Bouma, H. (1976). Eccentric vision: adverse interactions between line

segments. Vision Research, 16(1), 71-78.

Awh, E., Barton, B., & Vogel, E. K. (2007). Visual working memory represents a fixed number

of items regardless of complexity. Psychological Science, 18(7), 622-628.

doi:10.1111/j.1467-9280.2007.01949.x

Baddeley, A. D., & Hitch, G. J. (1974). Working memory. In G. H. Bower (Ed.), The Psychology

of Learning and Motivation (Vol. 8, pp. 47-89). New York: Academic Press.

Bahcall, D. O., & Kowler, E. (1999). Attentional interference at small spatial separations. Vision

Research, 39(1), 71-86.

Barton, B., Ester, E. F., & Awh, E. (2009). Discrete resource allocation in visual working

memory. Journal of Experimental Psychology. Human Perception and Performance,

35(5), 1359-1367. doi:10.1037/a0015792

110

Bays, P. M., Catalao, R. F. G., & Husain, M. (2009). The precision of visual working memory is

set by allocation of a shared resource. Journal of Vision, 9(10), 7.1-11.

doi:10.1167/9.10.7

Bays, P. M., & Husain, M. (2008). Dynamic shifts of limited working memory resources in

human vision. Science, 321(5890), 851-854. doi:10.1126/science.1158023

Bays, P. M., & Husain, M. (2009). Response to Comment on “Dynamic Shifts of Limited

Working Memory Resources in Human Vision.” Science, 323(5916), 877.

doi:10.1126/science.1166794

Bays, P. M., Wu, E. Y., & Husain, M. (2010). Storage and binding of object features in visual

working memory. Neuropsychologia. doi:10.1016/j.neuropsychologia.2010.12.023

Beck, D. M., & Kastner, S. (2007). Stimulus similarity modulates competitive interactions in

human visual cortex. Journal of Vision, 7(2), 19.1-12. doi:10.1167/7.2.19

Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10(4), 433-436.

Bundesen, C. (1990). A theory of visual attention. Psychological Review, 97(4), 523–547.

Chelazzi, L., Duncan, J., Miller, E. K., & Desimone, R. (1998). Responses of neurons in inferior

temporal cortex during memory-guided visual search. Journal of Neurophysiology, 80(6),

2918.

Chelazzi, L., Miller, E. K., Duncan, J., & Desimone, R. (1993). A neural basis for visual search

in inferior temporal cortex. Nature, 363(6427), 345–347.

Chelazzi, L., Miller, E. K., Duncan, J., & Desimone, R. (2001). Responses of neurons in

macaque area V4 during memory-guided visual search. Cerebral Cortex, 11(8), 761.

111

Corbetta, M., Kincade, J. M., Ollinger, J. M., McAvoy, M. P., & Shulman, G. L. (2000).

Voluntary orienting is dissociated from target detection in human posterior parietal

cortex. Nature Neuroscience, 3, 292–297.

Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in

the brain. Nature Reviews Neuroscience, 3(3), 215–229.

Cowan, N. (2001). The magical number 4 in short-term memory: a reconsideration of mental

storage capacity. The Behavioral and Brain Sciences, 24(1), 87-114; discussion 114-185.

Cowan, N., & Rouder, J. N. (2009). Comment on “Dynamic shifts of limited working memory

resources in human vision.” Science (New York, N.Y.), 323(5916), 877; author reply 877.

doi:10.1126/science.1166478

Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual

Review of Neuroscience, 18(1), 193–222.

Druzgal, T. J., & D'Esposito, M. (2003). Dissecting contributions of prefrontal cortex and

fusiform face area to face working memory. Journal of Cognitive Neuroscience, 15(6),

771-784. doi:10.1162/089892903322370708

Duncan, J. (1980). The locus of interference in the perception of simultaneous stimuli.

Psychological Review, 87(3), 272-300.

Duncan, J. (1984). Selective attention and the organization of visual information. Journal of

Experimental Psychology. General, 113(4), 501-517.

Duncan, J., & Humphreys, G. W. (1989). Visual search and stimulus similarity. Psychological

Review, 96(3), 433-58.

Eimer, M. (1996). The N2pc component as an indicator of attentional selectivity.

Electroencephalography and Clinical Neurophysiology, 99(3), 225–234.

112

Emrich, S. M., Ruppel, J. D. N., Al-Aidroos, N., Pratt, J., & Ferber, S. (2008). Out with the old:

inhibition of old items in a preview search is limited. Perception & Psychophysics, 70(8),

1552-1557. doi:10.3758/PP.70.8.1552

Emrich, S. M., Al-Aidroos, N., Pratt, J., & Ferber, S. (2009). Visual Search Elicits the

Electrophysiological Marker of Visual Working Memory. PLoS ONE, 4(11), e8042.

doi:10.1371/journal.pone.0008042

Folk, C. L., Remington, R. W., & Johnston, J. C. (1992). Involuntary covert orienting is

contingent on attentional control settings. Journal of Experimental Psychology. Human

Perception and Performance, 18(4), 1030-1044.

Fougnie, D., Asplund, C. L., & Marois, R. (2010). What are the units of storage in visual

working memory? Journal of Vision, 10(12), 27. doi:10.1167/10.12.27

Franconeri, S. L., Jonathan, S. V., & Scimeca, J. M. (2010). Tracking multiple objects is limited

only by object spacing, not by speed, time, or capacity. Psychological Science, 21(7),

920-925. doi:10.1177/0956797610373935

Fukuda, K., Awh, E., & Vogel, E. K. (2010). Discrete capacity limits in visual working memory.

Current Opinion in Neurobiology, 20(2), 177-182. doi:10.1016/j.conb.2010.03.005

Fukuda, K., & Vogel, E. K. (2009). Human variation in overriding attentional capture. The

Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 29(27),

8726-8733. doi:10.1523/JNEUROSCI.2145-09.2009

Fuster, J. M., & Alexander, G. E. (1971). Neuron activity related to short-term memory. Science,

173(3997), 652.

113

Fuster, J. M., & Jervey, J. P. (1981). Inferotemporal neurons distinguish and retain behaviorally

relevant features of visual stimuli. Science (New York, N.Y.), 212(4497), 952-955.

Fuster, J. M., & Jervey, J. P. (1982). Neuronal firing in the inferotemporal cortex of the monkey

in a visual memory task. The Journal of Neuroscience: The Official Journal of the

Society for Neuroscience, 2(3), 361-375.

Geng, J. J., Eger, E., Ruff, C. C., Kristjánsson, A., Rotshtein, P., & Driver, J. (2006). On-line

attentional selection from competing stimuli in opposite visual fields: effects on human

visual cortex and control processes. Journal of Neurophysiology, 96(5), 2601-2612.

doi:10.1152/jn.01245.2005

Ghose, G. M., & Maunsell, J. H. R. (2008). Spatial summation can explain the attentional

modulation of neuronal responses to multiple stimuli in area V4. The Journal of

Neuroscience: The Official Journal of the Society for Neuroscience, 28(19), 5115-5126.

doi:10.1523/JNEUROSCI.0138-08.2008

Griffin, I. C., & Nobre, A. C. (2003). Orienting attention to locations in internal representations.

Journal of cognitive neuroscience, 15(8), 1176-94.

Harrison, S. A., & Tong, F. (2009). Decoding reveals the contents of visual working memory in

early visual areas. Nature, 458(7238), 632-635. doi:10.1038/nature07832

Hilimire, M. R., Mounts, J. R. W., Parks, N. A., & Corballis, P. M. (2009). Competitive

interaction degrades target selection: an ERP study. Psychophysiology, 46(5), 1080-1089.

doi:10.1111/j.1469-8986.2009.00846.x

Hilimire, M. R., Mounts, J. R. W., Parks, N. A., & Corballis, P. M. (2010). Event-related

potentials dissociate effects of salience and space in biased competition for visual

representation. PloS One, 5(9). doi:10.1371/journal.pone.0012677

114

Hopf, J. M., Boelmans, K., Schoenfeld, M. A., Luck, S. J., & Heinze, H. J. (2004). Attention to

features precedes attention to locations in visual search: evidence from electromagnetic

brain responses in humans. The Journal of Neuroscience: The Official Journal of the

Society for Neuroscience, 24(8), 1822-1832. doi:10.1523/JNEUROSCI.3564-03.2004

Hopf, J. M., Luck, S. J., Girelli, M., Hagner, T., Mangun, G. R., Scheich, H., & Heinze, H. J.

(2000). Neural sources of focused attention in visual search. Cerebral Cortex (New York,

N.Y.: 1991), 10(12), 1233-1241.

Huang, L. (2010). Visual working memory is better characterized as a distributed resource rather

than discrete slots. Journal of Vision, 10(14), 8. doi:10.1167/10.14.8

Ihssen, N., Linden, D. E. J., & Shapiro, K. L. (2010). Improving visual short-term memory by

sequencing the stimulus array. Psychonomic Bulletin & Review, 17(5), 680-686.

doi:10.3758/PBR.17.5.680

Irwin, D. E., & Thomas, L. E. (2008). Visual sensory memory. In S. J. Luck & A. Hollingworth

(Eds.), Visual Memory (pp. 9-41). New York: Oxford University Press US.

Jiang, Y. V., Chun, M. M., & Olson, I. R. (2004). Perceptual grouping in change detection.

Perception & Psychophysics, 66(3), 446-453.

Jiang, Y. V., Olson, I. R., & Chun, M. M. (2000). Organization of visual short-term memory.

Journal of Experimental Psychology. Learning, Memory, and Cognition, 26(3), 683-702.

Johnson, J. S., Spencer, J. P., Luck, S. J., & Schöner, G. (2009). A dynamic neural field model of

visual working memory and change detection. Psychological Science, 20(5), 568-577.

doi:10.1111/j.1467-9280.2009.02329.x

115

Kapadia, M. K., Ito, M., Gilbert, C. D., & Westheimer, G. (1995). Improvement in visual

sensitivity by changes in local context: parallel studies in human observers and in V1 of

alert monkeys. Neuron, 15(4), 843-856.

Kapadia, M. K., Westheimer, G., & Gilbert, C. D. (1999). Dynamics of spatial summation in

primary visual cortex of alert monkeys. Proceedings of the National Academy of Sciences

of the United States of America, 96(21), 12073-12078.

Kastner, S., De Weerd, P., Desimone, R., & Ungerleider, L. G. (1998). Mechanisms of directed

attention in the human extrastriate cortex as revealed by functional MRI. Science (New

York, N.Y.), 282(5386), 108-111.

Kastner, S., De Weerd, P., Pinsk, M. A., Elizondo, M. I., Desimone, R., & Ungerleider, L. G.

(2001). Modulation of sensory suppression: implications for receptive field sizes in the

human visual cortex. Journal of Neurophysiology, 86(3), 1398-1411.

Kastner, S., & Ungerleider, L. G. (2001). The neural basis of biased competition in human visual

cortex. Neuropsychologia, 39(12), 1263-1276.

Kleiner, M., Brainard, D. H., Pelli, D., Ingling, A., Murray, R., & Broussard, C. (2007). What’s

new in Psychtoolbox-3. Perception, 36.

Levi, D. M. (2008). Crowding - an essential bottleneck for object recognition: a mini-review.

Vision research, 48(5), 635-654. doi:10.1016/j.visres.2007.12.009

Lin, P.-H., & Luck, S. J. (2009). The Influence of Similarity on Visual Working Memory

Representations. Visual Cognition, 17(3), 356-372. doi:10.1080/13506280701766313

Louie, E. G., Bressler, D. W., & Whitney, D. (2007). Holistic crowding: selective interference

between configural representations of faces in crowded scenes. Journal of Vision, 7(2),

24.1-11. doi:10.1167/7.2.24

116

Luck, S. J., & Hillyard, S. A. (1994a). Electrophysiological correlates of feature analysis during

visual search. Psychophysiology, 31(3), 291-308.

Luck, S. J., & Hillyard, S. A. (1994b). Spatial filtering during visual search: evidence from

human electrophysiology. Journal of Experimental Psychology. Human Perception and

Performance, 20(5), 1000-1014.

Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and

conjunctions. Nature, 390(6657), 279-281. doi:10.1038/36846

Magen, H., Emmanouil, T., McMains, S. A., Kastner, S., & Treisman, A. (2009). Attentional

demands predict short-term memory load response in posterior parietal cortex.

Neuropsychologia, 47(8-9), 1790-1798. doi:10.1016/j.neuropsychologia.2009.02.015

Makovski, T., Sussman, R., & Jiang, Y. V. (2008). Orienting attention in visual working memory

reduces interference from memory probes. Journal of experimental psychology, 34(2),

369-80.

Marois, R., & Ivanoff, J. (2005). Capacity limits of information processing in the brain. Trends

in Cognitive Sciences, 9(6), 296–305.

Mathôt, S., Hickey, C., & Theeuwes, J. (2010). From reorienting of attention to biased

competition: evidence from hemifield effects. Attention, Perception & Psychophysics,

72(3), 651-657. doi:10.3758/APP.72.3.651

McCarley, J. S., & Mounts, J. R. W. (2007). Localized attentional interference affects object

individuation, not feature detection. Perception, 36(1), 17-32.

McCarley, J. S., & Mounts, J. R. W. (2008). On the relationship between flanker interference and

localized attentional interference. Acta Psychologica, 128(1), 102-109.

doi:10.1016/j.actpsy.2007.10.005

117

McCarley, J. S., Mounts, J. R. W., & Kramer, A. F. (2004). Age-related differences in localized

attentional interference. Psychology and Aging, 19(1), 203-210. doi:10.1037/0882-

7974.19.1.203

McCollough, A. W., Machizawa, M. G., & Vogel, E. K. (2007). Electrophysiological measures

of maintaining representations in visual working memory. Cortex, 43(1), 77-94.

McNab, F., & Klingberg, T. (2008). Prefrontal cortex and basal ganglia control access to

working memory. Nature Neuroscience, 11(1), 103–108.

Mitchell, D. J., & Cusack, R. (2008). Flexible, capacity-limited activity of posterior parietal

cortex in perceptual as well as visual short-term memory tasks. Cerebral Cortex, 18(8),

1788-98.

Miyashita, Y., & Chang, H. S. (1988). Neuronal correlate of pictorial short-term memory in the

primate temporal cortex. Nature, 331(6151), 68-70. doi:10.1038/331068a0

Mounts, J. R. W. (2000). Evidence for suppressive mechanisms in attentional selection: feature

singletons produce inhibitory surrounds. Perception & Psychophysics, 62(5), 969-983.

Mounts, J. R. W., & Gavett, B. E. (2004). The role of salience in localized attentional

interference. Vision Research, 44(13), 1575-1588. doi:10.1016/j.visres.2004.01.015

Mounts, J. R. W., McCarley, J. S., & Terech, A. M. (2007). Attentional templates regulate

competitive interactions among attended visual objects. Perception & Psychophysics,

69(2), 209-217.

Movshon, J. A., Thompson, I. D., & Tolhurst, D. J. (1978). Spatial summation in the receptive

fields of simple cells in the cat's striate cortex. The Journal of Physiology, 283, 53-77.

Nakamura, K., & Kubota, K. (1995). Mnemonic firing of neurons in the monkey temporal pole

during a visual recognition memory task. Journal of Neurophysiology, 74(1), 162-178.

118

Nandy, A. S., & Tjan, B. S. (2007). The nature of letter crowding as revealed by first- and

second-order classification images. Journal of Vision, 7(2), 5.1-26. doi:10.1167/7.2.5

Nasr, S., Moeeny, A., & Esteky, H. (2008). Neural correlate of filtering of irrelevant information

from visual working memory. PloS One, 3(9), e3282. doi:10.1371/journal.pone.0003282

Pashler, H. (1988). Familiarity and visual change detection. Perception & Psychophysics, 44(4),

369–378.

Phillips, W. A. (1974). On the distinction between sensory storage and short-term visual

memory. Perception & Psychophysics, 16(2), 283–290.

Posner, M. I., & Cohen, Y. (1984). Components of visual orienting. Attention and performance

X: Control of language processes, 32, 531–556.

Posner, M. I., Walker, J. A., Friedrich, F. J., & Rafal, R. D. (1984). Effects of parietal injury on

covert orienting of attention. Journal of Neuroscience, 4(7), 1863.

Postle, B. R. (2006). Working memory as an emergent property of the mind and brain.

Neuroscience, 139(1), 23–38.

Postle, B. R., Berger, J. S., & D'Esposito, M. (1999). Functional neuroanatomical double

dissociation of mnemonic and executive control processes contributing to working

memory performance. Proceedings of the National Academy of Sciences of the United

States of America, 96(22), 12959-12964.

Postle, B. R., Druzgal, T. J., & D'Esposito, M. (2003). Seeking the neural substrates of visual

working memory storage. Cortex; a Journal Devoted to the Study of the Nervous System

and Behavior, 39(4-5), 927-946.

119

Schmidt, B. K., Vogel, E. K., Woodman, G. F., & Luck, S. J. (2002). Voluntazy and automatic

attentional control of visual working memory. Perception & Psychophysics, 64(5), 754-

763.

Serences, J. T., Ester, E. F., Vogel, E. K., & Awh, E. (2009). Stimulus-specific delay activity in

human primary visual cortex. Psychological Science, 20(2), 207-214. doi:10.1111/j.1467-

9280.2009.02276.x

Shulman, G. L., d'Avossa, G., Tansy, A. P., & Corbetta, M. (2002). Two attentional processes in

the parietal lobe. Cerebral Cortex, 12(11), 1124.

Sligte, I. G., Scholte, H. S., & Lamme, V. A. F. (2008). Are there multiple visual short-term

memory stores? PloS One, 3(2), e1699. doi:10.1371/journal.pone.0001699

Sligte, I. G., Vandenbroucke, A. R. E., Scholte, H. S., & Lamme, V. A. F. (2011). Detailed

sensory memory, sloppy working memory. Frontiers in Psychology, 1(0), 12.

doi:10.3389/fpsyg.2010.00175

Soto, D., Heinke, D., Humphreys, G. W., & Blanco, M. J. (2005). Early, involuntary top-down

guidance of attention from working memory. Journal of Experimental Psychology:

Human Perception and Performance, 31, 248–261.

Soto, D., Humphreys, G. W., & Rotshtein, P. (2007). Dissociating the neural mechanisms of

memory-based guidance of visual selection. Proceedings of the National Academy of

Sciences, 104(43), 17186.

Sperling, G. (1960). The information available in brief visual presentations. Psychological

Monographs: General and Applied, 741, 1–29.

Todd, J. J., & Marois, R. (2004). Capacity limit of visual short-term memory in human posterior

parietal cortex. Nature, 428(6984), 751-754. doi:10.1038/nature02466

120

Todd, J. J., & Marois, R. (2005). Posterior parietal cortex activity predicts individual differences

in visual short-term memory capacity. Cognitive, Affective, and Behavioral

Neuroscience, 5, 144–155.

Toet, A., & Levi, D. M. (1992). The two-dimensional shape of spatial interaction zones in the

parafovea. Vision Research, 32(7), 1349-1357.

Umemoto, A., Drew, T., Ester, E. F., & Awh, E. (2010). A bilateral advantage for storage in

visual working memory. Cognition, 117(1), 69-79. doi:10.1016/j.cognition.2010.07.001

Vogel, E. K., & Machizawa, M. G. (2004). Neural activity predicts individual differences in

visual working memory capacity. Nature, 428(6984), 748-751. doi:10.1038/nature02447

Vogel, E. K., McCollough, A. W., & Machizawa, M. G. (2005). Neural measures reveal

individual differences in controlling access to working memory. Nature, 438(7067), 500-

503. doi:10.1038/nature04171

Vogel, E. K., Woodman, G. F., & Luck, S. J. (2001). Storage of features, conjunctions and

objects in visual working memory. Journal of Experimental Psychology. Human

Perception and Performance, 27(1), 92-114.

Vogel, E. K., Woodman, G. F., & Luck, S. J. (2006). The time course of consolidation in visual

working memory. Journal of Experimental Psychology. Human Perception and

Performance, 32(6), 1436-1451. doi:10.1037/0096-1523.32.6.1436

Wilken, P., & Ma, W. J. (2004). A detection theory account of change detection. Journal of

Vision, 4(12), 1120-1135. doi:10:1167/4.12.11

Woodman, G. F., Arita, J. T., & Luck, S. J. (2009). A cuing study of the N2pc component: an

index of attentional deployment to objects rather than spatial locations. Brain Research,

1297, 101-111. doi:10.1016/j.brainres.2009.08.011

121

Xu, Y. (2007). The role of the superior intraparietal sulcus in supporting visual short-term

memory for multifeature objects. Journal of Neuroscience, 27(43), 11676-11686.

Xu, Y., & Chun, M. M. (2006). Dissociable neural mechanisms supporting visual short-term

memory for objects. Nature, 440(7080), 91-95. doi:10.1038/nature04262

Xu, Y., & Chun, M. M. (2009). Selecting and perceiving multiple visual objects. Trends in

Cognitive Sciences, 13(4), 167–174.

Yantis, S., & Johnson, D. N. (1990). Mechanisms of attentional priority. Journal of Experimental

Psychology. Human Perception and Performance, 16(4), 812-825.

Zhang, W., & Luck, S. J. (2008). Discrete fixed-resolution representations in visual working

memory. Nature, 453(7192), 233-235. doi:10.1038/nature06860

Zhang, W., & Luck, S. J. (2009). Sudden Death and Gradual Decay in Visual Working Memory.

Psychological Science, 20(4), 423-428. doi:10.1111/j.1467-9280.2009.02322.x

Documents

Competition in Visual Working Memory by Stephen M - T-Space