Working memory paradox: Using working memory distorts … · 2020. 6. 19. · Working memory...

Working memory paradox: Using working memory distorts working memoryKeisuke Fukuda1,2, April E. Pereira3, Joseph M. Saito2, Hiroyuki Tsubomi4, & Gi-Yeul Bae5

1University of Toronto Mississauga, 2University of Toronto, 3University of Waterloo,4University of Toyama, & 5Arizona State University

Base

line

Sim

ilarit

yJu

dgm

ent

Memory item Memory itemSame-side probe Opposite-side probe

1st probe pair2nd probe pair

Sim

ilarit

yJu

dgm

ent “Similar or not?”

“Remember”

“Remember”

“Recall” “Adjust” “Confidence?”

1 2 3

“Recall” “Adjust” “Confidence?”

1 2 3800ms

+800ms delay1600ms

+ 800ms delay Until response Until response Until response

Base

line

1500ms+500ms delay

2000ms+ 500ms delay

2000ms+ 500ms delay Until response Until response

“More similar?” “More similar?”

“Remember”

“Remember”

“Recall” “Adjust” “Confidence?”

1 2 3

“Recall” “Adjust” “Confidence?”

1 2 3Until response

-180 -90 0 90 1800

0.01

0.02

0.03

Res

pons

e pr

opor

tion

Res

pons

e pr

opor

tion

-180 -90 0 90 1800

0.01

0.02

0.03

Signed response offset (°)

Signed response offset (°)

BaselineSame-sideOpposite-side

Col

orSh

ape

Res

pons

e pr

opor

tion

-180 -90 0 90 1800

0.01

0.02

0.03

-180 -90 0 90 1800

0.01

0.02

0.03

Res

pons

e pr

opor

tion

Signed response offset (°) for HC responses

Res

pons

e pr

opor

tion

for

phys

ical

ly-id

entic

al p

robe

tria

ls

Signed response offset (°) for HC responses-180 -90 0 90 1800

0.01

0.02

0.03

-180 -90 0 90 1800

0.01

0.02

0.03

Res

pons

e pr

opor

tion

for

phys

ical

ly-id

entic

al p

robe

tria

ls

HC memory precision (кM)

Col

orSh

ape

D. Prediction: Similarity judgment biases VWM H. Prediction: Perceived similarity induces attraction bias

I. Attraction or repulsion? J. Physical or perceived similarity?

C. Exp 1: Inducing and cancelling VWM bias by similarity judgments G. Exp 2: Examining the underlying mechanism of VWM bias K. Exp 3: Predicting biasA. Introduction

B. Stimuli L. DiscussionE. VWM bias? F. Driven by low-confidence reports?

-180 -90 0 90 1800

0.01

0.02

0.03

-180 -90 0 90 1800

0.01

0.02

0.03

If using VWM for similarity judgments distorts VWM,the response offset distribution should be

systematically biased in the same-side probe condition.

Furthermore, the systematic bias should be‘cancelled out’

in the opposite-side probe condition.

Confidence report: 1 = High confidence, 2 = Low confidence, 3 = No confidenceProbe range: Similar probe = 16º-45º away from target, Dissimilar probe = 180º away from similar probe

Confidence report: 1 = High confidence, 2 = Low confidence, 3 = No confidenceProbe range: Similar probe = 16º-105º away from target

If a similar probe attracts VWM representations,VWM should be biased toward the probe

when it is judged as “similar” to VWM.

If perceived similarity drives the VWM bias,physically-identical probes should produce different

VWM bias depending on the perceived similarity.

Probe

“Similar!”

Attraction bias

Probe

“Dissimilar!”

Negligible bias

Probe

“Similar!”

Larger bias

Probe

“Dissimilar!”

Smaller bias

Signed response offset (°) for HC responses

Signed response offset (°) for HC responses

Res

pons

e pr

opor

tion

Res

pons

e pr

opor

tion

Similarity judgments bias VWM toward a similar probe.

VWM bias occurs even whenparticipants are highly confident (HC) in their VWM. VWM is attracted toward a probe.

Signed response offset (°) for HC responses

Signed response offset (°) for HC responses

Perceived similarity, but not physical similarity,drives VWM bias.

Individuals with lower VWM precisionare more vulnerable to VWM bias.

The primary goal of visual working memory (VWM) is to accurately maintain a limited amount of task-relevant visual information in mind so that it can be used to guide our behavior.

Studies have examined the robustness of VWM representations by imposing task-irrelevant distractions duringencoding and maintenance.

However, no studies have examined whether VWM representations remain intact when they are used to accomplish a directly-relevant task.

Considering that many tasks in real life require repeated usage of the same VWM representation (e.g., visual search), it is imperative to investigate whether VWM survives its usage without asystematic distortion.

Is VWM distorted when it isused for a task-relevantperceptual comparison?

Research Question:

VWM representation is attracted toward new visual input especially when the input is perceived to be similar to VWM representation (VWM bias).

Cancellation of VWM bias in theopposite-side probe condition (Exp. 1) and the lack of repulsion bias following dissimilar judgments (Exp. 2)demonstrate that VWM encoding noise cannot explain VWM bias, thusimplicating the causal role of similarity judgments.

Individuals with lower VWM precisionexhibit larger VWM bias.

These findings can be explained by ahypothesis that representationalintegration between VWM and new“perceived-to-be-similar” visual inputunderlies VWM bias.

Circular Color Space(Zhang & Luck, 2008)

Circular Shape Space(Li et al., 2017)

Li, A. Y., Liang, J. C., Lee, A. C. H., & Barense, M. D. (2019). The validated circular shape space: Quanti-fying the visual similarity of shape. J Exp Psychol Gen. doi:10.1037/xge0000693Zhang, W., & Luck, S. J. (2008). Discrete fixed-resolution representations in visual working memory. Nature, 453(7192), 233-235. doi:10.1038/nature06860

BaselineSame-sideOpposite-side

BaselineSame-sideOpposite-side

Mean signed response offsetSame-side: 6.5° ***Opposite-side: -1.5° *

Signed response offset(°):positive offset = offset toward the first similar probe

Mean signed response offsetSame-side: 6.4° ***Opposite-side: 0.0°

Mean signed response offsetSame-side: 6.1° ***Opposite-side: -1.3° *

BaselineSame-sideOpposite-side

Mean signed response offsetSame-side: 5.6° ***Opposite-side: -0.4°

p-value* < 0.05** < 0.01*** < 0.001

p-value* < 0.05** < 0.01*** < 0.001

BaselineSimilarDissimilar

Mean signed response offsetSimilar: 8.1° ***Dissimilar: 3.0° ***Sim > Dissimilar **

Mean signed response offsetSimilar: 7.6° ***Dissimilar: 2.9° ***Sim > Dissimilar **

BaselineSimilarDissimilar

BaselineSimilarDissimilar

BaselineSimilarDissimilar

Mean signed response offsetSimilar: 11.4° ***Dissimilar: 1.3°Sim > Dissimilar ***

Mean signed response offsetSimilar: 9.9° ***Dissimilar: 2.5° **Sim > Dissimilar **

Mea

n si

gned

resp

onse

offt

et (°

)fo

r HC

resp

onse

sVWM VWM VWM VWM

Signed response offset(°):positive offset = offset toward the first similar probe

Signed response offset(°):positive offset = offset toward probe

Signed response offset(°):positive offset = offset toward probe

0 30 60 90120-10

0

10

20

0 30 60 90120-10

0

10

20

P|sP,κP )Memory item:

p(xM|sM,κM)Probe item:

p(xP|sP,κP)Joint density:p(xM|sM,κM)p(x

Prob

abilit

y

X X X

Representational integration hypothesis:VWM bias is caused by representational

integration between VWM and perceptually-similar probe.

r = -0.37p < 0.001

r = -0.33p < 0.001

Color Shape

If so, individuals with lower VWM precision should show larger VWM bias due to

larger integration window.

Similar probe = 16º-45º away from targetDissimilar probe = 180º away from similar probe

N = 28 N = 16

N = 110