TEMPORAL DYNAMICS OF META-COGNITION IN A CONTINUOUS …msl/posters/lockeetal17b.pdf · 2017. 9. 8. · TEMPORAL DYNAMICS OF META-COGNITION IN A CONTINUOUS VISUOMOTOR TASK Shannon

TEMPORAL DYNAMICS OF META-COGNITIONIN A CONTINUOUS VISUOMOTOR TASK

Shannon M. Locke 1 - Michael S. Landy 1,2 - Pascal Mamassian 4 - Eero P. Simoncelli 1,2,3

New York University, NY: (1) Dept. of Psychology, (2) Center for Neural Science, (3) Courant Institute of Mathematical Sciences;(4) Laboratoire des Systemes Perceptifs, CNRS UMR 8248, Departement d’Etudes Cognitives, Ecole Normale Superieure, Paris, France

DÉPARTEMENTD’ÉTUDESCOGNITIVES

Exp. 1 Summary

Even in simple perceptual tasks, humans may rely onheuristics to assign confidence1,2. Yet, in a complexsensorimotor tracking task we found participants

monitored error (sub-optimally) to judge confidence.

Exp. 2 Summary

A robust recency e↵ect in the weighting of trackingerror was replicated when trial duration was uncertain.This result could be due to lossy accumulation of theerror signal or use of memory in assigning confidence.

Methods

1. track random-walk stimulus

target*

1 SD cursor

Dynamic Visual Localization With Moving Dot CloudsShannon M. Locke 1 - Michael S. Landy 1,2 - Pascal Mamassian 4 - Eero P. Simoncelli 1,2,3

New York University, NY: (1) Department of Psychology, (2) Center for Neural Science, (3) Courant Institute of Mathematical Sciences;(4) Laboratoire des Systèmes Perceptifs, CNRS UMR 8248, Département d’Études Cognitives, École Normale Supérieure, Paris, France

1. Task 2. Performance 3. Confidence

Acknowledgements: NSF BCS-1430262 Correspondence: [email protected]

Second Year Paper 25th September 2016

in temporal averaging and motor constraints, it is not possible to estimate motor noise

just yet and analysis will be restricted to the computational lag.

Results & Discussion

A cross-correlation analysis was performed to determine the lag between the target lo-

cation and the cursor placement for every single trial. The cross-correlation values were

normalised to produce a correlation coe�cient by subtracting the mean and dividing by

the standard deviation for each lag value examined. Cross-correlograms were first av-

eraged within subjects, with the peak used as the estimate of the subject’s preferred

tracking lag, � , for the model fitting(see Figure 4a). Averaging these cross-correlograms

indicates that normal, healthy adult is expected to have a � of approximately 400 ms.

Shown in Figure 4b is the distribution of peak lags across trials for each subject. The

histograms are all positively skewed and resemble those observed for reaction times in

traditional psychophysics tasks (Usher & McClelland, 2001). However, a one-sample t-

test on the Pearson moment coe�cient of skewness for each subject’s sample of peak lags

did not reveal a significant e�ect (t(4)=2.24 , p = 0.09). This result is not unexpected

given the small sample size of this pilot experiment. Subjects are encouraged by the

points system of the experiment to track the target quickly, but doing so may reduce

�2 0 20.2

0.4

0.6

0.8

1

400 ms lag

Tracking Lag (sec)

Cor

rela

tion

(A)

0.3 0.4 0.5 0.6 0.7 0.80

5

10

15

20

Peak Tracking Lag (sec)

Per

cent

(B)

Figure 4: A) Mean cross-correlations for individual subjects (black) and averaged across sub-jects (red). Peak lag for the red trace is 400 ms. B) Individual histograms showing the percentof traces with a particular peak lag (smoothed with a Gaussian filter with sd= 20 ms).

Page 17


0.2 0.3 0.4 0.5 0.60

10

20

30�c = 67ms

Peak Tracking Lag (sec)

Per

cent

cloudtarget

Figure 5: A comparison of the lag in tracking for one subject in a task where the target isinferred from the dot cloud (“cloud”, blue), and a task where the target is visible (“target”,orange). The dashed lines indicate peak of the mean cross-correlation from each of the tasks.The distance between these peaks corresponds to the time to compute the centroid, �centroid

(abbr. in figure), assuming that temporal lag due to sensory processing and motor executionare the same in both tasks.

the accuracy of their tracking for various reasons such as reduced time in estimating the

centroid or planning and executing a movement. Therefore, peak tracking time may be a

useful measure of speed in a speed-accuracy trade-o� analysis. Current attempts at this

analysis suggest that more data needs to be collected.

The same cross-correlation analysis was applied to the tracking data from the task

where the target was made visible (see Figure 5). As expected, the peak lag for the ex-

plicit target experiment is lower than for the peak lag for the experiment requiring the

subject to infer target location from the dot cloud. This indicates the subject takes ap-

proximately 70 ms to compute the centroid of the dot cloud and 300 ms to both process

the sensory information and execute a movement. Another noticeable di�erence between

the two distributions of peak lags is that the target visible distribution has a smaller vari-

ance, indicating that computing the centroid contributes considerably to the variability

in tracking delays. Viewing the results in another way, one could say that decreasing the

di�culty of the task by providing the target’s true location led to faster responses. This is

consistent with traditional decision-making experiments (Gold & Shadlen, 2007). Further

experiments are needed to see if manipulating the quality of the sensory information in

the dot cloud (i.e. the number of dots) will similarly a�ect tracking lag.

Tracking behaviour was first compared to the standard Kalman filter described in

Page 18


1 2 3 4 50

1

2

3

4

Standard Kalman Filter

Subject

RM

SE

(deg

)

Human TrackingLagged Kalman Filter

(A)

1 2 3 4 5

0

1

2

3

Subject

�R

MSE

(deg

)

Standard Kalman FilterLagged Kalman Filter

(B)

Figure 6: Comparison of errors as compared to the true target location for 1) the actual trackingbehaviour of the subject, 2) the unadjusted Kalman filter, and 3) a Kalman filter lagged by thesubject’s peak tracking lag. Error bars and shaded error region represent the 95% confidenceintervals.

Part 2. This model is equivalent to a human that has no temporal averaging or internal

sensory noise and can instantaneously place the cursor on the estimated target location

without any motor noise. Figure 6a shows that the human performance is approximately

three times worse than this model when assessed in terms of the Root Mean Square Error

(RMSE). If, however, the output of this Kalman filter is shifted by � estimate of the

corresponding subject, the performance is indistinguishable from that of the human4. It

is unlikely that more tracking trial will reduce the spread of errors as Subject 1 in the plot

completed 2.5 times more trials than the other subjects. Figure 6b plots the di�erence

between the subject’s tracking and the two models on a per trial basis. Again, it is not

possible to di�erentiate the lagged Kalman filter and human performance. This suggests

adding in the additional components of the model may be tricky if RMSE is used as the

metric of fit.

To conclude, a lagged version of the standard Kalman filter did very well at fitting

human performance. The average tracking lag was very consistent across subjects, and

the distributions of peak tracking lag tended to follow the pattern observed for reaction

times in non-tracking experiments.

4Do you think it is a problem I am using the same sequences to estimate tracking lag and assess thefit of the model?

Page 19

Part 2. Modelling Approach

This section outlines the development of a Bayesian ideal observer model for a tracking

task where subjects track a moving cloud of dots as moves along a one-dimensional ran-

dom walk trajectory (see Figure 1). We selected the Kalman filter, which is the optimal

recursive linear estimator suited to dynamic environments assuming all noise in the sys-

tem is Gaussian. This is the “decision” component of the model. The Kalman filter is

biologically plausible is because it does not require infinite memory, yet considers every

piece of sensory evidence given.

The decision, “where to move next?”, is answered by this model, but requires realistic

inputs and outputs. We consider several human perception factors that act on the sensory

input and motor outputs, as well as modify variables in the decision-making machinery. A

schematic diagram of the final model is shown at the end of this section (Figure 3). This

model gives, for a particular sensory input, the ideal tracking performance achievable.

(A)

0 2 4 6 8 10 12 14 16 18 20

�10

0

10

Time (sec)

Tar

get

µt(d

eg)

(B)

Figure 1: A) Example of a single display frame. White dots are sampled from a 2D Gaussiandistribution (red dot indicates mean and dashed line shows 1 SD, both not visible to subjects).Black dot is the cursor position as set by the subject. B) An example horizontal random-walktrajectory of the target.

6

Part

2.

Model

ling

Appro

ach

This

sect

ion

outl

ines

the

dev

elop

men

tof

aB

ayes

ian

idea

lob

serv

erm

odel

for

atr

acki

ng

task

wher

esu

bje

cts

trac

ka

mov

ing

clou

dof

dot

sas

mov

esal

ong

aon

e-dim

ensi

onal

ran-

dom

wal

ktr

aje

ctor

y(s

eeFig

ure

1).

We

sele

cted

the

Kal

man

filt

er,w

hic

his

the

opti

mal

recu

rsiv

elinea

res

tim

ator

suit

edto

dyn

amic

envi

ronm

ents

assu

min

gal

lnoi

sein

the

sys-

tem

isG

auss

ian.

This

isth

e“d

ecis

ion”

com

pon

ent

ofth

em

odel

.T

he

Kal

man

filt

eris

bio

logi

cally

pla

usi

ble

isbec

ause

itdoe

snot

requ

ire

infinit

em

emor

y,ye

tco

nsi

der

sev

ery

pie

ceof

senso

ryev

iden

cegi

ven.

The

dec

isio

n,“w

her

eto

mov

enex

t?”,

isan

swer

edby

this

mod

el,but

requ

ires

real

isti

c

inputs

and

outp

uts

.W

eco

nsi

der

seve

ralh

um

anper

cepti

onfa

ctor

sth

atac

ton

the

senso

ry

inputan

dm

otor

outp

uts

,as

wel

lasm

odify

vari

able

sin

the

dec

isio

n-m

akin

gm

achin

ery.

A

schem

atic

dia

gram

ofth

efinal

mod

elis

show

nat

the

end

ofth

isse

ctio

n(F

igure

3).

This

mod

elgi

ves,

for

apar

ticu

lar

senso

ryin

put,

the

idea

ltr

acki

ng

per

form

ance

achie

vable

.

(A)

02

46

810

1214

1618

20

�10010

Tim

e(s

ec)

Targetµt(deg)

(B)

Fig

ure

1:A

)E

xam

ple

ofa

singl

edis

pla

yfr

ame.

Whit

edot

sar

esa

mple

dfr

oma

2DG

auss

ian

dis

trib

uti

on(r

eddot

indic

ates

mea

nan

ddas

hed

line

show

s1

SD

,bot

hnot

visi

ble

tosu

bje

cts)

.B

lack

dot

isth

ecu

rsor

pos

itio

nas

set

byth

esu

bje

ct.B

)A

nex

ample

hor

izon

talra

ndom

-wal

ktr

aje

ctor

yof

the

targ

et.

6

Mean cross-correlograms of target and cursor:

subject’s appear to lag behind the stimulus by 400

ms (add legend). Human error versus model error:

Applying temporal lag to a Kalman filter Oh look, simple Kalman filter with

temporal lag does a good job at explaining

error (difference is between human and

model). (make fill versus no fill)Control experiment (tracking

visible target): Tracking lag not just motor response time, reflects something about computation (CCG). Include tracking lag

distribution for standard non-lagged model (simulations), this

shows the contribution of the exponential decay weighting

function to lag.

Result 1: ROC curves: show all Ss on plot, mean area (SEM) AUROC. Oh

look, there appears to be meta-cognitive

sensitivity going on!

Variable Sensory SD Variable Walk SD

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

127226

387741

906998

−1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0Mean Subtracted RMSE (deg)

Den

sity conf

HighLow

Metacognitive Sensitivity


0102030

0102030

0102030

0102030

0102030

0102030

127226

387741

906998


Cou

nt

confHighLow


Figure 2.3: ...

15


0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0127

226387

741906

998


Den

sity conf

HighLow



0102030

0102030

0102030

0102030

0102030

0102030

127226

387741

906998


Cou

nt

confHighLow


Figure 2.3: ...

15

mean subtracted RMSE

# re

spon

ses


A = 0.59

A = 0.78

A = 0.6

A = 0.67

A = 0.79

A = 0.76

A = 0.6

A = 0.68

A = 0.72

A = 0.63

A = 0.83

A = 0.72

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

127226

387741

906998

0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00P(Low Conf)

P(Hi

gh C

onf)

ROC Analysis of Metacognitive Sensitivity

Figure 2.4: Metacognitive ROC curves were computed non-parametrically (i.e. using the raw data)for each observer in each condition. The area under the curve was also computed non-parametricallyusing the trapezoidal method detailed in Macmillan and Creelman (2005) on pg. 64. An area of 0.5,or a curve that falls along the dashed line, indicates no metacognitive sensitivity. An area of 1, or acurve that follows the left and top border, indicates the upper limit of metacognitive sensitivity.

16


A = 0.59

A = 0.78

A = 0.6

A = 0.67

A = 0.79

A = 0.76

A = 0.6

A = 0.68

A = 0.72

A = 0.63

A = 0.83

A = 0.72

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

127226

387741

906998

0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00P(Low Conf)

P(Hi

gh C

onf)



16


A = 0.59

A = 0.78

A = 0.6

A = 0.67

A = 0.79

A = 0.76

A = 0.6

A = 0.68

A = 0.72

A = 0.63

A = 0.83

A = 0.72

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

127226

387741

906998

0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00P(Low Conf)

P(Hi

gh C

onf)



16

A = 0.70 +/- 0.04 A = 0.70 +/- 0.03

Secon

dY

earPap

er25

thSep

tember

2016


the Kalman filter is supplied with a prior estimate x0 = 0 and the associated error

covariance P0 = 0, which will be updated as measurements are made.

In the one-dimensional case of horizontal tracking, the Kalman gain is simply

Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)

where �cloud is the standard deviation of the generating Gaussian distribution for dot

locations and �walk is the standard deviation in the random walk. This Kalman gain term

describes the previous estimate of target location, xt�1, is combined with the incoming

measurements to update the estimate of target location xt:

xt = xt�1 + Kt(zt � xt�1) (3)

With each iteration of the Kalman filter loop, the error covariance term, Pt, is updated

as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)

Internal Sensory Noise

The dots presented in the task are intentionally high contrast, so the subjects will be

easily able to identify their locations. However, it is still likely that internal sensory noise

is also contributing to tracking behaviour. This can be estimated in a simple 2IFC visual

discrimination task, where two dots are sequentially presented and subjects have to judge

if the second dot was to the left or right of the first. The internal sensory noise estimate

would be calculated from the Just Noticeable Di�erence (JND)

�internal =JND�

2(5)

We could adjust the Kalman filter modifying equations 1 and 2. The estimate of the

Page 4





Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)





xt = xt�1 + Kt(zt � xt�1) (3)


as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)








�internal =JND�

2(5)


Page 4


in the following manner1

ct =

��

��

0 + �motor, if t � � � 0

xt�� + �motor, otherwise

(9)

We can estimate � by finding the delay which produces the highest correlation between

target location and mouse cursor in the tracking task. A simple addition to the main

tracking task, however, can reveal more about these temporal delays as well as provide

a way to estimate motor noise. If we place a dot at the true position of the target,

identifiable by its red colour, and ask subjects to track this dot and ignore white dot

cloud, we have removed the centroid computation step. The delay with the highest

target-cursor correlation could be expressed as

�visible = �sensory + �motor (10)

Thus, the di�erence in temporal lag between having the target invisible or not tells us

the time subjects are using to compute the centroid of the dot cloud2

�centroid = �invisible � �visible (11)

Additionally, we can computed the RMSE in tracking behaviour in the target visible

experiment, after shifting the cursor trace by �visible, as an estimate of �motor. The corre-

sponding RMSE in the main task would also include error in estimating the target, which

is why we wouldn’t use it to estimate motor noise.

1I haven’t thought of a good way to express the cursor placement prior to acting on sensory information.Subjects will have to place the cursor at the center of the screen (with some small spatial tolerance) soit should be around 0, but then they might move it about a bit, and then of course those movements arenot independent of each other...

2I’m not entirely convinced if this is correct. In this version of the task �dot = 0, and so the Kalmangain will be di�erent in the sense they will move closer to their estimate than in the version where thetarget is invisible. Would this correspond to a di�erent lag?

Page 6


Temporal Averaging

Dot clouds are presented very rapidly in the tracking task. The perceptual consequence

of this is that several dot clouds will appear on the screen together due to the temporal

averaging of the visual system. I can imagine modelling this using a temporal weighting

function which includes the previous dot locations in the computation of the centroid.

Let’s begin by redefining the centroid computation

gt =1

J

J�

j=1

�djt + �internal

�(12)

zt =

t�t�

i=0

�wigt�i�t

�(13)

where �t is the time step of the sampling and wi is the weight from the temporal weighting

function and must be such thatt

�t�

i=0

wi = 1 (14)

The first weighting function we will consider is one where all dots which appear to be

simultaneously presented on the screen are given equal weight (i.e. a step function):

wi =

��

��

1�blur�t , if i�t � �blur

0, otherwise

(15)

where �blur is the length of time over which the stimulus is temporally averaged. Alterna-

tive temporal weighting functions would weight dots according to when they where first

presented. For example, this could be done using a Gaussian function wi � N(0, �blur).

Another possibility is an exponential function, but this seems like it would be hard to

disentangle from the operation of the Kalman filter itself.

Page 7


Temporal Averaging






gt =1

J

J�

j=1


�(12)

zt =

t�t�

i=0

�wigt�i�t

�(13)



�t�

i=0

wi = 1 (14)



wi =

��

��


0, otherwise

(15)






Page 7


target via computing the centroid would be corrupted by Gaussian noise as follows

zt =1

J

J�

j=1


�(6)

where �internal � N(0, �internal), and the Kalman gain would also include the internal noise

in addition to the external noise from the dot cloud

Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud + �2internal

(7)

Motor Noise

In addition to the spatial blurring of the input caused by internal noise, there will also

be spatial blurring of the output due to motor noise. That is, the cursor ct is placed

at the target’s estimated location, but the movement is corrupted by Gaussian noise

�motor � N(0, �motor):

ct = xt + �motor (8)

An experiment that would provide an estimate of �motor is described in the next section.

Temporal Delays

It should be obvious that the cursor is not placed instantaneously on the target estimate

as soon as the dot cloud is displayed but after some time has elapsed. This temporal

lag, � , is likely the sum of several delays relating to the sensory acquisition of the display,

�sensory; computing the centroid of the dot cloud, �centroid; and the delay in formulating,

sending, and executing a motor plan, �motor. We can update equation 8 to reflect this lag

Page 5





Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)





xt = xt�1 + Kt(zt � xt�1) (3)


as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)








�internal =JND�

2(5)


Page 4





Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)





xt = xt�1 + Kt(zt � xt�1) (3)


as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)








�internal =JND�

2(5)


Page 4

SENSORY: DECISION MAKING: MOTOR:

compute centroidexecute movement

compute Kalman gainupdate error term

update estimate

priors

Figure 3: A schematic of the proposed Kalman filter model for tracking. The three components are: 1) gathering sensory evidence to estimatetarget location; 2) deciding where to move next; and finally, 3) executing the movement of the computer mouse to the intended location.

Page

14

Secon

dY

earPap

er25

thSep

tember

2016





Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)





xt = xt�1 + Kt(zt � xt�1) (3)


as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)








�internal =JND�

2(5)


Page 4





Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)





xt = xt�1 + Kt(zt � xt�1) (3)


as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)








�internal =JND�

2(5)


Page 4



ct =

��

��

0 + �motor, if t � � � 0


(9)


















Page 6


Temporal Averaging






gt =1

J

J�

j=1


�(12)

zt =

t�t�

i=0

�wigt�i�t

�(13)



�t�

i=0

wi = 1 (14)



wi =

��

��


0, otherwise

(15)






Page 7


Temporal Averaging






gt =1

J

J�

j=1


�(12)

zt =

t�t�

i=0

�wigt�i�t

�(13)



�t�

i=0

wi = 1 (14)



wi =

��

��


0, otherwise

(15)






Page 7



zt =1

J

J�

j=1


�(6)



Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2


(7)

Motor Noise







Temporal Delays






Page 5





Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)





xt = xt�1 + Kt(zt � xt�1) (3)


as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)








�internal =JND�

2(5)


Page 4





Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)





xt = xt�1 + Kt(zt � xt�1) (3)


as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)








�internal =JND�

2(5)


Page 4




update estimate

priors


Page

14

0 200 400 600 800 10000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Wei

ght

Time in the Past (ms)

Secon

dY

earPap

er25

thSep

tember

2016





Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)





xt = xt�1 + Kt(zt � xt�1) (3)


as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)








�internal =JND�

2(5)


Page 4





Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)





xt = xt�1 + Kt(zt � xt�1) (3)


as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)








�internal =JND�

2(5)


Page 4



ct =

��

��

0 + �motor, if t � � � 0


(9)


















Page 6


Temporal Averaging






gt =1

J

J�

j=1


�(12)

zt =

t�t�

i=0

�wigt�i�t

�(13)



�t�

i=0

wi = 1 (14)



wi =

��

��


0, otherwise

(15)






Page 7


Temporal Averaging






gt =1

J

J�

j=1


�(12)

zt =

t�t�

i=0

�wigt�i�t

�(13)



�t�

i=0

wi = 1 (14)



wi =

��

��


0, otherwise

(15)






Page 7



zt =1

J

J�

j=1


�(6)



Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2


(7)

Motor Noise







Temporal Delays






Page 5





Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)





xt = xt�1 + Kt(zt � xt�1) (3)


as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)








�internal =JND�

2(5)


Page 4





Kt =Pt�1 + �2

walk

Pt�1 + �2walk + �2

cloud

(2)





xt = xt�1 + Kt(zt � xt�1) (3)


as follows

Pt = (1 � Kt)(Pt�1 + �2walk) (4)








�internal =JND�

2(5)


Page 4




update estimate

priors


Page

14

position or velocity

Dots drawn independently

from 2D Gaussian every frame

(17ms).

Moving dot cloud stimulus:

Follows a !random walk:

Colour scheme: target (red), cursor (), mean across Ss (red), high conf (purple), low conf (pink)

●

●

●

●

●

●

0.5

0.6

0.7

0.8

0.9

0.5 0.6 0.7 0.8 0.9Variable Sensory SD

Varia

ble

Wal

k SD

A Comparison of AUROC values

2AFC Confidence Task: “Do you think your performance

was better or worse than average?”

Number of dots: 3 Cloud SD: …

Tracking duration: 20 sec

Tracking Task Experiment 1: Tracking Behaviour Experiment 2: Confidence

Number of dots: 3 Cloud SD: …(obvious) Walk SD: … (subtle)

Tracking duration: 20 sec

Motivation box?

Result 2: No difference in sensitivity even though one cue is obvious and

one is subtle.

−4 −3 −2 −1 0 1 2 3 40

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Target Position (deg)

Prop

ortio

n of

Rig

htw

ard

Judg

emen

ts

PFbyConf: sID = 690, nDots = 5, cloudSD = 2.5

Num

ber o

f Tria

ls

0

8

16

24

32

40

48

56

64

72

80Low ConfHigh Conf

Static : ROC curves: show all Ss on plot, mean area (SEM) AUROC. Oh look,

there appears to be meta-cognitive

sensitivity going on!

Optimal Kalman !Filter Model:!!

assumes knowledge of sensory and walk variance, weights previous sensory evidence according to an

exponential weighting function.

* target invisbleto observer

Time

Horizontal

Target

Velocity

2. report confidence

Relative to all trials in this session,do you think your performancein the current trial was better

or worse than average?

“better” ! high conf.“worse” ! low conf.

Exp. 1:di�culty manipulation

cloud size:(obvious), session A

5 levels, 1 – 3�

Target

Velocity

velocity stability:(subtle), session B5 levels, 0.05 – 0.25�/sec

Exp. 2:duration manipulation

trial length: 6, 10, or 14 sec

(tracking after 2 sec counted towards score)

0 2 4 6 8 10 12 14

Time (sec)

cursor change

Exp. 1 Results

RMSE

Low Conf.P(RMSE|“worse”)

High Conf.P(RMSE|“better”) av

g.

0.5 0.75 1.0

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1(n = 7)

Area = 0.71± 0.03

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1(n = 7)

Area = 0.74± 0.04

Cumulative P(RMSE|“worse”)

Cumulative

P(R

MSE|“better”) Variable Cloud Size Variable Velocity Stability

Meta-cognitive sensitivity is comparable for obvious &subtle di�culty manipulations, indicating participantsmonitored their tracking error to determine confidence

rather than relying on cues to trial di�culty.

Separation of the highand low confidenceerror distributionsindicates meta-

cognitive sensitivity.Index: the area underthe quantile-quantilecurve (like ROC).

0 2 4 6 8 10

0.5

0.6

0.7

Time in Trial (sec)

AreaUnder

Curve

Late error better predictor ofconfidence. Both optimal anddi�culty-heuristic strategies

predict flat functions.Can we encourage more equal

weighting of error withtemporal uncertainty?

Exp. 2 Results

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1(n = 7)

Area = 0.69± 0.03

Cumulative P(RMSE|“worse”)Cumulative

P(R

MSE|“better”)

Similar meta-cognitivesensitivity observed whentracking di�culty is notexplicitly manipulated,supporting Exp. 1 resultthat participants monitor

their tracking errorto judge confidence.

0 2 4 6 8 10 12 14

0.5

0.6

0.7 6 sec

10 sec 14 sec

Time in Trial (sec)

AreaUnder

CurveTemporal uncertainty did

not encourage more equalweighting of error over

time. Is the recency e↵ectthe only component ofthe weighting function?

Modelling:1. Internal Error Signal

Time

Weight

Model 1: Optimal

Time

Weight

Model 2: Recency E↵ect

Time

Weight

Model 3: Combined Model 4: Max Error

Weighted/Max Error

P(L

owConf)

2. Decision Linking Function

high ! “better”

low conf. ! “worse”

No. Our results arebest fit by a modelwhere error after 2sec counts, but errorclose to confidence

report is over-weighted.

Model Comparison (AIC scores):

M1 M2 M3 M4

Subject

param.model

(k = 2) (k = 3) (k = 5) (k = 2)

876 893 869 867

850 855 785 852

760 764 731 801

769 901 761 776

886 879 870 871

795 801 756 798

References: 1. Barthelme & Mamassian (2010) PNAS2. de Gardelle & Mamassian (2015) PLoS ONE

Funding: NSF BCS-1430262, Correspondence: [email protected]

Documents

TEMPORAL DYNAMICS OF META-COGNITION IN A CONTINUOUS …msl/posters/lockeetal17b.pdf · 2017. 9. 8. · TEMPORAL DYNAMICS OF META-COGNITION IN A CONTINUOUS VISUOMOTOR TASK Shannon