Scaling Laws in Cognitive Science

Scaling Laws in Cognitive Science

Christopher KelloCognitive and Information Sciences

Thanks to NSF, DARPA, and the Keck Foundation

Background and Disclaimer

Cognitive Mechanics…

Fractional Order Mechanics?

Reasons for FC in Cogsci• Intrinsic Fluctuations

• Critical Branching

• Lévy-like Foraging

• Continuous-Time Random Walks

= Disabled synapse = Unblamed synapses

= Enabled synapse = Blamed synapses

1. Choose a disabled synapse2. If , enable with probability ρ3. Set to

B

?

?

itiV ,

~B~B

1. Choose an enabled synapse2. If , disable with probability ρ3. Set to

BB

Spike triggers axonal & dendritic processes

~B

Sequ

ence

dPo

isson

Poiss

on+S

TDP

Source Reservoir

6.9 6.902 6.904 6.906 6.908 6.91x 10

5

0

10

20

30

40

50

60

70

80

90

100

Unit Time Interval

Spi

ke C

ount

SequencePoissonPoisson+STDP

0 1 2 3 4 5x 104Unit Time Interval

Spi

ke C

ount

SequencePoissonPoisson+STDP

CB on CB off

Intrinsic Fluctuations

• Neural activity is intrinsic and ever-present– Sleep, “wakeful rest”

• Behavioral activity also has intrinsic expressions– Postural sway, gait, any repetition

Lowen & Teich (1996), JASA

TN i

TN

TNTNTA

i

ii

2

21

TN i 1

Allan Factor Analyses Show Scaling Law Clustering

TTA

Intrinsic Fluctuations In Spike Trains

Intrinsic Fluctuations in LFPs

Beggs & Plenz (2003), J Neuroscience

Bursts of LFP Activity inRat Somatosensory Slice Preparations

Mazzoni et al. (2007), PLoS One

231 SSP

Burst Sizes Follow a 3/2 Inverse Scaling Law

Intrinsic Fluctuations in LFPs

Intact Leech Ganglia Dissociated Rat Hippocampus

Intrinsic Fluctuations in Speech

Ampl

itude

Time

“Bucket” “Bucket” “Bucket” “Bucket”

12

9

3

6

3

0

Trial Number

Power (dB)-40-20

0204060

Freq

uenc

y (K

Hz)

n n+1 n+2 n+3 n n+1 n+2 n+3

Buck Buck Buck Buck Ket Ket Ket Ket

Ampl

itude

Time

“Bucket” “Bucket” “Bucket” “Bucket”

12

9

3

6

3

0

Trial Number

Power (dB)-40-20

0204060

Freq

uenc

y (K

Hz)

n n+1 n+2 n+3 n n+1 n+2 n+3

Buck Buck Buck Buck Ket Ket Ket Ket

Intrinsic Fluctuations in Speech

0.15 kHz

6.05 kHz

13.15 kHz

Bucks KetsFr

eque

ncy

(KH

z)In

tens

ity (s

tand

ardi

zed)

Trial Number

0.15 kHz

6.05 kHz

13.15 kHz

Bucks KetsFr

eque

ncy

(KH

z)In

tens

ity (s

tand

ardi

zed)

Trial Number

Intrinsic Fluctuations in Speech

0.0 0.5 1.0 1.5 2.0Alpha

0

30

60

90

120

150

Freq

uenc

y

M = 1.06SD = 0.26-0.85

Log f

Log

S(f)

S(f) ~ 1/fα

Scaling Laws in Brain and Behavior

• How can we model and simulate the pervasiveness of these scaling laws?

– Clustering in spike trains

– Burst distributions in local field potentials

– Fluctuations in repeated measures of behavior

Critical Branching• Critical branching is a critical point between

damped and runaway spike propagation

1~prepostc SN

1sub 1c 1super

Damped Runaway

pre

post

Spiking Network Model

PSPj,t : Ij,t = ωj

PSPk,t+τk

?, itiV

itiV ,

tjtt

titi IeVV i,

)'(',,

PSPk,t+τk

τk

Incoming PSP

Update Membrane(and floor at zero)

Crossed Threshold?(and not in refractory)

Reset Membrane

Outgoing PSPs forenabled synapses

ωkτk

ωk

LeakyIntegrate

&Fire

Neuron

Source

Sink

Rese

rvoi

r

Critical Branching Algorithm

= Disabled synapse = Unblamed synapses

= Enabled synapse = Blamed synapses

1. Choose a disabled synapse2. If , enable with probability ρ3. Set to

B

?

?

itiV ,

~B~B

1. Choose an enabled synapse2. If , disable with probability ρ3. Set to

BB

Spike triggers axonal & dendritic processes

~B

Critical Branching Tuning

0 1000 2000 3000 4000 5000 6000Unit Time Interval X 10

Mea

n Lo

cal B

ranc

hing

Rat

io

SequencePoissonPoisson+STDP

Tuning ON Tuning OFF

Spike TrainsSe

quen

ced

Poiss

onPo

isson

+STD

P

Source Reservoir

Allan Factor Results

100 101 102 103

100

101

102

Counting Time (T)

Alla

n Fa

ctor

A(T

)

SequencePoissonPoisson+STDP

data5data6

CB off

TN i

TN

TNTNTA

i

ii

2

21

TN i 1

TTA

Sequ

ence

dPo

isson

Poiss

on+S

TDP

Source Reservoir

Neuronal Bursts

6.9 6.902 6.904 6.906 6.908 6.91x 105

0

10

20

30

40

50

60

70

80

90

100

Unit Time Interval

Spi

ke C

ount

SequencePoissonPoisson+STDP

Neuronal Avalanche Results

100 101 102 103 104 10510-8

10-6

10-4

10-2

100

Size

P(S

ize)

SequencePoissonPoisson+STDPdata4data5

Simple Response Series

Predictable Cues Unpredictable Cues

Spik

e Co

unt

Sour

ceRe

serv

oir

Time

1/f Noise in Simple Responses

Response Times Response Durations

10-4 10-3 10-2 10-1 10010-1

100

101

102

Frequency

Pow

er

Evenly Timed CuesRandomly Timed Cues

10-4 10-3 10-2 10-1 10010-1

100

101

102

Frequency

Pow

er

Evenly Timed CuesRandomly Timed Cues

Memory Capacity of Spike Dynamics

0 5 10 15 20 25 300.5

0.6

0.7

0.8

0.9

1

Time Lag

% C

orre

ct

BR ~ 1BR < 1 (~0.8)BR > 1 (~1.1)Random

0.7 0.8 0.9 1 1.10.68

0.69

0.7

0.71

0.72

0.73

0.74

Branching Ratio Bias

Mea

n %

Cor

rect

*Random

Critical Branching and FC

• The critical branching algorithm produces pervasive scaling laws in its activity.

FC might serve to:

– Analyze and better understand the algorithm

– Formalize the capacity for spike computation

– Refine and optimize the algorithm

Lévy-like Foraging𝑃 (𝑙 ) 𝑙−𝜇1<𝜇<3

Animal Foraging

𝑃 (𝑡𝑖 ) (𝑡𝑖+1 )−𝜇

𝜇 2

Memory Foraging

𝑃 (𝑡𝑖 ) (𝑡𝑖+1 )−𝜇

𝜇 2

Lévy-like Visual Search

Lévy-like Visual Search

100 101 102 103 104 105100

101

102

103

104

105

106

Tile Size

Alla

n Fa

ctor

Var

ianc

e

NaturalArtificialNaturalArtificial

Image

Eye

100 101 102 10310-6

10-5

10-4

10-3

10-2

10-1

100

Saccade Length

P(S

acca

de L

engt

h)

NaturalArtificial

Lévy-like Foraging Games

.05 .15 .25 .50

-2.2

-2.1

-2

-1.9

-1.8

-1.7Number of Resources Averaged

Resource Clustering

Slo

pe

Top 20 ScoresMiddle 20 ScoresBottom 20 Scores

25 50 100 150

-2.2

-2.1

-2

-1.9

-1.8

-1.7Degree of Clustering Averaged

Resource Quantity

Top 20 ScoresMiddle 20 ScoresBottom 20 Scores

“Optimizing” Search with Levy Walks• Lévy walks with μ ~ 2 are maximally efficient

under certain assumptions

• How can these results be generalized and applied to more challenging search problems?

Continuous-Time Random WalksIn general, the CTRW probability density obeys

Mean waiting time:

Jump length variance:

Human-Robot Search Teams

• Wait times correspond to times for vertical movements

• Tradeoff between sensor accuracy and scope

• Human-controlled and algorithm-controlled search agents in virtual environments

Conclusions

• Neural and behavioral activities generally exhibit scaling laws

• Fractional calculus is a mathematics suited to scaling law phenomena

• Therefore, cognitive mechanics may be usefully formalized as fractional order mechanics

Collaborators

• Gregory Anderson• Brandon Beltz• Bryan Kerster• Jeff Rodny• Janelle Szary

• Marty Mayberry• Theo Rhodes

• John Beggs• Stefano Carpin• YangQuan Chen• Jay Holden• Guy Van Orden