Introduction to ACT-RTutorial

21st Annual Conference Cognitive Science Society

ACT-R Home Page: http://act.psy.cmu.edu

John R. AndersonChristian Lebiere

Psychology DepartmentCarnegie Mellon

UniversityPittsburgh, PA 15213

ja+@cmu.educl+@cmu.edu

Dieter WallachInstitut fur Psychologie

Universitaet Basel Bernoullistr. 16 CH-4056 Basel

wallachd@ubaclu.unibas.ch

Tutorial Overview1. Introduction

2. Symbolic ACT-RDeclarativeProceduralLearning

3. Subsymbolic Performance in ACT-RActivation (Declarative)Utility (Procedural)

4. Subsymbolic Learning in ACT-RActivation (Declarative)Utility (Procedural)

5. ACT-R/PM

Note: For detailed (40-100 hrs) tutorial, visit ACT-R Education link. For software visit ACT-R Software link. For models visit Published ACT-R Models link.

ACT-R exemplifies what Newell meant when he spoke of a unified theory of cognition – i.e., a single system within which we can understand the wide range of cognition.

Arguments against Unified Theories 1. Modularity – behavioral and neural evidence. 2. Need for specialization - Jack of all trades, master of none.

Argument for Unified Theories 1. System Organization - We need to understand how the overall mental system works in order to have any real

understanding of the mind or any of its more specific

functions. 2. Mental plasticity – ability to acquire new competences.

Unified Theories of Cognition

+ 1. Behave as an (almost) arbitrary function of the environment (universality)+ 2. Operate in real time+ 3. Exhibit rational, i.e., effective adaptive behavior+ 4. Use vast amounts of knowledge about the environment+ 5. Behave robustly in the face of error, the unexpected, and the unknown+ 6. Use symbols (and abstractions)+ 7. Use (natural) language- 8. Exhibit self-awareness and a sense of self+ 9. Learn from its environment+ 10. Acquire capabilities through development- 11. Arise through evolution+ 12. Be realizable within the brain

Newell’s Constraints on a Human Cognitive Architecture

(Newell, Physical Symbol Systems, 1980)

ACT-R is explicitly driven to provide models for behavioral phenomena. The tasks to which ACT-R has been applied include:

1. Visual search including menu search2. Subitizing3. Dual tasking including PRP 4. Similarity judgements5. Category learning6. List learning experiments7. Paired-associate learning8. The fan effect9. Individual differences in working memory 10. Cognitive arithmetic11. Implicit learning (e.g. sequence learning)12. Probability matching experiments

The Missing Constraint: Making Accurate Predictions about Behavioral Phenomena.

13. Hierarchical problem solving tasks including Tower of Hanoi 14. Strategy selection including Building Sticks Task15. Analogical problem solving16. Dynamic problem solving tasks including military command and control17. Learning of mathematical skills including interacting with ITSs18. Development of expertise19. Scientific experimentation20. Game playing21. Metaphor comprehension22. Learning of syntactic cues23. Syntactic complexity effects and ambiguity effects24. Dyad Communication

A priori ACT-R models can be built for new domains taking knowledge representations and parameterizations from existing domains. These deliver parameter-free predictions for phenomena like time to solve an equation.

History of the ACT-framework

Predecessor HAM (Anderson & Bower 1973)

Theory versions ACT-E (Anderson, 1976)ACT* (Anderson, 1978)ACT-R (Anderson, 1993)ACT-R 4.0 (Anderson & Lebiere, 1998)

Implementations GRAPES (Sauers & Farrell, 1982)

PUPS (Anderson & Thompson, 1989)

ACT-R 2.0 (Lebiere & Kushmerick, 1993)ACT-R 3.0ACT-R 4.0 (Lebiere, 1998)ACT-R/PM (Byrne, 1998)

ACT-R : Information Flow

ConflictResolution

Retrieval Request

Transform

Goal

CurrentGoal

(Cortical

Activation)

ProceduralMemory

(Basal Ganglia& Frontal Cortex)

DeclarativeMemory

(Hippocampus

& Cortex)

GoalStack

(Frontal Cortex)

Retrieval

Result

PopPush

Production

Compilation

ACT-R

OUTSIDE WORLD

Action Perception

Popped

Goal

ACT-R: Information Flow

Addition-FactThree Seven

Four

addend1 sum

addend2

Declarative-Procedural Distinction

Procedural Knowledge: Production Rules

for retrie ving chunks to solve problems.

336

+848

4

IF the goal is to add n1 and n2 in a column

and n1 + n2 = n3

THEN set as a subgoal to write n3 in that column.

Productions serve to coordinate the retrieval of

information from declarative memory and the enviroment

to produce transformations in the goal state.

Declarative Knowledge: Chunks

Configurations of small numbers of elements

ACT-R: Knowledge Representation

PerformanceDeclarative Procedural

SymbolicRetrieval of

ChunksApplication of

Production Rules

SubsymbolicNoisy ActivationsControl Speed and

Accuracy

Noisy UtilitiesControl Choice

LearningDeclarative Procedural

SymbolicEncoding

Environment andCaching Goals

Compilation fromExample and Instruction

SubsymbolicBayesianLearning

BayesianLearning

ACT-R: Assumption Space

NAME

SSLOT1LOT1 Filler1Filler1

SSLOT2 LOT2 Filler2Filler2

SSLOTLOTNN

NEWCHUNK(

FillerNFillerN )

isa ADDITION-FACT

AADDENDDDEND11 TTHREEHREE

AADDENDDDEND22 FFOUROUR

SSUM UM

FACT3+4(

SSEVENEVEN )

isa

Chunks: Example

CHUNK-TYPE NAME SSLOT1LOT1 SSLOT2LOT2 SSLOTNLOTN( )

Chunks: Example(CLEAR-ALL)(CHUNK-TYPE addition-fact addend1 addend2 sum)(CHUNK-TYPE integer value)(ADD-DM (fact3+4

isa addition-fact addend1 three addend2 four sum seven) (three

isa integer value 3) (four

isa integer value 4) (seven

isa integer value 7)

ADDITION-FACT

FACT3+4ADDEND1 SUM

ADDEND2

THREE

FOUR

SEVEN

isa

isa

INTEGER

isa

VALUE VALUE

3 7

isa

Chunks: Example

VALUE

4

Chunks: Exercise I

Fact:

Encoding:

(Chunk-Type proposition agent action object)

The cat sits on the mat.

proposition

action

cat007

sits_on

mat

isa

fact007agent object

(Add-DM (fact007

isa proposition

agent cat007

action sits_on

object mat) )

Chunks: Exercise IIFact The black cat with 5 legs sits on the mat.

Chunks(Chunk-Type proposition agent action object)(Chunk-Type cat legs color)

(Add-DM (fact007 isa proposition

agent cat007action sits_onobject mat)

(cat007 isa catlegs 5

color black) )

proposition

action

cat007

sits_on

mat

isa

fact007agent object

cat

isa

color

5

black

legs

Chunks: Exercise III

Fact

ChunkThe rich young professor buys a beautiful and expensive city house.

(Chunk-Type proposition agent action object)(Chunk-Type prof money-status age)(Chunk-Type house kind price status)

(Add-DM (fact008 isa proposition

agent prof08action buysobject house1001

) (prof08 isa prof

money-status richage young

) (obj1001 isa house

kind city-houseprice expensivestatus beautiful

))

proposition

action

buys

isa

fact008agent object

prof

isa

prof08

age

young

rich

house

kind

city-house

obj1001

price

expensive

isa

status

beautiful

money-status

proceduralmemory

set of productions, organizedset of productions, organizedthrough reference to goalsthrough reference to goals

productions

• • modularitymodularity• • abstractionabstraction• • goal factoringgoal factoring• • conditional asymmetryconditional asymmetry

Productions

( p

==>

)

<Goal Transformation>

<External action>

condition part

delimiter

action part

name

<Goal pattern><Chunk retrieval >

Structure of productions

Psychological reality of productions

Taken from: Anderson, J.R. (1993). Rules of the mind. Hillsdale, NJ: LEA.

Error rates: Data & Model

Taken from: Anderson, J.R. & Lebiere, C. (1998). The atomic components of thought. Hillsdale, NJ: LEA.

chunk retrievalvariable prefix

(p add-numbers

=goal> isa add-column num1 =add1 num2 =add2 result nil

=fact> isa addition-fact addend1 =add1 addend2 =add2 sum =sum

= =>

=goal> result =sum)

production name

action description

goal pattern

>

head/slot separator

fact=

Add-numbers

note in the goal that the result is =sum

the goal is to add numbers in a column and =add1 is the first number and =add2 is the second number

and you remember an addition fact that =add1 plus =add2 equals =sum

??Add-numbers

IF

Then

(p add-numbers

=goal> isa add-column num1 =add1 num2 =add2 result nil

=fact> isa addition-fact addend1 =add1 addend2 =add2 sum =sum

= = >

=goal> result =sum

)

(first-goal isa add-colomn num1 three num2 four result nil)

(fact3+4 isa addition-fact addend1 three addend2 four sum seven)

(first-goal isa add-colomn num1 three num2 four result seven)

3+4

declarative memory

goal

left-hand side

Pattern matching

(fact2+3 isa add-fact addend1 two addend2 three sum five)

(fact3+1 isa add-fact addend1 three addend2 one sum four)

(fact0+4 isa add-fact addend1 zero addend2 four sum four)

=goal> isa find-sum addend2 =num2 sum =sum

(fact2+2 isa add-fact addend1 two addend2 two sum four)

(goal1 isa find-sum addend1 nil addend2 two sum four )

=fact> isa add-fact addend1 zero addend2 =num2 sum =sum

negation

— addend1

First-Goal 0.000 isa COUNT-FROM start 2 end 5

(P increment =goal> ISA count-from start =num1 =count> ISA count-order first =num1 second =num2==> !output! ( =num1) =goal> start =num2)

(P stop =goal> ISA count-from start =num end =num==> !output! ( =num) !pop!)

(add-dm (a ISA count-order first 1 second 2) (b ISA count-order first 2 second 3) (c ISA count-order first 3 second 4) (d ISA count-order first 4 second 5) (e ISA count-order first 5 second 6) (first-goal ISA count-from start 2 end 5))

Counting Example

Web Address: ACT-R Home Page

Published ACT-R Models Counting Example

Initial state

stack-manipulating actions

Goal Stack

!focus-on! =G4

G4

G1

!pop!

G1

G2G1

!push! =G2

G2

!push! =G3

G1

G3

Tower of Hanoi Demo

Web Address: ACT-R Home Page Published ACT-R Models Atomic Components of Thoughts Chapter 2 Model for Ruiz

Start-Tower IF the goal is to move a pyramid of size n to peg x and size n is greater than 1 THEN set a subgoal to move disk n to peg x and change the goal to move a pyramid of size n-1 to peg x

Final-Move IF the goal is to move a pyramid of size 1 to peg x THEN move disk 1 to peg x and pop the goal

Subgoal-Blocker IF the goal is to move disk of size n to peg x and y is the other peg and m is the largest blocking disk THEN post the goal of moving disk n to x in the interface and set a subgoal to move disk m to y

Move IF the goal is move disk of size n to peg x and there are no blocking disks THEN move disk n to peg x and pop the goal

Tower of Hanoi: Data & Models

Taken from: Anderson, J.R. & Lebiere, C. (1998). The atomic components of thought. Hillsdale, NJ: LEA.

Subsymbolic levelSummary

Computations on the subsymbolic level are responsible for

• which production ACT-R attempts to fire• how to instantiate the production• how long the latency of firing a production is• which errors are observed

As with the symbolic level, the subsymbolic level is not a static level, but is changing in the light of experience to allow the system to adapt to the statistical structure of the environment.

(goal1 isa add-column num1 Three num2 Four result nil )

FACT3+4Bi

FOUR

ADDITION-FACT

SEVEN

add

Chunks & Activation

end2

isa

Sji

Wj

THREE

Wj

=fact> isa addition-fact addend1 =add1 addend2 =add2 sum =sum

(p add-numbers =goal> isa add-column num1 =add1 num2 =add2 result nil

addend1 sum

Sji Sji

Ai=Bi+WjSji

Chunk Activation

baseactivatio

n

associativestrength

sourceactivation

activation ( )= +

Ai = Bi + Wj * Sji

Context activation

Activation makes chunks available to the degree that past experiences indicate that they will be useful at the particular moment:

• Base-level: general past usefulness • Context: relevance in the current context

j

*

Base-level Activation

The base level activation Bi of chunk Ci reflects a context-independent estimation of how likely Ci is to match a production, i.e. Bi is an estimate of the log odds that Ci will be used.

Two factors determine Bi:

• frequency of using Ci

• recency with which Ci was used

BBii = ln  = ln (( ))P(CP(Cii))P(CP(Cii))

baseactivation

associativestrength

sourceactivation

activation ( )= +*

Ai = Bi + Wj * Sji

Base-Level Activation & Noise

B(t) = - d * ln(t) + 1 + 2

Basel-level activation fluctuates and decays with time

initial expected base-level activation

decay with time, parameterd denotes the decay rate

transient noise 2, reflectingmoment-to-moment fluctuations

random noise in initial base-

level activation 1 at creation time

Source Activation

The source activations Wj reflect the amount of attention given to elements, i.e. fillers, of the current goal. ACT-R assumes a fixed capacity for goal elements, and that each element has an equal amount (W= Wi = 1).

(1) constant capacity for source activations(1) constant capacity for source activations(2) equally divided among the n goal elements: (2) equally divided among the n goal elements: constant/nconstant/n(3) W reflects an individual difference parameter(3) W reflects an individual difference parameter

baseactivatio

n

associativestrength

sourceactivation

activation

( )= + *

Ai = Bi + Wj * Sjij

Associative strength

The association strength SThe association strength Sjiji between chunks C between chunks Cjj and C and Ci i is a is a measure of how often Cmeasure of how often Cii was needed (retrieved) when C was needed (retrieved) when Cjj was was element of the goal, i.e. Selement of the goal, i.e. Sjiji estimates the log likelihood ratio of estimates the log likelihood ratio of CCjj being a source of activation if C being a source of activation if Ci i was retrieved.was retrieved.

baseactivatio

n

associativestrength

sourceactivation

activation

( )= +

( )P(Ni Cj)P(Ni)

Sji = ln

*

= S - ln(P(Ni|Cj))

Ai = Bi + Wj * Sji

Retrieval time

Chunks i to instantiate production p are retrieved sequentially

Time to retrieve a chunk as function of match score Mip andstrength of matching production Sp

Retrieval time is an exponential function of the sum of matchscore of the chunk and the production strength

TimeipRetrieval-timep =

i

Timeip = Fe-f(Mip + Sp)

Retrieval time

Fan effect

Lawyer

Church

Park

Fireman

In

Doctor Bank

Fan Effect DemoRetrieve-by-PersonIf the goal is to retrieve a sentence involving a person and a location and there is a proposition about that person in some locationThen store that person and location as the retrieved pair.

Retrieve-by-LocationIf the goal is to retrieve a sentence involving a person and a location and there is a proposition about some person in that locationThen store that person and location as the retrieved pair.

Mismatch-PersonIf the retrieved person mismatches the probeThen say no.

Mismatch-LocationIf the retrieved location mismatches the probeThen say no.

Match-BothIf the retrieved person and location both match the probeThen say yes.

Web Address: ACT-R Home Page Published ACT-R Models Atomic Components of Thought Chapter 3 Fan Effect Model

Fan Effect

Threshold

Chunks with an activation lower than threshold

can not be retrieved

Retrieval probability = 1

1 + e-(A-)/s

Equivalently: Odds of recall = e(A- )/s

recall is an exponential function of the distance betweenActivation Ai of Chunk Ci and threshold scaled by activationnoise s.

odds of recall decreases as a power function of time

These occur when the correct chunk falls below the activation thresholdfor retrieval and the intended production rule therefore cannot fire.

Errors of Omission

These occur when some wrong chunk is retrieved instead of the correctone and so the wrong instantiation fires.

Errors of Commission

Partial matching

==>

==>

partial matching is restricted to chunks with the same type asspecified in a production’s retrieval pattern

Partial matching

an amount reflecting the degree of mismatch Dip to a retrievalpattern of production p is subtracted from the activation level Ai

of a partially matching chunk i. The match score for the matchof chunk i to production p is:

Mip = Ai - DipDip is the sum for each slot of the degree of mismatch between

the value of the slot in chunk i and the respective retrieval pattern

Probability of retrieving chunk i as a match for production p:

eMip/t

Mjp/tej

t = 6 = 2 s

SUGAR FACTORY

SUGAR FACTORY

Sugar productiont = 2 * workerst - sugar productiont-1 [+/- 1000]

Negative correlation between knowledge and performance

100 200 300 400 500 600 700 800 900 1000 1100 1200

1000 1000 3000 5000 7000 9000 11000 12000 12000 12000 12000 12000 12000 2000 1000 2000 4000 6000 8000 10000 12000 12000 12000 12000 12000 12000 3000 1000 1000 3000 5000 7000 9000 11000 12000 12000 12000 12000 12000 4000 1000 1000 2000 4000 6000 8000 10000 12000 12000 12000 12000 12000 5000 1000 1000 1000 3000 5000 7000 9000 11000 12000 12000 12000 12000 6000 1000 1000 1000 2000 4000 6000 8000 10000 12000 12000 12000 12000 7000 1000 1000 1000 1000 3000 5000 7000 9000 11000 12000 12000 12000 8000 1000 1000 1000 1000 2000 4000 6000 8000 10000 12000 12000 12000 9000 1000 1000 1000 1000 1000 3000 5000 7000 9000 11000 12000 12000 10000 1000 1000 1000 1000 1000 2000 4000 6000 8000 10000 12000 12000 11000 1000 1000 1000 1000 1000 1000 3000 5000 7000 9000 11000 12000 12000 1000 1000 1000 1000 1000 1000 2000 4000 6000 8000 10000 12000

workers

production

sugar

Similarities: example

sim(a,b)=mina,b( )maxa,b( )

D = Mismatch Penalty * (1-sim(a, b))

aa

1221110 345678

9 1.08.090.10.110.1250.140.160.20.250.330.5Ratio Similarities:

Retrieval of encoded chunks

(GOALCHUNK isa transition state 2000 production 9000 worker nil) (Episode007 isa transition state 1000 production 8000 worker 5)MatchPartial Match

(p retrieve-episode =goal> isa transistion state =state production =production =episode> isa transition state =state production =production worker =worker==> goal> worker =worker))Lebiere, C., Wallach, D. & Taatgen, N. (1998). Implicit and explicit learning in ACT-R. In F. E. RitterAnd R. Young (Eds.) Proceedings of the Second European Conference on Cognitive Modeling, pp. 183-189.Nottingham: Nottingham University Press.

Control performance

0510152025

ACT-R D & FExperiment

Trial 41-80Trial 1-40

TargetStates

Concordance

1

baselinecorrectwrong0

.5

.25

.75

Problem solving vs. questionaire

ACT-R Experiment D & F

Transition from computation to retrieval

8060402000.00.20.40.60.81.0

Trials

Conflict resolution

In general, conflict resolution gives answers to two questions:

Which production out of a set of matching productionsis selected?

Which instantiation of the selected production is fired?

activationSequential instantiationNo backtracking

expected gainGoal factoringSuccess probabilityCosts

goal-specific

Expected Gain = –G

Probability ofgoal achievement Goal value

Cost ofgoal achievement

P C*

production-specific

Conflict resolution

Expected Gain =

q r• a b+

–GProbability of

goal achievement Goal valueCost of

goal achievement

P C*

Selection of Productions

q rP*

Probability of Goal Achievement

probability of the production working successfully

probability of achieving the goal if the production works successfully

Achieving a goal depends on the Achieving a goal depends on the joint probability joint probability of of the respective production being successful the respective production being successful andand subsequent rules eventually reaching the goal.subsequent rules eventually reaching the goal.

Production's matching/actions/subgoals Goal accomplished and popped have the intended effect successfully.

Costs of a production

amount of effort (in time) that a pro- duction will take

estimate of the amount of effort from when a pro- duction completes until the goal is achieved

Production costs are calculated as the sum of the effort associated with production pi and (an estimate of) the effort that subsequent productions pj..n take on the way to goal achievement.

a bC+

Production's costs ofmatching/actions/subgoals Costs of future productions

currentstate

Intendednextstate

goalstate…

{ {{ {

Conflict resolution

q rP*

a bC+

Goal valueG=20

p3

G'=17

!push!

G' = rG-b = .9 * 20 - 1 = 17

p3 parameters:q: 1r: .9a: .05b: 1

ACT-R values a goal less the more deeply it is ACT-R values a goal less the more deeply it is embeddedembeddedin uncertain subgoalsin uncertain subgoalsACT-R pops the goal with failure if no production above ACT-R pops the goal with failure if no production above the utility threshold (default: 0) can match (goal the utility threshold (default: 0) can match (goal abandonment) abandonment)

Noise in Conflict Resolution

Evaluation Ei of production i = P(i)*G-C(i)

Probability of choosing i among n applicable productions with Evaluation Ej

eEi/t

Ej/tej

2t =

Remember:

Boltzmann Equation

2-person Matrix Game

Players

Actions

Payoff matrix A2 B2A1 3, 7 8, 2B1 4, 6 1, 9

Player1, Player2

Actions A, B ...

Data sets

Erev & Roth (1998)

“ There is a danger that investigators will treat themodels like their toothbrushes, and each will use

its own model only on his own data.”

Diverse data sets re-analyzed

2 x 2 4 x 45 x 5

Suppes & Atkinson (1960) [SA2, SA8, SA3k, SA3u]Erev & Roth (1998) [SA3n]Malcom & Liebermann (1965)O'Neill (1987)Rapoport & Boebel (1992) [R&B10, R&B15]

(p player1-A =goal> isa decide player1 nil==> =goal> player1 A)

(p player1-B =goal> isa decide player1 nil==> =goal> player1 B)

game12 isa decide player1 A player2 B

Productions

Chunk

Model

2 4 6 03 3 1 5

1/3 2/3 1 01/2 1/2 1/6 5/6

Best Fits – Random Games

1-ParameterErev & Roth (1998)

Referencepoint=xmin

Reference point= 0

Model ACT-R; Average

Parameter S(1)=15 Par. priors=53Data set Random games Data

setRandom games

100*MSD 1.3 1.087 100*MSD 0.471game #1 0.304 0.4725 game 1 0.288game #2 0.89585 0.4403 game 2 0.163game #3 0.8994 1.235 game 3 0.289game #4 0.28565 0.4074 game 4 0.325game #5 2.0305 1.181 game 5 0.447game #6 0.6485 1.298 game 6 0.903game #7 2.201 0.6303 game 7 0.626game #8 0.4287 1.18 game 8 0.204game #9 3.589 2.523 game 9 1.006game #10 1.7195 1.504 game 10 0.546

Conflict resolution

Goal Match

testgoal pattern(1)

fire production(4)

Ep = 18.95Ep = 18.95

Match

retrieve chunk(s)

(3)

sele

ctio

n

Ep = 13.95

Ep = 17.30

evaluateconflict set(2)

Ep = 18.95Ep = 18.95

1. Lael Schooler: Statistical structure of the demands on declarative memory posed by the environment.

2. Christian Lebiere: Consequences for 20 years of practicing arithmetic facts.

3. Marsha Lovett: Selection among production rules is also sensitive to both the features of the current problem and the rule’s past history of success.

Learning as Subsymbolic Tuning to the Statistics of the Environment

Lael SchoolerDeclarative Memory:

Statistical Tuning

1. The goal of declarative memory is to makemost available those memory chunks that aremost likely to be needed at a particular pointin time.

2. The probability of a memory chunk beingrelevant depends on its past history of usageand the current context.

3. Log Odds = Log t

j

− d

j = 1

n

∑

⎛

⎝

⎜

⎜

⎞

⎠

⎟ + Context

1008060402000.0

0.1

0.2

(a) New York Times Retention

Days since Last Occurrence

Probabilitity on Day 101

543210-6

-5

-4

-3

-2

-1

(d) New York Times Retention

Log Days

Log Need Odds

Log Odds= - 1.95 - 0.73 Log Days R^2 = 0.993

Odds = .14 T-.73

1008060402000.00

0.02

0.04

0.06

0.08

0.10

0.12

(b) Parental Speech Retention

Utterances since Last occurrence

Probability in Utterance 101

543210-6

-5

-4

-3

-2(e) Parental Speech Retention

Log Utterances

Log Need Odds

Log Odds = - 1.70 - 0.77 Log Utterances R^2 = 0.984

Odds = .18 T-.77

1008060402000.0

0.1

0.2

0.3

(c) Mail Sources Retention

Days since Last Occurrence

Probability on Day 101

543210-5

-4

-3

-2

-1

0

(f) Mail Sources Retention

Log Days

Log Need Odds

Log Odds = - 1.09 - 0.83 Log Days R^2 = 0.986

-.83Odds = .34 T

Parameter learning:log( tj

-d) n

j=1

Lael Schooler’s Research

p(AIDS) = .018 New York Times p(AIDS| associate) p(AIDS| associate)

p(AIDS)virus .75 41.0

Associates spread .54 29.4patients .40 21.8health .27 14.6

Parental Speech p(play) = .0086p(play|game) p(play|game)

p(play).41 47.3

807060504030201000.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

strong contextweak context

(a) CHILDES standard

retention in utterances

need odds

807060504030201000.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

(b) New York Times standard

retention in days

need odds

Environmental Analyses of Context and Recency

5.04.54.03.53.02.52.01.51.00.50.0-7

-6

-5

-4

-3

-2

-1

0

retention in log utterances

log need odds

(c) CHILDES power

4.54.03.53.02.52.01.51.00.50.0-7

-6

-5

-4

-3

-2

-1

0

(d) New York Times power

retention in log days

log need odds

Lael Schooler Retrieval Odds Mirrors Odds of Occurring

Conclusions from Environmental Studies: Log Odds = Log

Proposal for ACT-R’s Declarative Memory: - Activation reflects Log Odds of Occurring

- Learning works as a Bayesian inference scheme to try to identify the right values of the factors determining odds of recall.

t j−d

j=1

n∑

⎛

⎝ ⎜

⎞

⎠ ⎟ + Context

TwelveEight

Four

addition-factsumaddend1

addend2

W SB

Sj ji ji

S

W

ji

j

i

Activation Structure

Ai = Bi + ∑j

Wj Sji Activatio n Equation

Bi=ln t

j

− d

j = 1

n

∑

⎛

⎝

⎜

⎜

⎞

⎠

⎟ Base-Level Learning

Sj i= S - ln((P(i| j )) Stre ngth Learning

Performa nce Str ucture

Mi = Ai - Dp Match Equat ion

Probability = e

M

i

/ t

e

M

j

/ t

j

∑

Chunk Choice

Time i = Fe-fMi Retrieval Time

Declarative Equations

What happens when the probabilistic world of informationretrieval hits the hard and unforgiving world of mathematics?

Christian Lebiere’sSimulation of Cognitive Arithmetic

Over 100,000 problems of each type (1+1 to 9+9; 1x1 to 9x9) over 20years.

Addition MultiplicationIF the goal is to find a + b and a + b = cTHEN the answer is c

IF the goal is to find a * b and a * b = cTHEN the answer is c

IF the goal is to find a + bTHEN set a subgoal to count

b units past a

IF the goal is to find a * bTHEN set a subgoal to add

a for b times

Critical Phenomena: Transition from computation to retrieval Errors due to partial matching and noise Errors due to retrieving wrong answers Effects of frequency distribution 1+1 is about three times

more frequent than 9+9

Retrieve

Compute

Small Large0

2

4

6

8

10

1st

4th

7th

10th

College

Problem Size Effect (Data)

Problem Size

RT (sec)

Problem Size Effect over Time

Small Large0

2

4

6

8

10

1st

4th

7th

10th

College

Problem Size Effect over Time

Problem Size

Response Time (sec)

Model

Effect of Argument Size on AccuracyFor Addition (4 year olds)

654321020

30

40

50

60

70

80

AugendAddend

Percentage Correct for Addition Retrieval in the First Cycle (1000 Problems)

Operand

Percentage Correct

654321020

30

40

50

60

70

80

Augend

Addend

Addition Retrieval

Operand

Percentage Correct

Percentage of Correct Retrieval per Operand Percentage Correct in Simulation

Data Model

Effect of Argument Size on AccuracyFor Multiplication (3rd Grade)

1086420

10

20

30

40

50

Multiplicand

Multiplier

Multiplication Computation

Argument

Error Percentage

Percentage of Correct Computations per Operand Percentage Errors in Multiplication Simulation

1086420

10

20

30

40

50

Multiplicand

Multiplier

Error Percentage for MultiplicationComputation in Cycle 3 (~4th Grade)

Argument

Error Percentage

Data Model

Conclusions aboutCognitive Arithmetic

Subsymbolic learning mechanisms that yield adaptiveretrieval in the world at large are behind the 20 yearstruggle that results in the mastery of cognitivearithmetic. Part of the reason why it is a struggle isthat there is noise in the system. However, moredeeply, two things about the arithmetic domain failto match up with the assumptions our memorysystem makes about the world:

1. Precise matching is required.2. High interference between competing memories.

Making Choices: Conflict Resolution

Expected Gain = E = PG-C

Probability of choosing i = e

E

i

/ t

e

E

j

/ t

j

∑

P=

Successe =s α+mFailure =s +n

SuccessesSuccesses +Failures

P is expected probability of successG is value of goalC is expected cost

t reflects noise in evaluation and is like temperature in the Bolztman equation

α is prior successesm is experienced successes is prior failuresn is experienced failures

Procedural Learning

Undershoot

More Successful

Overshoot

More Successful

Looks

Undershoot

10 Undershoot

0 Overshoot

10 (5) Undershoot

10 (15) Overshoot

Looks

Overshoot

10 (15) Undershoot

10 (5) Overshoot

0 Undershoot

10 Overshoot

INITIAL STATE

desired:

current:

building:

UNDERSHOOT UNDERSHOOTOVERSHOOT

desired:

current:

building:

desired:

current:

building:

desired:

current:

building:

possible first moves

a b c

a b c a b c a b c

Building Sticks Task (Lovett)

0

0

0

0

0

1

1

1

1

1

3

3

3

3

3

High

Against

Low

Against

Neutral Low

Toward

High

Toward

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Proportion Choice More Successful Operator

Test Problem Bias

Observed Data

Biased Condition

Extreme-Biased Condition

0

0

0

0

0

1

1

1

1

1

3

3

3

3

3

High

Against

Low

Against

Neutral Low

Toward

High

Toward

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Test Problem Bias

0 0

0

00

1 1

1

11

3 3

3

3 3

High

Against

Low

Against

Neutral Low

Toward

High

Toward

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Proportion Choice More Successful Operator

Test Problem Bias

0 0

0

0 0

1 1

1

1 1

3 3

3

3 3

High

Against

Low

Against

Neutral Low

Toward

High

Toward

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Test Problem Bias

Predictions of Decay-Based ACT-R

(2/3) (5/6)

Lovett & Anderson, 1996

Build Sticks DemoDecide-Under If the goal is to solve the BST task and the undershoot difference is less than the overshoot differenceThen choose undershoot.

Decide-Over If the goal is to solve the BST task and the overshoot difference is less than the undershoot differenceThen choose overshoot.

Force-Under If the goal is to solve the BST taskThen choose undershoot.

Force-Over If the goal is to solve the BST taskThen choose overshoot.Web Address:

ACT-R Home Page Published ACT-R Models Atomic Components of Thought Chapter 4 Building Sticks Model

ACT-R model probabilities before and afterproblem-solving experience in Experiment 3

(Lovett & Anderson, 1996)

ProductionPrior

Probabilityof Success

Final Value

67% Condition 83% Condition

Force-Under

More Successful

Context Free

.50 .60 .71

Force-Over

Less Successful

Context Free

.50 .38 .27

Decide-Under

More Successful

Context Sensitive

.96 .98 .98

Decide-Over

Less Successful

Context Sensitive

.96 .63 .54

Successes ( t ) = t

j

− d

j = 1

m

∑ Success Discounting

Failures ( t ) = t

j

− d

j = 1

n

∑ Failure Discounting

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

U, U O, U U, O O, O

Outcome on Trial N-2, N-1

Data

Decay Model

No Decay Model

Decay of Experience

Note: Such temporal weighting is critical in the real world.

• But, what happens when there is more than onecritical choice per problem?

-How is credit/ blame assigned by human problemsolvers?

-How well does ACT-R's learning mechanism handlethis more complex case?

-In ACT-R all choices leading to goal resolution areequally weighted.

-But, is there evidence for a goal gradient?

Credit-Assignment in ACT-R

INITIAL STATE

UNDERSHOOTOVERSHOOT

desired:

current:

building:

desired:

current:

building:

desired:

current:

building:

ab

c

ab

c a b c

add cadd b

delete c add c

desired:

current:

building:

ab

c

desired:

current:

building:

a b c

delete a

desired:

current:

building:

a b c

delete a

desired:

current:

building:

a b c

delete c

desired:

current:

building:

a b c

add a

desired:

current:

building:

a b c

add cadd a

desired:

current:

building:

a b c

add a

desired:

current:

building:

a b c

desired:

current:

building:

a b c

desired:

current:

building:

ab

c

delete a

MAINTAIN REVERSEMAINTAIN REVERSE

75%

75% 75%

Building Sticks Task 2 Levels

Choice Learning

0.400

0.450

0.500

0.550

0.600

0.650

0.700

0.750

0.800

1 2 3 4

Block

1st - 100% 1st-75%2nd-75% 2nd 50%1st-25% 2nd-25%

It would be trivial to create a system that would do well at this task simply by eliminating the noise and getting rid of the discounting of past experience. However, this again makes the error of assuming that the human mind evolved for optimal performance at our particular laboratory task.

In the real world noise is important both to explore other options and to avoid getting caught in traps.

The discounting of experience also allows us to rapidly update in the presence of the changing world.

Christian Lebiere and Robert West have shown that these features arecritical to getting good performance in games as simple as rocks-papers-scissors.

Adapting to a Variable and Changing World

AC

T-R

/PM

CognitionLayer

Perceptual/Motor Layer

DeclarativeMemory

ProductionMemory

Environment

VisionModule

icon

MotorModule

AuditionModule

audicon

SpeechModule

attentiontarget of attention(chunks)

pixels

raw audio

clicks,keypresses,

etc.

attentiontarget ofattention(chunks)

audio

Martin-Emerson-Wickens Task

Zur Anzeige wird der QuickTime™ Dekompressor “Photo - JPEG”

benötigt.

Martin-Emerson& Wickens (1992):The vertical visual

field and implicationsfor the head-up

display

Perform compensatory tracking,keeping the crosshair on target

Respond to choice stimuli asrapidly as possible

Choice stimulus appears at various distances from target(vertical separation)

Tracking requires eye to be onthe crosshair

Eye must be moved to see stimulus

Choice response & tracking move-ments are bottlenecked throughsingle motor module

(Dual-)Task

Model

Find-Target-Oval IF the target hasn't been located and the oval is at locationTHEN mark the target at location

Attend-Cursor IF the target has been found and the state has not been set and the pointer is at location and has not been attended to and the vision module is freeTHEN send a command to move the attention to location and set the state as "looking"

Attend-Cursor-Again IF the target has been found and the state is "looking" and the pointer is at location and has not been attended to and the vision module is freeTHEN send a command to move the attention to location

Start-Tracking IF the state is "looking" and the object focused on is a pointer and the vision module is freeTHEN send a command to track the pointer and update the state to "tracking"

MEW Productions

Move-Cursor IF the state is "tracking" and the target is at location and the motor module is freeTHEN send a command to move the cursor to location

Stop-Tracking IF the state is "tracking" and there is an arrow on screen that hasn't been attended toTHEN move the attention to that location and update the state to "arrow"

Right-Arrow IF the state is "arrow" and the arrow is pointing to the right and the motor module is freeTHEN send a command to punch the left index finger and clear the state

Left-Arrow IF the state is "arrow" and the arrow is pointing to the left and the motor module is freeTHEN send a command to punch the left middle finger and clear the state

VM RS

+ AV P

VM Init

AV RS Cognition

Motor

Vision

Audition

VM

AV

VM Feature Prep

AV Feature Prep AV Init AV Detect Speech

VM Detect

Schedule chart for Schumacher, et al. (1997) perfect time-sharing model. VM = visual-Manual ask, AV = auditory-verbal task, RS = response selection.

Location Discrim. Tone Discrim.

0

50

100

150

200

250

300

350

400

450

500

Task

Single-task

Dual-task

ACT-R/PM simulation of Schumacher, et al. (1997) perfect time-sharing results.