7. Larkin - Cognitive Modeling

8/7/2019 7. Larkin - Cognitive Modeling

http://slidepdf.com/reader/full/7-larkin-cognitive-modeling 1/31

1

COGNITIVE MODELING AND ROBUST

DECISION MAKING14 March 2011

Dr. Willard Larkin for Dr. Jun Zhang

Program Manager

AFOSR/RSL

AFOSR

Distribution A: Approved for public release; distribution is unlimited. 88ABW-2011-0784



2

PORTFOLIO OVERVIEW

Program Manager: Jun Zhang, Ph.D. (Briefing by Willard Larkin)

BRIEF DESCRIPTION OF PORTFOLIO:

• Experiments and modeling of high-order cognitive processes

for human performance on complex tasks• Computational principles for symbiosis of mixed human-machine systems with allocating and coordinating requirements

SUB-AREAS IN PORTFOLIO:

2313/B: Mathematical Modeling of Cognition and Decision2311/H: Human System Interface and Robust Decision Making



3

Program Components, Goals, andStrategy

Mathematical Modeling of Cognition and Decision(Subtask 2313/B)

Advance mathematical & computational models of human cognition,

especially reasoning, planning, problem solving and decision making.• (Includes Sec. of Defense PBD709 Topic, “Information Fusion and Decision

Science,” emphasizing math foundations of machine learning)

Human-System Interface(Subtask 2311/H)

Advance research on mixed human - machine systems to aid inference,

communication, prediction, planning, scheduling, and decision making.

• Includes “Robust Decision Making” Discovery Challenge Thrust (DCT)

Primary Strategy:• Forge useful connections among experts in math, computation,neuroscience, and cognitive behavior.

• Seek algorithms for adaptive intelligence inspired by brain science



4

TechHorizonsPriority Key Technology Areas

(COGNITIVE MODELING AND ROBUST DECISION-MAKING, RSL)

• Autonomous systems• Autonomous reasoning and learning

• Resilient autonomy

• Complex adaptive systems

• V&V for complex adaptive systems

• Collaborative/cooperative control

• Autonomous mission planning

• Cold-atom INS

• Chip-scale atomic clocks

• Ad hoc networks

• Polymorphic networks

• Agile networks

• Laser communications

•Frequency-agile RF systems

• Spectral mutability• Dynamic spectrum access

• Quantum key distribution

• Multi-scale simulation technologies

• Coupled multi-physics simulations

• Embedded diagnostics

• Decision support tools

• Automated software generation

• Sensor-based processing

• Behavior prediction and anticipation

• Cognitive modeling

• Cognitive performance augmentation

• Human-machine interfaces



5

(2) Reinforcement Learning : Balance short- and long-term goalsin executing sequential, continuous tasks.

Examples of Algorithms forCognition and Decision

(1) Information Accumulation: Seek optimal trades between speedand accuracy for decisions under time pressure.

(3) Categorization & Classification: Generalize from past examplesto future encounters by optimally regulating complexity anddata-fitting performance.

(4) Causal Reasoning and Bayesian inference: Seek optimalfusion of prior knowledge with new evidence for reasoning andprediction under uncertainty, ambiguity, and risk.



6

All Projects – 6 Clusters(For program review purposes only)

1. Mathematical Foundation of Decision Under Uncertainty

2. Neural Basis of Decision Making Under Time Pressure

3. Optimal Planning/Control via Reinforcement Learning4. Robust Classification and Prediction (includes Machine

Learning Sub-program)

5. Memory, Context, and Causal Reasoning

6. Vision, Communication, and Autonomous Systems

Goal: Supporting hum an-in-the-loop in various cognitive domains



7

Recent Transitions

To IARPA and US Army: Mathematical implementations of Cultural

Consensus Theory to improve forecasting technology and informationaggregation for analysts. IARPA Program: “Integrated Cognitive-NeuroscienceArchitectures for Understanding Sensemaking” (ICArUS) Program (Dr. W.Batchelder, U.C. Irvine.)

To HRL Laboratories: Techniques for spike-time encoding anddecoding of information in analog signals. (Dr. Aurel Lazar, Columbia Univ.POC: Peter Petrie, Malibu CA.)

To AFRL/RH: Techniques for implementing generative and persistentmodels for information sampling and learning. (Dr. B. Love, U. Texas, Austin..POCs: S. Douglas and C. Myers, AFRL/RH.)

To AFRL/Rome Lab: An AFRL-RI funded project with G. Kreiman(Harvard): Implementing kernel machine techniques for modeling feedback invision systems. (Dr. T. Poggio (MIT). POCs: Todd Howlett and Yuri Luzanov.



8

CLUSTER one

Mathematical Foundations of Decision Making and Reasoning Under Uncertainty

1. Narens (UC Irvine): New axioms of subjective probability.

2. Chichilnisky (Columbia): Topological methods for unexpected events with

catastrophic risk (“Black Swan” theory).

3. Halpern (Cornell): Theory for games with deficient awareness.

4. Schweickert (Purdue): Conditional independence and selective influence.

5. Luce (UC Irvine): Conjoint structures for multi-modal sensory scaling.

6. *Bringsjord (RPI): Fusing analogical and deductive reasoning.

Scientific Goal: Develop formal foundations for cognitive systems.



9

Coherence:assumed in the subjective,not the objective domain

SUBJECTIVE RATIONALITY:

Foundations of Subjectively RationalDecisions (Louis Narens, UCI)

Scientific Challenge:• Emotions and contexts affecthuman decisions.• Design a decision theory as closeas possible to rational decisiontheory, but allow for such effects.

New Axioms & Theorems fora Calculus of SubjectiveRationality:

Generalizes notion of a finitelyadditive probability measure on anEvent Space X with a Pseudo-Complement Such that:

Intuitively: is iis the largest eventthat “seems like” the complement of A



10

CLUSTER two

1. McClelland (Stanford): MURI on neural basis of decision making

with Ditterich (UC Davis): multi-alternative perceptual decisions.

2. Pouget (Rochester): optimal cue integration for decision making.

3. *Lee (UC Irvine): adaptive threshold for speed-accuracy tradeoff.

4. *Ratcliff (OSU): change detection and performance monitoring.

5. T. Zhang (Johns Hopkins): hippocampal spatial memory.

6. Chua (Berkeley): memristor models for neurons and synapses.

Neural Basis of Decisions under Time Pressure

Former AFOSR Program Manager Jerome Busemeyer started this theme. It hasnow become main-stream.



11

McClelland MURI (2007)Neural Basis of Decision Making

• Proposes accumulators of noisy evidence , y1, y2, with leakage , andmutual inhibition :

dy 1/dt = I1-gy 1–bf (y 2 )+x1

dy 2 /dt = I2-gy 2 –bf (y 1)+x2

f (y) = [y]+

• In time controlled tasks, chooseresponse 1 if y1-y2 > 0

I1 I2

y1 y2

GOAL: Build a lattice of alternative models, motivated byneurological data, for decision-making under time pressure;test them in human tasks.

A GENERALIZED DRIFT-DIFFUSION MODEL:

Time (sec.)

EVIDENCE LEVEL

Y1

Y2

“LEAKY COMPETING ACCUMULATORS

”

Y1 Y2



12

( ) ( ) ( )d t R t R t

( ) (1 )

t

d t kS e

Alternative formulationsfor “Accumulators of

Competing Evidence”

can be distinguished byqualitative datasignatures.

Inhibition DominatesLeakage Dominates

McClelland MURI (2007)Neural Basis of Decision Making



13

How does reward for fast decisionsaffect how information is integrated ?

1. Reward acts as an information input from the rewardcue onset to the end of the integration period

2. Reward influences the state of the informationaccumulators before the stimulus onset

3. Reward introduces a fixed offset in the decisionvariable.

Three Alternative Hypotheses:



14

Reward Bias Signatures Plotted againstDecision Latency

Data for 4 Individuals

REWARD BIASUNDERESTIMATESOPTIMAL BIAS;BOTH DECLINETO A NON-ZEROPLATEAU

Data lead to a clear choice among the three alternative models



15

Qualitative Data Signatures Rule Out H1and H3 (McClelland, cont.)

X X

“REWARD SETS A

FIXED OFFSET”

“REWARD MIMICS

INFORMATION INPUT’”

“REWARD PRE-

CONFIGURES THEACCUMULATORS”



16

1. Barto (U Mass): domain competency and skill learning.

2. Tenenbaum (MIT): causal modeling of representation learning.

3. Love (UT Austin): feature selection and information sampling.

4. Jones (Colorado): kernel-based representation for RL.

5. Mahadevan (U Mass): graph-theoretic methods for RL.

6. Qian (Columbia): attention and sensori-motor control.

Achieving Optimal Planning and Control via

Reinforcement Learning

GOAL: Build on past success of the reinforcement learning algorithm(which cuts across optimization, machine learning, animal learning, andneuroscience), push new frontiers , e.g., intrinsic motivation, hierarchicalplanning, feature selection, attention.

CLUSTER three



17

Automated Representation Discovery inSequential Decision Problems

Scientific Challenge:To avoid human trial-and-error, discover how togenerate novel representations of complex tasks,such as Markov Decision Problems (MDPs), andother stochastic problems, to facilitate their solution.

A Recursive Basis FunctionDecomposition for a “Taxi” Problem

Sridhar MahadevanUMass Amherst

Osentoski and Mahadevan,AAMAS 2010

A Graph-Theoretic Approach for Task Hierarchies:

A State Graph of a“Taxi” Subtask

http://en.wikipedia.org/wiki/File:Uma_seal.png

http://www.cs.umass.edu/faculty/directory/mahadevan_sridhar



18

Automated Basis Function ConstructionSpeeds Reinforcement Learning

Convergence Results Comparing FourGraph Laplacian Approaches

260 Basis functionsfrom a joint State-Action graph

264 Basis functionsfrom a State graph

Number of Learning Iterations ina 4-Room “Gridworld” Problem

Steps to Goa

l

S. Osentoski and S. Mahadevan, UMass, Amherst

“GRIDWORLD”



19

1. Zhu (UW Madison): unsupervised and semi-supervised learning

2. Wang (Wright State): semi-supervised learning for structured prediction

3. Nosofsky (Indiana): rule-based categorization

4. Batchelder (UC Irvine): aggregation, cultural consensus model

Robust Classification and Prediction

Includes Sub-Program inMathematical Foundations of

Machine Learning(from PBD 709 funds)

CLUSTER four



20

The Scientific Challenge:

• Informants or respondents share culture-specificknowledge, assumptions and beliefs

• They respond to questionnaire items, but we do notknow which answers, if any, represent their special

shared knowledge -- Nor do we know, a priori,anyone’s “cultural competency,” response biases, or

the “cultural saliency” of the questions we ask.

• This problem inverts the usual psychometric testingsituation in which the answer key is already known.

The Problem:

Develop mathematical models of the response process from which“culture -specific answers” and informant characteristics can be

inferred to identify belief structures and to forecast behaviors.

Wm. BatchelderUC Irvine

Mathematical Development ofCultural Consensus Theory

Algorithms transitioned to IARPA and Army

http://en.wikipedia.org/wiki/File:Ucirvine-seal.png



21

Information Fusion and Decision Science(PBD709 Topic from SECDEF)

1. Lafferty (CMU): statistical learning for structured high dimensional data

2. Poggio (MIT): hierarchical kernels for visual object recognition

3. Xu (Syracuse): kernel learning and refinement in RKHS

4. Bertsekas (MIT): large-scale convex optimization and approximate DP

5. T. Zhang (Rutgers): multi-stage convex relaxation in machine learning

6. *Casazza (U Missouri): frames theory and compressive sensing

Mathematical Foundations of Machine Learning

GOAL: Investigate fundamental mathematical concepts invoked inmachine learning; extend them to the domain of cognitive informationprocessing.



22

Frames and Quantitative RedundancyP.Casazza (Missouri), M. Fickus (AFIT)

Prof. Casazza

Scientific Challenge:

MISSOURIAFIT PRINCETON

Develop new mathematics for dimension-reduction problems encountered in patternrecognition and information fusion.

P.G. Casazza, M. Fickus, D.G. Mixon, Y. Wang and Z.Zhou, Constructing tight fusion frames, to appear in

Applied Computational Harmonic Analysis.

P.G. Casazza, M. Fickus, D.G. Mixon and J.C. Tremain, The Bourgain-Tzafriri conjecture and concrete constructions of non-pavableprojections, to appear in Operators and Matrices.

Recent Publications:

Approach:

FRAME THEORY -- a rigorous infrastructure forstudy of redundant linear measurements –

developed mainly by Peter Casazza

http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WB3-50860KF-1&_user=1082852&_coverDate=06%2F08%2F2010&_rdoc=1&_fmt=high&_orig=search&_origin=search&_sort=d&_docanchor=&view=c&_searchStrId=1493301230&_rerunOrigin=google&_acct=C000051401&_version=1&_urlVersion=0&_userid=1082852&md5=abacaeeb51c62367953c6bc3f7dba6d6&searchtype=a

http://arxiv.org/abs/1006.2677









http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WB3-50860KF-1&_user=1082852&_coverDate=06%2F08%2F2010&_rdoc=1&_fmt=high&_orig=search&_origin=search&_sort=d&_docanchor=&view=c&_searchStrId=1493301230&_rerunOrigin=google&_acct=C000051401&_version=1&_urlVersion=0&_userid=1082852&md5=abacaeeb51c62367953c6bc3f7dba6d6&searchtype=a

http://en.wikipedia.org/wiki/File:PrincetonShield.png

http://en.wikipedia.org/wiki/File:Air_Force_Institute_of_Technology.png

http://en.wikipedia.org/wiki/File:University_of_Missouri_seal.png



23

A Joint Initiative with AFRL/RH andGeorge Mason University

A Center of Excellence in Neuro-Ergonomics, Technology and Cognition

Started 15 July 2011Convened 14 September 2010 at George Mason University

Prof. RajaParasuraman

Goal:Stimulate productive new collaborations withscientists in AFRL’s Human Effectiveness

Directorate on a broad range of problems incognitive science

GENETIC DETERMINANTS, NEUROADAPTIVE TRAINING, TRUST IN AUTOMATION

NEUROIMAGING, ATTENTION, MULTI-TASKING, MEMORY, SPATIAL COGNITION

3D S i l Vi li i

http://www.google.com/imgres?imgurl=http://www.princeton.edu/~jasonf/Pictures/AFOSR%2520logo.jpg&imgrefurl=http://www.princeton.edu/~jasonf/links.htm&h=402&w=409&sz=54&tbnid=5utaCfVVR0oP_M:&tbnh=123&tbnw=125&prev=/images%3Fq%3Dafosr%2Blogo&zoom=1&q=afosr+logo&hl=en&usg=__QxhKSmFDWveoUq0DSpAyYqrrWq4=&sa=X&ei=PvhSTaLgJInLgQf_-4mLCA&ved=0CCAQ9QEwAw

http://en.wikipedia.org/wiki/File:Gmuseal.svg



24

3D Spatial Visualization(Glenn Gunzelmann AFRL 711HPW/RH)

TASK: Imagine steps Up, Down, Left,

or Right, one by one, along a randompath in a 3D grid. Detect each return toyour prior positions.

Grid path is held in“The Mind’s Eye”

Scientific Questions:• What are the constraints and limits of

mental navigation?

• Do individuals differ in this ability?

• How and why is the ability fragile?

• Can a computational model (ACT-R)account for the phenomena?



25

Path Visualization Performance:Depends on No. of Steps between Re-Visits

19151173

100

90

80

70

60

Percent Correct

PERFORMANCE FOR 7 X 7 GRID

Number of Path Segments (Lag)Separating Return Visits

• Individual performance varieswidely – some people areextraordinarily competent.

• The ACT-R model for one sizegrid correctly predicts data for

other sizes.

• Increasing path length boostsassociative interference, butthis has a very minor role in thelag effects shown here.

• The ACT-R activation decayrate parameter best explainsindividual differences.

Gunzelmann, et al. 711HPW AFRL/RH



26

CLUSTER six

1. Pizlo (Purdue): robotic navigation with symmetric-based shape recognition

2. Tyler (SKERI): surface representation as intermediate vision

3. Yu (Indiana): development of symbolic processing in communication

4. Damasio/Yau/ (USC/Geometric Informatics): Ricci-flow method for aligning

brain imaging data

5. Myung (Ohio State): Cognitive Modeling Repository

6. Lazar (Columbia): Spike-time encoding & reconstruction of visual displays

Novel Approaches to Vision, Communication,

& Autonomous Systems

NOVEL APPROACHES TO: Topics such as intermediate representation in vision,emergence of symbolic representation for communication, analysis of brain imagingdata, cognitive and biomimetic approaches to machine intelligence.

Spike Time Encoding and Decoding of



27

Spike-Time Encoding and Decoding ofVisual Scenes (Aurel Lazar, Columbia U.)

Scientific Challenge:

INITIAL STAGES OF VISION

GENERATE NEURAL SPIKE TRAINS

SUBSEQUENT STAGESASSUMED INVERTIBLE

Invent a computationalengine to encode visual

scenes and to reconstructthem from spike-timing data

Analog Signal Recovery from a Model



28

Bipolar Neuron modelconverts analog signalinto a series of “On”

and “Off” spikes at

threshold crossings.

Analog Signal Recovery from a ModelBipolar Cell & Spike-Timing Analysis

ON ---

OFF --

Signal Recovery

Bipolar cell modelwith thresholdingand feedback

Aurel A. Lazar, Columbia University



29

Video Stream Recovery from SpikingNeurons– Using Hodgkin-Huxley Model

ORIGINAL RECONSTRUCTED

Aurel A. Lazar, Columbia University

Progress:

Reconstruction minimizes a 3-component costfunction that prevents over-fitting to noise,measures reconstruction error, and regulates thetrade between smoothness and match to data,

• First demonstrationof natural scenerecovery fromspiking neuronmodels based upon

an architecture thatincludes visualreceptive fields andneural circuits withfeedback.

• Scalable decodingalgorithms weredemonstrated on aparallel computingplatform.

Transitions to HRL Labs and others

SUMMARY:



30

SUMMARY:Transformational Impacts & Opportunities

4

• Coherent investments in the development ofcomputational algorithms for adaptive intelligence

• Program represents a new level of mathematicallysophisticated cognitive science:

• Responds to 6 high-priority topics in AF TechHorizons

• Several research areas with extraordinary promise:

• Novel math frameworks to characterize risk, uncertainty, & subjectivity• Autonomous methods to optimize complex reinforcement learning• Robust enhancements of classification, recognition, and reasoning

• New cohort of interdisciplinary theorists and experimentalists• Embraces wide-ranging ideas and non-traditional expertise• Supports multiple collaborations between Universities and AFRL



31

Questions?

Thank you for your attention

Willard Larkin, Program Manager, AFOSR/RSL

703-696-7793

[email protected]

SPECIAL THANKS TO LT. IAN PRUDHOMMEFOR HELP WITH THIS PRESENTATION

mailto:[email protected]

mailto:[email protected]

Documents

7. Larkin - Cognitive Modeling