TOPIC 1: COGNITIVE ARCHITECTURETHE FIXED PARTS OF COGNITIVE

PROCESSES

No matter what your favorite computational theory may be, it always assumes certain fixed

properties of the system within which it functions

AMONG THE ARCHITECTURAL PROPERTIES ASSUMED BY ANY COMPUTATIONAL

THEORY:● The form of representations. Computational

theories typically assume that representations consist of symbol structures. But how do they represent magnitudes? By numerals or analogs?

● Available operations. The building blocks of processes that combine to form algorithms.

● Constraints Information flow and on subprocesses (e.g., encapsulated modules?)

● Fixed capacities determined by the innate structures of the mind/brain, as opposed to those that depend on their niche environment.

AN EVEN MORE FUNDAMENTAL REASON WHY IT IS ESSENTIAL TO GET THE COGNITIVE ARCHITECTURE

RIGHT IS THE CRITICAL ROLE OF "COGNITIVE CAPACITY"

Because an organism may remain in its ecological or social niche, only a small fraction of its behavioral repertoire is ever actually observed. But an adequate explanatory theory must reveal an organism's structure, or its cognitive capacity. To do that it must account for the organism’s entire potential behavioral repertoire.

That’s why “variance accounted for” is a poor measure of a theory’s explanatory

value.

OBSERVED REGULARITY VERSUS CAPACITY:

THE DIFFERENCE BETWEEN EXPLANATIONS THAT APPEAL TO MENTAL ARCHITECTURE AND THOSE THAT APPEAL TO REPRESENTATIONS OR TACIT

KNOWLEDGE

The parable of the found mysterious box:

Walking through a field one day, a cognitive scientist comes upon a mysterious box. The box has a meter that records some aspect of its behavior. After many days of recording, the scientist finds some robust behavioral regularity of the box’s recording. What does it tell him about the nature of the box or about its intrinsic causal properties?

OBSERVED PATTERNS OF THE MYSTERY BOX

What does this behavior pattern tell us about the nature of the box?

(Single blip precedes double blip)

(Double blip precedes single blip)

time

AN ILLUSTRATIVE EXAMPLE: MYSTERY BOX

What does this behavior pattern tell us about the nature of the box?

Careful study reveals that pattern #2 only occurs in this special context when it is preceded by pattern A

IT TELLS US (VERY NEARLY) NOTHING ABOUT THE NATURE OF THE SYSTEM

UNDER STUDY

● Why? Because the observed behavior, although it is an objective true record, is but a small part of what the box is capable of.

● The sample we observed is attributable to the environment – including what the box is used for – rather than to the fixed structure (the architecture) or the nature of the box.

HOW CAN AN OBJECTIVE RECORD OF BEHAVIOR NOT TELL YOU ABOUT THE NATURE OF A SYSTEM?

WHAT MORE IS THERE?● In this example, what the scientist found happens to be

a box that is transmitting English messages in international Morse code.

● Two short blips is a code for i, one blip is an e and a long-short-long-short sequence is a c. Thus the regularity that the scientist found can be

explained by the spelling rule in English: “i before e except after c”!

● Nothing inside the box can explain that pattern because the box has a capacity not revealed by its usual behavior. It could transmit very many unobserved patterns because it has the capacity to do so.

THE MORAL: REGULARITIES IN

BEHAVIOR MAY BE DUE TO EITHER:

1. The inherent nature of the system or its structure or architecture.

2. The content of what the system represents (what it “knows”).

WHY IT MATTERS:

A great many regular patterns of behavior reveal nothing more about human nature than that people do what follows rationally from what they believe. An example from language understandingThe example of human conditioning

ANOTHER EXAMPLE WHERE IT MATTERS:

THE STUDY OF MENTAL IMAGERY

Application of the architecture vs knowledge distinction to understanding what goes on when we reason using mental images

EXAMPLES OF BEHAVIOR REGULARITIES ATTRIBUTABLE TO

TACIT KNOWLEDGE

● Color mixing, conservation of volume● The effect of image size ?● Scanning mental images ?

WHAT COLOR WOULD YOU SEE WHEN THE FOLLOWING TWO COLOR

FILTERS OVERLAP?

?

WHERE WOULD THE WATER GO IF YOU POURED IT OVER A FULL BEAKER OF

SUGAR?

Is there conservation of volume in your image? If not, why not?

STUDIES OF MENTAL SCANNINGDOES IT SHOW THAT IMAGES HAVE METRICAL SPACE?

(Pylyshyn & Bannon. Described in Pylyshyn, 1981)

Conclusion: The image scanning effect is Cognitively Penetrable i.e., it depends on goals and beliefs, or on Tacit Knowledge.

Do mental images have size?Imagine a very small mouse. Can you see its whiskers? Now imagine a huge mouse. Can you see its whiskers?

Which is faster?

PART 2: ATTENTION AND SELECTION

The next set of slides concern how information is filtered and encoded in

order to alleviate the information overload faced by perceptual systems

THERE ARE MANY OTHER MORE DETAILED PROPERTIES OF COGNITIVE ARCHITECTURE

THAT HAVE BEEN STUDIEDOne of the most studied concerns the input systems:●How does the mind take in information (through perception)?●Since there is much more information in the perceived world than the brain is able to assimilate, how does it deal with this mismatch?●Among the mechanisms for dealing with speed mismatch (say in computers) are:

Buffering and fast shortcut ways of dealing with buffer contents (modularity of input systems)

Filtering (selective attention): Selection by location or what else? Encoding in terms of more efficient codes (more efficient for our

cognitive architecture)

OTHER FINDINGS RELATING TO COGNITIVE ARCHITECTURE: INPUT SYSTEMS

● Modularity of inputs. There is a large part of the first stages in vision that are encapsulated and insensitive to what the organism knows or believes (called Early Vision)

● Selection of inputs to be encoded (Selective Attention)● A major area of study in Cognitive Science is the question

of which properties are encoded in early vision (e.g., VAL)● Start with the observation that we see (in the sense of

information getting through the eye) much more than we are able to deal with, so some filtering is necessary.

● How and by what properties is information filtered?

BROADBENT’S FILTER THEORY

Broadbent, D. E. (1958). Perception and Communication. London: Pergamon Press.

Limited Capacity Channel

Effectors

Store of conditional probabilities of past events (in LTM)

Filt

erMotor planner

Ver

y Sh

ort T

erm

Sto

re

Sens

es

Rehearsal loop

ALONG WHAT DIMENSIONS IS HUMAN INFORMATION PROCESSING CAPACITY

LIMITED?● Channel capacity: Shannon-Hartley Theorem

● This formula says that the capacity of a channel to transmit information depends entirely on such physical properties as its bandwidth in cycles/sec

● But that does not work for human perception/memory. It seems that our capacity has to be measured in the number of coding units or “chunks” (George Miller’s classic paper “Magic Number 7 plus or minus 2”)

EXAMPLE OF THE USE OF CHUNKING

• To recall a string of binary bits – e.g., 00101110101110110101001

• People can recall a string of about 8 binary integers. If they learn a binary encoding rule (000, 011, 102, 113) they can recall about 8 such chunks or 18 binary bits. If they learn a 3:1 chunking rule (called the Octal number system) they can recall a 24 bit string, etc

WHAT DOES Selective Attention SELECT? WHAT IS THE BASIS FOR VISUAL

SELECTION?● If attention is selection, what does visual attention select?

An obvious answer is places. We can select places by moving our eyes so our gaze lands on different places.

When places are selected, are they selected automatically?

Must we always move our eyes to change what we attend to? Studies of Covert Attention-Movement: Posner (1980).

How does attention switch from one place to another?

Is it always the case that we attend to places? Can we attend to any other property? Can we select on the basis of color, depth, spatial frequency, affordances, or the property a painting has of having been painted by Da Vinci (A property to which Bernard Berenson was able to attend extremely well). cf Gibson

COVERT MOVEMENTS OF ATTENTION

Example of an experiment using a cue-validity paradigm for showing that the locus of attention moves without eye movements and for estimating its speed. Posner, M. I. (1980). Orienting of Attention. Quarterly Journal of Experimental Psychology, 32, 3-25.

THE OBJECT-BASED VIEW OF ATTENTION SELECTION

● There are good reasons for supposing that attention attaches itself to objects rather than locations

WE CAN SELECT A SHAPE EVEN WHEN IT IS INTERTWINED AMONG OTHER SIMILAR

SHAPES

Are the green items the same? On a surprise test at the end, subjects were not able to recognize shapes that had been present but had not been attended – i.e., in this case they had not appeared in green.

IF YOU DO NOT ATTEND TO A VISUAL PATTERN YOU MAY NOT

SEE IT!

● Inattentional Blindness Mack, A., & Rock, I. (1998). Inattentional blindness. Cambridge, MA: MIT Press.

● Change blindness

ANOTHER NEGATIVE ATTENTION EFFECT: INATTENTIONAL BLINDNESS

INATTENTIONAL BLINDNESS The main task is to report which of two arms of the + is longer.

Subjects fixated their gaze at the intersection of the + lines After many trials a critical trial occurs in which a small square appears. On only this trial subjects were asked if anything different appeared

25% of subjects failed to see the square when it was presented in the parafovea (2° from fixation).

But 65% failed to see it when it was at fixation!

When the background cross was made 10% as large, Inattentional Blindness increased from 25% to 66%.

It is not known whether this apparent “blindness” is due to concentration of attention on the primary task, or whether there is inhibition of outside regions.

WHAT IS ATTENTION IS FOR? TREISMAN’S Attention as Glue HYPOTHESIS

The purpose of visual attention The purpose of visual attention is to is to bindbind properties together in properties together in order to recognize objectsorder to recognize objects

HOW ARE CONJUNCTIONS OF FEATURES DETECTED?

Under these conditions Conjunction Errors are very frequent

Read the vertical line of digits in the following brief display

What color was the N? What color was the O? What letters were in red?

RAPID VISUAL SEARCH (TREISMAN)

Find the following simple figure in the next slide:

This search is called a “popout” and does not require focused attention

“OBJECTS” ENDURE OVER TIME & SPACE

Several studies have shown that what counts as the same object endures over time and over changes in location – regardless of what properties these objects have and regardless of whether the properties change;

This gives what we have been calling a “visual object” a real physical-object character and partly justifies our calling it an “object”.

MULTIPLE OBJECT TRACKING● One of the clearest cases illustrating object-based

attention is Multiple Object Tracking

● Keeping track of individual visual objects requires a mechanism for individuating, selecting, accessing and maintaining the identity of individuals over time These are the functions we have proposed are carried out

by the mechanism of visual indexes (FINSTs)

We have been using a variety of methods for studying visual indexing, including subitizing, subset selection for search, and Multiple Object Tracking (MOT).

MULTIPLE OBJECT TRACKING● In a typical experiment, 8 simple identical objects are

presented on a screen and 4 of them are briefly distinguished in some visual manner – usually by flashing them on and off.

● After these 4 “targets” have been briefly identified, all objects resume their identical appearance and move randomly. The subjects’ task is to keep track of which ones had earlier been designated as targets.

● After a period of 5-10 seconds the motion stops and subjects must indicate, using a mouse, which objects were the targets.

● People are very good at this task (80%-98% correct). The question is: How do they do it?

MULTIPLE OBJECT TRACKING EXPERIMENTS

● Examples of Multiple Object Tracking displays used in our experiments can be viewed at FINST.com and also http://ruccs.rutgers.edu/faculty/pylyshyn/DemoPage.html