Cognitive Neuroscience Section 4

Perceptual categorization Perception, attention, and memory are all interrelated. From the perspective of memory, perception is seen as memory updating by new sensory experience. Perception involves sensory analysis within the framework of previous experience as stored in memory. The perceptual categorization of sensory input is guided by previously established memory. For perception to be guided implies that it is an active, not a passive, process. But this implication is not universally accepted. For the case of vision, we will consider whether it is more appropriate to view perception as being passive or active.

The difference between passive and active perception in computational terms may be that passive processing is strictly feed-forward (top) and active processing involves feed-back (bottom) from higher “down-stream” stages to lower “up-stream” stages.

The argument for passive visual perception Hypothesis:

The visual stimulus operates like a stamp or imprint. All the information needed for visual perception is presented to the retina from the external world: environmental light patterns impinging on the retina determine retinal activity; these light patterns are faithfully reproduced in retina, LGN, & V1.

Implication: Visual perception only requires progressive processing of visual information, and so depends only on feedforward processes.

Neural Mechanism: Low-level visual features are detected in V1, and progressively more elaborate features are detected in higher visual areas. Visual recognition occurs when high-level symbolic features are compared with features stored in memory.

Evidence: The bottom-up projection from retina to V1 and higher visual areas is retinotopic, suggesting faithful reproduction of light patterns. Electrophysiological recording shows that V1 neurons are sensitive to low-level visual features & higher-level visual neurons are sensitive to more complex visual features. Lesions along path from retina to V1 produce blindness for the part of the visual field corresponding to the lesioned cells; lesions in visual areas outside V1 produce "mind-blindness" -- lack of visual comprehension.

The argument for active visual perception Hypothesis:

The visual sensorium is constantly changing, and real-world visual stimuli are ambiguous and indeterminate. Not all the information needed for visual perception is presented to the retina from the external world, and retinal activity does not specify the light pattern to be perceived.

Implication: Visual perception requires information supplied by the brain, and so depends on feed-back (top-down) as well as feed-forward (bottom-up) processes.

Neural Mechanism: Low-level activity patterns in V1 undergo progressive elaboration through an iterative feedforward-feedback cycle involving higher visual areas. Visual recognition occurs when high-level visual areas produce patterns from memory representing hypotheses that are consistent with low-level activity patterns.

Evidence: The flux of photons at the retina is highly variable in time. Top-down inputs to V1 are more prevalent than bottom-up inputs. Visual perception can occur without visual stimulation, as in imagery.

ART The Adaptive Resonance Theory (ART) is a class of computer models that captures some of the cardinal features of active visual perception. Grossberg & Carpenter developed ART beginning in the 1980s. ART models can be trained to perform pattern recognition, i.e. after learning, they recognize a category of input patterns. The ART Architecture In the ART model, input is delivered to a “feature representation field” (F1), which interacts with a “category representation field” (F2).

The ART search cycle: 1. An input pattern I registers itself as a short-term memory activity pattern x in

field F1 (frame a) 2. Pattern x is transformed into a compressed pattern T in field F2 (frame a) 3. Competition occurs among nodes in F2 for the strongest match with T (frame a) 4. Activation of memory trace y occurs at the node in F2 having the strongest

match (frame a)

5. Memory trace y is treated as a hypothesis to be tested by matching its top-down

pattern V against pattern x that selected it (frame b) 6. Pattern V is fed back from F2 to F1 where it is matched against pattern x (frame

b)

7. Those portions of pattern x that match top-down pattern V are suppressed. The portion of x that is not suppressed is the residual activity pattern x* (frame b)

8. Residual activity pattern x* represents a pattern of critical features in the current input x that are different from what was hypothesized as y (frame b)

9. Steps 2-8 are repeated iteratively: residual pattern x* is fed forward from F1 to F2, where another memory trace emerges through competition and feeds back a new pattern V* to F1 (frame b)

10. Scenario 1 (an adequate match is found): after a number of iterations, V converges to x and x* gets smaller and smaller; if x* becomes sufficiently small, the final y represents an adequately matched memory pattern which is the percept of I (frame b)

11. Scenario 2 (no adequate match is found): if the input is too novel to satisfy the matching criterion, V does not converge to x and x* remains large. F2 is reset (frame c), and a new memory trace y* is created corresponding to x* (frame d)

Application of the concept of active perception to the cortex 1. We infer that perception by the brain involves a similar iterative matching between a set of sensory impressions and pre-established cognits:

a) If an adequate match occurs, the matching cognit becomes the percept b) If no adequate match occurs, a new cognit is created and becomes the

percept 2. Perceptual categorization of sensory information does not require consciousness, and we are not normally aware of the different processes, executed in parallel, that underlie perception. 3. Perceptual processing is usually guided by selective attention. Attentional perception is conscious and is executed sequentially. However, this does not mean that we are conscious of all the steps involved in perceptual processing. Perhaps, we are only aware of the results. Attention may be viewed as an aid to categorical perception à it often leads to formation of a new category or to re-categorization. Without selective attention, sensory systems would be either overwhelmed or oblivious to important sensory details. In other words, the capacity of these systems for processing sensory information is limited.

4. Mood can affect perceptual categorization by influencing attention. E.g. depression often dulls perception, & produces anhedonia (the absence of pleasure from experience that would normally be pleasurable) and negative mis-categorizations (e.g. hypochondria). Emotional connotations can affect perceptual categorization regardless of whether we are conscious of them or not.

The Gestalt Sensory information arriving at the cortex is organized according to sets of spatial and temporal relations between elementary sensory features. These relations define the informational structure that become stored in memory through perception. The Gestalt school of psychology studied the structured patterns of visual images. They sought to explain how we are able to identify regularities in the sensory world, e.g. visual objects. The term “gestalt” has come to mean a pattern of elements unified as a whole with properties that cannot simply be derived from the parts. Basic questions addressed by Gestalt psychology:

a) how do we perceive objects as individual entities? b) how do we segregate objects from others around them? c) how do we segregate objects from the background? d) how is the identity of objects preserved despite discontinuities, distortions, or

occlusions?

Figure-Ground Perception Visual perception involves the recognition of objects (figure) as distinct from their backgrounds (ground). Objects appear to “stand out” from the background. Figure-ground perception in vision usually depends on edge assignment and how that effects shape perception. It may be bistable, meaning that either of two (stable) figures may be perceived. This may occur when a visual pattern is too ambiguous for the visual system to recognize it with a single unique interpretation. The differentiation between foreground and background of a sensory scene also exists for perception in the other sensory modalities, such as hearing and touch.

Gestalt principles of organization (Fuster, Figure 4.2): a) proximity b) similarity c) continuation d) closure

Gestalt principles apply at the psychophysical level, i.e. they are valid for describing perceptual phenomena, but not necessarily the underlying neural phenomena. Past attempts by Gestalt psychologists to theorize about cortical phenomena were largely unsuccessful. The generalization of Gestalt principles to cognitive function would help to define cognitive structure in terms of relationships among components. Such a generalization would have to apply to:

a) other sensory modalities than vision b) higher levels of abstraction c) temporal as well as spatial organization

If it could be shown that Gestalt principles apply not only to perceptual phenomena, but also to perceptual cognits, this would help to establish an isomorphism between the structure of cognition and the structure of cortical networks.

Cortical dynamics of perception Perception may be viewed as the sensory activation of a perceptual cognit in posterior cortex that represents in its structure a pattern of relationship existing in the sensory environment. Perceptual processing consists of categorizing sensory information according to the memory structure that has been built up by prior experience. A. Principles of categorical perception by cognit activation

1) Perception consists of the categorization of sensory input according to prior experience.

2) Perception takes place within a complex, hierarchically organized system of cortical networks (perceptual cognits) that represent established memory.

3) Established perceptual cognits both guide perception and are modified by it. 4) Sensory activity has spatial structure that may be similar to the cognit structure

to varying degrees. 5) Cognits are activated according to the similarity of their structure with that of the

sensory activity. 6) Cognit activation is degenerate – a given cognit may be activated by a variety of

sensory activity patterns that are similar to its structure.

B. Putative steps in categorical perception by cognit activation 1) A sensory activity pattern may activate multiple cognits that share common

features. 2) The cognit that is activated the most represents the categorical perception

(recognition) of the input pattern. 3) Sensory patterns that represent familiar objects activate cognits at higher levels

of abstraction, up to the semantic or symbolic level, eventually allowing identification.

4) Sensory activity patterns that represent unfamiliar objects undergo more elaborate analysis, involving iterative matching, at lower levels, eventually leading to creation of a new cognit at a higher level.

5) Multiple sensory activity patterns may be analyzed in this way in parallel.

C. Convergence and divergence 1) Upward convergence in perceptual hierarchies promotes the association of

higher-level cognits with multiple lower-level cognits. 2) Upward divergence promotes the association of a lower-level cognit with

multiple higher-level cognits. D. Cortical areas contributing to perception

1) Primary sensory cortex: involved in constructing sensory activity patterns based on low-level feature detection

2) Unimodal sensory cortex: involved in categorization of sensory activity patterns 3) Transmodal association cortex

a. limbic & paralimbic cortex: evaluation of emotional associations of percepts (in cooperation with amygdala)

b. lateral frontal cortex: evaluation of motor associations, e.g. affordance (in Gibson’s theory of ecological perception, an affordance is a perceived possibility for action)

c. multimodal convergence cortex: evaluation of poly-sensory associations

E. Symbols 1) A symbol may be viewed as an abstract, perceptual category that is represented

by a high-level perceptual cognit having profuse connectivity with lower-level cognits.

2) Symbols, including words and other linguistic structures, are encoded in cognits of the posterior (unimodal & transmodal) association cortex.

3) Symbolic cognits may consist of nodes of convergence. Damasio proposed that convergence zones (nodes) are high-level networks in association cortex that represent association patterns, i.e. they code for specific patterns of sensory activities. A symbol is a pattern of association of sensory activities. When the symbolic cognit is activated, it may re-activate the lower-level sensory cognits from which it was formed.

4) A symbolic cognit is highly degenerate in that it may be activated by many different lower-level cognits. It may be amodal, or may be associated with only one or more modalities. The symbol “airplane” may be considered bimodal – it is defined by visual and/or auditory features, but not by taste.

5) Symbolic cognits are distributed across regions of higher association cortex, including inferior, medial, and superior temporal cortex, Wernicke’s area, and posterior parietal cortex.

F. Perceptual constancy 1) The property of degeneracy points to a possible way that the brain is able to

create perceptual constancy – the ability to recognize the same perceptual entity when the sensory components vary, e.g. to recognize a visual object at different distances and viewing angles.

2) The degree of perceptual constancy evidenced by neurophysiological recording increases with ascending levels of the perceptual hierarchy. The degree of perceptual constancy in an area may reflect the degree of abstraction of categorical representation.

3) Neurons showing constancy for visual objects are found in inferior temporal cortex. Neuronal populations showing constancy for graphic symbols such as letters & words are found in Wernicke’s area.

Perceptual binding Perceptual binding refers to the joining together of the associated sensory features of a perceptual object into a gestalt. A cognit for the Gestalt represents the perceptual object. The object is categorized, and perception of the object occurs, when the cognit is activated. A neural binding mechanism is needed to join together the neural activity in different parts of the cognit.

What is the neural mechanism of neural binding? A. Evidence from neuroelectric activity Binding has been proposed to operate by the phase synchronization of oscillatory activity from columnar modular assemblies. Assembly oscillations are best observed in the LFP, EEG, and MEG. Oscillations are classified by frequency: delta: 0-3 Hz theta: 4-7 Hz alpha: 8-12 Hz beta: 13-30 Hz gamma: 31-100 Hz Beta & gamma activity together are also called High-Frequency (HF) activity. When the waves of oscillation in different assemblies are aligned in time, they are called phase-synchronized. Assemblies do not communicate by LFP waves. They send pulse activity back and forth along axonal pathways. However, LFP phase-synchronization may be a sign of functional binding of assemblies. In perception, the binding together of distributed assemblies in a cognit may occur by phase-synchronization of HF oscillatory activity.

The study by Tallon-Baudry et al (1995) supports this idea: when subjects are presented with Kanizsa triangle images, the level of HF activity at 300 ms poststimulus in EEG over occipital lobes appears to correspond to the degree of perceived spatial coherence of the triangle (Fig. 4.4).

B. Evidence from functional neuroimaging PET & fMRI (neuroimaging) studies are useful for detecting the activation of cognits. They do not directly image active networks, but rather the “ghosts” of heavily activated nodes (epicenters) of excitatory neuronal activity in the networks. They show the most heavily activated nodes of distributed memory networks. In neuroimaging studies of visual perception, the epicenters of cognits for object categories appear in unimodal visual and multimodal association areas (Fig. 4.5). Fig. 4.6A is compatible with the idea that objects are represented at several hierarchical levels, from sensory to symbolic. Color perception has activation maxima in inferior occipital cortex, whereas color words and color imagery have maxima further anterior in inferior temporal cortex.

Perception-action cycle Perception may lead to other acts of cognition, including action. Sensory and motor structures are interconnected all the way from spinal cord up to cortex. In the cortex, posterior sensory areas are interconnected with frontal areas of comparable hierarchical rank. Networks at any stage of perceptual representation are reciprocally connected with networks at the corresponding stage of executive representation.

An animal’s behavior consists of a succession of adaptive motor acts that are guided by perceived changes in its external and internal environments. Since those acts have effects on those environments, perception and action follow one another in a perception-action cycle: perceptual signals lead to actions that then lead to new perceptual signals, and so on. Sensorimotor integration operates in loops at different hierarchical levels. Instinctual, reflexive, and well-learned integration operates at subcortical levels (e.g. spinal cord, brainstem). Higher processes, such as goal-directed behavior and language, engage the perceptual & executive networks of the cerebral cortex. Perception and action are integrated in loops of cortico-cortical activity flow. In goal-directed behavior, sensorimotor cycling occurs at multiple cortical levels at the same time. In order for goals to be achieved, this cycling must involve orderly sequences of percepts and motor acts. The organization of these events may have a complex temporal structure.

Integration across time is a basic function of the prefrontal cortex. This association area sits at the top of the executive hierarchy, so it is the highest executive structure involved in the perception-action cycle. The prefrontal cortex is responsible for the execution of schemas (temporal gestalts) of behavior. But in order to carry out this function, it must depend on perceptual input from posterior sensory cortical areas. It also must send integrative signals to the sensory systems in the processes of attention and working memory that are necessary for the orderly sequencing of behavior and cognition. Part of the signaling sent from prefrontal to posterior cortex consists of inhibitory feedback to suppress extraneous perceptual inputs into the perception-action cycle.